Smart home voice interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to the field of smart home, and discloses a smart home voice interaction method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a received voice request of a target user, and analyzing a control intention in the voice request; the control intention is compared with request entries in a preset retrieval enhancement generation database, whether the control intention is clear or not is judged, and the retrieval enhancement generation database comprises a home equipment manual, the request entries related to equipment and answer content corresponding to the request entries; and when the control intention is not clear, guiding the target user to correct the voice request until the control intention in the corrected voice request is determined to be clear. According to the smart home voice interaction method disclosed by the invention, the problem that man-machine interaction is difficult to realize smooth and natural communication similar to that between people is solved, so that conversation interaction closer to a real scene is carried out with a customer, and the conversation experience of the customer is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of smart homes, and in particular to a smart home voice interaction method, device, electronic device and storage medium. Background Art

[0002] As an important entry point for users to control home appliances, smart home voice interaction greatly facilitates users to control home appliances. However, current intelligent voice interaction technology can only achieve a shallow level of human-computer interaction. For more complex communication scenarios, such as multi-round dialogues, close contextual connections, logical reasoning, emotional expression, etc., human-computer interaction is difficult to achieve the smooth and natural communication between people, resulting in a low interaction experience. Summary of the invention

[0003] The purpose of the present invention is to at least provide a smart home voice interaction method, which can at least solve the technical problem that human-computer interaction is difficult to achieve smooth and natural communication like human-to-human communication, and at least achieve a smoother and more natural interaction effect.

[0004] To solve the above technical problems, at least one embodiment of the present application provides a smart home voice interaction method, including:

[0005] In response to receiving a voice request from a target user, parsing a control intent in the voice request;

[0006] Comparing the control intention with the request items in a preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes a home appliance manual, device-related request items, and answer contents corresponding to the request items;

[0007] When the control intention is unclear, the target user is guided to correct the voice request until the control intention in the corrected voice request is determined to be clear.

[0008] In some optional embodiments, in response to receiving a voice request from a target user, parsing a control intent in the voice request includes:

[0009] In response to receiving a voice request from a target user, determining whether the content of the voice request belongs to the field of smart home;

[0010] When the content of the voice request does not belong to the smart home field, based on the voice request, a generalized instruction related to the smart home field is generated by using a preset semantic understanding model;

[0011] The control intent in the generalized instruction is parsed.

[0012] In some optional embodiments, before the step of responding to receiving a voice request from a target user and determining whether the content of the voice request belongs to the field of smart home, the step further includes:

[0013] Establishing a training data set for smart home scenarios based on the acquired home device usage data of the target user, wherein the home device usage data includes at least one of a conversation instance between the user and the smart home and the behavior habit data of the target user;

[0014] The initial semantic understanding model is trained using the training data set until a preset training standard is met, thereby obtaining the preset semantic understanding model.

[0015] In some optional embodiments, the semantic understanding model is a pre-trained large model based on the Transformer architecture.

[0016] In some optional embodiments, guiding the target user to correct the voice request includes:

[0017] Searching for request items related to the voice request in a database based on the search enhancement generation, and voice broadcasting the request items related to the voice request;

[0018] Receive feedback information from the target user on a request item related to the voice request, and determine whether a new request item needs to be retrieved and broadcasted to the target user based on the feedback information.

[0019] In some optional embodiments, it also includes:

[0020] When the control intention is clear, a control instruction corresponding to the control intention is generated.

[0021] In some optional embodiments, it also includes:

[0022] When it is impossible to directly answer the target user's question based on the local knowledge of the search enhancement generation database, searching for relevant answering content on the Internet according to the target user's question;

[0023] The reply contents retrieved from the Internet are integrated to generate a final response text.

[0024] At least one embodiment of the present application further provides a smart home voice interaction device, including:

[0025] An intention parsing module, configured to, in response to receiving a voice request from a target user, parse the control intention in the voice request;

[0026] An intention judgment module, used to compare the control intention with the request items in a preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes a home appliance manual, device-related request items, and answer content corresponding to the request items;

[0027] The correction guidance module is used to guide the target user to correct the voice request when the control intention is unclear, until the control intention in the corrected voice request is determined to be clear.

[0028] In some optional embodiments, the intention parsing module includes:

[0029] A judgment unit, configured to judge, in response to receiving a voice request from a target user, whether the content of the voice request belongs to the field of smart home;

[0030] A generalized instruction generating unit, configured to generate a generalized instruction related to the smart home field based on the voice request by using a preset semantic understanding model when the content of the voice request does not belong to the smart home field;

[0031] The intention parsing unit is used to parse the control intention in the generalized instruction.

[0032] In some optional embodiments, the smart home voice interaction device further includes:

[0033] A data set creation module, configured to establish a training data set for a smart home scenario based on the acquired home device usage data of the target user, wherein the home device usage data includes at least one of a conversation instance between the user and the smart home and the behavior habit data of the target user;

[0034] The model training module is used to train the initial semantic understanding model using the training data set until the preset training standard is met to obtain the preset semantic understanding model.

[0035] In some optional embodiments, the correction guidance module includes:

[0036] A search unit, configured to search for a request item related to the voice request based on the search enhancement generation database, and voice broadcast the request item related to the voice request;

[0037] The feedback information receiving unit is used to receive feedback information of the target user on the request item related to the voice request, and determine whether it is necessary to retrieve a new request item and broadcast it to the target user based on the feedback information.

[0038] In some optional embodiments, it also includes:

[0039] When the control intention is clear, a control instruction corresponding to the control intention is generated.

[0040] In some optional embodiments, the smart home voice interaction device further includes:

[0041] A retrieval unit, configured to retrieve relevant reply content on the Internet according to the question of the target user when it is impossible to directly reply to the question of the target user based on the local knowledge of the retrieval enhancement generation database;

[0042] The response text generation unit is used to synthesize the response content retrieved from the Internet to generate a final response text.

[0043] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned smart home voice interaction.

[0044] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned smart home voice interaction when executed by a processor.

[0045] The smart home voice interaction method provided by the embodiment of the present application provides a standard reference for the user's request intention, expression correction, etc. by using a pre-established retrieval enhancement generation database. The method can use the home appliance manual, device-related request items and corresponding answer content in the retrieval enhancement generation database to assist in judging the clarity of the control intention, which not only enables the system to understand the user's request more comprehensively, but also provides it with a rich knowledge base to deal with various possible request scenarios, thereby enhancing the intelligence of the system. At the same time, the interactive method of the present application allows correction guidance when the control intention is unclear, which shows the system's adaptability to uncertainty and changes and improves its flexibility. By comparing the control intention with the request items in the preset retrieval enhancement generation database, the method can more accurately judge the clarity of the control intention. This helps to reduce erroneous operations caused by misunderstanding the user's intention and improves the accuracy of the control instruction. Further, by guiding the user to correct the unclear voice request, the method can gradually approach the user's true intention, thereby ensuring that the control instruction finally executed is consistent with the user's expectations. In the interactive method of the present application, when it is detected that the control intention is unclear, the system does not simply reject or ignore the request, but actively guides the user to correct the expression of the request, making the human-computer interaction smoother and more natural, and more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0047] Figure 1 It is a schematic diagram of the voice interaction process in the prior art;

[0048] Figure 2 This is a process of a smart home voice interaction method provided by an embodiment of the present application Figure 1 ;

[0049] Figure 3 This is a process of a smart home voice interaction method provided by an embodiment of the present application Figure 2 ;

[0050] Figure 4 This is a schematic diagram of a smart home voice interaction device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0051] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0052] In order to facilitate understanding of the embodiments of the present application, relevant content about smart home voice interaction is first introduced here.

[0053] Smart home voice interaction is an important entry point for users to control home appliances. It is common for users to not understand what people are saying. The main reason for this phenomenon is the limitations of voice recognition technology:

[0054] 1) Imperfections of ASR (automatic speech recognition) technology: Although ASR technology is relatively mature, in actual applications, there is still a problem of low recognition rate. This is mainly due to various factors such as speech complexity, noise interference, inaccurate pronunciation, etc. ASR technology has limited adaptability to specific accents, dialects or speech speeds, which may also lead to recognition errors.

[0055] 2) Challenges of NLP (Natural Language Processing) Technology: NLP technology is responsible for converting speech into text and then further understanding the meaning of the text. However, due to the complexity and diversity of natural language, NLP technology often encounters difficulties when processing spoken text. Spoken text often contains ambiguity, homophones, grammatical errors and other problems, which pose challenges to NLP technology.

[0056] At present, the smart home voice interaction technology route is usually as follows Figure 1 The process can be broken down into the following main steps:

[0057] 1. Voice input: It is the starting point of the entire process and represents the information input by the user through voice.

[0058] 2. Speech Recognition (ASR): In this step, the input speech signal is converted into a processable data form. ASR (Automatic Speech Recognition) technology is the key to achieve this conversion.

[0059] 3. Natural Language Understanding (NLU): The data obtained after ASR processing is fed into the Natural Language Understanding module. In this stage, the data is further analyzed and interpreted to understand its meaning.

[0060] 4. Voice database: The system uses a voice database to assist in understanding or generating speech. The database contains various voice samples, vocabulary, grammar and other information.

[0061] 5. Natural Language Generation (NLG): After understanding the user's intention, the system will use natural language generation technology (NLG) to convert the information into a form that can be perceived by hearing, that is, generate speech.

[0062] 6. Text-to-speech (TTS): After NLG generates text, text-to-speech technology (TTS) is used to convert the text into speech, thereby converting machine-generated text into human-audible speech.

[0063] 7. Voice output: Finally, the voice processed by TTS is output and presented in a way that humans can understand. This step is the end of the entire process and is also the result that users can finally hear.

[0064] It can be seen that the above voice processing process is a single-line process, that is, the system processes the information input by the user. If it cannot be processed, it usually gives a corresponding prompt, for example, prompting the user to repeat the voice control again. Especially when the user uses complex sentence structures, slang or terminology in specific fields, it is more difficult to accurately understand the user's control intention.

[0065] In order to solve the technical problem that the above-mentioned human-computer interaction is difficult to achieve smooth and natural communication similar to that between people, the present invention proposes a smart home voice interaction method. The implementation details of the smart home voice interaction method of this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0066] Embodiment 1:

[0067] The smart home voice interaction method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 2 As shown, including:

[0068] Step 110, in response to receiving a voice request from a target user, parsing a control intent in the voice request;

[0069] In this embodiment, the executor of the smart home voice interaction method can be a central control device or a smart speaker. The voice request is a control instruction or query request issued by the target user to the central control device through voice input. The smart speaker uses voice recognition technology to convert the pre-processed voice request into text form, performs semantic analysis on the recognized text, identifies the keywords, phrases and sentence structures therein, and then parses the control intention of the target user. Based on the results of the semantic analysis, the control intention of the target user is further determined using a preset intention recognition model or algorithm. The intention recognition model can be built based on machine learning or deep learning technology to identify different types of control intentions by training a large amount of sample data.

[0070] Step 120, comparing the control intention with the request items in a preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes a home appliance manual, device-related request items, and answer content corresponding to the request items;

[0071] In this embodiment, the preset retrieval-augmented generation database (RAG database) is a pre-built and optimized database, which contains information in the home appliance manual, device-related request items, and answer content or control instructions corresponding to these request items. After the smart home system receives the user's control intention, it will compare it with the request items in the preset retrieval-augmented generation database (RAG database).

[0072] By comparing, the system can determine whether the user's control intention is clear, that is, whether a matching request entry can be found in the RAG database. If a matching request entry is found, it means that the user's control intention is clear, and the system can perform corresponding operations according to the corresponding answer content or control instructions. If no matching request entry is found, it means that the user's control intention may not be clear enough or ambiguous, and further confirmation with the user or provision of relevant help information may be required.

[0073] The process of matching request items can be achieved by calculating the similarity between the control intent and the request items or by matching them using a machine learning model. The similarity calculation can be based on the matching degree of keywords, phrases or sentence structures. The machine learning model can learn the association between different control intents and request items by training a large amount of sample data. If the control intent completely matches or is highly similar to a request item in the database, the control intent is considered to be clear; if the control intent does not match any request item in the database or the similarity is low, the control intent is considered to be unclear.

[0074] For example, a smart home system includes smart light bulbs, smart curtains and other devices, and a RAG database has been built. The system interprets the user's control intent as "turn on the lights in the living room". The request entries in the RAG database include the following:

[0075] Request item 1: "Turn on the lights in the living room", corresponding answer content or control instruction: "Send a turn-on instruction to the smart light bulb".

[0076] Request item 2: "Close the curtains in the bedroom", corresponding answer content or control instruction: "Send a closing instruction to the smart curtains".

[0077] ...(other request items and corresponding answer content or control instructions)

[0078] Comparison process: After the system receives the user's control intention "turn on the light in the living room", it will compare it with the request items in the RAG database. The system found request item 1 that matches "turn on the light in the living room". Therefore, the system determines that the user's control intention is clear, and performs the operation according to the corresponding answer content or control instruction "send an on instruction to the smart bulb". If the target user says: "Turn on that light". Since "that light" in the voice request is a vague reference, it cannot be directly matched to any request item in the retrieval enhancement generation database. Therefore, it is judged that the control intention is unclear.

[0079] Through the application of RAG technology, the smart home system can accurately understand and process the user's control intentions and perform corresponding operations. This technology not only improves the intelligence level of the system, but also provides users with a more convenient and efficient smart home experience.

[0080] Step 130: When the control intention is unclear, guide the target user to correct the voice request until the control intention in the corrected voice request is determined to be clear.

[0081] In this embodiment, feedback and suggestions can be provided to the target user in the form of voice or text to guide the target user to correct the unclear voice request. When the smart home system determines that the control intention is unclear, it needs to generate corresponding feedback information to guide the target user to make corrections. Feedback information can include the following types:

[0082] 1) Direct prompt: Directly tell the target user the ambiguities in their voice request and suggest that they use more explicit language to describe it. For example: "Which room's light are you talking about? Please specify the room name."

[0083] 2) Option prompt: Provide a series of possible options for the target user to choose from to clarify their control intention. For example: "Do you want to turn on the lights in the living room, bedroom, or kitchen?"

[0084] 3) Example prompts: Give some example voice requests for the target user to refer to and imitate their expressions. For example: "You can say: 'Turn on the lights in the living room.'"

[0085] After that, the generated feedback information is output to the target user in the form of voice or text. If voice output is adopted, the feedback information needs to be converted into voice signals using text-to-speech (TTS) technology; if text output is adopted, the feedback information can be directly displayed on the display screen of the smart speaker or central control device.

[0086] After the target user receives the feedback information, the target user is guided to correct the voice request, which can be achieved by repeating the process of steps 110 and 120, that is, receiving the voice request of the target user again, parsing the control intent and comparing and judging with the search enhancement generation database. If the control intent in the corrected voice request is still unclear, feedback information is continuously generated to guide the correction until the control intent is clear.

[0087] In some optional embodiments, it further includes: when the control intention is clear, generating a control instruction corresponding to the control intention. In this embodiment, if the control intention in the corrected voice request is clear, or the intention of the target user when issuing the control instruction for the first time is clear, the corresponding operation is performed or the user's query is answered. This achieves precise control of the device based on the user's wishes.

[0088] In summary, the smart home voice interaction method provided by this embodiment provides a standard reference for the user's request intention, expression correction, etc. by using a pre-established retrieval enhancement generation database. The method can use the home appliance manual, device-related request items and corresponding answer content in the retrieval enhancement generation database to assist in judging the clarity of the control intention, which not only enables the system to understand the user's request more comprehensively, but also provides it with a rich knowledge base to deal with various possible request scenarios, thereby enhancing the intelligence of the system. At the same time, the interaction method of the present application allows correction guidance when the control intention is unclear, which shows the system's adaptability to uncertainty and changes and improves its flexibility. By comparing the control intention with the request items in the preset retrieval enhancement generation database, the method can more accurately judge the clarity of the control intention. This helps to reduce erroneous operations caused by misunderstanding the user's intention and improves the accuracy of the control instructions. Further, by guiding the user to correct the unclear voice request, the method can gradually approach the user's true intention, thereby ensuring that the control instruction finally executed is consistent with the user's expectations. The interactive method of the present application, when it is detected that the control intention is unclear, the system does not simply reject or ignore the request, but actively guides the user to correct the expression of the request, making the human-computer interaction smoother and more natural, effectively improving the intelligence level of the smart home system and the user's interactive experience.

[0089] In some optional embodiments, in response to receiving a voice request from a target user, parsing the control intent in the voice request includes: in response to receiving a voice request from a target user, determining whether the content of the voice request belongs to the smart home field; when the content of the voice request does not belong to the smart home field, based on the voice request, generating a generalized instruction related to the smart home field using a preset semantic understanding model; and parsing the control intent in the generalized instruction.

[0090] In this embodiment, when the system receives a voice request from a target user, it first determines whether the content of the voice request belongs to the field of smart home. This determination process involves understanding and classifying the voice content, such as determining whether the user is asking about the status of a smart home device, wants to control a device, or is discussing a topic unrelated to smart home.

[0091] Generating generalized instructions related to the smart home field means correcting the user's imprecise expression into a conventional expression in the smart home field. If the content of the voice request does not belong to the smart home field, the system will try to use the preset semantic understanding model to generate a generalized instruction related to the smart home field based on this request to make it meet the control or query requirements of smart home devices. Finally, the control intent in the generalized instruction is parsed.

[0092] For example, if the target user says to the smart speaker: "I have to go on a business trip tomorrow and need to get up early", the processing system of the smart speaker will determine that it does not belong to the direct control request in the smart home field, because the target user is talking about the work and rest plan, not controlling or querying the smart home device.

[0093] After that, the system will use the preset semantic understanding model to generate a generalized instruction related to the smart home field based on this request. For example, the system may think that since the target user mentioned "need to get up early", he may want to set a function to remind him to get up early. For example, based on the statistics of the target user's work and rest time in the past, it is found that the target user usually gets up early at 5:30. Therefore, the system will generate a generalized instruction, such as "set an alarm to remind you at 5:30 tomorrow".

[0094] Finally, the system will parse the control intent in the generalized instruction "Set an alarm to remind you at 5:30 tomorrow" and identify the specific operation that the target user wants to perform.

[0095] Through the above process, even if the original voice request of the target user does not directly involve the control of the smart home device, the system can generate and execute the corresponding smart home control operation according to the user's intention.

[0096] In some optional embodiments, the response to receiving a voice request from a target user, before determining whether the content of the voice request belongs to the field of smart home, further includes: establishing a training data set for a smart home scenario based on the acquired home device usage data of the target user, wherein the home device usage data includes at least one of a conversation instance between the user and the smart home and the behavioral habit data of the target user; and training an initial semantic understanding model using the training data set until a preset training standard is met to obtain the preset semantic understanding model.

[0097] In this embodiment, the preset semantic understanding model is a model that has been specially trained and fine-tuned for the smart home field to better understand the control intention of the target user. The training process mainly includes the following two aspects:

[0098] Step 1: Dataset Preparation

[0099] First, we collect and organize conversation examples related to smart home through research, such as controlling home appliances, checking the weather, setting alarms, and other daily tasks. In addition, we can also include user behavior data to help the model better understand user habits and preferences. We then create a training dataset specifically for smart home scenarios using the conversations.

[0100] Step 2: Fine-tune the large model

[0101] Select a pre-trained open source large model, such as Qianwen, Zhipu, etc., and then fine-tune the model using the dataset prepared in the previous step. The goal of fine-tuning is to make the model better adapt to the specific scenarios of smart homes. Through supervised learning, the model learns how to correctly understand and respond to user voice commands.

[0102] For example, a smart home system includes smart light bulbs, smart door locks, and smart thermostats. The target user is a business person who travels frequently and is used to turning off all lights and appliances before leaving home and setting the thermostat to energy-saving mode.

[0103] Create a training dataset:

[0104] The system first collects examples of the target user’s previous conversations with the smart home system, such as “turn off the lights in the living room” and “adjust the thermostat to energy-saving mode”.

[0105] At the same time, the system will also collect data on the target user's behavioral habits, such as he usually turns off all lights around 10 pm and sets the thermostat to energy-saving mode before leaving home. This data will be used to build a training data set for smart home scenarios.

[0106] Train the semantic understanding model:

[0107] The system uses this training data set to train an initial semantic understanding model. During the training process, the model tries to understand the user intentions behind these conversation instances and behavioral habit data. For example, when the model hears a voice request like "I'm going out", it may associate it with the user's previous behavioral habits (turning off lights and appliances, setting the thermostat to energy-saving mode) and predict the actions the user wants to perform.

[0108] Through multiple iterations of training, the prediction accuracy of the model will gradually improve. The preset semantic understanding model is obtained. Once the prediction accuracy of the model reaches the preset training standard (such as an accuracy rate of more than 90%), the system will save it as the preset semantic understanding model. The model will be used to process the user's voice requests in the future. For example, when the user says "I'm going out", the model can accurately understand the user's intention and generate corresponding smart home control instructions (turn off all lights and appliances, set the thermostat to energy-saving mode).

[0109] In some optional embodiments, the semantic understanding model is a pre-trained large model based on the Transformer architecture.

[0110] Specifically, the current smart home voice interaction still uses early deep learning models (such as RNN, LSTM, etc.) for natural language understanding. This embodiment uses a pre-trained large model (LLM) based on the Transformer architecture, so that it can have stronger processing capabilities when processing complex voice interaction scenarios. The pre-trained large model based on the Transformer architecture can perform small-sample learning, which can quickly adapt to new voice command tasks. It can not only handle natural language understanding (NLU) tasks, but also natural language generation (NLG) tasks, such as multi-language communication, translation, etc.

[0111] In some optional embodiments, guiding the target user to correct the voice request includes: searching for request items related to the voice request in a database based on the retrieval enhancement, and voice broadcasting the request items related to the voice request; receiving feedback information from the target user on the request items related to the voice request, and determining whether it is necessary to retrieve new request items and broadcast them to the target user based on the feedback information.

[0112] In this embodiment, when a user's voice request is received but cannot be directly understood or executed (perhaps because the user's expression is not clear enough or there is no completely matching request item in the system), the system will search for request items related to the user's voice request based on the retrieval enhancement generation database (RAG database). These related request items may be semantically similar to the user's voice request, or operations that the system believes the user may want to perform.

[0113] Next, the system will present the relevant request items found to the user through voice broadcast, letting the user know what the system has understood that may be related to his / her intention, and giving the user an opportunity to confirm or correct his / her voice request.

[0114] Next, receive user feedback and determine whether new request items need to be retrieved. The system will receive user feedback on the related request items that were broadcasted. The feedback information can be the user confirming that a certain item is exactly the operation he / she wants to perform, or the user indicating that none of these items meet his / her intentions. If the user confirms an item, the system can perform the corresponding operation based on this item. If the user indicates that none of these items meet his / her intentions, the system may continue to retrieve new request items and broadcast them to the user again until an item that meets the user's intentions is found or the user gives up correcting the voice request.

[0115] Through the guidance method of this embodiment, the system can process the user's voice request more flexibly. Even if the user's expression is not clear or accurate enough, the system can find an operation that meets the user's intention by guiding the user to correct the request.

[0116] In some optional embodiments, it also includes: when it is impossible to directly reply to the target user's question based on the local knowledge of the retrieval enhancement generation database, retrieving relevant reply content on the Internet according to the target user's question; and generating a final response text by integrating the reply content retrieved from the Internet.

[0117] In this embodiment, the application of RAG technology in the voice interaction system has many advantages over the current voice database. Traditional voice databases are static and difficult to update in real time. RAG can retrieve the latest information from external knowledge bases in real time or regularly, and can update its knowledge base in time to maintain the timeliness of information. Traditional voice databases lack context awareness and can only conduct single-round conversations and the context is inconsistent. By combining retrieval technology and context understanding, RAG can better understand the context of users' questions and provide more appropriate answers. The accuracy of answers can be improved by retrieving information from multiple sources. RAG technology can provide a source for generating answers, enhancing the credibility and verifiability of the answers.

[0118] In addition, current voice interaction systems usually rely on local databases, which are limited by storage capacity. The use of RAG technology for online query in this embodiment breaks through this limitation, and can access almost unlimited online resources at any time, thereby expanding the knowledge boundary of the system. In dealing with unknown problems, online query can play a supplementary role for questions that cannot be answered in the local database. The system can search for relevant answers through the Internet, which greatly improves the system's ability to solve problems and improves user satisfaction.

[0119] The current intelligent voice interaction technology can only achieve a shallow level of human-computer interaction. For more complex communication scenarios, such as multi-round dialogues, close contextual connections, logical reasoning, emotional expression, etc., human-computer interaction cannot be as smooth and natural as human-to-human communication. The smart home voice interaction method based on the big model in this embodiment has a more powerful semantic understanding and logical reasoning ability, which can deeply understand the customer's questions and accurately identify their intentions and needs, thereby providing more accurate and personalized answers. Provide more natural and fluent dialogue content in terms of emotional expression. In addition, the big model and RAG technology fine-tuned according to the smart home data set can answer smart home questions more accurately and deeply. If there are questions beyond the cognition of the big model, you can also query and reply to the user online. When the user's language request does not completely match the control instruction, a generalized instruction is generated through the semantic understanding of the fine-tuned professional big model. If the intention is unclear, guide the user to correct his request expression, give a clear and accurate request expression, and achieve the purpose of controlling the smart home.

[0120] Embodiment 2:

[0121] Based on the above embodiments, this embodiment provides an application example, providing a smart home voice interaction system. The interaction flow chart of the system is as follows: Figure 3 As shown, the implementation process includes the following key steps:

[0122] Step 1: Dataset Preparation

[0123] First, we collect and organize conversation examples related to smart home through research, such as controlling home appliances, checking the weather, setting alarms, and other daily tasks. In addition, we can also include user behavior data to help the model better understand user habits and preferences. We then create a training dataset specifically for smart home scenarios using the conversations.

[0124] Step 2: Fine-tune the large model

[0125] Select a pre-trained open source large model, such as Qianwen, Zhipu, etc., and then fine-tune the model using the dataset prepared in the previous step. The goal of fine-tuning is to make the model better adapt to the specific scenarios of smart homes. Through supervised learning, the model learns how to correctly understand and respond to user voice commands.

[0126] Step 3: Integrate knowledge base RAG technology

[0127] In order to enhance the model's ability to handle complex problems, the retrieval-augmented generation (RAG) technology is introduced. RAG technology combines the two methods of retrieval and generation, allowing the model to use additional data sources when generating responses. First, a local knowledge base containing information such as smart home device manuals and FAQs is built, and this information is associated with the model's input through RAG technology to provide more accurate answers.

[0128] Step 4: Realize online query function

[0129] For questions that cannot be answered directly by the local knowledge base or fine-tuned model, the system has the ability to query the Internet. When the question cannot be found in RAG, the model is allowed to access the Internet to obtain information. At the same time, the accuracy of the query results must be ensured and integrated into the final response.

[0130] Step 5: Guided Interaction

[0131] Since users are not very familiar with the functions of smart home products, they are accustomed to controlling home appliances through voice according to their own understanding and past operating habits. However, user language requests cannot completely match specific control instructions and generalized instructions, and smart home products cannot accurately understand user intentions. Some products standardize some fixed feedback. Although they meet the user's interaction needs, they do not correctly activate the corresponding smart functions according to user requests. Guided interaction includes the following two aspects:

[0132] (1) When the content of the user's request is not in the field of smart home, the semantic understanding of the fine-tuned professional large model is used to generate generalized instructions that are close to the field of smart home.

[0133] (2) Determine the intention of the user request. If the intention is unclear, guide the user to correct his or her request and give a clear and accurate request statement to achieve the purpose of controlling the smart home.

[0134] Step 6: Build speech recognition and synthesis modules

[0135] In addition to the core dialogue generation module of the large model, it is necessary to connect the speech recognition (ASR) and speech synthesis (TTS) modules. ASR is used to convert the user's voice instructions into text, while TTS is responsible for converting the text responses generated by the model into voice output. The selection and integration of these two modules directly affects the naturalness of the user experience.

[0136] In summary, the system of this embodiment can not only efficiently process user commands, but also respond to a wider range of needs through RAG technology and online query functions, providing users with a more intelligent and convenient life experience. Compared with the existing smart furniture interaction system, it has at least the following advantages:

[0137] 1. At present, the natural language understanding of smart home voice interaction still uses early deep learning models (such as RNN, LSTM, etc.). The present invention uses a pre-trained large model (LLM) based on the Transformer architecture, which has a huge number of parameters, with billions or even hundreds of billions of parameters, which enables it to have stronger capabilities in processing complex voice interaction scenarios. It is capable of few-sample learning and can quickly adapt to new voice command tasks with only a small number of examples. It can not only handle natural language understanding (NLU) tasks, but also natural language generation (NLG) tasks, such as multi-language communication, translation, etc.

[0138] 2. The application of RAG (retrieval enhanced generation) technology in the voice interaction system adopted by the present invention has many advantages over the current voice database. Traditional voice databases are static and difficult to update in real time. RAG can retrieve the latest information from external knowledge bases in real time or regularly, and can update its knowledge base in time to maintain the timeliness of information. Traditional voice databases lack context awareness and can only conduct single-round conversations and the context is inconsistent. By combining retrieval technology and context understanding, RAG can better understand the context of users' questions and provide more appropriate answers. The accuracy of answers can be improved by retrieving information from multiple sources. RAG technology can provide a source for generating answers, enhancing the credibility and verifiability of the answers.

[0139] 3. Current voice interaction systems usually rely on local databases, which are limited by storage capacity. Online query breaks this limitation and can access almost unlimited online resources at any time, thereby expanding the knowledge boundary of the system. When dealing with unknown problems, online query can play a supplementary role in solving problems that cannot be answered in the local database. The system can search for relevant answers through the Internet, which greatly improves the system's problem-solving ability and improves user satisfaction.

[0140] 4. Smooth and natural interactive guidance: When the user's request is not in the smart home field, the semantic understanding of the fine-tuned professional large model is used to generate generalized instructions that are close to the smart home field. The user's request is judged for its intention. If the intention is unclear, the user is guided to correct his or her request expression and give a clear and accurate request expression to achieve the purpose of controlling the smart home.

[0141] Therefore, the smart home voice interaction system of this embodiment solves the following problems:

[0142] 1. The natural language comprehension is limited, and only shallow conversations can be carried out. The smart home voice interaction system of this embodiment can deeply understand the customer's questions, accurately identify their intentions and needs, and thus provide more accurate answers.

[0143] 2. When dealing with complex problems or problems with ambiguous semantics, its accuracy is not high. The smart home voice interaction system of this embodiment has stronger logical reasoning ability, context understanding ability, and multi-round dialogue ability.

[0144] 3. The answering method is relatively simple, mechanical, stiff, and lacks a natural and smooth conversation experience. The smart home voice interaction system of this embodiment has enhanced emotional expression, more natural and smooth conversation content, and can interact with customers in a more real scene, thereby improving the customer's conversation experience.

[0145] Embodiment three:

[0146] Another embodiment of the present application relates to a smart home voice interaction device. The implementation details of the smart home voice interaction device of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this solution. The schematic diagram of the smart home voice interaction device of this embodiment can be as follows Figure 4 As shown, it includes an intention analysis module 410, an intention judgment module 420 and a correction guidance module 430.

[0147] The intention parsing module 410 is used to parse the control intention in the voice request in response to receiving the voice request of the target user;

[0148] The intention determination module 420 is used to compare the control intention with the request items in the preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes the home appliance manual, the request items related to the equipment, and the answer content corresponding to the request items;

[0149] The correction guidance module 430 is used to guide the target user to correct the voice request when the control intention is unclear, until the control intention in the corrected voice request is determined to be clear.

[0150] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0151] Embodiment 4:

[0152] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the smart home voice interaction method in the above-mentioned embodiments.

[0153] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0154] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0155] Embodiment five:

[0156] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0157] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0158] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A smart home voice interaction method, characterized in that: include: In response to receiving a voice request from a target user, parsing a control intent in the voice request; Comparing the control intention with the request items in a preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes a home appliance manual, device-related request items, and answer contents corresponding to the request items; When the control intention is unclear, the target user is guided to correct the voice request until the control intention in the corrected voice request is determined to be clear.

2. The smart home voice interaction method according to claim 1, characterized in that: The step of, in response to receiving a voice request from a target user, parsing a control intent in the voice request includes: In response to receiving a voice request from a target user, determining whether the content of the voice request belongs to the field of smart home; When the content of the voice request does not belong to the smart home field, based on the voice request, a generalized instruction related to the smart home field is generated by using a preset semantic understanding model; The control intent in the generalized instruction is parsed.

3. The smart home voice interaction method according to claim 2, characterized in that: In response to receiving the voice request of the target user, before determining whether the content of the voice request belongs to the field of smart home, the method further includes: Establishing a training data set for smart home scenarios based on the acquired home device usage data of the target user, wherein the home device usage data includes at least one of a conversation instance between the user and the smart home and the behavior habit data of the target user; The initial semantic understanding model is trained using the training data set until a preset training standard is met, thereby obtaining the preset semantic understanding model.

4. The smart home voice interaction method according to claim 2, characterized in that: The semantic understanding model is a pre-trained large model based on the Transformer architecture.

5. The smart home voice interaction method according to claim 1, characterized in that: The guiding the target user to correct the voice request includes: Searching for request items related to the voice request in a database based on the search enhancement generation, and voice broadcasting the request items related to the voice request; Receive feedback information from the target user on a request item related to the voice request, and determine whether a new request item needs to be retrieved and broadcasted to the target user based on the feedback information.

6. The smart home voice interaction method according to claim 1, characterized in that: Also includes: When the control intention is clear, a control instruction corresponding to the control intention is generated.

7. The smart home voice interaction method according to any one of claims 1 to 6, characterized in that: Also includes: When it is impossible to directly answer the target user's question based on the local knowledge of the search enhancement generation database, searching for relevant answering content on the Internet according to the target user's question; The reply contents retrieved from the Internet are integrated to generate a final response text.

8. A smart home voice interaction device, characterized in that: include: An intention parsing module, configured to, in response to receiving a voice request from a target user, parse the control intention in the voice request; An intention judgment module, used to compare the control intention with the request items in a preset search enhancement generation database to determine whether the control intention is clear, wherein the search enhancement generation database includes a home appliance manual, device-related request items, and answer content corresponding to the request items; The intention guidance module is used to guide the target user to correct the voice request when the control intention is unclear, until the control intention in the corrected voice request is determined to be clear.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the smart home voice interaction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the smart home voice interaction method according to any one of claims 1 to 7 is implemented.

Images