Voice control method, system, device and medium for autonomous learning home scene

Abstract

The present invention discloses a voice control method, system, device and medium in an autonomous learning home scene, the method comprising: receiving a user voice command sent by a sound pickup device and the spatial information of the sound pickup device; recognizing the voice command, querying the user's custom configured intention control logic in the corpus, and directly executing the intention control logic if there is one; if not, querying the pre-stored intention library for the intention control logic that matches the voice recognition result and the spatial information of the sound pickup device, and directly executing the intention control logic if there is one, and sending a query to the user whether the execution is correct, obtaining the user's feedback information on whether the intention control logic is executed correctly, and if it is correct, entering the intention control logic into the user's custom configured intention control logic in the corpus that matches the user's voice recognition result. The present invention solves the problem that the machine cannot accurately judge the user's intention when the user's voice command is ambiguous in the smart home scene.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a voice control method, system, device and medium in an autonomous learning home scene. Background Art

[0002] After receiving the user's voice command, the smart home voice control first converts the speech to text through ASR, and then performs word segmentation on the text to obtain the [space] information and [device information] in the user's voice command.

[0003] If the user's voice command does not contain [space] information, the industry generally practices either to directly control all devices of the same type in the user's home by default, or to ask the user through a second clarification which [space] device they want to control. When the user says the command "turn on the light" in different space ranges, their real intention is different.

[0004] For example, when a user says "turn on the lights" (without clear spatial information) when just entering the house, the real intention is to turn on the lights in the entrance and living room; and when the user says "turn on the lights" when entering the master bedroom or lying on the bed, the real intention is only to turn on the lights in the bedroom.

[0005] Current NLU and NLP technologies are not yet able to accurately understand the differences in user intentions for the same ambiguous command in different scenarios. Summary of the invention

[0006] In view of the above problems, the present invention provides a voice control method, system, device and computer storage medium for autonomous learning in a home scenario, which solves the problem that the machine cannot accurately judge the user's intention when the user's voice command is ambiguous in a smart home scenario.

[0007] In order to achieve the above technical effects, the technical solution adopted by the present invention is:

[0008] In one aspect, the present invention provides a method for autonomously learning voice control in a home scene, the method comprising:

[0009] Step 1: receiving a user voice command sent by a sound pickup device and information about the space where the sound pickup device is located;

[0010] Step 2: Perform voice recognition processing on the voice command, query whether there is a user-defined intention control logic that matches the result of the voice recognition in the pre-stored corpus, and if so, directly execute the intention control logic, which includes the type, space and target action of the target device that matches the result of the voice recognition, and end the process; if not, execute step 3;

[0011] Step 3: Check whether there is a system default intention control logic that matches the result of the voice recognition and the spatial information of the sound pickup device in the pre-stored intention library. If yes, directly execute the intention control logic and ask the user whether the execution is correct, and then go to step 4; if no, go to step 5;

[0012] Step 4: Obtain feedback information from the user on whether the intention control logic is executed correctly. If it is correct, enter the intention control logic into the user-defined intention control logic in the corpus that matches the result of the user's speech recognition; if it is incorrect, execute step 5;

[0013] Step 5: Remind the user that there is a problem in being unable to execute the voice command.

[0014] As a preferred solution, the speech recognition processing includes at least one of the following methods: voiceprint recognition, ASR recognition, ASR recognition + word segmentation processing, ASR recognition + NLU understanding.

[0015] As a preferred solution, the result of the speech recognition processing includes at least one of the following corpora: user voiceprint information, the type or name of the target device, the space where the target device is located, and the action that the target device needs to perform.

[0016] As a preferred solution, the second step also includes: based on the result of voice recognition processing on the voice instruction, pre-determining whether the voice instruction meets the requirements for querying in the corpus, if so, performing subsequent query steps, if not, directly executing the fifth step.

[0017] On the other hand, the present invention provides a voice control system in an autonomous learning home scene, the system comprising:

[0018] A collection module, used to receive the user voice command sent by the sound pickup device and the space information where the sound pickup device is located;

[0019] A recognition module, used for performing voice recognition processing on the voice command;

[0020] A first matching module, used to search a pre-stored corpus for a user-defined intention control logic that matches the result of the speech recognition;

[0021] A first execution module, configured to execute the intention control logic of the user-defined configuration queried by the first matching module;

[0022] A second matching module is used to search the pre-stored intention library for a system default intention control logic that matches the result of the speech recognition and the spatial information of the sound pickup device;

[0023] A second execution module, used for executing the system default intention control logic queried by the second matching module;

[0024] A confirmation module, used for confirming with the user whether the execution is correct after the second execution module completes the execution of the system default intention control logic, and entering the correctly executed intention control logic into the user-defined intention control logic in the corpus that is consistent with the result of the user's speech recognition;

[0025] The reminder module is used to remind the user that there is a problem of being unable to execute voice commands.

[0026] As a preferred solution, the recognition module includes at least one of the following modules: a voiceprint recognition module, an ASR recognition module, an ASR recognition + word segmentation processing module, and an ASR recognition + NLU understanding module.

[0027] As a preferred solution, the result of the speech processing of the recognition module includes at least one of the following corpora: user voiceprint information, the type or name of the target device, the space where the target device is located, and the action that the target device needs to perform.

[0028] As a preferred embodiment, the system also includes a prediction module for pre-judgment of whether the voice instruction meets the query requirements in the corpus based on the speech recognition results obtained by the recognition module. If so, it is output to the first matching module; if not, it is output to the reminder module.

[0029] On the other hand, the present invention provides a voice control device for autonomous learning home scenarios, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the voice control method as described above when executing the program.

[0030] On the other hand, the present invention further provides a computer storage medium having a computer program stored thereon, wherein the program implements the voice control method as described above when executed by a processor.

[0031] Compared with the prior art, the beneficial effects of the present invention include:

[0032] Based on real life scenarios and conversation habits, the machine is helped to better understand the real intention behind a vague command in different spaces through user feedback. At the same time, considering the differences in language habits and spaces of each person, a set of NLU learning logic is stored for each historical data of the conversation between the user and the machine to achieve a personalized effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0034] Figure 1 The present invention is a flowchart of a smart home voice control method based on user habits according to an embodiment of the present invention.

[0035] Figure 2 The structure block diagram of the smart home voice control system based on user habits according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0037] The terms in the embodiments of the present invention are explained as follows:

[0038] ASR: Automatic Speech Recognition is a technology that converts human speech into text.

[0039] NLP: Natural Language Processing, is a technology that uses natural language to communicate with computers. It studies the use of electronic computers to simulate the human language communication process, so that computers can understand and use the natural languages ​​of human society such as Chinese and English, and realize natural language communication between humans and computers to replace part of human mental labor, including querying information, answering questions, extracting literature, compiling information, and processing all natural language information.

[0040] The present invention provides a voice control method in an autonomous learning home scene, which is applied in a smart home ecosystem. The smart home ecosystem includes a smart home application (APP), a voice collection device and multiple smart home devices. The voice collection device communicates with the smart home application in a wireless or wired manner, and the smart home device and the smart home application also communicate in a wireless or wired manner. When a user wants to control a smart home device by voice, the user sends a voice message, and then the voice collection device collects the voice information sent by the user. When the voice collection device collects the voice information sent by the user, the voice information is sent to the smart home application.

[0041] The embodiment of the present invention provides a voice control method and system in an autonomous learning home scene, and the system is used to implement the voice control method. It can be understood that the system is the above-mentioned smart home application (APP), and the voice collection device in this embodiment adopts a sound pickup device (smart speaker, voice assistant, etc.).

[0042] Please refer to Figure 1 As shown, an embodiment of the present invention provides a voice control method in an autonomous learning home scene, comprising the following steps:

[0043] Step 1: Receive the user's voice command and the spatial information of the pickup device sent by the pickup device;

[0044] It should be noted that the sound pickup device can be a smart speaker, voice assistant, etc. After the sound pickup device logs into the system, the system automatically stores the information of the corresponding sound pickup device, including ID information, location information, etc. Therefore, when the system obtains the voice command of the sound pickup device, it can automatically identify the location information of the sound pickup device. In a broad sense, it is to obtain the location information of the sound pickup device while obtaining the voice command. For example, the user says "turn on the light", "turn on the air conditioner", "turn on the TV", and the "light", "air conditioner", and "TV" should be smart home devices with intelligent control capabilities.

[0045] Step 2: Perform voice recognition processing on the voice command, query whether there is a user-defined intention control logic that matches the result of the voice recognition in the pre-stored corpus, and if so, directly execute the user-defined intention control logic, wherein the user-defined intention control logic includes the type, space and target action of the target device that matches the result of the voice recognition, and end the process; if not, execute step 3;

[0046] The target device type is the type of the target smart home device, such as air conditioner, light or TV, and the target action is the action performed by the corresponding target smart home device controlled by the user's voice command, such as turning on or off (light), raising or lowering (air conditioner temperature), etc.

[0047] Specifically, in this step, the speech recognition processing may include at least one of the following methods: voiceprint recognition, ASR recognition, ASR recognition + word segmentation processing, ASR recognition + NLU understanding.

[0048] In order to determine the user's identity, the smart home device the user wants to control, and the action to be performed on the smart home device to be controlled, an electroacoustic instrument can be used to perform voiceprint recognition on the user's voice command. Voiceprint information has the same role as fingerprints in identity recognition. According to the preset voice recognition algorithm, the user's voiceprint information in the voice command can be recognized.

[0049] When the system recognizes the user's voiceprint information, it queries the pre-stored corpus based on the voiceprint information to see whether there is any user-defined intention control logic that matches the voiceprint information. If the user has completely entered the "target device type, space, and target action" that matches the voiceprint information in the corpus, then at this time, the above information can be directly queried based on the voiceprint information. By controlling the target device of this type in the space to perform the target action, the user-defined intention control logic can be completed.

[0050] If the user lacks any of the "target device type, space and target action" that matches the voiceprint information in the corpus, the system can choose to perform ASR recognition and simple word segmentation processing on the voice command, or ASR recognition + NLU understanding to obtain the target device type, space and target action in the voice command, and can also implement the intention control logic configured by the user.

[0051] In some cases, the user's voice command lacks information about the space where the target device is located. For example, the user only said "turn on the light", but it is not known whether it means "turn on the light in the living room" or "turn on the light in the bedroom". In this case, it is impossible to know based on the user's habits, that is, the corpus also lacks information about the space where the user's customized device corresponding to the voice command is located. At this time, we proceed to the third step.

[0052] Furthermore, this step may also include: judging in advance whether the voice instruction meets the requirements for querying in the corpus based on the result of voice recognition processing on the voice instruction, and if so, performing the subsequent query step, and if not, directly executing the fifth step. This situation is for voice instructions that are not supported by the corpus, such as voice instructions issued by users without permission, and directly reporting an error to remind the user to issue the correct voice instruction, and the error reporting can be performed by the sound pickup device.

[0053] Step 3: Check in the pre-stored intent library whether there is a system default intent control logic that matches the result of speech recognition and the spatial information of the sound pickup device. If there is, directly execute the intent control logic and ask the user whether the execution is correct, and then go to step 4; if not, go to step 5;

[0054] The contents of the intent library can be found in Table 1 below:

[0055] Corpus Room where the sound pickup equipment is located Intent (device execution action) Turn on the lights living room Hallway light + living room light Turn on the lights Master Bedroom Bedroom Lights

[0056] When the user's voice command lacks information about the space where the target device is located, but only has the type of target device and the target action, for example, if the user says "turn on the lights", and the lights in the living room and the master bedroom are in the same smart home ecosystem of the user, it is difficult to determine whether the user's real intention is to "turn on the lights in the living room" or "turn on the lights in the master bedroom". At this time, the user's space can be determined based on the spatial information where the sound pickup device is located. The sound pickup device is generally set to be valid only in a certain space, such as the living room or the master bedroom. Therefore, when the voice command is issued by the sound pickup device in the living room, it can be determined that the user is in the living room at this time. Combined with the type of target device and the target action in the user's voice command, the system default intention control logic is obtained, that is, when the space where the sound pickup device is located is the living room, the lights in the living room (including the entrance lights + the living room lights) are turned on, or, when the space where the sound pickup device is located is the master bedroom, the bedroom lights are turned on.

[0057] This will solve the problem in smart home scenarios where the machine cannot accurately judge the user's intention when the user's voice commands are ambiguous.

[0058] Furthermore, after executing the system default intention control logic, the system may also ask the user: Is the execution correct? The inquiry is issued by the sound pickup device.

[0059] Step 4: Obtain feedback from the user on whether the intent control logic is executed correctly. If it is correct, enter the intent control logic into the user-defined intent control logic in the corpus that matches the user's speech recognition result; if it is incorrect, execute step 5;

[0060] In this step, if the user answers that the execution is correct, the voice command is associated with the system's default intent control logic and sent to the custom-configured intent control logic under the user account, stored in the corresponding corpus, and the intent control logic is directly executed for the same voice command next time, without asking the user again whether the execution is correct. If the user answers that the execution is incorrect, the system's default intent control logic will not be executed next time;

[0061] Step 5: Remind the user that there is a problem with not being able to execute voice commands.

[0062] This step can be implemented by the sound pickup device, reminding the user that the type, space and target action of the target device that the voice command wants to control can be customized in the system (this step is not necessary and can be omitted).

[0063] Please refer to Figure 2 As shown, an embodiment of the present invention provides a voice control system in an autonomous learning home scene, the system comprising:

[0064] The collection module 11 is used to receive the user voice command sent by the sound collection device and the space information where the sound collection device is located;

[0065] A recognition module 12, used for performing voice recognition processing on the voice command;

[0066] A first matching module 13 is used to search a pre-stored corpus for a user-defined intention control logic that matches the result of the speech recognition;

[0067] A first execution module 14, configured to execute the intention control logic of the user-defined configuration queried by the first matching module 13;

[0068] The second matching module 15 is used to search the pre-stored intention library for the system default intention control logic that matches the result of the speech recognition and the spatial information of the sound pickup device;

[0069] A second execution module 16, used to execute the system default intention control logic queried by the second matching module;

[0070] The confirmation module 17 is used to confirm with the user whether the execution is correct after the second execution module completes the execution of the system default intention control logic, and enter the correctly executed intention control logic into the user-defined intention control logic in the corpus that matches the result of the user's speech recognition;

[0071] The reminder module 18 is used to remind the user that there is a problem of being unable to execute the voice command.

[0072] Among them, the recognition module 12 may specifically include at least one of the following modules: a voiceprint recognition module, an ASR recognition module, an ASR recognition + word segmentation processing module, and an ASR recognition + NLU understanding module.

[0073] The result of the speech processing by the recognition module 12 may include at least one of the following corpora: user voiceprint information, the type or name of the target device, the space where the target device is located, and the action that the target device needs to perform.

[0074] The system may also include a prediction module 19 for pre-judging whether the voice instruction meets the query requirements in the corpus based on the voice recognition result obtained by the recognition module 12. If so, it is output to the first matching module 13; if not, it is output to the reminder module 18 to directly report an error.

[0075] In addition, an embodiment of the present invention also provides a voice control device for an autonomous learning home scenario, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the methods of the above-mentioned embodiments when executing the program.

[0076] In addition, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the methods of the above embodiments are implemented.

[0077] The present invention starts from real life scenarios and conversation habits, and helps the machine better understand the real intention behind a vague command in different spaces through user feedback. At the same time, considering the differences in language habits and spaces of each person, a set of NLU learning logic is stored for each historical data of the conversation between the user and the machine to achieve the effect of different faces for different people.

[0078] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims. In addition, the parts not involved in the present invention are the same as the prior art or can be implemented by the prior art.

Claims

1. A voice control method for autonomous learning in a home scene, applied in a smart home ecosystem, wherein the smart home ecosystem includes a smart home application, a voice acquisition device and a plurality of smart home devices, wherein the voice acquisition device communicates with the smart home application in a wireless or wired manner, and the smart home device communicates with the smart home application in a wireless or wired manner, wherein: The control method comprises: Step 1: receiving the user voice command sent by the sound pickup device and the space information where the sound pickup device is located; after the sound pickup device logs into the system, the system automatically stores the information of the corresponding sound pickup device, including ID information and space information, and the system logged into by the sound pickup device is the system used to implement the voice control method; The second step: perform voice recognition processing on the voice command, query whether there is a user-defined intention control logic that matches the result of the voice recognition in the pre-stored corpus, if yes, directly execute the intention control logic, the intention control logic includes the type, space and target action of the target device that matches the result of the voice recognition, and end the process; if no, execute the third step; the target device type is the type of the target smart home device, and the target action is the action performed by the corresponding target smart home device controlled by the user voice command; the voice recognition processing includes at least one of the following methods: voiceprint recognition, ASR recognition, ASR recognition + word segmentation processing, ASR recognition + NLU understanding; the result of the voice recognition processing includes at least one of the following corpora: user voiceprint information, type or name of the target device, space where the target device is located The system can select the type of target device, space and target action to be performed by the target device in the voice command, and can also realize the execution of the user-defined intention control logic; the second step also includes: according to the result of voice recognition processing on the voice command, pre-determining whether the voice command meets the requirements for querying in the corpus, if it meets the requirements, then proceed to the subsequent query step, if not, then directly execute the fifth step; Step 3: Check in the pre-stored intent library whether there is a system default intent control logic that matches the result of the voice recognition and the spatial information of the sound pickup device. If yes, directly execute the intent control logic and ask the user whether the execution is correct, and then go to step 4; if no, go to step 5; Step 4: Obtain feedback information from the user on whether the intention control logic is executed correctly. If it is correct, enter the intention control logic into the user-defined intention control logic in the corpus that is consistent with the result of the user's voice recognition; if it is incorrect, execute step 5; in this step, if the user answers that it is executed correctly, the voice command is associated with the system's default intention control logic and sent to the intention control logic customized under the user account, and stored in the corresponding corpus. Next time, for the same voice command, the intention control logic is directly executed without asking the user again whether it is executed correctly; if the user answers that it is not executed correctly, the system's default intention control logic will not be executed next time; Step 5: Remind the user that there is a problem in being unable to execute the voice command. This step is implemented by the sound pickup device to remind the user that the type, space and target action of the target device that the voice command wants to control can be customized in the system.

2. A voice control system for autonomous learning home scenes, characterized in that: The voice control system is used to implement the voice control method according to claim 1, and the voice control system includes: A collection module, used to receive the user voice command sent by the sound pickup device and the space information where the sound pickup device is located; A recognition module, used for performing voice recognition processing on the voice command; A first matching module, used to search a pre-stored corpus for a user-defined intention control logic that matches the result of the speech recognition; A first execution module, configured to execute the intention control logic of the user-defined configuration queried by the first matching module; A second matching module is used to search the pre-stored intention library for a system default intention control logic that matches the result of the speech recognition and the spatial information of the sound pickup device; A second execution module, used for executing the system default intention control logic queried by the second matching module; A confirmation module, used for confirming with the user whether the execution is correct after the second execution module completes the execution of the system default intention control logic, and entering the correctly executed intention control logic into the user-defined intention control logic in the corpus that is consistent with the result of the user's speech recognition; A reminder module is used to remind the user that there is a problem of being unable to execute voice commands; The recognition module includes at least one of the following modules: a voiceprint recognition module, an ASR recognition module, an ASR recognition + word segmentation processing module, and an ASR recognition + NLU understanding module; The result of the speech processing by the recognition module includes at least one of the following corpora: user voiceprint information, the type or name of the target device, the space where the target device is located, and the action that the target device needs to perform; It also includes a pre-judgment module, which is used to pre-judge whether the voice instruction meets the query requirements in the corpus based on the voice recognition result obtained by the recognition module. If so, it is output to the first matching module; if not, it is output to the reminder module.

3. A voice control device for autonomous learning in a home scene, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the voice control method as claimed in claim 1 is implemented.

4. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the voice control method as claimed in claim 1 is implemented.

Images