A voice interaction method, device and equipment

Abstract

The present invention provides a voice interaction method, device, equipment and computer-readable storage medium; the method includes: recognizing voice data to obtain a recognition result; determining the target cooking step to be hit according to the recognition result and the cooking content of the cooking step in the recipe; selecting the target skill to respond to the voice data from the response skills configured for the target cooking step according to the recognition result; and providing the target skill. Through the present invention, user needs can be accurately judged, thereby providing accurate services for users.

Description

Technical Field

[0001] The present invention relates to the field of intelligent electrical appliance control technology, and in particular to a voice interaction method, device, equipment and computer-readable storage medium. Background Art

[0002] With the popularization and use of smart kitchen appliances, more and more users cook by interacting with smart kitchen appliances. However, during the interaction process, smart kitchen appliances often misjudge the user's voice intention and perform wrong operations. Summary of the invention

[0003] In view of this, the main purpose of the present invention is to provide a voice interaction method, device, equipment and computer-readable storage medium, which can accurately judge user needs and thus provide accurate services to users.

[0004] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0005] An embodiment of the present invention provides a voice interaction method, including:

[0006] Recognize voice data and obtain recognition results;

[0007] Determining a target cooking step according to the recognition result and the cooking content of the cooking step in the recipe;

[0008] According to the recognition result, selecting a target skill for responding to the voice data from the response skills configured for the target cooking step;

[0009] Provide the target skill.

[0010] An embodiment of the present invention provides a voice interaction device, including:

[0011] A recognition unit, used for recognizing voice data and obtaining a recognition result;

[0012] A determination unit, configured to determine a hit target cooking step according to the recognition result and the cooking content of the cooking step in the recipe;

[0013] A selection unit, configured to select a target skill for responding to the voice data from the response skills configured for the target cooking step according to the recognition result;

[0014] A providing unit is used to provide the target skill.

[0015] In the above solution, the determining unit is used to:

[0016] Determining a hit candidate cooking step according to a matching degree between a keyword representing the recognition result and a cooking content of the cooking step in the recipe;

[0017] determining the number of steps between the alternative cooking step and the current cooking step in the recipe;

[0018] The target cooking step is selected from the candidate cooking steps according to the priority coefficient corresponding to the step number.

[0019] In the above solution, the selection unit is used to:

[0020] According to the recognition result, a response skill matching the recognition result is selected from the response skills configured for the target cooking step as the target skill.

[0021] In the above solution, the selection unit is further used for:

[0022] According to the recognition result, selecting an alternative skill matching the recognition result from the response skills configured for the target cooking step;

[0023] The target skill is selected from the candidate skills according to the accuracy of the recognition result and the hit rate of the candidate skills.

[0024] In the above solution, the selection unit is further used for:

[0025] According to the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state is selected from the response skills configured for the target cooking step as the target skill.

[0026] In the above solution, the selection unit is further used for:

[0027] If it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has been executed, a question-and-answer skill associated with the cooking operation and matching the recognition result and / or a prompt skill is selected as the target skill.

[0028] In the above solution, the selection unit is further used for:

[0029] If it is determined according to the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has not been executed, the equipment operation skill associated with the cooking operation and matching the recognition result is selected as the target skill.

[0030] In the above solution, the prompting skills include at least one of the following:

[0031] Cooking time reminder skills;

[0032] Tips, tricks, and skills for cooking steps.

[0033] In the above scheme, the providing unit is also used for:

[0034] When executing the device operation skill, if the cooking device is a controlled device, the cooking device is controlled to perform the corresponding device operation; or, if the cooking device is not a controlled device, an operation prompt for controlling the device operation is output. In the above scheme, the voice interaction device also includes a guiding unit, which is used to:

[0035] If it is determined based on the recognition result that the hit target cooking step is not configured with a response skill that matches the recognition result, the guiding skill is used as the target skill to respond to the target data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next voice.

[0036] In the above solution, the guiding unit is also used for:

[0037] If the target cooking step is not determined based on the recognition result and the cooking content in the recipe, the guidance skill is used as the target skill to respond to the target data; wherein, under the guidance skill, a guidance voice is output to guide the user to say the next step voice.

[0038] In the above solution, the voice interaction device further includes an entertainment unit, which is used to:

[0039] When it is determined according to the current cooking state that the device is in the process of automatically cooking or when the last cooking step is waiting to be completed, entertainment skills are executed according to the multimedia information hit by the recognition result.

[0040] In the above scheme, the entertainment skills include:

[0041] Audio entertainment skills, used to output audio;

[0042] and / or,

[0043] Film and television entertainment skills, used to output film and television.

[0044] The embodiment of the present invention further provides a voice interaction device, including:

[0045] A memory for storing executable instructions;

[0046] The processor is used to implement the voice interaction method provided in the embodiment of the present invention when executing the executable instructions stored in the memory.

[0047] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program; wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the voice interaction method provided by the embodiment of the present invention.

[0048] The embodiments of the present invention have the following beneficial effects:

[0049] The embodiment of the present invention obtains recognition results by recognizing voice data, and determines the target cooking steps to be hit according to the recognition results and the cooking content of the cooking steps in the recipe. Compared with providing voice interaction in the cooking process by combining the voice data alone, the embodiment of the present invention combines the cooking content in the recipe currently being cooked to determine the steps to be executed by the voice data sent by the current user. Due to the added reference content, voice interaction can be performed more accurately. On the other hand, a single cooking step is configured with one or more skills, but the voice data that hits these skills may be the same. At this time, the results will be further recognized and the target skills will be selected for provision. Compared with configuring a single skill, more dimensional needs can be provided to users, and more fine-grained responses to users' needs can be achieved, thereby improving the accuracy of voice interaction and providing users with a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a process implementation of a voice interaction method provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of a voice interaction system provided by an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of a voice interaction system provided by an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of the composition of a knowledge base provided by an embodiment of the present invention;

[0054] Figure 5 A schematic diagram of the interaction between a kitchen appliance and the cloud provided by an embodiment of the present invention;

[0055] FIG6( a ) is a schematic diagram of interaction when a question-and-answer skill is triggered according to an embodiment of the present invention;

[0056] FIG6( b ) is a schematic diagram of interaction when a device operation skill is triggered according to an embodiment of the present invention;

[0057] FIG6( c ) is a schematic diagram of interaction when an audio entertainment skill is triggered according to an embodiment of the present invention;

[0058] Figure 7 A schematic diagram of a process implementation of a voice interaction method provided by an embodiment of the present invention;

[0059] Figure 8 A schematic diagram of a voice interaction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0061] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0063] See also Figure 1 , Figure 1 A schematic diagram of a process implementation of a voice interaction method provided by an embodiment of the present invention, the method comprising:

[0064] Step 101, recognize voice data and obtain recognition results.

[0065] like Figure 2 As shown, Figure 2 A schematic diagram of the composition structure of a voice interaction system provided in an embodiment of the present invention, the voice interaction system 200 includes a cloud 201 and a device 202, and the cloud and the device are communicatively connected.

[0066] In the embodiment of the present invention, the device 202 may be a smart kitchen appliance, such as an induction cooker, a microwave oven, a refrigerator, an oven, an electric rice cooker, an electric stew pot, or other local devices.

[0067] Step 101 may be executed by a local device or a cloud device.

[0068] In one possible example, after the device establishes a connection with the cloud, the device may obtain voice data from the user and upload the voice data to the cloud. After receiving the voice data, the cloud performs automatic voice recognition on the voice data to obtain multiple recognition results, wherein each recognition result has a different accuracy rate. An accuracy rate threshold is pre-set to filter out recognition results with an accuracy rate lower than the accuracy rate threshold. After filtering, one or more recognition results in step 101 may be obtained.

[0069] Step 102, determining the target cooking step to be hit according to the recognition result and the cooking content of the cooking step in the recipe.

[0070] Step 102 may be executed by a local device or a cloud device.

[0071] In one embodiment, before executing step 101, the recipe currently being cooked is determined, for example, the recipe currently being cooked is determined according to a manual input or voice input by a user.

[0072] A recipe may include one or more cooking steps, and cooking is completed after these cooking steps are completed.

[0073] For another example, the cloud determines the recipe that the user currently needs to cook by interacting with the device, and sends the cooking steps of the recipe to the device one by one, so that the device broadcasts the cooking steps to the user.

[0074] In a possible example, after obtaining the recognition result, if the ratio of the frequency of a word in the recognition result to the number of recognition results is greater than a preset value, the word is used as a keyword to characterize the recognition result. For example, if the preset value is 0.75, and the word "heating" appears 4 times in 5 recognition results, and the ratio of its frequency to the number of recognition results (0.8) is greater than 0.75, then the word can be used as a keyword. Calculate the matching degree between the keyword and the cooking content corresponding to one or more cooking steps of the current recipe. Select one or more cooking steps whose matching degree exceeds the first matching threshold as alternative cooking steps; or sort the cooking steps from large to small according to the matching degree, and select one or more cooking steps with the highest ranking as alternative cooking steps, such as selecting the cooking steps with the top 20% or top 30% matching degree as alternative cooking steps; and then select the target cooking step from the alternative cooking steps. Optionally, the matching method can select synonym matching. There may be 0, 1 or more keywords in the recognition result.

[0075] Step 103: Select a target skill for answering the voice data from the response skills configured for the target cooking step according to the recognition result.

[0076] Step 103 may be executed by a local device or a cloud device.

[0077] In one possible example, a recipe library is pre-established, and one or more response skills are configured for one or more cooking steps of each recipe in the recipe library to answer voice data. The response skills may include question-and-answer skills, device operation skills, prompt skills, guidance skills, entertainment skills, etc. For example, the steamed egg recipe includes a total of 3 cooking steps. A corresponding skill may be configured for each of the 3 cooking steps, or only one or two cooking steps may be configured with skills. When configuring a skill for a cooking step, only one skill may be configured, or multiple skills may be configured. For example, both the question-and-answer skill and the entertainment skill may be configured for the cooking step of "separating egg white and yolk". When the question-and-answer skill of this cooking step is triggered, the cloud or device answers the questions raised by the user regarding this cooking step; when the entertainment skill of this cooking step is triggered, the cloud or device plays entertainment items such as related music or videos. The cloud may determine the voice intent based on the recognition result, and further, according to the voice intent, the response skill adapted to the voice intent may be selected from the response skills configured for the target cooking step to answer the voice data.

[0078] Step 104, providing target skills. Step 104 can be performed by a local device or a cloud device.

[0079] The following describes how to provide a target skill by taking the case where the target skill is a question-and-answer skill or a device operation skill and is executed in the cloud as an example.

[0080] If the target skill is a question-and-answer skill, the cloud obtains the answer to the question that matches the recognition result based on the target cooking steps, and sends the answer to the device so that the device announces the answer to the user. Optionally, the cloud locates related questions in the question-and-answer library based on the cooking content of the target cooking steps of the recipe; matches the related questions with the recognition results one by one, and selects the answer corresponding to the related question with the highest matching degree as the answer to the question. Among them, the matching method can be selected from complete matching or partial matching, etc.

[0081] If the target skill is a device operation skill, the cloud determines whether the device has the device operation skill (such as heating); if it does, the cloud sends an operation instruction to the device to enable the device to perform the corresponding operation (heating).

[0082] It can be seen that the embodiment of the present invention obtains recognition results by recognizing voice data, and determines the target cooking steps to be hit according to the recognition results and the cooking content of the cooking steps in the recipe. Compared with providing voice interaction in the cooking process by combining the voice data alone, in the embodiment of the present invention, the cooking content in the recipe currently being cooked is combined to determine the steps executed by the voice data sent by the current user. Due to the added reference content, voice interaction can be performed more accurately. On the other hand, a single cooking step is configured with one or more skills, but the voice data that hits these skills may be the same. At this time, the results will be further recognized and the target skills will be selected for provision. Compared with configuring a single skill, more dimensional needs can be provided to users, and more fine-grained responses to users' needs can be achieved, thereby improving the accuracy of voice interaction and providing users with a better user experience.

[0083] In an optional embodiment, step S102 may include: determining the hit alternative cooking steps based on the matching degree between the keywords representing the recognition results and the cooking content of the cooking steps in the recipe; determining the number of steps between the alternative cooking steps and the current cooking steps in the recipe; and selecting the target cooking step from the alternative cooking steps based on the priority coefficient corresponding to the number of steps.

[0084] After obtaining the recognition result, the cloud or device can further obtain one or more keywords representing the recognition result, match the one or more keywords with the cooking content corresponding to each cooking step of the recipe, and select one or more cooking steps with a matching degree exceeding a first matching threshold as candidate cooking steps. Optionally, the matching method can select synonym matching.

[0085] Then, determine the number of steps between the alternative cooking step and the current cooking step. If there are G cooking steps in the recipe, G is a positive integer, then the number of steps ranges from 0 to G. For example, the current cooking step is the second step in the recipe, and there are 3 alternative cooking steps, namely the first step, the third step, and the fourth step in the recipe, then the number of steps between them and the current cooking step is 1, 1, and 2, respectively.

[0086] Finally, the priority coefficient of each candidate cooking step is calculated according to the correspondence rule between the priority coefficient and the number of steps, and the candidate cooking step with the highest priority coefficient is selected as the target cooking step.

[0087] It should be noted that different numbers of steps correspond to different priority coefficients, and the smaller the number of steps, the higher the corresponding priority coefficient. The correspondence rule between the priority coefficient and the number of steps can be preset, for example, it can be set to: y = 1-x / G. Among them, y is the priority coefficient, x is the number of steps, and G is the total number of cooking steps in the recipe.

[0088] It can be seen that in the embodiment of the present invention, the target cooking step is determined by matching the recognition result corresponding to the voice data with the cooking content of the cooking step in the recipe and the priority coefficient of the cooking step. Compared with directly taking the current cooking step as the target cooking step, the positioning of the target cooking step is more accurate, the occurrence of misjudgment can be avoided, and the accuracy of voice interaction is improved.

[0089] In an optional embodiment, step S103 may include: according to the recognition result, selecting a response skill that matches the recognition result as the target skill from the response skills configured for the target cooking step.

[0090] It should be noted that when the cloud or device configures response skills for cooking steps, it also configures relevant trigger words for each response skill. For example, the trigger words "heating", "high temperature", "thawing", etc. are configured for the device operation skill "heating". When the cloud recognizes that the voice data includes these trigger words, it is possible to trigger the device operation skill "heating", causing the device to perform a heating operation.

[0091] In one possible example, after obtaining one or more recognition results with an accuracy rate higher than the accuracy rate threshold through step 101, the keywords in the recognition results, such as "heating", are matched one by one with the trigger words corresponding to the various response skills configured for the target cooking step to obtain one or more response skills with a matching rate exceeding the second matching threshold. Finally, the target skill for answering the voice data is selected from the response skills with a matching rate exceeding the second matching threshold. The matching method may be a full match, a partial match, a word segmentation match, a synonym match, etc. Preferably, the target skill may be the response skill with the highest matching rate.

[0092] It can be seen that in the embodiment of the present invention, by configuring one or more response skills for a single cooking step and selecting a matching response skill for the target cooking step based on the recognition result, the needs of users in different dimensions can be met and the user experience can be improved. Even for the same voice data that can hit different skills, further distinction can be made based on the recognition result to obtain the skill that best matches the voice data, paving the way for providing accurate services later.

[0093] In an optional embodiment, based on the recognition result, a response skill matching the recognition result is selected from the response skills configured for the target cooking step as the target skill, which may include: based on the recognition result, selecting an alternative skill matching the recognition result from the response skills configured for the target cooking step; and selecting the target skill from the alternative skills based on the accuracy of the recognition result and the hit rate of the alternative skills.

[0094] The hit rate of the candidate skill indicates the probability of the candidate skill being used. The hit rate of each response skill is obtained based on historical statistical data. For example, historical statistical data show that within a period of time, a response skill is selected as a candidate skill K times and is selected as a target skill L times, then the hit rate of the response skill is L / K.

[0095] In one possible example, after obtaining the keyword according to the recognition result and determining the target cooking step, the keyword is matched with the trigger words corresponding to each response skill configured for the target cooking step, and one or more response skills whose matching rate exceeds the second matching threshold are used as candidate skills. Obtain the hit rate corresponding to one or more candidate skills. Multiply or add the accuracy rates corresponding to the multiple recognition results and the hit rates corresponding to the one or more candidate skills respectively, select the recognition result corresponding to the result with the largest product / sum as the target recognition result, and select the candidate skill corresponding to the result with the largest product / sum as the target skill.

[0096] For example, the recognition results with accuracy rates higher than the accuracy threshold are recognition result 1 and recognition result 2, and the corresponding accuracy rates are accuracy rate 1 and accuracy rate 2, respectively, and the hit rates of the candidate skills are hit rate 1 of skill 1 and hit rate 2 of skill 2, respectively. The recognition result and candidate skill corresponding to the maximum product are selected from the four product results obtained by accuracy rate 1×hit rate 1, accuracy rate 1×hit rate 2, accuracy rate 2×hit rate 1, and accuracy rate 2×hit rate 2. It should be noted that the target skill can also be selected by other methods, and the embodiment of the present invention does not limit the specific method for selecting the target skill.

[0097] It can be seen that in the embodiment of the present invention, the target skill is selected according to the accuracy of the recognition result and the hit rate of the alternative skills, which fully considers the probability of use of each skill, improves the accuracy of correctly selecting the target skill, and can meet the needs of users in most scenarios. In addition, while determining the target skill, the target recognition result can also be determined, so that accurate services can be provided to users based on the target recognition result. For example, when the target skill is determined to be a device operation skill and the target recognition result is "preheat for 3 minutes", the device can perform the corresponding preheating operation.

[0098] In an optional embodiment, based on the recognition result, a response skill that matches the recognition result is selected as the target skill from the response skills configured for the target cooking step. It may also include: based on the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state is selected as the target skill from the response skills configured for the target cooking step.

[0099] In one possible example, when the target cooking step is configured with a response skill that matches the recognition result, if the response skill that matches the recognition result is a device operation skill, it is necessary to determine whether to select the device operation skill as the target skill in combination with the current cooking state of the device. That is, if the device operation skill that matches the recognition result has been executed, the device operation skill cannot be selected as the target skill; if the device operation skill that matches the recognition result has not been executed, the device operation skill can be selected as the target skill.

[0100] In addition, if the response skill that matches the recognition result is a question-and-answer skill, you can directly select the question-and-answer skill as the target skill; if you are currently waiting for the device to cook automatically or complete the last cooking step, you can trigger the entertainment skill based on the recognition result.

[0101] It can be seen that the embodiment of the present invention combines the specific cooking status to select the appropriate response skill to answer the voice data, rather than directly selecting the response skill that matches the recognition result as the target skill, thereby avoiding the problem that the selected target skill does not match the current cooking status and fails to achieve the expected effect, thereby improving the user experience.

[0102] In an optional embodiment, if it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has been executed, a question-and-answer skill associated with the cooking operation and matching the recognition result, and / or a prompt skill is selected as the target skill.

[0103] In one possible example, the cloud is used as the execution subject, and the cloud matches the recognition result and the response skill configured for the target cooking step, and determines that the response skill matching the recognition result is the device operation skill (such as heating). At the same time, the cloud obtains the current cooking state of the device, and determines that the device operation skill (such as heating) has been executed, so there is no need to trigger the device operation skill again. From the response skills associated with the cooking operation other than the device operation skill configured for the target cooking step, the question-and-answer skill matching the recognition result and / or the prompt skill are selected as the target skill to respond to the voice data.

[0104] For example, the recognition result corresponding to the voice data is "microwave heating", and the skill that matches the recognition result is the device operation skill. The microwave oven is currently in the heating state. The cloud confirms that the voice intention may be to ask about the heating time, select the question and answer skill, and / or, the prompt skill as the target skill. Trigger the question and answer skill that matches the target cooking step, and answer through the device "The second step needs to be heated for 2 minutes"; and / or, trigger the prompt skill that matches the target cooking step, such as triggering the cooking time prompt skill, prompting the user "The second step needs to be heated for 2 minutes", and triggering the cooking step trick prompt skill. After that, the user is prompted "The heating time is short and the taste is better." It should be noted that the question and answer skills and prompt skills in this example can also be replaced by other skills, and this application does not limit this.

[0105] Optionally, the prompt skill includes at least one of the following: a cooking time prompt skill; a cooking step trick prompt skill. Among them, the cooking time prompt skill is used to prompt the user the time required for the target cooking step, and / or to prompt the user that the target cooking step will be completed within a period of time. For example, the user is prompted that "the first step takes 3 minutes", or the user is prompted that "there are 3 minutes left to complete the heating". The cooking step trick prompt skill is used to prompt some tricks about the target cooking step. For example, eggs will be softer after being beaten with water.

[0106] It can be seen that in the embodiment of the present invention, when the device operation skill hit by the voice data has been executed, the method of not responding to the voice data will not be adopted, but other skills related to cooking operations and matching the recognition results of the voice data will be provided to the user for response. In this way, it can help users solve questions about recipes and help users master cooking time and cooking tips, which is conducive to users improving their cooking skills and improving user experience.

[0107] In an optional embodiment, if it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has not been executed, the equipment operation skill associated with the cooking operation and matching the recognition result is selected as the target skill.

[0108] In one possible example, the cloud is used as the execution subject, and the cloud matches the recognition result and the response skill configured for the target cooking step, and determines that the response skill that matches the recognition result is the device operation skill. At the same time, the cloud obtains the current cooking status of the device, and determines that the device operation skill is not executed, and then selects the device operation skill as the target skill. For example, the recognition result corresponding to the voice data is "microwave heating", and the skill that matches the recognition result is the device operation skill, and the microwave oven is not currently performing the heating operation, then the device operation skill (heating) is selected as the target skill. In this way, when it is determined that the skill that matches the recognition result is the device operation skill and the device operation skill is not executed, the device operation skill can be triggered in time. In response to the voice data.

[0109] In an optional embodiment, after selecting a response skill that matches the recognition result as the target skill from the response skills configured for the target cooking step based on the recognition result, it may include: if it is determined based on the recognition result that the hit target cooking step is not configured with a response skill that matches the recognition result, taking a guiding skill as the target skill for answering the target data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next voice.

[0110] For example, if the matching rates of the keywords in the recognition result and the trigger words corresponding to the response skills configured for the target cooking step are all lower than the second matching threshold, it is determined that the target cooking step is not configured with a response skill matching the recognition result. The guiding skill is used as the target skill trigger, and a guiding voice is output to guide the user to say the next voice.

[0111] In one embodiment, the guiding voice may be a prompting voice, for example, a prompting voice to re-enter.

[0112] In another embodiment, the guiding voice may be an inquiry voice.

[0113] The outputting of the guiding voice for guiding the user to speak the next voice comprises: outputting the inquiry voice for guiding the user to speak the next voice according to the recognition result.

[0114] For example, the inquiry voice is used to prompt whether the recognition result of the previous voice is correct.

[0115] For another example, the output guides the user to say the next voice, including: according to the current recipe and / or the current cooking status, outputs an inquiry voice asking whether it is necessary to ask about the current cooking status or about the content of the current recipe.

[0116] In this way, even if there is no response skill that matches the recognition result, the user can be guided to speak the next voice through prompt voice, inquiry voice and other guiding voices, so that the device can perform corresponding operations according to the user's next voice.

[0117] In an optional embodiment, step S104 may include: when executing the device operation skill, if the cooking device is a controlled device, controlling the cooking device to perform the corresponding device operation; or, if the cooking device is not a controlled device, outputting an operation prompt for controlling the device operation.

[0118] For example, with the cloud as the execution subject, when executing the device operation skill of heating, if the cooking device is a controlled device, the cloud sends a heating instruction to the cooking device to control the cooking device to perform the heating operation. If the cooking device is not a controlled device, the cloud outputs an operation prompt "Please press the heating button" through the device to prompt the user to control the cooking device.

[0119] It can be seen that the embodiments of the present invention fully cover controlled devices and uncontrolled devices. Even for uncontrolled devices, the user will be prompted to perform relevant operations, so that the cooking steps can be carried out smoothly, improving the user experience.

[0120] For example, the non-controlled device may be a non-networked cooking device, and the controlled device is a networked controlled device. In this way, the networked cooking device can receive control instructions sent by the gateway or server to provide control skills, while the non-networked cooking device requires manual operation by the user.

[0121] In an optional embodiment, after step S104, it also includes: if the hit target cooking step is not determined based on the recognition result and the cooking content in the recipe, the guiding skill is used as the target skill for responding to the target data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next step voice.

[0122] When the accuracy of all speech recognition results in step 101 is lower than the accuracy threshold, or the matching degree between the cooking content corresponding to the current recipe cooking step and the keywords in the recognition result is lower than the first matching threshold, the target cooking step cannot be determined. At this time, the device outputs a guiding voice to guide the user to say the next voice, such as prompting the user "Please re-enter the voice". In this way, even if the target cooking step cannot be determined, the problem of not being able to determine the target cooking step can be solved by guiding the user to perform the next operation. For example, the user is guided to re-enter the voice, and when the user re-enters a clearer voice, the target cooking step is determined.

[0123] In another optional embodiment, after step S104, the method further includes: determining that the device is in the process of automatically cooking or waiting for the last cooking step to be completed according to the current cooking state, and executing entertainment skills according to the multimedia information hit by the recognition result. The entertainment skills include: audio entertainment skills for outputting audio; and / or film and television entertainment skills for outputting film and television.

[0124] While waiting for the device to cook automatically, such as waiting for the rice cooker to cook soup (and possibly adding ingredients later), or while waiting for the final cooking step to be completed, such as waiting for the oven to bake apples after completing all other steps, if the cloud or device determines that the cooking state is one of the above two situations, the keyword is obtained based on the recognition result. The keyword is matched with the name of the online audio or online video. If the highest matching rate is a certain audio, the audio is played; if the highest matching rate is a certain video, the video is played.

[0125] It can be seen that in the embodiment of the present invention, the skill triggered by the recognition result can be determined in combination with the cooking state. Even if the recognition result may trigger other skills such as question-and-answer skills or device operation skills, when the device is in a certain cooking state, only the entertainment skill is triggered.

[0126] Next, an exemplary application of the embodiment of the present invention in an actual cooking scenario will be described.

[0127] See also Figure 3 , Figure 3 The structure diagram of a voice interaction system provided by an embodiment of the present invention is shown in FIG. 300. The voice interaction system 300 includes a cloud 301, kitchen appliances 302, and a network 303. The cloud is connected to the kitchen appliances, and both the cloud and the kitchen appliances can access the network. The kitchen appliances include induction cookers, microwave ovens, refrigerators, ovens, rice cookers, electric stew pots, etc., and the kitchen appliances have functions such as recipe step broadcasting, intelligent question and answer, automatic cooking, music playback, and video playback.

[0128] Deploy the knowledge base in the cloud in advance, such as Figure 4 As shown, the knowledge base 400 includes three parts: recipe base 401, question and answer base 402 and skill base 403. The knowledge base is updated as needed or regularly. The recipe base includes various recipes. A question and answer base is established based on the questions involved in each step of each recipe, so that relevant questions and answers can be searched in the question and answer base by using the specific steps of a specific recipe as an index.

[0129] One or more skills are configured for each step of each recipe in the recipe library. The skills include question-and-answer skills, equipment operation skills, entertainment skills, prompt skills, and guidance skills. When configuring the skills, the cloud also configures relevant trigger words for each skill. For example, the trigger words "heating", "high temperature", and "thawing" are configured for the equipment operation skill "heating". When the voice data received by the cloud includes these trigger words, it is possible to trigger the equipment operation skill "heating", so that the kitchen appliance performs the heating operation. The cloud obtains the hit rate of each skill based on statistical data, and stores the hit rate of each skill and the trigger words related to each skill while configuring the skills, thereby establishing a skill library. Among them, the hit rate represents the probability that the skill related to a certain step is used, and the hit rate of each skill related to each step is different.

[0130] When the question-and-answer skill is hit, the cloud or kitchen appliance searches for answers related to the user's question in the question-and-answer library and answers it; when the device operation skill is hit, the kitchen appliance can perform operations, such as heating; when the entertainment skill is hit, the kitchen appliance can play music or videos, etc.; when the prompt skill is hit, the kitchen appliance prompts the cooking time or cooking tips; when the guidance skill is hit, the kitchen appliance outputs a guidance voice to guide the user's operation. For example, the first step of a recipe is configured with a question-and-answer skill and a device operation skill, and the second step is configured with an entertainment skill. Among them, the question that the question-and-answer skill configured for the first step may involve is "How long does it take to heat milk?", the device operation skill is heating, and the entertainment skill related to the second step is playing music related to the current recipe (baked apples), such as playing "Little Apples".

[0131] Kitchen appliances can be woken up by specific voice data, for example, an oven can be woken up by the voice "oven".

[0132] like Figure 5 As shown, Figure 5 A schematic diagram of the interaction between a kitchen appliance and the cloud provided by an embodiment of the present invention. Figure 5In the example, after the kitchen appliance is awakened, it automatically establishes a connection with the network and the cloud, obtains the user's voice data a, and uploads the obtained voice data a to the cloud. The cloud performs voice recognition on the voice data a, obtains the corresponding text, and determines the associated words related to the recipe name in the text. Then, the cloud searches the recipe library in the cloud or the network based on the associated words, obtains the recipe with the highest matching degree with the associated words, and takes the recipe with the highest matching degree as the target recipe. For example, when the oven has been awakened, the oven uploads the received voice data "cake" to the cloud. After receiving the voice data, the cloud recognizes the voice data through automatic voice recognition technology and obtains the text of "cake". After that, the cloud searches the recipe library with "cake" as the associated word, and takes the recipe with the highest matching degree with "cake" in the search results as the target recipe.

[0133] After obtaining the target recipe, the cloud determines the cooking content corresponding to the cooking steps of the target recipe, and sends the cooking content corresponding to the cooking steps to the kitchen appliance, so that the kitchen appliance broadcasts the cooking content corresponding to the cooking steps of the target recipe.

[0134] After one step is finished and before the next step is announced, the kitchen appliance can perform corresponding operations (such as heating, freezing, playing songs, etc.) according to the received voice data. The kitchen appliance uploads the acquired voice data b to the cloud. The cloud recognizes the voice data b, determines the skills that match the recognition result, and triggers the response skills. As shown in FIG6(a), FIG6(a) is an interactive schematic diagram when the question-and-answer skill is triggered according to an embodiment of the present invention. In FIG6(a), when the question-and-answer skill is triggered, the cloud searches for the corresponding answer in the question-and-answer library or the network according to the question corresponding to the voice data b, and sends the answer to the kitchen appliance so that the kitchen appliance answers the user's question. As shown in FIG6(b), FIG6(b) is an interactive schematic diagram when the device operation skill is triggered according to an embodiment of the present invention. In FIG6(b), when the device operation skill is triggered, the cloud determines whether the kitchen appliance has the device operation skill corresponding to the voice data b. If it has, the cloud sends an operation instruction to the kitchen appliance so that the kitchen appliance performs the operation corresponding to the voice data b. If it does not have, the cloud sends a broadcast instruction to the kitchen appliance so that the kitchen appliance broadcasts to the user "Sorry, ** does not have the *** skill". As shown in FIG6(c), FIG6(c) is a schematic diagram of the interaction when the audio entertainment skill is triggered provided by an embodiment of the present invention. In FIG6(c), when the audio entertainment skill is triggered, the cloud sends a search request to the network using the name of the target recipe as the search term, receives the audio link retrieved by the network using the recipe name as the search term, and sends the audio link to the kitchen appliance. The kitchen appliance accesses the network through the audio link, obtains the audio corresponding to the audio link, and then plays the audio.

[0135] The cloud recognizes the voice data b, determines the skills that match the recognition results, and triggers the response skills. Figure 7 The process implementation diagram of the voice interaction method shown is implemented as follows:

[0136] Step 701, converting the voice data b into text to obtain N recognition results; wherein N is a positive integer.

[0137] Step 702: Determine the recognition results whose accuracy rates exceed the accuracy rate threshold among the N recognition results as the candidate recognition results, and determine the keywords according to the frequency of occurrence of each word in the candidate recognition results.

[0138] Step 703, determining the target cooking steps in the target recipe that match the keyword.

[0139] Step 704, matching the trigger words of the skills configured for the target cooking step with the keywords to determine the candidate skills.

[0140] Step 705 , selecting a target skill from the candidate skills according to the accuracy of the candidate recognition results and the hit rate of the candidate skills, and selecting a target recognition result from the candidate recognition results.

[0141] Step 706, triggering the target skill.

[0142] For example, the target recipe is "cake". After broadcasting the first step of the target recipe "separating egg whites and egg yolks", the cloud obtains a piece of voice data through the oven. The cloud recognizes the voice data and converts it into text, obtaining 10 recognition results. Among them, there are two candidate recognition results with an accuracy rate exceeding the accuracy threshold (70%), namely, recognition result 1: "How to deal with eggs" (accuracy rate is 85%); recognition result 2: "How to deal with egg whites" (accuracy rate is 70%). In these two candidate recognition results, only the word "how to deal with" appears twice, and "how to deal with" is used as the keyword. The cloud matches the keywords and the cooking content corresponding to each step of the recipe to obtain multiple candidate cooking steps, such as the second step and the third step. The step with the smallest difference in the number of candidate cooking steps from the current step is used as the target step, such as the second step. The cloud matches the trigger words of each skill configured for the target cooking step with the keywords, and determines the skills with a matching rate higher than the set threshold as candidate skills, such as determining that the candidate skills in the second step are question-and-answer skills and equipment operation skills (specifically heating). The cloud multiplies or adds the accuracy of the candidate recognition results and the hit rate of the candidate skills, selects the skill with the largest product or the result corresponding to the largest product as the target skill, such as the equipment operation skill, and selects the candidate recognition result corresponding to the largest product or the result as the target recognition result, such as "how to deal with eggs". The equipment operation skill is triggered, and a heating operation instruction is sent to the kitchen appliance so that the kitchen appliance performs the heating operation.

[0143] It should be noted that in the embodiment of the present invention, the steps of the recipe are announced by the kitchen appliance. After the terminal (mobile phone, iPad, PDA, wearable electronic device, etc.) is connected to the cloud, smart kitchen appliance and network in the voice interaction system, the steps of the recipe can also be announced by the terminal. If the above-mentioned knowledge base is configured in the recipe teaching application of the terminal, and each step of each recipe in the knowledge base is also configured with corresponding skills, the voice data can also be responded to by the terminal, such as answering the user's questions through the terminal; sending operation instructions to the kitchen appliance through the terminal to make the kitchen appliance perform operations; playing songs through the terminal, etc.

[0144] It can be seen that the embodiment of the present invention configures skills for each step of each recipe and establishes a knowledge base including a recipe library, a question-and-answer library, and a skill library, so as to accurately determine the voice intent and select appropriate skills for the target cooking steps to respond to the voice data. In this way, based on the clear user needs, more accurate services are provided to users, the misrecognition rate is reduced, and a better user experience is provided to users.

[0145] See also Figure 8 , Figure 8 A schematic diagram of a voice interaction device provided by an embodiment of the present invention. Figure 8 As shown, the voice interaction device provided by the embodiment of the present invention includes: a recognition unit 801, which is used to recognize voice data and obtain a recognition result; a determination unit 802, which is used to determine the hit target cooking step according to the recognition result and the cooking content of the cooking step in the recipe; a selection unit 803, which is used to select a target skill for responding to the voice data from the response skills configured for the target cooking step according to the recognition result; and a providing unit 804, which is used to provide the target skill.

[0146] In some embodiments, the determination unit 802 is used to: determine the hit alternative cooking steps based on the matching degree between the keywords representing the recognition results and the cooking content of the cooking steps in the recipe; determine the number of steps between the alternative cooking steps and the current cooking steps in the recipe; and select the target cooking step from the alternative cooking steps based on the priority coefficient corresponding to the number of steps.

[0147] In some embodiments, the selection unit 803 is used to: select, according to the recognition result, from the response skills configured for the target cooking step, a response skill that matches the recognition result as the target skill.

[0148] In some embodiments, the selection unit 803 is further used to: select an alternative skill that matches the recognition result from the response skills configured for the hit target cooking step according to the recognition result; and select a target skill from the alternative skills according to the accuracy of the recognition result and the hit rate of the alternative skills.

[0149] In some embodiments, the selection unit 803 is further used to: select, according to the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state from the response skills configured for the target cooking step as the target skill.

[0150] In some embodiments, the selection unit 803 is also used to: if it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has been executed, select the question and answer skill associated with the cooking operation and matching the recognition result, and / or, prompt the skill as the target skill.

[0151] In some embodiments, the selection unit 804 is further used to: if it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has not been executed, select the equipment operation skill associated with the cooking operation and matching the recognition result as the target skill.

[0152] In some embodiments, the prompt skill comprises at least one of the following: a cooking time prompt skill; a cooking step trick prompt skill.

[0153] In some embodiments, providing unit 804 is also used to: when executing device operation skills, if the cooking device is a controlled device, control the cooking device to perform corresponding device operations; or, if the cooking device is not a controlled device, output operation prompts for controlling device operations.

[0154] In some embodiments, the voice interaction device also includes a guiding unit 805, which is used to: if it is determined based on the recognition result that the hit target cooking step is not configured with a response skill that matches the recognition result, use the guiding skill as the target skill for answering the target data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next voice.

[0155] In some embodiments, the guidance unit 805 is also used for: if the target cooking step is not determined based on the recognition result and the cooking content in the recipe, the guidance skill is used as the target skill for responding to the target data; wherein, under the guidance skill, a guidance voice is output to guide the user to say the next step voice.

[0156] In some embodiments, the voice interaction device further includes an entertainment unit 806 for:

[0157] According to the current cooking state, it is determined that when the device is waiting to cook automatically or when the last cooking step is waiting to be completed, the entertainment skill is executed according to the multimedia information hit by the recognition result.

[0158] In some embodiments, the entertainment skills include: audio entertainment skills for outputting audio; and / or, video entertainment skills for outputting video.

[0159] An embodiment of the present invention provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the voice interaction method provided by the embodiment of the present invention.

[0160] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0161] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0162] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0163] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A voice interaction method, characterized in that: The method comprises: Recognize voice data and obtain recognition results; Determining a target cooking step according to the recognition result and the cooking content of the cooking step in the recipe; According to the recognition result, selecting a target skill for responding to the voice data from the response skills configured for the target cooking step; providing said target skills; The step of determining the target cooking step to be hit according to the recognition result and the cooking content of the cooking step in the recipe comprises: Determining a hit candidate cooking step according to a matching degree between a keyword representing the recognition result and a cooking content of the cooking step in the recipe; determining the number of steps between the alternative cooking step and the current cooking step in the recipe; The target cooking step is selected from the candidate cooking steps according to the priority coefficient corresponding to the step number.

2. The method according to claim 1, characterized in that The selecting, according to the recognition result, a target skill for responding to the voice data from the response skills configured for the target cooking step comprises: According to the recognition result, a response skill matching the recognition result is selected from the response skills configured for the target cooking step as the target skill.

3. The method according to claim 2, characterized in that The step of selecting, according to the recognition result, a response skill matching the recognition result from the response skills configured for the target cooking step as the target skill comprises: According to the recognition result, selecting an alternative skill matching the recognition result from the response skills configured for the target cooking step; The target skill is selected from the candidate skills according to the accuracy of the recognition result and the hit rate of the candidate skills.

4. The method according to claim 2, characterized in that: The step of selecting, according to the recognition result, a response skill matching the recognition result from the response skills configured for the target cooking step as the target skill comprises: According to the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state is selected from the response skills configured for the target cooking step as the target skill.

5. The method according to claim 4, characterized in that The step of selecting, according to the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state from the response skills configured for the target cooking step as the target skill, includes: If it is determined based on the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has been executed, a question-and-answer skill associated with the cooking operation and matching the recognition result and / or a prompt skill is selected as the target skill.

6. The method according to claim 4, characterized in that The step of selecting, according to the current cooking state and the recognition result, a response skill that matches the recognition result and is consistent with the current cooking state from the response skills configured for the target cooking step as the target skill, includes: If it is determined according to the current cooking state that the equipment operation skill configured for the target cooking step matching the recognition result has not been executed, the equipment operation skill associated with the cooking operation and matching the recognition result is selected as the target skill.

7. The method according to claim 5, characterized in that The prompting skills include at least one of the following: Cooking time reminder skills; Tips, tricks, and skills for cooking steps.

8. The method according to claim 7, characterized in that The providing of the target skills comprises: When executing the device operation skill, if the cooking device is a controlled device, the cooking device is controlled to perform the corresponding device operation; or, if the cooking device is not a controlled device, an operation prompt for controlling the device operation is output.

9. The method according to claim 2, characterized in that: The method further comprises: If it is determined based on the recognition result that the hit target cooking step is not configured with a response skill that matches the recognition result, the guiding skill is used as the target skill to respond to the target data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next step voice.

10. The method according to claim 1, characterized in that The method further comprises: If the target cooking step is not determined based on the recognition result and the cooking content in the recipe, the guiding skill is used as the target skill to respond to the voice data; wherein, under the guiding skill, a guiding voice is output to guide the user to say the next step.

11. The method according to claim 1, characterized in that: The method further comprises: When it is determined according to the current cooking state that the device is in the process of automatically cooking or when the last cooking step is waiting to be completed, entertainment skills are executed according to the multimedia information hit by the recognition result.

12. The method according to claim 11, wherein: The entertainment skills include: Audio entertainment skills, used to output audio; and / or, Film and television entertainment skills, used to output film and television.

13. A voice interaction device, characterized in that: include: A recognition unit, used for recognizing voice data and obtaining a recognition result; A determination unit, configured to determine a hit target cooking step according to the recognition result and the cooking content of the cooking step in the recipe; The step of determining the target cooking step to be hit according to the recognition result and the cooking content of the cooking step in the recipe comprises: Determining a hit candidate cooking step according to a matching degree between a keyword representing the recognition result and a cooking content of the cooking step in the recipe; determining the number of steps between the alternative cooking step and the current cooking step in the recipe; selecting the target cooking step from the candidate cooking steps according to the priority coefficient corresponding to the number of the step; A selection unit, configured to select a target skill for responding to the voice data from the response skills configured for the target cooking step according to the recognition result; A providing unit is used to provide the target skill.

14. A voice interaction device, characterized in that: At least comprising: a processor and a memory for storing executable instructions that can be run on the processor, wherein: When the processor is used to run the executable instructions, the executable instructions execute the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute the method according to any one of claims 1 to 12.

Images