Voice instruction processing method and device, electrical equipment and medium

Abstract

The invention discloses a voice instruction processing method and device, electrical equipment and a medium. The method comprises the following steps: acquiring a target voice instruction; an instruction text of the target voice instruction is processed based on a preset large language model, a rewritten instruction text is obtained, and the preset large language model is used for conducting intention reasoning on the instruction text so as to rewrite the instruction text; and determining a target type of the rewriting instruction text, and determining an instruction processing strategy corresponding to the rewriting instruction text based on the target type. According to the scheme, the response rate of the voice instruction can be improved, and user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of speech processing, and particularly relates to a speech instruction processing method and device, an electrical appliance and a medium. BACKGROUND

[0002] With the continuous development of science and technology, the intelligence of electrical appliances is also increasingly high. Many electrical appliances (such as air conditioners, televisions, etc.) can support voice interaction. When a user controls an electrical appliance through a voice instruction, the recognition may fail due to noise or ambiguous instruction intent contained in the voice instruction, thereby failing to effectively control the electrical appliance and reducing the user experience. SUMMARY

[0003] In view of the above technical problems in the related art, the embodiments of the present application provide a speech instruction processing method and device, an electrical appliance and a medium to improve the success rate of voice control of the electrical appliance and improve the user experience.

[0004] In a first aspect, the embodiments of the present application provide a speech instruction processing method, comprising:

[0005] obtaining a target speech instruction;

[0006] processing an instruction text of the target speech instruction based on a preset large language model to obtain a rewritten instruction text, wherein the preset large language model is used for intent reasoning on the instruction text to rewrite the instruction text;

[0007] determining a target type of the rewritten instruction text, and determining an instruction processing strategy corresponding to the rewritten instruction text based on the target type.

[0008] In some embodiments, the obtaining of the target speech instruction comprises:

[0009] receiving a to-be-processed speech instruction of a user;

[0010] performing speech recognition on the to-be-processed speech instruction to obtain a recognition result;

[0011] if the recognition result is a recognition failure, taking the to-be-processed speech instruction as the target speech instruction.

[0012] In some embodiments, before the processing of the instruction text of the target speech instruction based on the preset large language model, the method further comprises:

[0013] obtaining prompt engineering data corresponding to a speech instruction rewriting task;

[0014] inputting the prompt engineering data into the preset large language model, so that the preset large language model performs at least interference information filtering and intent reasoning on the input voice instruction based on the prompt engineering data to obtain a rewritten instruction text with complete sentence structure.

[0015] In some embodiments, the determination of the target type of the rewritten instruction text comprises:

[0016] determining a target confidence corresponding to the rewritten instruction text output by the preset large language model;

[0017] if the target confidence is greater than or equal to a preset confidence threshold, determining that the target type is a first type, the first type indicating that the rewritten instruction text can be directly executed;

[0018] if the target confidence is less than the preset execution threshold, determining that the target type is a second type, the second type indicating that the rewritten instruction text cannot be directly executed.

[0019] In some embodiments, the determination of the instruction processing strategy corresponding to the rewritten instruction text based on the target type comprises:

[0020] if the target type is the second type, determining a sub-type of the rewritten instruction text based on a preset instruction word table, the preset instruction word table comprising a preset sentence structure corresponding to an instruction intent, necessary word slots corresponding to the preset sentence structure, and word slots of fixed collocations of the instruction intent;

[0021] determining an instruction processing strategy corresponding to the rewritten instruction text based on the sub-type;

[0022] wherein the sub-type comprises a first sub-type for representing a contradictory instruction, a second sub-type for representing a parameter missing instruction, and a third sub-type other than the first type and the second type.

[0023] In some embodiments, the determination of the instruction processing strategy corresponding to the rewritten instruction text based on the sub-type comprises:

[0024] if the sub-type is the first sub-type, determining a preset contradictory instruction definition table corresponding to a first instruction intent of the rewritten instruction text based on the first instruction intent, the preset contradictory instruction definition table comprising at least one piece of contradictory data corresponding to the first instruction intent;

[0025] matching the rewritten instruction text with the preset contradictory instruction definition table to determine target contradictory data corresponding to the rewritten instruction text from the at least one piece of contradictory data;

[0026] based on the target contradictory data, output first interaction data for describing the target contradictory data to a user, the first interaction data being used to perform at least one round of dialogue with the user to correct the target contradictory data.

[0027] In some embodiments, the determining, based on the sub-type, of the instruction processing strategy corresponding to the rewritten instruction text comprises:

[0028] If the sub-type is the second sub-type, determining, based on a second instruction intent of the rewritten instruction text, a preset parameter missing instruction definition table corresponding to the second instruction intent, the preset parameter missing instruction definition table comprising at least one preset sentence pattern structure under the second instruction intent and a combination of missing word slots corresponding to each preset sentence pattern structure.

[0029] matching the rewritten instruction text with the preset parameter missing instruction definition table to determine a target word slot missing in a target preset sentence pattern structure corresponding to the rewritten instruction text;

[0030] based on the target word slot, output second interaction data for describing that the target word slot is missing in the target preset sentence pattern structure to a user, the second interaction data being used to perform at least one round of dialogue with the user to complete the target word slot.

[0031] In some embodiments, the determining, based on the sub-type, of the instruction processing strategy corresponding to the rewritten instruction text comprises:

[0032] If the sub-type is the third sub-type, output default interaction data to the user to prompt that the target voice instruction cannot be executed.

[0033] In a second aspect, an embodiment of the present application provides a voice instruction processing apparatus, comprising:

[0034] an acquisition module configured to acquire a target voice instruction;

[0035] a first processing module configured to process an instruction text of the target voice instruction based on a preset large language model to obtain a rewritten instruction text, wherein the preset large language model is used to perform intent reasoning on the instruction text to rewrite the instruction text;

[0036] a second processing module configured to determine a target type of the rewritten instruction text and determine, based on the target type, an instruction processing strategy corresponding to the rewritten instruction text.

[0037] In a third aspect, an electric appliance is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the voice instruction processing method when executing the program.

[0038] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the program implements the steps of the voice instruction processing method when executed by a processor.

[0039] The one or more technical solutions provided by the embodiments of the present application at least achieve the following technical effects or advantages:

[0040] In the voice instruction processing method provided by the embodiments of the present application, the target voice instruction is obtained, the instruction text of the target voice instruction is processed based on a preset large language model to obtain a rewritten instruction text, the preset large language model is used to perform intent reasoning on the instruction text to rewrite the instruction text, the target type of the rewritten instruction text is determined, and based on the target type, the instruction processing strategy corresponding to the rewritten instruction text is determined. According to the present application, the instruction text of the target voice instruction can be rewritten by using the preset large language model, and in the case that the target voice instruction has noise or the instruction is ambiguous, the influence caused by these problems can be eliminated as much as possible through rewriting, the accuracy of the rewritten instruction text is improved, in addition, different instruction processing strategies can be further executed according to different types for the rewritten instruction text, the response rate of the instruction is improved, and thus the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 A flowchart of a voice instruction processing method provided by the embodiments of the present application is provided.

[0043] Figure 2 A schematic diagram of a voice instruction processing flow provided by the embodiments of the present application is provided.

[0044] Figure 3 A schematic diagram of a voice instruction processing device provided by the embodiments of the present application is provided.

[0045] Figure 4 A schematic diagram of an electric appliance provided by the embodiments of the present application is provided. DETAILED DESCRIPTION

[0046] The embodiment of the present specification provides a voice instruction processing method, device, electrical equipment and medium, comprising: obtaining a target voice instruction; processing the instruction text of the target voice instruction based on a preset large language model to obtain a rewritten instruction text, wherein the preset large language model is used for intention reasoning on the instruction text to rewrite the instruction text; determining the target type of the rewritten instruction text, and determining the instruction processing strategy corresponding to the rewritten instruction text based on the target type.

[0047] The scheme in the embodiment of the present specification can rewrite the instruction text of the target voice instruction through the preset large language model, and when the target voice instruction has noise or the instruction is ambiguous, the influence caused by these problems can be eliminated as much as possible through rewriting, the accuracy of the rewritten instruction text is improved, in addition, different instruction processing strategies can be further executed according to the different types of the rewritten instruction text, the response rate of the instruction is improved, and thus the user experience is improved.

[0048] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0050] The embodiment of the present specification provides a voice instruction processing method, as shown in Figure 1 The embodiment of the present specification provides a flow chart of a voice instruction processing method, and the method comprises the following steps:

[0051] Step S101: obtaining a target voice instruction;

[0052] Step S102: processing the instruction text of the target voice instruction based on a preset large language model, to obtain a rewritten instruction text, wherein the preset large language model is used for intent reasoning on the instruction text to rewrite the instruction text.

[0053] Step S103: determining the target type of the rewritten instruction text, and determining the instruction processing strategy corresponding to the rewritten instruction text based on the target type.

[0054] The method provided by the embodiments of the present specification can be applied to an electric appliance, which can be an air conditioner, a television, a smart speaker, etc., can also be applied to a server in communication connection with the electric appliance, and can also be applied to a system composed of the electric appliance and the server, which is not limited here.

[0055] In step S101, when a user performs voice control on an electric appliance, the user can issue a voice instruction to the electric appliance. For example, when the electric appliance is an air conditioner, the user issues a voice instruction "increase the temperature to 26 degrees Celsius" to the air conditioner. The target voice instruction can be any voice instruction issued by the user, a voice instruction containing noise, a voice instruction with unclear pronunciation, a voice instruction with ambiguous intent, etc.

[0056] In some embodiments, the target voice instruction can be obtained by the following steps: receiving a to-be-processed voice instruction of a user; performing voice recognition on the to-be-processed voice instruction to obtain a recognition result; if the recognition result is a recognition failure, taking the to-be-processed voice instruction as the target voice instruction.

[0057] When a user interacts with an electric appliance, the user can control the electric appliance through multiple rounds of interaction, wherein the voice instruction issued by the user in each round of interaction can be taken as a to-be-processed voice instruction.

[0058] In the embodiments of the present specification, voice recognition on the to-be-processed voice instruction can be implemented in multiple ways. In some embodiments, in order to quickly process the to-be-processed voice instruction and facilitate model deployment, a lightweight voice recognition model can be used to implement processing of the to-be-processed voice instruction. When performing voice recognition on the to-be-processed voice instruction, a Java NLU (Natural Language Understanding) process can be performed, including but not limited to converting the to-be-processed voice into text, performing word segmentation, intent recognition, etc. on the text, and finally obtaining a control instruction that can be executed by the electric appliance.

[0059] If the speech recognition result of the to-be-processed voice instruction is successful, the electrical appliance device can be controlled to directly perform the operation corresponding to the instruction. However, when the to-be-processed voice instruction contains a large amount of interference information or the user instruction intention is ambiguous, the speech recognition result can be a recognition failure. If a bottom-up process of voice interaction is directly entered at this time, such as replying to the user that "cannot be executed", "does not understand" or directly not replying, it will lead to low interaction efficiency and poor user experience. In order to avoid the above problems, in the embodiment of the present specification, when the speech recognition result of the to-be-processed voice instruction is a recognition failure, the to-be-processed voice instruction with a recognition failure can be taken as a target voice instruction, and the target voice instruction can be rewritten through a preset large language model, so as to increase the recognition rate of the target voice instruction and improve the success rate of voice interaction.

[0060] It should be noted that the large language model is a deep learning model trained based on massive text data, which can not only generate natural language text, but also deeply understand the meaning of the text and process various natural language tasks. The pre-training model of the large language model can include natural language understanding models, natural language generation models, etc. For example, the large language model can be a natural language generation model, such as Generative Pre-Trained Transformer (GPT), PaLM (Pathways Language Model), LLaMA (Large Language Model Meta AI), ChatGLM (a dialogue robot supporting Chinese and English), Moss (Multilingual Open-Source Semantic Search, a dialogue large language model), etc. In the embodiment of the present specification, the preset large language model can be a pre-trained large language model, which can be deployed on the electrical appliance device side or the cloud side.

[0061] In step S102, in order to process the target voice instruction, the target voice instruction is rewritten by calling the preset large language model. It should be understood that since the preset large language model can complete multiple natural language tasks, in order to improve the accuracy of the preset large language model in rewriting the voice instruction, the embodiment of the present specification can further include the following steps before calling the preset large language model to process the target language instruction: obtaining prompt engineering data corresponding to the voice instruction rewriting task; inputting the prompt engineering data into the preset large language model, so that the preset large language model filters at least interference information and reasons the intention of the input voice instruction based on the prompt engineering data, to obtain a rewritten instruction text with complete sentence structure.

[0062] Prompt engineering involves designing, experimenting with, and optimizing input prompts to guide a model in producing language output that meets specific requirements. Prompt engineering can include selecting appropriate vocabulary, grammar, context, and topic, as well as employing various techniques and strategies to influence the model's generative behavior and results. By optimizing prompt engineering, large language models can be made more accurate, controllable, and adaptable to diverse tasks and application scenarios.

[0063] In the embodiments of this specification, the task that the preset large language model needs to perform is to rewrite the voice instructions. The corresponding prompt engineering data can be manually written or generated by the model, which is not limited here. In some embodiments, the prompt engineering data of the preset large language model includes but is not limited to: a task description; a historical voice instruction that successfully interacts with the device; a complete voice interaction instruction vocabulary, which includes but is not limited to the control intention of the device, the corresponding sentence structure, word slots and generalized statements, wherein the sentence structure that can be executed in the vocabulary can be defined as a complete structure; a real voice instruction with noise; a real instruction with pure noise; a real instruction with a complete sentence structure without noise; an online real noise data annotation result, including a clear distinction between noise and instructions.

[0064] In order to better understand the prompting of engineering data in the embodiments of this specification, a specific example of prompting engineering data is given below. The prompting of engineering data includes the following prompts:

[0065] Tip 1: You will play the role of a corpus rewriter. You need to analyze the user's voice commands and rewrite the text for device control.

[0066] Tip 2: Equipment control includes: temperature setting;

[0067] Tip 3: The sentence structure of temperature setting command is: A = function name + operation + range + unit, B = range + unit, function name is a, operation is b, range is c, unit is d;

[0068] Tip 4: In Tip 3, Function Name = Temperature, Operation = Adjust to, Set to, Change to, Range = 16-30, Unit = Degrees;

[0069] Tip 5: In addition to the sentence structures involved in Tips 3 and 4, you can use modal particles and auxiliary words, including: please help me, please put;

[0070] Tip 6: Tips 3, 4, and 5 are defined as instructions for complete sentence structures;

[0071] Tip 7: Example of a voice command containing noise: Please, I am a sheep, the temperature is 25 degrees, why don’t you name it?

[0072] Prompt 8: Pure noise instruction example: disturb the dog house you how to shop black oh;

[0073] Prompt 9: No noise voice instruction example: please set the temperature to twenty-five degrees;

[0074] Prompt 10: Process the input voice instruction, if it contains noise, rewrite the instruction, and the rewritten result should meet sentence structure A or sentence structure B. If a, b, c, and d are missing due to the original content, please supplement them according to the provisions of prompt 2 and the input context. Output the rewritten result, and let the rewritten result be X.

[0075] Prompt 11: Here is an example: the input is "please me a sheep temperature twenty-five degrees how not to name", and the output is "temperature twenty-five degrees";

[0076] Prompt 12: Now please judge the user's voice instruction based on the above settings. Please directly give the rewritten result. If you cannot rewrite or judge, directly output the original voice instruction.

[0077] In the embodiments of the present specification, the prompt engineering data can make the preset large language model more accurately rewrite the voice instruction. When the preset large language model processes the voice instruction, it can at least filter the interference information and infer the intent of the voice instruction in the voice instruction, wherein the interference information filtering can be filtering out the noise in the voice instruction, and the intent inference can be inferring the control intent of the voice instruction to the device. Taking an air conditioner as an example, the intent of the voice instruction can include but is not limited to adjusting the temperature, adjusting the wind speed, adjusting the mode, timing on-off, and turning off. The preset large language model can also complete the voice instruction when the voice instruction is not clear, ambiguous or incomplete in sentence structure, to obtain a rewritten instruction with complete sentence structure. It should be noted that when processing the voice instruction, the intent inference and completion can also be based on the context of the voice instruction.

[0078] It should be noted that the rewritten instruction text output by the preset large language model can be an instruction text without interference information and reorganized sentence structure, for example, generating the rewritten instruction text according to the sentence structure specified in the prompt engineering data. In some embodiments, the preset large language model can output a rewritten instruction text with complete sentence structure.

[0079] In step S102, when processing the instruction text of the target voice instruction through the preset large language model, the target voice instruction can be directly input into the preset large language model, or the target voice instruction can be first converted into an instruction text, and then the instruction text is input into the preset large language model. The preset large language model rewrites the instruction text to obtain the rewritten instruction text corresponding to the target voice instruction.

[0080] In step S103, considering that the rewritten instruction text output by the preset large language model still cannot be directly executed, in the embodiments of the present specification, in order to further process the rewritten instruction text that cannot be directly executed and improve the interaction success rate of the voice instruction, different instruction processing strategies can be adopted for rewritten instruction texts of different target types.

[0081] The target type of the rewritten instruction text can include a first type that can directly execute the instruction and a second type that cannot directly execute the instruction. That is, for the first type, the rewritten instruction text is an instruction with clear and explicit intent, and the device can directly execute the operation corresponding to the instruction, and for the second type, the rewritten instruction text still has problems, causing the device to be unable to execute the instruction.

[0082] The target type of the rewritten instruction text can be determined in various ways. In some embodiments, the target type can be determined by the following steps: determining the target confidence of the rewritten instruction text output by the preset large language model; if the target confidence is greater than or equal to a preset confidence threshold, determining that the target type is the first type, which indicates that the rewritten instruction text can be directly executed; and if the target confidence is less than the preset execution threshold, determining that the target type is the second type, which indicates that the rewritten instruction text cannot be directly executed.

[0083] Specifically, when outputting the rewritten instruction text, the preset large language model can output the confidence of the rewritten instruction text at the same time, and compare the confidence with a preset confidence threshold. The preset confidence threshold can be set according to actual needs, for example, the preset confidence threshold can be 0.9, 0.95, etc., which is not limited here. When the confidence is greater than or equal to the preset confidence threshold, it indicates that the reliability and accuracy of the rewritten instruction text is high, at this time, the target type of the rewritten instruction text is the first type, and the corresponding instruction processing strategy can be to directly execute the instruction. When the confidence is less than the preset confidence threshold, it indicates that the accuracy of the rewritten instruction text is low and cannot be directly executed, and the target type of the rewritten instruction text is the second type, which needs to be further analyzed and processed.

[0084] In the embodiments of the present specification, for the second type of rewritten instruction text, step S103 can be implemented in the following manner: if the target type is the second type, determining the sub-type of the rewritten instruction text based on a preset instruction word table, the preset instruction word table including a preset sentence pattern structure corresponding to an instruction intent, necessary word slots corresponding to the preset sentence pattern structure, and word slots fixedly combined with the instruction intent; determining the instruction processing strategy corresponding to the rewritten instruction text based on the sub-type; wherein the sub-type includes a first sub-type for representing a contradictory instruction, a second sub-type for representing a parameter missing instruction, and a third sub-type other than the first type and the second sub-type.

[0085] Specifically, the preset instruction word table can be a voice interaction instruction word table in the prompt engineering data, or a separately set instruction word table different from the voice interaction instruction word table, which is not limited here. When constructing the preset instruction word table, for each instruction intent, a preset sentence pattern structure under each instruction intent, necessary word slots corresponding to the preset sentence pattern structure, and word slots fixedly combined with the instruction intent can be established. For example, if the instruction intent corresponding to the preset instruction word table includes temperature adjustment, the preset sentence pattern structure of temperature adjustment can be operation + range + unit, the necessary word slots can be operation, range, and unit, and the word slots fixedly combined with temperature adjustment can be temperature, Celsius, etc.

[0086] In some embodiments, matching the rewritten instruction text with the preset instruction word table can first determine the instruction intent corresponding to the rewritten instruction text, and then determine the preset sentence pattern structure corresponding to the instruction intent, the necessary word slots corresponding to the preset sentence pattern structure, and the word slots fixedly combined with the instruction intent in the preset instruction word table, and one-to-one match with the rewritten instruction text, to determine the sub-type of the rewritten instruction text according to the matching result.

[0087] In some embodiments, the rewritten instruction text and the word slots fixedly combined with the instruction intent are matched. If the rewritten instruction text contains content inconsistent with the fixedly combined word slots, the rewritten instruction text corresponds to a contradictory instruction, and the sub-type of the rewritten instruction text is the first sub-type. Still using the above example of the instruction intent being temperature adjustment, the fixedly combined word slots for temperature adjustment include temperature. If the rewritten instruction text contains wind speed, it indicates that the function name in the rewritten instruction text is contradictory to the function name corresponding to the instruction intent, so the sub-type of the rewritten instruction text is the first sub-type.

[0088] In some embodiments, after determining the instruction intention of the rewritten instruction text, a preset sentence pattern structure matching the rewritten instruction text can be determined from at least one preset sentence pattern structure corresponding to the instruction intention, and then the necessary word slots in the preset sentence pattern structure are compared with the rewritten instruction text one by one. If one or more necessary word slots are missing in the rewritten instruction text, it can be determined that the rewritten instruction text corresponds to a parameter missing instruction, i.e., the sub-type of the rewritten instruction text is the second sub-type. Still using the above example of the instruction intention being temperature adjustment, the necessary word slots can include operation, function name, range, and unit. If the rewritten instruction text is "adjust temperature", the rewritten instruction text is missing the two necessary word slots of range and unit, and the sub-type of the rewritten instruction text is the second sub-type.

[0089] In some embodiments, if the rewritten instruction text does not belong to the first sub-type nor the second sub-type, the sub-type of the rewritten instruction text is taken as the third sub-type.

[0090] In the embodiments of the present specification, different instruction processing strategies can be adopted for rewritten instruction texts of different sub-types. Next, the instruction processing strategies for the above three sub-types are described.

[0091] The first

[0092] If the sub-type of the rewritten instruction text is the first sub-type, the instruction processing strategy corresponding to the rewritten instruction text can be implemented by the following steps: determining a preset contradictory instruction definition table corresponding to the first instruction intention of the rewritten instruction text based on the first instruction intention of the rewritten instruction text, the preset contradictory instruction definition table including at least one piece of contradictory data corresponding to the first instruction intention; matching the rewritten instruction text with the preset contradictory instruction definition table to determine target contradictory data corresponding to the rewritten instruction text from the at least one piece of contradictory data; and outputting first interaction data for describing the target contradictory data to the user based on the target contradictory data, the first interaction data being used to perform at least one round of dialogue with the user to correct the target contradictory data.

[0093] In the embodiments of the present specification, for each instruction intention, a preset contradictory instruction definition table corresponding to the instruction intention can be constructed in advance. The preset contradictory instruction definition table can include contradictory data under the instruction intention, which can include but is not limited to the intention name of the instruction intention, the word slots included under the instruction intention, the type of contradiction between the word slots, the trigger point of the contradiction, and the contradiction guide, etc.

[0094] For example, still taking the instruction intention of temperature adjustment as an example, the word slots contained in the instruction intention are operation, function name, range, and unit. The types of contradictions between the word slots can include contradictions between the function name and the unit, contradictions between the function name, the range, and the unit, and the like. For example, if the rewritten instruction text is “adjust the temperature to 26 meters”, the contradiction type corresponding to the rewritten instruction text is a contradiction between the function name and the unit. For another example, if the rewritten instruction text is “adjust the temperature to 3 gears”, the contradiction type corresponding to the rewritten instruction text is a contradiction between the function name, the range, and the unit. Of course, other types of contradictions between the word slots can also be included, which will not be exemplified one by one here.

[0095] The trigger point of the contradiction can be used to describe the cause of the contradiction. For example, the trigger point of the contradiction can include that the range in the rewritten instruction text exceeds the preset range. For example, if the rewritten instruction text is “adjust the temperature to 40 degrees Celsius”, but the preset range of temperature adjustment is 16-30 degrees Celsius, the adjustment temperature in the rewritten instruction text exceeds the upper limit of the preset range, triggering the generation of the contradiction. For another example, if the rewritten instruction text is “adjust the temperature to 26 meters”, the trigger point of the contradiction is that the unit is inconsistent with the unit of the instruction intention. It should be noted that the trigger point of the contradiction can be set according to actual conditions, which will not be exemplified one by one here.

[0096] The contradiction indication can be a solution provided based on the type of contradiction between the word slots and / or the trigger point of the contradiction. For example, when the type of contradiction between the word slots is a contradiction between the function name and the unit, the corresponding contradiction indication can be to modify the unit to the unit corresponding to the instruction intention. For another example, when the trigger point of the contradiction is that the range in the rewritten instruction text exceeds the preset range, the corresponding contradiction indication can be to adjust the range to be within the preset range.

[0097] In the embodiments of the present specification, for the rewritten instruction text of the first sub-type, the first instruction intention of the rewritten instruction text can be obtained in the process of processing the target voice instruction by the preset large language model, or can be obtained by re-recognizing the intention of the rewritten instruction text, which is not limited here. After obtaining the first instruction intention, a preset contradiction instruction definition table corresponding to the first instruction intention is determined, and the rewritten instruction text is matched with the preset contradiction instruction definition table to determine the target contradiction data existing in the rewritten instruction text. That is, by matching the rewritten instruction text with the corresponding contradiction data, the specific contradiction type, contradiction trigger point, and / or contradiction indication existing in the rewritten instruction text can be located.

[0098] Due to the contradiction in the rewritten instruction text, the instruction cannot be executed. In order to eliminate the contradiction in the instruction, the first interaction data can be generated based on the target contradiction data, wherein the first interaction data can be contradiction content clarification, asking the user for the real intention, and the like generated based on the contradiction of the rewritten instruction text.

[0099] For example, the rewritten instruction text is "adjust the temperature to 40 degrees Celsius", the target contradiction data determined by the preset contradiction instruction definition table includes: the contradiction trigger point is "the range exceeds the preset range", and the contradiction guide is "adjust the range to within the preset range". The first interaction data generated based on the target contradiction data can be "the current set temperature exceeds the upper limit of temperature adjustment, please set the temperature to 16-30 degrees Celsius".

[0100] The first interaction data can be output by the electrical appliance, so that the user modifies the instruction based on the first interaction data. It should be noted that the voice instruction fed back by the user for the first interaction data can be processed again through the process of the voice instruction processing method provided by the embodiments of the present specification, so as to obtain the accurate control instruction of the user for the electrical appliance through one or more rounds of dialogue between the user and the electrical appliance, and improve the interaction success rate of the voice instruction.

[0101] The second type

[0102] If the sub-type of the rewritten instruction text is the second sub-type, the instruction processing strategy corresponding to the rewritten instruction text can be implemented through the following steps: based on the second instruction intent of the rewritten instruction text, a preset parameter missing instruction definition table corresponding to the second instruction intent is determined, the preset parameter missing instruction definition table includes at least one preset sentence pattern structure under the second instruction intent, and a combination of missing word slots corresponding to each preset sentence pattern structure; the rewritten instruction text is matched with the preset parameter missing instruction definition table, and a target word slot missing in a target preset sentence pattern structure corresponding to the rewritten instruction text is determined; based on the target word slot, second interaction data for describing that the target word slot is missing in the target preset sentence pattern structure is output to the user, and the second interaction data is used to perform at least one round of dialogue with the user to complete the target word slot.

[0103] In the embodiments of the present specification, for each instruction intent, a preset parameter missing instruction definition table corresponding to the instruction intent can be constructed in advance, and the preset parameter missing instruction definition table can include at least one preset sentence pattern structure under the instruction intent, a combination of missing word slots corresponding to each preset sentence pattern structure, and can also include the intent name of the instruction intent, the word slots included under the instruction intent, etc.

[0104] Still taking the temperature regulation instruction intention as an example, the preset sentence structure under the instruction intention can include A: function name + operation + range + unit, B: range + unit. For each preset sentence structure, there is a corresponding combination of missing word slots, for example, for sentence structure A, the combination of missing word slots can include ①: missing function name, ②: missing operation, ③: missing range, ④: missing unit, ⑤: missing function name and operation, ⑥: missing range and unit… It should be understood that the combination of missing word slots can be set according to actual needs, which will not be exemplified here.

[0105] In the embodiments of the present specification, for the rewriting instruction text of the second sub-type, the second instruction intention of the rewriting instruction text can be obtained in the process of processing the target voice instruction by the preset large language model, or can be obtained by re-identifying the intention of the rewriting instruction text, which is not limited here. After obtaining the second instruction intention, the preset parameter missing instruction definition table corresponding to the second instruction intention is determined, and the rewriting instruction text is matched with the preset parameter missing instruction definition table, so as to determine the target preset sentence structure corresponding to the rewriting instruction text and the missing target word slot. For example, the rewriting instruction text is “adjust the temperature”, the corresponding target preset sentence structure is the above sentence structure A, and the missing target word slot is range and unit.

[0106] Since part of the parameters are missing in the rewriting instruction text, the electric appliance cannot directly execute it. In order to fill in the missing parameters, the second interaction data can be generated based on the missing target word slot, wherein the second interaction data can be to ask the user for the specific value of the missing target word slot. For example, the rewriting instruction text is “adjust the temperature”, and the generated second interaction data can be “how many degrees Celsius do you need to adjust the temperature to?”.

[0107] The second interaction data can be output by the electric appliance, so that the user supplements the missing parameters based on the second interaction data. It should be noted that the voice instruction feedback by the user to the second interaction data can be processed again through the process of the voice instruction processing method provided by the embodiments of the present specification, so as to realize the accurate control instruction of the electric appliance by the user through one or more rounds of dialogue between the user and the electric appliance, and improve the interaction success rate of the voice instruction.

[0108] The third type

[0109] If the sub-type of the rewriting instruction text is the third sub-type, the instruction processing strategy corresponding to the rewriting instruction text can be realized by the following steps: outputting default interaction data to the user to prompt that the target voice instruction cannot be executed.

[0110] In the embodiments of the present specification, if the rewriting instruction text is of the third sub-type, it indicates that the instruction cannot be processed, and a bottom-up process can be entered, that is, default interaction data is output. The default interaction data can be pre-set, for example, "instruction cannot be executed", "please repeat again as I did not hear clearly" and the like.

[0111] For better understanding of the voice instruction processing method provided by the embodiments of the present specification, please refer to Figure 2 FIG. 1 is a schematic diagram of a voice instruction processing flow provided by the embodiments of the present specification, and specifically includes the following steps:

[0112] Step S201: The electrical appliance device receives a voice instruction of a user;

[0113] Step S202: The voice instruction is processed through a JAVA NLU flow;

[0114] When the processing result of the voice instruction indicates that the voice instruction is a complete and clear instruction, step S203 is executed, and when the processing result indicates that the voice instruction is an incomplete and clear instruction, step S204 is executed:

[0115] Step S203: The electrical appliance device is controlled to execute the instruction obtained after processing;

[0116] Step S204: A preset large language model is called to filter interference information and infer the intention of the voice instruction, and the processed voice instruction and the confidence are output;

[0117] Step S205: It is judged whether the confidence is greater than or equal to a threshold value;

[0118] If yes, step S203 is executed; if no, step S206 is executed;

[0119] Step S206: The processed voice instruction is classified;

[0120] If the classification result indicates that the instruction is a contradictory instruction, step S207 is executed; if the classification result indicates that the instruction is a parameter missing instruction, step S209 is executed; if the classification result indicates that the instruction is other instruction, step S211 is executed;

[0121] Step S207: Contradictory data is determined;

[0122] Step S208: First interaction data is generated based on the contradictory data, and a multi-round dialogue flow is entered;

[0123] Step S209: The missing target word slot is determined;

[0124] Step S210: Second interaction data is generated based on the missing target word slot, and a multi-round dialogue flow is entered;

[0125] Step S211: enter the bottom-up process.

[0126] To sum up, the scheme in the embodiments of the present specification can perform de-interference processing and intent reasoning on a target voice instruction through a preset large language model, output a rewritten instruction text, and execute different instruction processing decisions according to the confidence of the rewritten instruction text. Among them, the rewritten instruction text with high confidence is an instruction with complete structure and clear and explicit intent, which can directly enter the execution process. The rewritten instruction text with low confidence is classified to determine the sub-type of the rewritten instruction text. For contradictory instructions and parameter missing instructions, multi-round dialogue clarification is initiated to the user through the generation of interaction data to confirm the instruction intent issued by the user, so as to modify and / or complete the instruction, improve the success rate of interaction, and improve the user experience.

[0127] Based on the same inventive concept, the embodiments of the present application provide a voice instruction processing device, as shown in Figure 3 The device comprises:

[0128] The acquisition module 301 is configured to acquire a target voice instruction.

[0129] The first processing module 302 is configured to process an instruction text of the target voice instruction based on a preset large language model, to obtain a rewritten instruction text, wherein the preset large language model is configured to perform intent reasoning on the instruction text to rewrite the instruction text.

[0130] The second processing module 303 is configured to determine a target type of the rewritten instruction text, and determine an instruction processing strategy corresponding to the rewritten instruction text based on the target type.

[0131] In some embodiments, the acquisition module 301 is configured to:

[0132] receive a voice instruction to be processed of a user;

[0133] perform voice recognition on the voice instruction to be processed to obtain a recognition result;

[0134] If the recognition result is a recognition failure, the voice instruction to be processed is taken as the target voice instruction.

[0135] In some embodiments, the device further comprises:

[0136] The prompt engineering data acquisition module is configured to acquire prompt engineering data corresponding to a voice instruction rewriting task.

[0137] The third processing module is configured to input the prompt engineering data into the preset large language model, so that the preset large language model performs at least interference information filtering and intent reasoning on the input voice instruction based on the prompt engineering data, and obtains a rewritten instruction text with complete sentence structure.

[0138] In some embodiments, the second processing module 303 is configured to:

[0139] determine a target confidence degree corresponding to the rewritten instruction text output by the preset large language model;

[0140] if the target confidence degree is greater than or equal to a preset confidence threshold, determine that the target type is a first type, and the first type indicates that the rewritten instruction text can be directly executed;

[0141] if the target confidence degree is less than the preset execution threshold, determine that the target type is a second type, and the second type indicates that the rewritten instruction text cannot be directly executed.

[0142] In some embodiments, the second processing module 303 is configured to:

[0143] if the target type is the second type, determine a sub-type of the rewritten instruction text based on a preset instruction word table, the preset instruction word table including a preset sentence structure corresponding to an instruction intent, necessary word slots corresponding to the preset sentence structure, and word slots of fixed collocations of the instruction intent;

[0144] determine an instruction processing strategy corresponding to the rewritten instruction text based on the sub-type;

[0145] wherein the sub-type includes a first sub-type for representing a contradictory instruction, a second sub-type for representing a parameter missing instruction, and a third sub-type other than the first sub-type and the second sub-type.

[0146] In some embodiments, the second processing module 303 is configured to:

[0147] if the sub-type is the first sub-type, determine a preset contradictory instruction definition table corresponding to a first instruction intent of the rewritten instruction text based on the first instruction intent, the preset contradictory instruction definition table including at least one piece of contradictory data corresponding to the first instruction intent;

[0148] match the rewritten instruction text with the preset contradictory instruction definition table, and determine target contradictory data corresponding to the rewritten instruction text from the at least one piece of contradictory data;

[0149] Based on the target contradictory data, first interaction data for describing the target contradictory data is output to the user, and the first interaction data is used to perform at least one round of dialogue with the user to correct the target contradictory data.

[0150] In some embodiments, the second processing module 303 is configured to:

[0151] If the sub-type is the second sub-type, based on a second instruction intention of the rewritten instruction text, a preset parameter missing instruction definition table corresponding to the second instruction intention is determined, the preset parameter missing instruction definition table including at least one preset sentence pattern structure under the second instruction intention and a combination of missing word slots corresponding to each preset sentence pattern structure.

[0152] The rewritten instruction text is matched with the preset parameter missing instruction definition table to determine a target word slot missing in a target preset sentence pattern structure corresponding to the rewritten instruction text.

[0153] Based on the target word slot, second interaction data for describing that the target word slot is missing in the target preset sentence pattern structure is output to the user, and the second interaction data is used to perform at least one round of dialogue with the user to complete the target word slot.

[0154] In some embodiments, the second processing module 303 is configured to:

[0155] If the sub-type is the third sub-type, default interaction data is output to the user to prompt that the target voice instruction cannot be executed.

[0156] Regarding the above device, the specific functions of each module have been described in detail in the embodiments of the voice instruction processing method provided by the embodiments of the present disclosure, and will not be described in detail here.

[0157] Based on the same inventive concept, the embodiments of the present disclosure provide an electric appliance device, referring to Figure 4 As shown in the figure, the electric appliance device includes a memory 404, a processor 402, and a computer program stored in the memory 404 and executable on the processor 402, and the processor 402 implements any one of the embodiments of the voice instruction processing method when executing the program.

[0158] In the embodiments of the voice instruction processing method, Figure 4In particular embodiments, bus architecture (represented generally by the bus 400) can include any number of interconnecting buses and bridges, the bus 400 linking together various circuits such as one or more processors represented by the processor 402, and memory represented by the memory 404. The bus 400 can also link together various other circuits which can include, among other things, peripheral devices, voltage stabilizers and power management circuits, all of which are well known in the art, and therefore, not further described herein. The bus interface 405 provides an interface between the bus 400 and the receiver 401 and the transmitter 403. The receiver 401 and the transmitter 403 can be the same component, i.e., a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 402 is responsible for managing the bus 400 and general processing, while the memory 404 can be used for storing data used by the processor 402 in executing operational processes.

[0159] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored on or transferred over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions can also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as technology evolves, the "functionalities" described herein can be implemented by various combinations of digital and analog circuits.

[0160] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other manners. The described device embodiments are merely illustrative, and the division of units can be different from the above. For example, the units can be combined or integrated into another system, or some features can be ignored or not implemented. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, and can be in electrical or other forms.

[0161] The units described as separate components can or can not be physically separate, and the components of the control device can or can not be physical units, i.e., can be located in one place or distributed on multiple units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0162] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0163] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for processing voice commands, characterized in that: include: Get the target voice command; Processing the instruction text of the target voice instruction based on a preset large language model to obtain a rewritten instruction text, wherein the preset large language model is used to perform intent inference on the instruction text to rewrite the instruction text; A target type of the rewriting instruction text is determined, and based on the target type, an instruction processing strategy corresponding to the rewriting instruction text is determined.

2. The method according to claim 1, wherein The acquiring of the target voice instruction includes: Receive the user's voice command to be processed; Performing voice recognition on the voice instruction to be processed to obtain a recognition result; If the recognition result is recognition failure, the to-be-processed voice instruction is used as the target voice instruction.

3. The method according to claim 1, wherein Before processing the instruction text of the target voice instruction based on the preset large language model, the method further includes: Obtaining prompt engineering data corresponding to the voice command rewriting task; The prompt engineering data is input into the preset large language model, so that the preset large language model performs at least interference information filtering and intention reasoning on the input voice instruction based on the prompt engineering data to obtain a rewritten instruction text with a complete sentence structure.

4. The method according to claim 1, wherein Determining the target type of the rewriting instruction text includes: Determining a target confidence level corresponding to the rewriting instruction text output by the preset large language model; If the target confidence is greater than or equal to a preset confidence threshold, determining that the target type is a first type, the first type indicating that the rewriting instruction text can be directly executed; If the target confidence is less than the preset execution threshold, the target type is determined to be a second type, and the second type indicates that the rewritten instruction text cannot be directly executed.

5. The method according to claim 4, wherein The determining, based on the target type, an instruction processing strategy corresponding to the rewriting instruction text includes: If the target type is the second type, determining the subtype of the rewriting instruction text based on a preset instruction vocabulary, the preset instruction vocabulary including a preset sentence structure corresponding to the instruction intent, a necessary word slot corresponding to the preset sentence structure, and a word slot fixedly collocated with the instruction intent; Determining, based on the subtype, an instruction processing strategy corresponding to the rewriting instruction text; The subtypes include a first subtype for characterizing contradictory instructions, a second subtype for characterizing parameter-missing instructions, and a third subtype other than the first subtype and the second subtype.

6. The method according to claim 5, wherein The determining, based on the subtype, an instruction processing strategy corresponding to the rewriting instruction text includes: If the subtype is the first subtype, determining a preset conflicting instruction definition table corresponding to the first instruction intent based on the first instruction intent of the rewriting instruction text, wherein the preset conflicting instruction definition table includes at least one conflicting data corresponding to the first instruction intent; Matching the rewriting instruction text with the preset contradiction instruction definition table, and determining target contradiction data corresponding to the rewriting instruction text from the at least one contradiction data; Based on the target contradiction data, first interaction data for describing the target contradiction data is output to the user, where the first interaction data is used to perform at least one round of dialogue with the user to correct the target contradiction data.

7. The method according to claim 5, wherein The determining, based on the subtype, an instruction processing strategy corresponding to the rewriting instruction text includes: If the subtype is the second subtype, determining a preset parameter missing instruction definition table corresponding to the second instruction intent based on the second instruction intent of the rewriting instruction text, wherein the preset parameter missing instruction definition table includes at least one preset sentence structure under the second instruction intent and a combination of missing word slots corresponding to each preset sentence structure; Matching the rewriting instruction text with the preset parameter missing instruction definition table to determine the target word slot missing in the target preset sentence structure corresponding to the rewriting instruction text; Based on the target word slot, second interaction data is output to the user for describing the lack of the target word slot in the target preset sentence structure, and the second interaction data is used to perform at least one round of dialogue with the user to complete the target word slot.

8. The method according to claim 5, wherein The determining, based on the subtype, an instruction processing strategy corresponding to the rewriting instruction text includes: If the subtype is the third subtype, default interaction data is output to the user to prompt the user that the target voice command cannot be executed.

9. A voice command processing device, characterized in that: include: An acquisition module is used to acquire target voice commands; a first processing module, configured to process the instruction text of the target voice instruction based on a preset large language model to obtain a rewritten instruction text, wherein the preset large language model is used to perform intent inference on the instruction text to rewrite the instruction text; The second processing module is used to determine a target type of the rewriting instruction text, and based on the target type, determine an instruction processing strategy corresponding to the rewriting instruction text.

10. An electrical device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images