Intelligent voice interaction method, electronic device and system

Abstract

The application discloses an intelligent voice interaction method, comprising: inputting user voice into an automatic speech recognition (ASR) model to extract voice text; determining whether a custom hotword exists in the voice text; in the case where the custom hotword exists, determining whether to replace the custom hotword with a domain instruction based on a comparison model; if the custom hotword is replaced with the domain instruction, outputting the replaced domain instruction; if the custom hotword is not replaced with the domain instruction, performing domain classification on the voice text by using a domain model, outputting a simple instruction if the voice text is determined to be a simple instruction, and inputting a difficult instruction recognized from the voice text into a large language model and the domain model, performing priority sorting on domain classification matched by the domain model, and outputting a plurality of domain instructions with high priority based on the large language model.

Description

Technical Field

[0001] This application relates to the field of intelligent voice processing technology, and in particular to an intelligent voice interaction method, electronic device and system. Background Technology

[0002] Voice interaction is frequently used in applications such as smart speakers, smart screens, robots, smart cameras, and smart home products. For example, after a user wakes up a smart screen with their voice, the device starts recording audio. When a user says "turn on the air conditioner" to a smart home product, the device executes the command to start the air conditioner. However, edge-domain IoT devices often experience recognition errors in understanding homophones, polysemous words, and the intent of commands with multiple layers of meaning, leading to incorrect responses to voice commands. Summary of the Invention

[0003] This application provides an intelligent voice interaction method to solve the problem that IoT devices in the edge domain may have errors in recognizing the intent of voice, resulting in an inability to respond correctly to voice commands.

[0004] In a first aspect, embodiments of this application provide an intelligent voice interaction method, including: The user's voice input is automatically recognized using an ASR (Automatic Speech Recognition) model to extract the speech text. Determine whether custom hot words exist in the spoken text: In the presence of custom hot words, a comparison model is used to determine whether to replace the custom hot words with domain instructions; If the custom hot word is replaced with a domain instruction, the replaced domain instruction is output. If the custom hot words are not replaced with domain instructions, then the domain model is used to classify the speech text into a domain. If the speech text is determined to be a simple instruction, then the simple instruction is output. If the speech text is determined to be a difficult instruction, the difficult instructions identified from the speech text are input into the large language model and the domain model. The domain classifications matched by the domain model are prioritized, and multiple domain instructions with higher priority are selected and output based on the large language model.

[0005] Optionally, determining whether to replace the custom hot words with domain instructions based on a comparison model includes: The comparison models include the Transformer model, the Multilayer Perceptron (MLP), and a statistical module. The speech text is converted into text features based on the Transformer model. Based on the MLP, the text features are input to obtain the confidence score of the instruction label that can be replaced with a domain instruction; Based on the statistical module, the confidence level of the instruction tags corresponding to different domain instructions is calculated. When the confidence level of the instruction tag of the replaced domain instruction exceeds the replacement accuracy threshold, it is determined that the domain instruction should be used to replace the custom hot word. If the confidence level of the instruction label of the replaced domain instruction is lower than the replacement accuracy threshold, it is determined that the domain instruction will not be used to replace the custom hot word; When the confidence level of the instruction label of the replaced domain instruction is equal to the replacement accuracy threshold, it is determined to be an uncertain case. Based on the domain model, the confidence level of the instruction label of whether it can be replaced by the domain instruction is calculated in parallel: the confidence level of the first instruction label of replacing the custom hot word with the domain instruction and the confidence level of the second instruction label of not replacing the custom hot word with the domain instruction are calculated, and the higher of the two is used to determine whether to replace the custom hot word with the domain instruction.

[0006] Optionally, prioritizing the domain classifications matched by the domain model includes: The vector representation of the spoken text is obtained based on the Tokenizer; The vector representation is input into the domain model to perform domain classification, wherein the domain model includes a BERT model, which is trained using a cross-entropy loss function based on multiple labels including hard commands, chat commands, media commands, furniture commands, and alarm clock commands. The domain model outputs the matched domain classification and the corresponding skill matching confidence score; Based on the skill matching confidence level, the priority of the corresponding domain categories is sorted.

[0007] Optionally, based on the large language model, select several domain instructions with higher priority for output, including: The large language model obtains the skill matching confidence between the difficult instruction and each domain category based on the prompt word, removes domains with low skill matching confidence, and generates corresponding domain instructions based on the remaining multiple domain categories, which are then output as the top-priority domain instructions; or The skill matching confidence scores of the domain model are sorted from high to low, and multiple domain instructions are filtered out by the jump filter principle.

[0008] Optionally, if the skill matching confidence of the domain model output is sorted from high to low according to skill matching confidence, and only one domain instruction is selected from the filtered output based on the jump filtering principle, then the domain instruction with higher skill matching confidence among the filtered domain instructions will be used as the expanded output domain instruction.

[0009] Optionally, the replacement accuracy threshold includes: an upper threshold and a lower threshold. The confidence level of the instruction label of the replaced domain instruction exceeds the replacement accuracy threshold, including: the confidence level of the instruction label of the replaced domain instruction exceeds the upper threshold; The replacement domain instruction's instruction label confidence level is lower than the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is lower than the lower threshold. The confidence score of the instruction label of the replaced domain instruction is equal to the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is equal to any value between the upper threshold and the lower threshold.

[0010] Optionally, based on the large language model, select several domain instructions with higher priority for output, including: Select the domain skills to be executed first based on the business logic of IoT products.

[0011] Secondly, embodiments of this application provide an electronic device, including: Processor and memory; The memory stores computer-readable instructions, which, when executed by the processor, implement the above method.

[0012] Thirdly, embodiments of this application provide an intelligent interactive system, including: Voice interaction devices are used to acquire user voice. A processor is used to execute computer-readable instructions to implement the methods described above.

[0013] Optionally, the system also includes a display screen: The display screen is used to display the execution result or semantic recognition result of the user command.

[0014] In this embodiment, after extracting speech text based on the ASR model, it is determined whether there are custom hot words. If so, a comparison model is used to determine whether to replace the custom hot words with domain instructions to obtain user instructions that the IoT device can recognize. For difficult instructions, the corresponding domain category is first matched based on the domain model, and then the domain instruction corresponding to the higher priority domain category is selected through the large language model and output to the IoT device. This solves the semantic understanding bias that occurs when the IoT device directly performs semantic recognition based on the received speech, which leads to errors in understanding the user's speech intent. For custom hot words, it attempts to replace them with domain instructions that the IoT device can recognize. For difficult instructions, it combines the capabilities of the domain model and the large language model to perform in-depth intent analysis, thereby improving the accuracy of user instruction processing. Attached Figure Description

[0015] Figure 1 A schematic diagram illustrating a method for intelligent voice interaction is provided for embodiments of this application; Figure 2 This is a schematic diagram of another intelligent voice interaction method provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the training process of the domain model provided in this application embodiment; Figure 4 This is a schematic diagram of large language model inference provided in an embodiment of this application; Figure 5 The intelligent interactive system provided in the embodiments of this application. Detailed Implementation

[0016] The present application will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application. Any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the protection scope of the present application.

[0017] If domain segmentation is used directly to process user-inputted speech, it may confuse user-defined hot words with instructions corresponding to homophones, and may also lead to semantic comprehension errors.

[0018] For example, if a user customizes the robot vacuum cleaner as a "spotlight" and says "turn on the spotlight," the smart home system may not be able to understand whether the user wants to turn on the robot vacuum cleaner or the light, which presents a problem of user semantic recognition difficulty.

[0019] For example, if a user customizes the projector to "movie," when the user says "turn on the movie," the smart home system may not be able to determine whether they want to turn on the projector or turn on the movie on the TV. In this case, there is a difficulty in understanding the user's intention.

[0020] To address the aforementioned problems, embodiments of this application provide an intelligent voice interaction method, including: S101: Automatic Speech Recognition (ASR) model for user voice input, extracting speech text; S102: Determine whether there are custom hot words in the voice text: S103: In the case of custom hot words, determine whether to replace the custom hot words with domain instructions based on the comparison model; S104: If the custom hot word is replaced with a domain instruction, the replaced domain instruction is output; S105: If the custom hot words are not replaced using domain instructions, then the speech text is classified using a domain model. S106: If the voice text is determined to be a simple instruction, then the simple instruction is output; S107: If the speech text is determined to be a difficult instruction, the difficult instructions identified from the speech text are input into the large language model and the domain model. The domain classifications matched by the domain model are prioritized and the multiple domain instructions with higher priority are selected and output based on the large language model.

[0021] Custom hot words refer to frequently used words defined by the user.

[0022] In S101, the ASR model may include, but is not limited to, any of the following algorithms: Whisper ASR, DeepSpeech, Kaldi, Wav2vec, SpeechBrain, FunAudio-ASR and Vosk, etc.

[0023] The comparison model in this embodiment may include a Transformer model, a Multilayer Perceptron (MLP), and a statistical module. The Transformer model converts the speech text into text features. The MLP inputs the text features to obtain the confidence level of whether they can be replaced with domain instructions. The statistical module replaces the custom hot words with domain instructions when the confidence level of the replaced domain instructions exceeds the replacement accuracy threshold. When the confidence level of the replaced domain instructions is lower than the accuracy threshold, the replaced domain instructions are not executed, and the process jumps directly to the step of performing domain classification on the speech text using a domain model.

[0024] See Figure 2 As shown in the figure, this application provides an intelligent voice interaction method, including: S201: Automatically recognizes user voice input using an ASR model to extract speech text; S202: Determine whether there are custom hot words in the spoken text: S203: In the case of custom hot words, determine whether to replace the custom hot words with domain instructions based on the comparison model; S204: If the custom hot word is replaced with a domain instruction, the replaced domain instruction is output; S205: If the custom hot words are not replaced using domain instructions, then the speech text is classified using a domain model. S206: If it cannot be determined whether the custom hot word can be replaced by a domain instruction, then the confidence score of the instruction label that can be replaced by a domain instruction is calculated in parallel based on the domain model.

[0025] When the confidence level of the instruction label of the replaced domain instruction exceeds the replacement accuracy threshold, it is determined that the domain instruction should be used to replace the custom hot word. If the confidence level of the instruction label of the replaced domain instruction is lower than the replacement accuracy threshold, it is determined that the domain instruction will not be used to replace the custom hot word.

[0026] When the confidence of the replaced domain instruction is equal to the replacement accuracy threshold, it is determined to be an uncertain case. Based on the domain model, the confidence of the instruction label that can be replaced by the domain instruction is calculated in parallel: the confidence of the first instruction label of replacing the custom hot word with the domain instruction and the confidence of the second instruction label of not replacing the custom hot word with the domain instruction are calculated, and the higher of the two is used to determine whether to replace the custom hot word with the domain instruction.

[0027] The contrast model can be trained based on voice text and user commands to replace custom hot words.

[0028] For example, a training set can include the following: Original audio text: "Open abc"; The replaced domain instruction is "Turn on the air conditioner", where abc is a user-defined hot word for the air conditioner.

[0029] The original voice text "How's the weather today?" was replaced with the domain command "How's the weather today, Mom?", a custom hot word set by the user for their mother.

[0030] For example, the comparison model could be a binary classification model. See also Figure 2 As shown, during the training process of the comparison model: Input sample data: "Open abc" and "Open the air conditioner" Output whether the result can be replaced: 1 (the representation can be replaced using domain instructions). Input sample data: "How's the weather today?" and "How's the weather today?"; Output whether the result can be replaced: 0 (the representation cannot be replaced by the domain instruction).

[0031] During the training process of the model, if the custom hot words correspond to multiple domain instructions, the replacement results can be iterated through to determine whether they can be replaced.

[0032] For example, in the case where there are two domain instructions corresponding to "movie": 1. Movie playback software within multimedia software; 2. Projector.

[0033] By comparing the models, two domain instructions can be used to replace the custom hot word "movie" respectively, and the confidence of the instruction label of the replaced domain instruction can be used to determine whether the replacement is possible.

[0034] The aforementioned instruction label confidence can be obtained by extracting a vector representation from the speech text using Transformer, inputting this vector representation into an MLP to obtain the corresponding instruction label confidence, and then comparing the instruction label confidence with the replacement accuracy threshold using a statistics module.

[0035] For example, the replacement accuracy threshold mentioned above could be 95%.

[0036] Optionally, the replacement accuracy thresholds include: an upper threshold and a lower threshold. The confidence level of the instruction label of the replaced domain instruction exceeds the replacement accuracy threshold, including: the confidence level of the instruction label of the replaced domain instruction exceeds the upper threshold; The replacement domain instruction's instruction label confidence level is lower than the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is lower than the lower threshold. The confidence score of the instruction label of the replaced domain instruction is equal to the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is equal to any value between the upper threshold and the lower threshold.

[0037] For example, the aforementioned replacement accuracy threshold may include an upper threshold and a lower threshold. Replacement is performed when the confidence of the instruction label is higher than the upper threshold, and no replacement is performed when the confidence of the instruction label is lower than the lower threshold. When the confidence of the instruction label is between the upper and lower thresholds, it is determined that the confidence of the instruction label is equal to the replacement accuracy threshold. It is uncertain whether the domain instruction can be used to replace the custom hot word. At this time, the process jumps to the step of performing domain classification on the speech text using a domain model and the step of replacing the custom hot word with a domain instruction. These two steps are executed in parallel. The first instruction label confidence of replacing the custom hot word with a domain instruction and the second instruction label confidence of not replacing the custom hot word with a domain instruction are calculated, and the higher of the two is used as the judgment result.

[0038] For example, the aforementioned statistics module can be a hard-coded statistics function module used to calculate the replacement accuracy threshold corresponding to the confidence level of the instruction label. For instance, the upper threshold could be 0.7, indicating that the statistics module calculates that when the confidence level of the instruction label is greater than 0.7, the replacement accuracy is higher than 95%, and the lower threshold could be 0.3, indicating that when the confidence level of the instruction label is less than 0.3, the non-replacement accuracy is higher than 95%. The statistics module determined that when the confidence level of the instruction label is between 0.3 and 0.7, it is uncertain whether it can be replaced.

[0039] In this embodiment of the application, the domain model can adopt the BERT model, see [link to relevant documentation]. Figure 3 As shown, the domain model can be trained based on various instruction labels. These labels can include media instructions, furniture instructions, weather instructions, alarm clock instructions, casual conversation instructions, and difficult instructions. Difficult instructions can represent complex instructions containing multiple sub-instructions.

[0040] After the user inputs a query, the Tokenizer processes the query to obtain the corresponding vector representation. This vector representation is then input into the domain model (BERT), which outputs multiple instruction labels. The cross-entropy loss function is then used to train the domain model based on the prediction loss.

[0041] The probability distribution of multiple instruction labels predicted by the domain model is compared with the real instruction labels labeled in the training set, and the cross-entropy loss is calculated.

[0042] For a vector representation of a single query, the cross-entropy loss formula is:

[0043] Where y i This represents the confidence level of the true instruction labels labeled in the training set. It is the confidence score of the instruction label output by the domain model.

[0044] See Figure 3 As shown, the confidence level of the instruction label for the media instruction is 0, the confidence level of the instruction label for the home instruction is 0.5, the confidence level of the instruction label for the weather instruction is 0.5, the confidence level of the instruction label for the difficulty instruction is 1.0, the confidence level of the instruction label for the alarm clock instruction is 0, and the confidence level of the instruction label for the chat instruction is 0.

[0045] Based on the LOSS corresponding to different instruction labels, the parameters of the domain model can be optimized, thereby improving the domain model's accuracy in classifying speech and text.

[0046] See Figure 4 As shown, the large language model obtains the skill matching confidence of the difficult instructions with each domain based on the prompt words, thereby obtaining the skill matching confidence of the current speech text with the corresponding domain, and realizing the output of multiple domain instructions with higher priority: The large language model obtains the skill matching confidence between the difficult instruction and each domain category based on the prompt word, and removes domains with low skill matching confidence. Based on the remaining multiple domain classifications, corresponding domain instructions are generated, and these are the domain instructions with the highest priority output.

[0047] See Figure 4 As shown, the skill matching confidence scores for media instructions are 0.2, home instructions are 0.15, weather instructions are 0.01, difficult instructions are 0.6, alarm clock instructions are 0.04, and casual conversation instructions are 0.00. Since the skill matching confidence score for difficult instructions is greater than 0.5, the large language model is triggered to prioritize domain classification.

[0048] For example, large language models can be prioritized in, but not limited to, the following manner: 1) First priority sorting method: Sort the domain model outputs from high to low based on skill matching confidence, and then filter the multiple domain instructions output using the jump filter principle. For example, the jump filtering principle is to obtain the skill matching confidence based on jumps less than 0.2, and to filter when the jump is greater than 0.2.

[0049] Assuming the sorted skill matching confidence sequence is [0.5, 0.35, 0.1, 0.1, 0.1], since the difference between 0.5 and 0.35 is 0.15, which is less than 0.2, the domain classification corresponding to 0.35 is retained. Since the difference between 0.35 and 0.1 is 0.25, which is greater than 0.2, 0.1 is removed, and subsequent judgments are terminated.

[0050] 2) Second priority sorting method: When there is only 1 domain instruction filtered according to the first priority sorting method, the number of domain categories will be automatically expanded to 2.

[0051] If the number of remaining categories after filtering is less than 2, it will be automatically expanded to 2. For example, [0.5, 0.2, 0.2, 0.2, 0.2], according to the filtering rules, only 0.5 can be kept. By automatically expanding to 2, 0.2 can also be kept.

[0052] 3) Third priority sorting method: After retaining the two domain instructions according to the second priority sorting method, the large language model is used, combined with the prompt, to determine whether to output one or two domain instructions. For example, for difficult instructions containing multiple semantics, two domain instructions are output.

[0053] 4) Fourth priority ranking method: Select multiple domain instructions with high priority for the large language model, and select the domain skills to be executed first based on the business logic of the IoT product: For example, for robot products that primarily play music, media-related skills should be prioritized. For products that primarily control customized home environments, home-related skills should be prioritized.

[0054] When the identified domain classification includes multiple domain classifications, the large model selects the IoT device (i.e., voice interaction device) that matches the domain classification according to the pre-set business logic, and executes the function corresponding to the domain classification.

[0055] Based on the same inventive concept, embodiments of this application provide an electronic device, including: Processor and memory; The memory stores computer-readable instructions, which, when executed by the processor, implement the above method.

[0056] This application provides an intelligent interactive system, including: Voice interaction devices are used to acquire user voice. A processing device for executing computer-readable instructions to implement the above-described method.

[0057] Optionally, the system also includes a display screen: The display screen is used to display the execution result or semantic recognition result of the user command.

[0058] See Figure 5As shown, the intelligent interactive system in this application embodiment may include a voice interaction device and a processing device. The voice interaction device and the processing device are connected via a communication link. The aforementioned voice interaction device may include, but is not limited to, any one of a smart central control screen, a smart speaker, a smart wearable device, a smart camera, a smart door lock, or a service robot. The processing device may include, but is not limited to, a local host (e.g., a local server), a remote server, or a cloud platform.

[0059] In this embodiment, after extracting speech text based on the ASR model, it is determined whether there are custom hot words. If so, a comparison model is used to determine whether to replace the custom hot words with domain instructions to obtain user instructions that the IoT device can recognize. For difficult instructions, the corresponding domain category is first matched based on the domain model, and then the domain instruction corresponding to the higher priority domain category is selected through the large language model and output to the IoT device. This solves the semantic understanding bias that occurs when the IoT device directly performs semantic recognition based on the received speech, which leads to errors in understanding the user's speech intent. For custom hot words, it attempts to replace them with domain instructions that the IoT device can recognize. For difficult instructions, it combines the capabilities of the domain model and the large language model to perform in-depth intent analysis, thereby improving the accuracy of user instruction processing.

[0060] The memory can be random access memory, read-only memory, non-volatile, programmable ROM, erasable PROM, electrically erasable memory, flash memory, optical memory, and registers, etc. The processor can be a general-purpose processor, which is a processor that performs specific steps and / or operations by reading and executing computer programs stored in the memory. The general-purpose processor may use the memory during the execution of these steps and / or operations. The general-purpose processor can be a central processing unit, ASIC, and FPGA, etc. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor.

[0061] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0062] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0063] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An intelligent voice interaction method, characterized in that, The method includes: The user's voice input is automatically recognized using an ASR (Automatic Speech Recognition) model to extract the speech text. Determine whether custom hot words exist in the spoken text: In the presence of custom hot words, a comparison model is used to determine whether to replace the custom hot words with domain instructions; If the custom hot word is replaced with a domain instruction, the replaced domain instruction is output. If the custom hot words are not replaced with domain instructions, then the domain model is used to classify the speech text into a domain. If the speech text is determined to be a simple instruction, then the simple instruction is output. If the spoken text is determined to be a difficult instruction, then the difficult instructions identified from the spoken text are input into the large language model and the domain model. The domain classifications matched by the domain model are prioritized, and the large language model selects the highest-priority domain instructions for output. The step of determining whether to replace the custom hot words with domain instructions based on a comparison model includes: The comparison models include the Transformer model, the Multilayer Perceptron (MLP), and a statistical module. The speech text is converted into text features based on the Transformer model. Based on the MLP, the text features are input to obtain the confidence score of the instruction label that can be replaced with a domain instruction; Based on the statistical module, the confidence level of the instruction tags corresponding to different domain instructions is calculated. When the confidence level of the instruction tag of the replaced domain instruction exceeds the replacement accuracy threshold, it is determined that the domain instruction should be used to replace the custom hot word. If the confidence level of the instruction label of the replaced domain instruction is lower than the replacement accuracy threshold, it is determined that the domain instruction will not be used to replace the custom hot word; When the confidence of the instruction label of the replaced domain instruction is equal to the replacement accuracy threshold, it is determined to be an uncertain case. Based on the domain model, the confidence of the instruction label of whether it can be replaced by the domain instruction is calculated in parallel: the confidence of the first instruction label of replacing the custom hot word with the domain instruction and the confidence of the second instruction label of not replacing the custom hot word with the domain instruction are calculated, and the higher of the two is used to determine whether to replace the custom hot word with the domain instruction.

2. The method as described in claim 1, characterized in that, The step of prioritizing the domain classifications matched by the domain model includes: The vector representation of the spoken text is obtained based on the Tokenizer; The vector representation is input into the domain model to perform domain classification, wherein the domain model includes a BERT model, which is trained using a cross-entropy loss function based on multiple labels including hard commands, chat commands, media commands, furniture commands, and alarm clock commands. The domain model outputs the matched domain classification and the corresponding skill matching confidence score; Based on the skill matching confidence level, the priority of the corresponding domain categories is sorted.

3. The method as described in claim 2, characterized in that, Based on a large language model, multiple domain instructions with higher priority are selected for output, including: The large language model obtains the skill matching confidence between the difficult instruction and each domain category based on the prompt word, removes domains with low skill matching confidence, and generates corresponding domain instructions based on the remaining multiple domain categories, which are then output as the top-priority domain instructions; or The skill matching confidence scores of the domain model are sorted from high to low, and multiple domain instructions are filtered out by the jump filter principle.

4. The method as described in claim 3, characterized in that, If the skill matching confidence of the domain model is sorted from high to low, and the domain instruction output is only one after filtering by the jump filter principle, then the domain instruction with the higher skill matching confidence among the filtered domain instructions will be used as the expanded output domain instruction.

5. The method as described in claim 1, characterized in that, The replacement accuracy threshold includes: an upper threshold and a lower threshold. The confidence level of the instruction label of the replaced domain instruction exceeds the replacement accuracy threshold, including: the confidence level of the instruction label of the replaced domain instruction exceeds the upper threshold; The replacement domain instruction's instruction label confidence level is lower than the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is lower than the lower threshold. The confidence score of the instruction label of the replaced domain instruction is equal to the replacement accuracy threshold, including: The confidence level of the instruction label of the replaced domain instruction is equal to any value between the upper threshold and the lower threshold.

6. The method as described in claim 1, characterized in that, Based on a large language model, multiple domain instructions with higher priority are selected for output, including: Select the domain skills to be executed first based on the business logic of IoT products.

7. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 6.

8. An intelligent interactive system, characterized in that, include: Voice interaction devices are used to acquire user voice. A processing device for executing computer-readable instructions to implement the method as described in any one of claims 1 to 6.

9. The system as described in claim 8, characterized in that, The system also includes a display screen: The display screen is used to display the execution result of user commands or the semantic recognition result.

Images