Voice instruction processing method and device, electronic equipment and storage medium

Abstract

The invention provides a voice instruction processing method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a user wake-up voice assistant, obtaining a corresponding audio clip based on a voice instruction of a user, converting the audio clip into a first audio text based on a preset first online recognition channel, and storing the first audio text in a server; obtaining an online dialogue management result corresponding to the audio clip based on the first audio text, and sending the first audio text to a generative large model in a preset second online identification channel based on the preset second online identification channel to obtain a large model processing result corresponding to the audio clip; and performing local dialogue management on the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result, determining a to-be-executed target result according to the local dialogue management result based on a preset priority processing strategy, and executing the target result on the voice assistant.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a voice command processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous breakthroughs in deep learning technology, especially the increase in the scale and complexity of neural network models, large generative models have emerged. These models can learn rich language patterns and knowledge representations through large-scale data training.

[0003] At present, with the development of generative large models, generative large language models (LLMs) are increasingly being applied in in-vehicle voice assistants.

[0004] However, there are many conflicts and mutual restrictions between the generative big model and traditional semantics. For example, the results of the generative big model and traditional semantics in recognizing the user's voice commands conflict with each other, and because traditional semantics and big models often have a linear order of processing, the timeliness of the generative big model's responses is generally low, and the execution efficiency and accuracy of the voice assistant are low. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a voice command processing method, device, electronic device and storage medium, which converts an audio clip through a first online recognition channel to obtain a first audio text, obtains an online dialogue management result of the first online recognition channel and a large model processing result of the generative large model of the second online recognition channel respectively, performs local dialogue management on the online dialogue management result and the large model processing result, determines and executes the target result to be executed based on the priority processing strategy, realizes the integration of the generative large model and traditional semantics, solves the conflicts and limitations between the generative large model and traditional semantics, can process the user's voice commands in parallel, improves the timeliness of the generative large model reply, and also improves the execution efficiency and execution accuracy of the voice assistant.

[0006] In a first aspect, an embodiment of the present application provides a method for processing a voice command, the method comprising:

[0007] In response to the user waking up the voice assistant, acquiring a corresponding audio clip based on the user's voice command;

[0008] The audio segment is converted into a first audio text based on a preset first online recognition channel, and an online conversation management result corresponding to the audio segment is obtained based on the first audio text;

[0009] Sending the first audio text to the generative big model in the second online recognition channel based on a preset second online recognition channel to obtain a big model processing result corresponding to the audio segment;

[0010] Performing local dialogue management on the online dialogue management result and the large model processing result to obtain corresponding local dialogue management results;

[0011] Based on a preset priority processing strategy, the target result to be executed is determined according to the local dialogue management result, and the target result is executed on the voice assistant.

[0012] In a possible implementation, after the method acquires the corresponding audio segment based on the user's voice command in response to the user waking up the voice assistant, it further includes:

[0013] Converting the audio segment into a second audio text based on a preset local recognition channel, and obtaining a local semantic understanding result corresponding to the audio segment based on the second audio text;

[0014] The performing local dialogue management on the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result includes:

[0015] Local dialogue management is performed on the local semantic understanding result, the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result.

[0016] In a possible implementation, the determining a target result to be executed based on the local dialog management result based on a preset priority processing strategy, and executing the target result on the voice assistant includes:

[0017] Determining whether the local semantic understanding result is recognized based on the local dialogue management result;

[0018] If the local semantic understanding result is identified, the local semantic understanding result is determined as the target result, the target result is executed on the voice assistant, and the online dialogue management result and the large model processing result are discarded and stopped waiting;

[0019] In a possible implementation, the method further includes:

[0020] If the local semantic understanding result is not identified, determining whether the online dialogue management result is identified based on the local dialogue management result;

[0021] In response to identifying the online dialogue management result based on the local dialogue management result, the online dialogue management result is determined as a target result for execution, the target result is executed on the voice assistant, and the waiting for the large model processing result is discarded and stopped.

[0022] In a possible implementation, the method further includes:

[0023] In response to not identifying the online dialogue management result based on the local dialogue management result, executing the large model processing result on the voice assistant.

[0024] In a possible implementation, in response to the user waking up the voice assistant, acquiring a corresponding audio clip based on the user's voice command;

[0025] In response to the user waking up the voice assistant, detecting a voice command of the user based on a preset voice activity detection algorithm;

[0026] An audio segment corresponding to the voice instruction is obtained based on the detection result.

[0027] In a possible implementation, the first online recognition channel includes an online speech recognition algorithm and an online management algorithm; the online management algorithm includes an online natural language understanding algorithm and an online dialogue management algorithm; the converting the audio segment into a first audio text based on the preset first online recognition channel, and obtaining an online dialogue management result corresponding to the audio segment based on the first audio text, includes:

[0028] Converting the audio segment into a corresponding first audio text based on the online speech recognition algorithm, and sending the first audio text to the online natural language understanding algorithm and the online dialogue management algorithm;

[0029] The first audio text is processed based on the online natural language understanding algorithm and the online dialogue management algorithm to obtain a corresponding online dialogue management result.

[0030] In a possible implementation, the local recognition channel includes a local speech recognition algorithm and a local natural language understanding algorithm; the converting the audio segment into a second audio text based on the preset local recognition channel, and obtaining a local semantic understanding result corresponding to the audio segment based on the second audio text, includes:

[0031] Converting the audio segment into a corresponding second audio text based on the local speech recognition algorithm, and sending the second audio text to the local natural language understanding algorithm;

[0032] The second audio text is processed based on the local natural language understanding algorithm to obtain a corresponding local natural language understanding result.

[0033] In a possible implementation, the sending the first audio text to the generative big model in the second online recognition channel based on the preset second online recognition channel to obtain the big model processing result corresponding to the audio segment includes:

[0034] In response to sending the first audio text to the vehicle terminal, sending the first audio text to the generative large model at the vehicle terminal;

[0035] The first audio text is processed based on the generative big model to obtain a corresponding big model processing result.

[0036] In a second aspect, an embodiment of the present application further provides a voice command processing device, the device comprising:

[0037] A first acquisition module, configured to, in response to a user waking up the voice assistant, acquire a corresponding audio clip based on a voice command of the user;

[0038] A second acquisition module is used to convert the audio segment into a first audio text based on a preset first online recognition channel, and obtain an online conversation management result corresponding to the audio segment based on the first audio text;

[0039] A third acquisition module is used to send the first audio text to the generative big model in the second online recognition channel based on a preset second online recognition channel to obtain a big model processing result corresponding to the audio segment;

[0040] A fourth acquisition module, used to perform local dialogue management on the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result;

[0041] The first determination module is used to determine the target result to be executed according to the local dialogue management result based on a preset priority processing strategy, and execute the target result on the voice assistant.

[0042] In a possible implementation, the device further includes:

[0043] The fifth acquisition module is used to convert the audio segment into a second audio text based on a preset local recognition channel, and obtain a local semantic understanding result corresponding to the audio segment based on the second audio text;

[0044] The fourth acquisition module is specifically used for:

[0045] Local dialogue management is performed on the local semantic understanding result, the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result.

[0046] In a possible implementation manner, the first determining module is specifically configured to:

[0047] Determining whether the local semantic understanding result is recognized based on the local dialogue management result;

[0048] If the local semantic understanding result is identified, the local semantic understanding result is determined as the target result, the target result is executed on the voice assistant, and the online dialogue management result and the large model processing result are discarded and stopped waiting.

[0049] In a possible implementation, the device further includes:

[0050] A first recognition module, configured to determine whether the online dialogue management result is recognized based on the local dialogue management result if the local semantic understanding result is not recognized;

[0051] The second determination module is used to determine the online dialogue management result as the target result for execution in response to identifying the online dialogue management result based on the local dialogue management result, execute the target result on the voice assistant, and abandon and stop waiting for the large model processing result.

[0052] In a possible implementation, the device further includes:

[0053] The second recognition module is used to execute the large model processing result on the voice assistant in response to the online dialogue management result not being recognized based on the local dialogue management result.

[0054] In a possible implementation manner, the first acquisition module is specifically configured to:

[0055] In response to the user waking up the voice assistant, detecting a voice command of the user based on a preset voice activity detection algorithm;

[0056] An audio segment corresponding to the voice instruction is obtained based on the detection result.

[0057] In a possible implementation, the first online recognition channel includes an online speech recognition algorithm and an online management algorithm; the online management algorithm includes an online natural language understanding algorithm and an online dialogue management algorithm; the second acquisition module is specifically used to:

[0058] Converting the audio segment into a corresponding first audio text based on the online speech recognition algorithm, and sending the first audio text to the online natural language understanding algorithm and the online dialogue management algorithm;

[0059] The first audio text is processed based on the online natural language understanding algorithm and the online dialogue management algorithm to obtain a corresponding online dialogue management result.

[0060] In a possible implementation, the local recognition channel includes a local speech recognition algorithm and a local natural language understanding algorithm; the fifth acquisition module is specifically configured to:

[0061] Converting the audio segment into a corresponding second audio text based on the local speech recognition algorithm, and sending the second audio text to the local natural language understanding algorithm;

[0062] The second audio text is processed based on the local natural language understanding algorithm to obtain a corresponding local natural language understanding result.

[0063] In a possible implementation manner, the third acquisition module is further used to:

[0064] In response to sending the first audio text to the vehicle terminal, sending the first audio text to the generative large model at the vehicle terminal;

[0065] The first audio text is processed based on the generative big model to obtain a corresponding big model processing result.

[0066] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the voice command processing method as described in any one of the first aspects.

[0067] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the voice instruction processing method described in any one of the first aspects are executed.

[0068] A voice command processing method, device, electronic device and storage medium provided in an embodiment of the present application respond to a user waking up a voice assistant, obtain a corresponding audio clip based on the user's voice command, convert the audio clip into a first audio text based on a preset first online recognition channel, and obtain an online dialogue management result corresponding to the audio clip based on the first audio text, send the first audio text to a generative large model in the second online recognition channel based on a preset second online recognition channel, obtain the large model processing result corresponding to the audio clip, perform local dialogue management on the online dialogue management result and the large model processing result, obtain the corresponding local dialogue management result, determine the target result to be executed according to the local dialogue management result based on a preset priority processing strategy, and execute the target result on the voice assistant. In the present application, an audio clip is converted into a first audio text through a first online recognition channel, and an online dialogue management result of the first online recognition channel and a large model processing result of the generative large model of the second online recognition channel are obtained respectively, local dialogue management is performed on the online dialogue management result and the large model processing result, and the target result to be executed is determined and executed based on the priority processing strategy, thereby realizing the integration of the generative large model and traditional semantics, resolving the conflicts and limitations between the generative large model and traditional semantics, and being able to process the user's voice commands in parallel, thereby improving the timeliness of the generative large model's responses, and also improving the execution efficiency and accuracy of the voice assistant.

[0069] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 is a flow chart of a method for processing a voice command according to an embodiment of the present application;

[0072] Figure 2 It is a schematic diagram of the overall flow of the voice command processing method;

[0073] Figure 3 is a flow chart of a method for processing a voice command according to another embodiment of the present application;

[0074] Figure 4 is a flow chart of a method for processing a voice command according to another embodiment of the present application;

[0075] Figure 5 is a structural diagram of a voice command processing device according to an embodiment of the present application;

[0076] Figure 6 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0077] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0078] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0079] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0080] Taking into account the continuous breakthroughs in deep learning technology, especially the increase in the scale and complexity of neural network models, generative large models have come into being. These models can learn rich language patterns and knowledge representations through large-scale data training.

[0081] At present, with the development of generative big models, generative big models are increasingly being applied to in-vehicle voice assistants. However, there are many conflicts and mutual restrictions between generative big models and traditional semantics. For example, the results of the generative big models and traditional semantics in recognizing the user's voice commands conflict with each other, and because traditional semantics and big models often have a linear order of processing, the timeliness of the generative big model's responses is generally low, and the execution efficiency and accuracy of the voice assistant are low.

[0082] To address this problem, the present application provides a voice command processing method, which converts an audio clip through a first online recognition channel to obtain a first audio text, obtains an online dialogue management result of the first online recognition channel and a large model processing result of the generative large model of the second online recognition channel, respectively, performs local dialogue management on the online dialogue management result and the large model processing result, determines and executes the target results to be executed based on a priority processing strategy, realizes the integration of the generative large model and traditional semantics, resolves the conflicts and limitations between the generative large model and traditional semantics, can process the user's voice commands in parallel, improves the timeliness of the generative large model's responses, and also improves the execution efficiency and accuracy of the voice assistant.

[0083] Figure 1 FIG. 1 is a flow chart of a method for processing voice commands according to an embodiment of the present application. Figure 1 As shown, the voice command processing method of the embodiment of the present application may specifically include:

[0084] S101. In response to a user waking up a voice assistant, a corresponding audio clip is obtained based on the user's voice command.

[0085] S102: Convert the audio segment into a first audio text based on a preset first online recognition channel, and obtain an online dialogue management result corresponding to the audio segment based on the first audio text.

[0086] S103: Send the first audio text to the generative big model in the second online recognition channel based on the preset second online recognition channel to obtain the big model processing result corresponding to the audio segment.

[0087] S104: Perform local dialogue management on the online dialogue management results and the large model processing results to obtain corresponding local dialogue management results.

[0088] S105. Based on a preset priority processing strategy, determine the target result to be executed according to the local dialogue management result, and execute the target result on the voice assistant.

[0089] In the above-mentioned voice command processing method, the audio clip is converted through the first online recognition channel to obtain the first audio text, and the online dialogue management result of the first online recognition channel and the big model processing result of the generative big model of the second online recognition channel are obtained respectively. Local dialogue management is performed on the online dialogue management result and the big model processing result, and the target result to be executed is determined and executed based on the priority processing strategy, thereby realizing the fusion between the generative big model and the traditional semantics, resolving the conflicts and limitations between the generative big model and the traditional semantics, and being able to process the user's voice commands in parallel, thereby improving the timeliness of the generative big model's response, and also improving the execution efficiency and accuracy of the voice assistant.

[0090] The above exemplary steps of the embodiment of the present application are described below with reference to specific examples:

[0091] S101, in response to a user waking up a voice assistant, obtaining a corresponding audio clip based on the user's voice command.

[0092] In an embodiment of the present application, the voice command is the voice command issued by the user after waking up the voice assistant, for example, "open the car window". The user wakes up the voice assistant to start a voice conversation, for example, calling the name of the voice assistant. After the user wakes up the voice assistant to start a voice conversation, the user issues a voice command, and the corresponding audio clip is obtained based on the user's voice command for subsequent processing.

[0093] As a possible implementation, in response to the user waking up the voice assistant, the user's voice command is detected based on a preset voice activity detection algorithm; and an audio clip corresponding to the voice command is obtained based on the detection result. The voice activity detection algorithm is a preset voice activity detection (VAD) algorithm. Specifically, when the user wakes up the voice assistant and starts a voice conversation, the VAD algorithm is started to detect the user's voice command, thereby obtaining the audio clip corresponding to the voice command, for example, Figure 2 As shown, Figure 2 The audio in represents the user's voice command.

[0094] S102: Convert the audio segment into a first audio text based on a preset first online recognition channel, and obtain an online dialogue management result corresponding to the audio segment based on the first audio text.

[0095] In the embodiment of the present application, the first online recognition channel is a cloud recognition channel that performs recognition processing on the audio clip, for example, Figure 2As shown, cloud ASR (Automatic Speech Recognition)-cloud NLU (Natural Language Understanding)-cloud DM (Dialog Management) represents the first online recognition channel, the first audio text is the text corresponding to the audio segment, and the online dialogue management result is the result of dialogue management in the cloud. The audio segment is converted into the first audio text based on the first online recognition channel, and the online dialogue management result corresponding to the audio segment is obtained based on the first audio text for subsequent processing. For example, Figure 2 As shown, the cloud DM outputs the online dialogue management result. It should be noted that the first online recognition channel includes an online speech recognition algorithm and an online management algorithm; the online management algorithm includes an online natural language understanding algorithm and an online dialogue management algorithm.

[0096] S103: Send the first audio text to the generative big model in the second online recognition channel based on the preset second online recognition channel to obtain the big model processing result corresponding to the audio segment.

[0097] In the embodiment of the present application, the second online recognition channel is a cloud recognition channel that performs recognition processing on the audio clip, for example, Figure 2 As shown, the cloud ASR-vehicle-big model represents the second online recognition channel, the big model is the generative big model, and the big model processing result is the result of the big model processing in the cloud. Based on the second online recognition channel, the second audio text is sent to the generative big model in the second online recognition channel to obtain the big model processing result corresponding to the audio segment for subsequent processing. Figure 2 As shown, the large model outputs the large model processing results.

[0098] S104, performing local dialogue management on the online dialogue management results and the large model processing results to obtain corresponding local dialogue management results.

[0099] In the embodiment of the present application, local dialogue management means performing dialogue management locally, performing local dialogue management on the online dialogue management result obtained in step S102 and the large model processing result obtained in step S103, and obtaining corresponding local dialogue management results for subsequent processing.

[0100] It should be noted that, in response to the user waking up the voice assistant, after obtaining the corresponding audio segment based on the user's voice command, the audio segment is converted into a second audio text based on a preset local recognition channel, and the local semantic understanding result corresponding to the audio segment is obtained based on the second audio text. The local recognition channel is a local recognition channel that performs recognition processing on the audio segment, for example, Figure 2As shown, local ASR-local NLU-local DM represents the local recognition channel; the second audio text is the text into which the audio segment is converted in the local recognition channel, and the local semantic understanding result is the result of the local natural language understanding processing. The local recognition channel includes a local speech recognition algorithm and a local natural language understanding algorithm. Optionally, the audio segment is converted into a corresponding second audio text based on the local speech recognition algorithm and sent to the local natural language understanding algorithm; the second audio text is processed based on the local natural language understanding algorithm to obtain a corresponding local natural language understanding result. For example, Figure 2 As shown, based on the local ASR, the audio segment is converted into a second audio text, and sent to the local NLU for understanding and processing to obtain a local natural language understanding result.

[0101] As a possible implementation, local dialogue management is performed on the local semantic understanding result, the online dialogue management result, and the large model processing result to obtain a corresponding local dialogue management result. Specifically, local dialogue management is performed on the online dialogue management result obtained in step S102, the large model processing result obtained in step S103, and the local semantic understanding result obtained above to obtain a corresponding local dialogue management result for subsequent processing.

[0102] Thus, the integration of traditional semantics and big models is completed.

[0103] S105, based on a preset priority processing strategy, determine the target result to be executed according to the local dialogue management result, and execute the target result on the voice assistant.

[0104] In an embodiment of the present application, the priority processing strategy is a preset strategy for processing the priorities of local semantic understanding results, online dialogue management results, and large model processing results. The target result is the result to be executed determined according to the priority processing strategy. The target result to be executed is determined according to the local dialogue management result based on the priority processing strategy and is executed on the voice assistant.

[0105] As a possible implementation method, it is determined whether the local semantic understanding result is recognized based on the local dialogue management result; if the local semantic understanding result is recognized, the local semantic understanding result is determined as the target result, and the target result is executed on the voice assistant, and the online dialogue management result and the large model processing result are discarded and stopped waiting. Among them, the target result is one of the three results of the online dialogue management result, the large model processing result and the local semantic understanding result that is determined to be executed.

[0106] Specifically, after receiving the local semantic understanding results, online dialogue management results and large model processing results, the local DM can identify these results in sequence to obtain the recognition results, that is, the local dialogue management results. This local dialogue management result represents the success or failure of the local DM to recognize each result. For example, this local dialogue management result can indicate that the local semantic understanding result is recognized. When the local semantic understanding result is recognized, the local semantic understanding result is determined as the target result to be executed on the voice assistant. At this time, the user's voice command has been executed through the local semantic understanding result, and there is no need for the online dialogue management results and the large model processing results. Therefore, the online dialogue management results and the large model processing results are discarded and the waiting for is stopped.

[0107] It should be noted that the "recognition" here can be understood as successful recognition, or in other words, the result itself understands the intention of the user's voice command. For example, when the local semantic understanding result understands the user's intention, it is determined that there is a local semantic understanding result, that is, it can be recognized. When the content of the result cannot be read, for example, the string of the content of the local semantic understanding result cannot be read. Figure 2 If the local DM is successfully read, it means that the local semantic understanding result does not understand the user's intention, that is, there is no local semantic understanding result, that is, the local semantic understanding result is not recognized, and the local semantic understanding result cannot be executed. Figure 2 As shown, the local DM determines whether it exists, that is, identifies the local semantic understanding result. If so, the local semantic understanding result is executed.

[0108] Furthermore, if the local semantic understanding result is not identified, it is determined whether the online dialogue management result is identified based on the local dialogue management result; in response to the online dialogue management result being identified based on the local dialogue management result, the online dialogue management result is determined as the target result for execution, the target result is executed on the voice assistant, and the waiting for the large model processing result is discarded and stopped.

[0109] Specifically, when the local dialogue management result indicates that the local semantic understanding result has not been identified, that is, the local semantic understanding result has not been identified. At this time, it is necessary to continue to confirm the identification of other results, that is, to continue to judge whether the online dialogue management result has been identified, that is, whether the local dialogue management result indicates that the online dialogue management result has been identified. If it is determined that the online dialogue management result has been identified, the online dialogue management result is determined as the target result to be executed on the voice assistant. At this time, the user's voice command has been executed through the online dialogue management result, and there is no need for the large model processing result. Therefore, the large model processing result is discarded and stopped waiting. For example, Figure 2As shown, the local DM determines whether a cloud semantic result, namely, an online dialogue management result, exists, that is, identifies the cloud semantic result. If so, the online dialogue management result is executed.

[0110] Further, in response to failure to identify an online dialogue management result based on the local dialogue management result, the large model processing result is executed on the voice assistant.

[0111] It should be noted that when the local dialogue management result indicates that the online dialogue management result is not recognized, the online dialogue management result is not recognized. At this time, the large model processing result remains, and the large model processing result is directly executed on the voice assistant. Figure 2 As shown, the local DM determines whether it exists, that is, identifies the cloud semantic result, that is, the online dialogue management result. If not, the large model processing result is directly executed.

[0112] Therefore, the priority processing strategy is that the local semantic understanding result is greater than the online dialogue management result, which is greater than the large model processing result. That is, the local semantic result has the highest priority, followed by the online dialogue management result, and finally the large model processing result. By identifying and judging the corresponding target result, the execution efficiency and accuracy of the voice command are improved. Such a priority processing strategy can improve the response speed of the voice assistant.

[0113] The voice command processing method provided in the embodiment of the present application responds to the user waking up the voice assistant, obtains the corresponding audio clip based on the user's voice command, converts the audio clip into a first audio text based on a preset first online recognition channel, and obtains the online dialogue management result corresponding to the audio clip based on the first audio text, sends the first audio text to the generative large model in the second online recognition channel based on the preset second online recognition channel, obtains the large model processing result corresponding to the audio clip, performs local dialogue management on the online dialogue management result and the large model processing result, obtains the corresponding local dialogue management result, determines the target result to be executed according to the local dialogue management result based on the preset priority processing strategy, and executes the target result on the voice assistant. The voice command processing method of the present application converts the audio clip into the first audio text through the first online recognition channel, obtains the online dialogue management result of the first online recognition channel and the large model processing result of the generative large model of the second online recognition channel, performs local dialogue management on the online dialogue management result and the large model processing result, determines and executes the target result to be executed based on the priority processing strategy, realizes the integration of the generative large model and traditional semantics, solves the conflicts and limitations between the generative large model and traditional semantics, can process the user's voice commands in parallel, improves the timeliness of the generative large model's response, and also improves the execution efficiency and accuracy of the voice assistant. Furthermore, if Figure 3As shown, in step S103 in the above embodiment, “converting the audio segment into a first audio text based on a preset first online recognition channel, and obtaining an online conversation management result corresponding to the audio segment based on the first audio text”, specifically may include the following steps:

[0114] S301, converting the audio clip into a corresponding first audio text based on an online speech recognition algorithm, and sending it to an online natural language understanding algorithm and an online dialogue management algorithm.

[0115] In the embodiment of the present application, the first audio text is the audio text converted from the audio clip locally, the audio clip in the above embodiment is converted into the corresponding first audio text according to the online speech recognition algorithm, and the first audio text is sent to the online natural language understanding algorithm and the online dialogue management algorithm. Figure 2 As shown, the audio clip is sent to the online speech recognition algorithm, namely the cloud ASR, the audio clip is converted into a first audio text based on the cloud ASR, and the first audio text is sent to the online natural language understanding algorithm (namely the cloud NLU) and the online dialogue management algorithm (namely the cloud DM) at the same time.

[0116] S302: Process the first audio text based on an online natural language understanding algorithm and an online dialogue management algorithm to obtain a corresponding online dialogue management result.

[0117] In the embodiment of the present application, the first audio text in step S301 is processed based on the online natural language understanding algorithm and the online dialogue management algorithm to obtain the corresponding online dialogue management result for subsequent processing. Figure 2 As shown, after the first audio text is sent to the cloud NLU and the cloud DM at the same time, the first audio text is processed based on the cloud NLU and the cloud DM to obtain an online dialogue management result.

[0118] Further, such as Figure 4 As shown, step S103 in the above embodiment, “sending the first audio text to the generative large model in the second online recognition channel based on the preset second online recognition channel to obtain the large model processing result corresponding to the audio segment”, may specifically include the following steps:

[0119] S401, in response to sending the first audio text to the vehicle terminal, the vehicle terminal sends the first audio text to the generative large model.

[0120] S402: Process the first audio text according to the generative big model to obtain a corresponding big model processing result.

[0121] In the embodiment of the present application, the first audio text obtained by the online speech recognition algorithm in the above embodiment is sent to the vehicle terminal, and the first audio text is sent to the generative large model at the vehicle terminal, and the first audio text is processed according to the generative large model to obtain the corresponding large model processing result. Figure 2 As shown, the first audio text obtained by the cloud ASR processing is sent to the vehicle terminal, and the first audio text is sent to the large model at the vehicle terminal, and the first audio text is processed according to the large model.

[0122] It should be noted that there is no need to use the vehicle terminal as a relay, and the first audio text can be directly sent to the generative large model for processing to obtain the corresponding large model processing result.

[0123] Figure 5 is a flow chart of a voice instruction processing device according to an embodiment of the present application. Figure 5 As shown, the voice instruction processing device 500 of the embodiment of the present application may specifically include:

[0124] The first acquisition module 501 is used to respond to the user waking up the voice assistant and acquire the corresponding audio segment based on the user's voice command.

[0125] The second acquisition module 502 is used to convert the audio segment into a first audio text based on a preset first online recognition channel, and obtain an online dialogue management result corresponding to the audio segment based on the first audio text.

[0126] The third acquisition module 503 is used to send the first audio text to the generative large model in the second online recognition channel based on the preset second online recognition channel to obtain the large model processing result corresponding to the audio segment.

[0127] The fourth acquisition module 504 is used to perform local dialogue management on the online dialogue management results and the large model processing results to obtain corresponding local dialogue management results.

[0128] The first determination module 505 is used to determine the target result to be executed based on the preset priority processing strategy and the local dialogue management result, and execute the target result on the voice assistant.

[0129] In a possible implementation, the voice instruction processing device further includes:

[0130] A fifth acquisition module, configured to convert the audio segment into a second audio text based on a preset local recognition channel, and obtain a local semantic understanding result corresponding to the audio segment based on the second audio text;

[0131] The fourth acquisition module is specifically used for:

[0132] Perform local dialogue management on the local semantic understanding results, online dialogue management results and large model processing results to obtain corresponding local dialogue management results.

[0133] In a possible implementation manner, the first determining module is specifically configured to:

[0134] Determine whether the local semantic understanding result is recognized based on the local dialogue management result;

[0135] If the local semantic understanding result is identified, the local semantic understanding result is determined as the target result, the target result is executed on the voice assistant, and the online dialogue management result and the large model processing result are discarded and stopped waiting.

[0136] In a possible implementation, the voice instruction processing device further includes:

[0137] A first recognition module, configured to determine whether an online dialogue management result is recognized based on the local dialogue management result if no local semantic understanding result is recognized;

[0138] The second determination module is used to determine the online dialogue management result as the target result for execution in response to identifying the online dialogue management result based on the local dialogue management result, execute the target result on the voice assistant, and discard and stop waiting for the large model processing result.

[0139] In a possible implementation, the voice instruction processing device further includes:

[0140] The second recognition module is used to execute the large model processing result on the voice assistant in response to the online dialogue management result not being recognized based on the local dialogue management result.

[0141] In a possible implementation manner, the first acquisition module is specifically configured to:

[0142] In response to the user waking up the voice assistant, detecting the user's voice command based on a preset voice activity detection algorithm;

[0143] The audio segment corresponding to the voice command is obtained based on the detection result.

[0144] In a possible implementation, the first online recognition channel includes an online speech recognition algorithm and an online management algorithm; the online management algorithm includes an online natural language understanding algorithm and an online dialogue management algorithm; the second acquisition module is specifically used to:

[0145] Converting the audio clip into a corresponding first audio text based on an online speech recognition algorithm, and sending the first audio text to an online natural language understanding algorithm and an online dialogue management algorithm;

[0146] The first audio text is processed based on an online natural language understanding algorithm and an online dialogue management algorithm to obtain a corresponding online dialogue management result.

[0147] In a possible implementation, the local recognition channel includes a local speech recognition algorithm and a local natural language understanding algorithm; the fifth acquisition module is specifically used to:

[0148] Converting the audio segment into a corresponding second audio text based on a local speech recognition algorithm, and sending the second audio text to a local natural language understanding algorithm;

[0149] The second audio text is processed based on a local natural language understanding algorithm to obtain a corresponding local natural language understanding result.

[0150] In a possible implementation manner, the third acquisition module is further used to:

[0151] In response to sending the first audio text to the vehicle computer, sending the first audio text to the generative large model at the vehicle computer;

[0152] The first audio text is processed based on the generative big model to obtain a corresponding big model processing result.

[0153] The voice command processing device provided in the embodiment of the present application responds to the user waking up the voice assistant, obtains the corresponding audio clip based on the user's voice command, converts the audio clip into a first audio text based on a preset first online recognition channel, and obtains the online dialogue management result corresponding to the audio clip based on the first audio text, sends the first audio text to the generative big model in the second online recognition channel based on the preset second online recognition channel, obtains the big model processing result corresponding to the audio clip, performs local dialogue management on the online dialogue management result and the big model processing result, obtains the corresponding local dialogue management result, determines the target result to be executed according to the local dialogue management result based on the preset priority processing strategy, and executes the target result on the voice assistant. The voice command processing device of the present application converts an audio clip through a first online recognition channel to obtain a first audio text, obtains an online dialogue management result of the first online recognition channel and a large model processing result of the generative large model of the second online recognition channel, respectively, performs local dialogue management on the online dialogue management result and the large model processing result, determines and executes the target result to be executed based on a priority processing strategy, realizes the integration of the generative large model and traditional semantics, resolves the conflicts and limitations between the generative large model and traditional semantics, can process the user's voice commands in parallel, improves the timeliness of the generative large model's replies, and also improves the execution efficiency and accuracy of the voice assistant.

[0154] like Figure 6As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-mentioned voice command processing method.

[0155] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the above-mentioned voice instruction processing method can be executed.

[0156] Corresponding to the above-mentioned voice instruction processing method, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned voice instruction processing method are executed.

[0157] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0158] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the deployment method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0161] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing voice commands, characterized in that: The method comprises: In response to a user waking up the voice assistant, acquiring a corresponding audio clip based on the user's voice command; The audio segment is converted into a first audio text based on a preset first online recognition channel, and an online conversation management result corresponding to the audio segment is obtained based on the first audio text; Sending the first audio text to the generative big model in the second online recognition channel based on a preset second online recognition channel to obtain a big model processing result corresponding to the audio segment; Performing local dialogue management on the online dialogue management result and the large model processing result to obtain corresponding local dialogue management results; Based on a preset priority processing strategy, the target result to be executed is determined according to the local dialogue management result, and the target result is executed on the voice assistant.

2. The method according to claim 1, characterized in that The method further includes, after the user wakes up the voice assistant in response to acquiring the corresponding audio segment based on the user's voice command: Converting the audio segment into a second audio text based on a preset local recognition channel, and obtaining a local semantic understanding result corresponding to the audio segment based on the second audio text; The performing local dialogue management on the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result includes: Local dialogue management is performed on the local semantic understanding result, the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result.

3. The method according to claim 2, characterized in that The method of determining a target result to be executed based on the local dialogue management result based on a preset priority processing strategy, and executing the target result on the voice assistant includes: Determining whether the local semantic understanding result is recognized based on the local dialogue management result; If the local semantic understanding result is identified, the local semantic understanding result is determined as the target result, the target result is executed on the voice assistant, and the online dialogue management result and the large model processing result are discarded and stopped waiting.

4. The method according to claim 3, characterized in that The method further comprises: If the local semantic understanding result is not identified, determining whether the online dialogue management result is identified based on the local dialogue management result; In response to identifying the online dialogue management result based on the local dialogue management result, the online dialogue management result is determined as a target result for execution, the target result is executed on the voice assistant, and the waiting for the large model processing result is discarded and stopped.

5. The method according to claim 4, characterized in that The method further comprises: In response to not identifying the online dialogue management result based on the local dialogue management result, executing the large model processing result on the voice assistant.

6. The method according to claim 1, characterized in that In response to the user waking up the voice assistant, acquiring a corresponding audio clip based on the user's voice command; In response to the user waking up the voice assistant, detecting a voice command of the user based on a preset voice activity detection algorithm; An audio segment corresponding to the voice instruction is obtained based on the detection result.

7. The method according to claim 2, characterized in that The first online recognition channel includes an online speech recognition algorithm and an online management algorithm; the online management algorithm includes an online natural language understanding algorithm and an online dialogue management algorithm; the first online recognition channel based on the preset first online recognition channel converts the audio segment into a first audio text, and obtains an online dialogue management result corresponding to the audio segment based on the first audio text, including: Converting the audio segment into a corresponding first audio text based on the online speech recognition algorithm, and sending the first audio text to the online natural language understanding algorithm and the online dialogue management algorithm; The first audio text is processed based on the online natural language understanding algorithm and the online dialogue management algorithm to obtain a corresponding online dialogue management result.

8. The method according to claim 2, characterized in that: The local recognition channel includes a local speech recognition algorithm and a local natural language understanding algorithm; the method of converting the audio segment into a second audio text based on the preset local recognition channel, and obtaining a local semantic understanding result corresponding to the audio segment based on the second audio text, includes: Converting the audio segment into a corresponding second audio text based on the local speech recognition algorithm, and sending the second audio text to the local natural language understanding algorithm; The second audio text is processed based on the local natural language understanding algorithm to obtain a corresponding local natural language understanding result.

9. The method according to claim 8, characterized in that The step of sending the first audio text to the generative large model in the second online recognition channel based on the preset second online recognition channel to obtain the large model processing result corresponding to the audio segment includes: In response to sending the first audio text to the vehicle terminal, sending the first audio text to the generative large model at the vehicle terminal; The first audio text is processed based on the generative big model to obtain a corresponding big model processing result.

10. A voice command processing device, characterized in that: The device comprises: A first acquisition module, configured to, in response to a user waking up the voice assistant, acquire a corresponding audio clip based on a voice command of the user; A second acquisition module is used to convert the audio segment into a first audio text based on a preset first online recognition channel, and obtain an online conversation management result corresponding to the audio segment based on the first audio text; A third acquisition module is used to send the first audio text to the generative big model in the second online recognition channel based on a preset second online recognition channel to obtain a big model processing result corresponding to the audio segment; A fourth acquisition module, used to perform local dialogue management on the online dialogue management result and the large model processing result to obtain a corresponding local dialogue management result; The first determination module is used to determine the target result to be executed according to the local dialogue management result based on a preset priority processing strategy, and execute the target result on the voice assistant.

11. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the voice instruction processing method as described in any one of claims 1 to 9 are performed.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the voice instruction processing method as described in any one of claims 1 to 9 are executed.

Images