Voice interaction method, server and readable storage medium

Abstract

The invention discloses a voice interaction method. The method comprises the following steps: receiving voice request fragments forwarded by a vehicle in sequence; and performing natural language processing on the first voice request fragment set to obtain a first recognition result and a first understanding result, the first voice request fragment set comprising voice request fragments received at the first moment. And performing natural language processing on a second voice request fragment set to obtain a second recognition result and a second understanding result, the second voice request fragment set comprising voice request fragments received at a second moment following the first voice request fragment set, and the second moment being later than the first moment. And performing natural language recognition on a received voice request to obtain a third recognition result, the voice request being obtained by splicing voice request fragments. And completing voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. Therefore, the accuracy of the system in processing the composite sentences is improved.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a voice interaction method, a server and a non-volatile computer-readable storage medium containing a computer program. Background Art

[0002] In the related art, the in-vehicle intelligent system can perform natural language processing on the user's voice request during the process of receiving the user's voice request, so that the system can quickly respond to the user's voice request. However, when the user's voice request is more complex, the inference length increases, the inference effect of the in-vehicle intelligent system decreases, and the user experience is affected. Summary of the invention

[0003] The present application provides a voice interaction method, a server, and a non-volatile computer-readable storage medium containing a computer program.

[0004] The present application provides a voice interaction method, the method comprising:

[0005] Receive voice request segments forwarded sequentially by the vehicle;

[0006] Performing natural language processing on a first voice request segment set to obtain a first recognition result and a first understanding result, wherein the first voice request segment set includes the voice request segment received at a first moment;

[0007] performing natural language processing on a second voice request segment set to obtain a second recognition result and a second understanding result, wherein the second voice request segment set includes the voice request segment received at a second time point subsequent to the first voice request segment set, and the second time point is later than the first time point;

[0008] Performing natural language recognition on the received voice request to obtain a third recognition result, wherein the voice request is obtained by concatenating the voice request segments;

[0009] The voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0010] In this way, the server receives the voice request segments forwarded by the vehicle in sequence. Next, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result. The first voice request segment set includes the voice request segment received at the first moment. Then, the server performs natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result. The second voice request segment set includes the voice request segment received at the second moment and continuing the first voice request segment set. The second moment is later than the first moment. The server then performs natural language recognition on the received voice request to obtain a third recognition result. The voice request is obtained by splicing the voice request segments. Finally, the server completes the voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, in the end-to-end architecture, the streaming natural language recognition and streaming natural language understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, reducing the resources and time required to wait for the entire voice request to be completed, thereby greatly shortening the response time of the server. In addition, through phased natural language recognition and natural language understanding of different sets of voice request fragments, the large language model can accurately understand the user's intentions and combine the natural language understanding results of different stages, thereby improving the accuracy of the system when processing long or complex sentences.

[0011] In some implementations, performing natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result includes:

[0012] Performing natural language recognition on the first voice request segment set to obtain the first recognition result;

[0013] According to a first preset mark, the first recognition result is marked to obtain a target marked speech recognition result;

[0014] Natural language understanding is performed on the target mark speech recognition result to obtain the first understanding result.

[0015] In this way, the server performs natural language recognition on the first voice request segment set to obtain a first recognition result. Next, the server performs tagging processing on the first recognition result according to the first preset tag to obtain a target tag voice recognition result. Finally, the server performs natural language understanding on the target tag voice recognition result to obtain a first understanding result. In this way, by using the first preset tag to tag the first natural language recognition result, the large language model can accurately recognize the input natural language recognition result and accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0016] In certain embodiments, the method further comprises:

[0017] The first understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

[0018] In this way, the server marks the first understanding result according to the second preset mark to obtain the target marked speech understanding result. In this way, by marking the natural language understanding result with the second preset mark, the large language model can optimize the natural language understanding process, ignore the natural language understanding results marked with the second preset mark, reduce the accumulation of errors in the dialogue process, and thus improve the overall understanding accuracy. In addition, the amount of information that needs to be considered can be reduced so that the system can process subsequent inputs more quickly, thereby improving the overall response speed.

[0019] In some embodiments, performing natural language understanding on the target mark speech recognition result to obtain the first understanding result includes:

[0020] Performing slot recognition on the target mark speech recognition result to obtain a slot recognition result;

[0021] Performing application program interface prediction on the target mark speech recognition result to obtain a predicted application interface;

[0022] According to the slot identification result and the predicted application interface, the predicted application interface is selected to perform application program interface parameter filling to obtain the first understanding result.

[0023] In this way, the server performs slot recognition on the target tag speech understanding result to obtain a slot recognition result. Next, the server performs application program interface prediction on the target tag speech understanding result to obtain a predicted application interface. Finally, the server selects the predicted application interface to perform application program interface parameter filling based on the slot recognition result and the predicted application interface to obtain a first understanding result. In this way, through slot recognition and application program interface prediction, the system can more accurately understand the user's request and select the most appropriate application interface to perform the task, thereby reducing misunderstandings and errors and providing a good user experience.

[0024] In some embodiments, performing natural language understanding on the target mark speech recognition result to obtain the first understanding result includes:

[0025] Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech recognition result to obtain the first understanding result.

[0026] In this way, based on the pre-trained large language model, the server performs natural language understanding on the target labeled speech recognition result to obtain a first understanding result. In this way, by performing natural language understanding on the target labeled speech recognition result through the pre-trained large language model, the accuracy and efficiency of natural language understanding can be improved.

[0027] In some implementations, completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request includes:

[0028] Concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result;

[0029] Performing matching processing on the splicing recognition result and the third recognition result;

[0030] The voice interaction is completed according to the result of the matching process.

[0031] In this way, the server concatenates the first recognition result and the second recognition result to obtain a concatenated recognition result. Then, the server matches the concatenated recognition result with the third recognition result. Finally, the server completes the voice interaction according to the result of the matching process. In this way, by concatenating the multiple recognition results obtained in the process of receiving the voice request and matching them with the third recognition result, the server can accurately understand the user's complex voice request and effectively perform the corresponding operation.

[0032] In some implementations, completing the voice interaction according to the result of the matching process includes:

[0033] When the splicing recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result;

[0034] Sending the target output result to the vehicle to complete the voice interaction; or

[0035] In the case that the concatenated recognition result and the third recognition result do not match, performing natural language understanding on the voice request to obtain the target output result;

[0036] The target output result is sent to the vehicle to complete the voice interaction.

[0037] In this way, when the spliced ​​recognition result and the third recognition result match, the server obtains the target output result according to the first understanding result and the second understanding result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Or, when the spliced ​​recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain the target output result. Then, the server sends the target output result to the vehicle to complete the voice interaction. In this way, when processing complex voice commands, by flexibly handling the matching of the spliced ​​recognition result and the third recognition result, the system can understand and respond to the user's voice commands more accurately, and can also make effective corrections when understanding deviations occur, which helps to improve the overall performance and user experience of the voice interaction system.

[0038] In some implementations, obtaining a target output result according to the first understanding result and the second understanding result includes:

[0039] Screening the first understanding result to obtain a first sub-goal output result, wherein the first sub-goal output result includes a first application interface and a first application program interface parameter filling result;

[0040] Screening the second understanding result to obtain a second sub-goal output result, wherein the second sub-goal output result includes a second application interface and a second application program interface parameter filling result;

[0041] The target output result is obtained according to the first sub-target output result and the second sub-target output result.

[0042] In this way, the server screens and processes the first understanding result to obtain the first sub-target output result, which includes the first application interface and the first application interface parameter filling result. Next, the server screens and processes the second understanding result to obtain the second sub-target output result, which includes the second application interface and the second application interface parameter filling result. Finally, the server obtains the target output result based on the first sub-target output result and the second sub-target output result. In this way, when processing complex voice requests, by screening and processing the understanding results in steps, the system can more accurately extract and integrate the natural language understanding results, thereby generating more accurate target output results, which helps to improve the understanding and response quality of the voice interaction system.

[0043] An embodiment of the present application provides a server, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0044] An embodiment of the present application provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0045] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 It is one of the flowcharts of the voice interaction method of certain embodiments of the present application;

[0048] Figure 2 It is one of the flowcharts of processing voice requests in certain embodiments of the present application;

[0049] Figure 3 This is a second schematic diagram of a flow chart of processing a voice request in certain embodiments of the present application;

[0050] Figure 4 This is the second flow chart of the voice interaction method of certain implementation modes of the present application;

[0051] Figure 5 This is the third flow chart of the voice interaction method of certain implementation modes of the present application;

[0052] Figure 6 This is a fourth flowchart of a voice interaction method according to certain embodiments of the present application;

[0053] Figure 7 This is a fifth flow chart of a voice interaction method according to certain embodiments of the present application;

[0054] Figure 8 This is the sixth flow chart of the voice interaction method of certain implementation modes of the present application;

[0055] Fig. 9 is a flowchart of a matching process in certain embodiments of the present application;

[0056] Fig.10 This is the seventh flow chart of the voice interaction method of certain implementation modes of the present application;

[0057] Fig.11 This is the eighth flow chart of the voice interaction method of certain implementation modes of the present application;

[0058] Fig.12This is the ninth flow chart of the voice interaction method of certain embodiments of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0060] In related technologies, in order to improve response speed and user experience, the system can perform natural language processing (NLP) in real time during the process of receiving user voice requests, that is, during voice activity detection (VAD). This allows the system to understand and respond to user requests before the user's voice command is completely finished, thereby achieving faster interaction.

[0061] However, when the user's voice request becomes more complex, such as containing multiple sub-commands or requiring the understanding of long sentences, the system's reasoning length increases. In this case, the reasoning effect of the in-vehicle intelligent system may decrease because the system requires more time and computing resources to process and understand complex voice requests. This delay and performance degradation may affect the user experience because the user may have to wait longer for a response, or the system may not accurately understand the user's intentions.

[0062] Voice activity detection refers to a technique used to determine the presence of a speech signal. In speech processing and communication systems, VAD is used to identify the beginning and end of speech activity so that only the portions containing speech can be processed, ignoring periods of silence or background noise.

[0063] Based on the above questions, please refer to Figure 1 , the present application embodiment provides a voice interaction method, the method comprising:

[0064] 01: Receive the voice request segments forwarded by the vehicle in sequence;

[0065] 02: Perform natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result;

[0066] 03: Perform natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result;

[0067] 04: Perform natural language recognition on the received voice request to obtain a third recognition result;

[0068] 05: Complete voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0069] The embodiment of the present application also provides a server, including a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive voice request fragments forwarded sequentially by the vehicle. And natural language processing is performed on the first voice request fragment set to obtain a first recognition result and a first understanding result. And natural language processing is performed on the second voice request fragment set to obtain a second recognition result and a second understanding result. The processor is also used to perform natural language recognition on the received voice request to obtain a third recognition result. And voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0070] The embodiment of the present application also provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a language processing module, a language recognition module and a processing module. The receiving module is used to receive voice request fragments forwarded sequentially by the vehicle. The language processing module is used to perform natural language processing on a first set of voice request fragments to obtain a first recognition result and a first understanding result. The language processing module is used to perform natural language processing on a second set of voice request fragments to obtain a second recognition result and a second understanding result. The language recognition module is used to perform natural language recognition on the received voice request to obtain a third recognition result. The processing module is used to complete the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0071] Specifically, see Figure 2 In classic technology, the process of human-computer interaction can be divided into five stages, including receiving user voice requests, voice activity detection, automatic speech recognition, natural language understanding (NLU), and robot understanding instructions. Among them, the time required for receiving user voice requests may be A1, the time required for voice activity detection may be A2, the time required for automatic speech recognition may be A3, the time required for natural language understanding may be A4, and the time required for the robot to understand the instructions may be A5. The time it takes for the server to complete this human-computer interaction is A1+A2+A3+A4+A5.

[0072] Robot understanding instructions refers to the large language model receiving instructions (natural language understanding results). The instruction may be a piece of program code, and the robot that interacts with humans needs to understand this program code, which takes a certain amount of time.

[0073] However, this classic voice interaction technology, that is, after receiving a complete voice request, the complete voice request is processed by natural language, the system response time is slow and the interaction experience is poor. Based on the above problems, the related art provides a voice interaction method that can improve the response speed, that is, in the process of receiving the user's voice request, the voice request fragment is processed in real time. Figure 2 As shown, it can be found that when the server receives the first voice request segment, it will perform natural speech recognition processing and natural language understanding processing. When the voice request is received, the natural language recognition processing and natural language understanding processing of each voice request segment are also completed. Subsequently, the server performs sound activity detection, and performs natural language recognition processing and result selection processing on the complete voice request. In this way, the time required for the server to receive the user's voice request is also A1, the time required for sound activity detection is also A2, the time required for natural language recognition processing on the complete voice request is also A3, and the time required for result selection processing may be A6. The time for the server to complete this human-computer interaction is A1+A2+A3+A6, and the response speed is faster.

[0074] Result selection processing refers to selecting the natural language understanding results obtained according to the voice request fragments through matching and screening processing, selecting the appropriate natural language understanding results and sending them to the vehicle to complete the voice interaction. At the same time, since the program code of each natural language understanding result obtained according to different voice request fragments has been understood by the robot, the response time is reduced.

[0075] Compared with the related art, when facing complex voice requests, the reasoning effect of the in-vehicle intelligent system is reduced and the user experience is reduced due to the increase in the length of the voice request that needs to be inferred. The voice interaction method provided by the embodiment of the present application can more accurately understand the user's intention through phased natural language recognition and natural language understanding of different voice request fragment sets, and respond to the voice request in combination with the natural language understanding results of different stages, thereby improving the accuracy of the system in processing long or complex sentences.

[0076] A voice request segment refers to a voice segment obtained by intercepting the received voice request based on a certain interval when the server first receives the voice request forwarded by the vehicle. For example, if the voice request sent by the user is "open the car window and turn on the air conditioner", the server may intercept two voice request segments at different times, the first voice request segment is "open", and the second voice request segment is "car window".

[0077] The voice request segment set refers to the voice request segment set that can output the application interface and the application interface parameter filling result. In other words, if the user inputs "open the car window and turn on the air conditioner", then the voice request segment set is "open the car window" and "open the car window".

[0078] The first voice request segment set refers to the first voice request segment set that can output the application interface and the application interface parameter filling result, including all voice request segments received within the first moment, and the first moment is used to indicate the moment when the application interface and the application interface parameter filling result can be output. In other words, if the user inputs "open the car window and turn on the air conditioner", then the first voice request segment set is "open the car window", which is composed of the first voice request segment and the second voice request segment.

[0079] The second voice request segment set can refer to all voice request segment sets received after the first voice request segment set. It is a large concept, including all voice request segments received within the second moment. The second moment is used to indicate any moment after the first moment when the application interface and the application program interface parameter filling result can be output. In other words, if the user inputs "open the car window and turn on the air conditioner", then the second voice request segment set is "turn on the air conditioner", which is composed of the fourth voice request segment and the fifth voice request segment. If the user inputs "open the car window, turn on the air conditioner, and play music", then the second voice request segment set is "turn on the air conditioner" and "play music", "turn on the air conditioner" is composed of the third voice request segment and the fourth voice request segment, and "play music" is composed of the sixth voice request segment and the seventh voice request segment. It should be noted that "play music" can be called the third voice request segment set in terms of timing.

[0080] See also Figure 3, the server can receive the voice request fragments forwarded by the vehicle in sequence, that is, process the voice stream in real time, instead of waiting for the entire voice request to end before processing. The following example illustrates the voice interaction method of the implementation mode of the present application. If the voice request given by the user is "open the car window and turn on the air conditioner", the server will receive multiple voice request fragments. The voice request fragment received at time T0 is "open", the voice request fragment received at time T1 is "car window", the voice request fragment received at time T2 is "and", the voice request fragment received at time T3 is "open", and the voice request fragment received at time T4 is "air conditioner".

[0081] Next, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result. The first voice request segment set includes all voice request segments received at the first moment. Continuing with the above example, the server will perform natural language processing on the received voice request segments, that is, at time T0, the server will perform natural language processing on the voice request segment "Open", and obtain a natural language recognition result of "Open" and a natural language understanding result of "Unclear". It can be found that no application interface and application interface parameter filling results are output at time T0, so the voice request segment set at time T0 is not the first voice request segment set. At time T1, the server will perform natural language processing on the voice request segment set of the voice request segment "open" and the voice request segment "window", and obtain the natural language recognition result of "open the window" and the natural language understanding result of "WindowOpen, {device: window}". It can be found that at time T1, there is an output application interface and application interface parameter filling result, and this voice request segment set is the first output application interface and application interface parameter filling result in this voice request, so the voice request segment set at time T1 is the first voice request segment set, the natural language recognition result "open the window" is the first recognition result, and the natural language understanding result "WindowOpen, {device: window}" is the first understanding result.

[0082] Then, the server performs natural language processing on the second set of voice request segments to obtain a second recognition result and a second understanding result. The second set of voice request segments includes all voice request segments received at the second moment in time that are subsequent to the first set of voice request segments, and the second moment is later than the first moment. Continuing with the above example, after the server obtains the first set of voice request segments, it no longer needs to consider the voice request segments before that moment, that is, moment T2, and no longer needs to perform natural language processing on the voice request segments received at moments T0 and T1. Then the set of voice request segments that need to be processed by natural language at moment T2 is “and”, and the natural language recognition result “and” and the natural language understanding result “Noise” are obtained. In this way, the second set of voice request segments is the set of voice request segments at moment T4, and the second recognition result obtained is “and turn on the air conditioner”, and the second understanding result is “AcOpen, {device: air conditioner}”.

[0083] After a certain period of time, if no voice request fragments forwarded by the vehicle are received, the server determines that the user has completed his voice request. The server will perform natural language recognition on the received voice request to obtain a third recognition result. The voice request is obtained by splicing all the voice request fragments. In other words, there is only one third natural language recognition result, which is obtained by performing natural language recognition processing on the final voice request. Continuing with the above example, the second natural language recognition result is "open the car windows and turn on the air conditioner."

[0084] Finally, the server completes the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0085] In summary, in the voice interaction method and server provided in the embodiment of the present application, the server receives the voice request segments forwarded sequentially by the vehicle. Then, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result, and the first voice request segment set includes the voice request segment received at the first moment. Then, the server performs natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result, and the second voice request segment set includes the voice request segment received at the second moment and continuing the first voice request segment set, and the second moment is later than the first moment. The server then performs natural language recognition on the received voice request to obtain a third recognition result, and the voice request is obtained by splicing the voice request segments. Finally, the server completes the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, in the end-to-end architecture, the streaming natural language recognition and streaming natural language understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, reducing the resources and time required to wait for the entire voice request to be completed, thereby greatly shortening the response time of the server. In addition, through phased natural language recognition and natural language understanding of different sets of voice request fragments, the large language model can accurately understand the user's intentions and combine the natural language understanding results of different stages, thereby improving the accuracy of the system when processing long or complex sentences.

[0086] See also Figure 4 In some implementations, step 02 (performing natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result) includes:

[0087] 021: Perform natural language recognition on the first voice request segment set to obtain a first recognition result;

[0088] 022: Marking the first recognition result according to the first preset mark to obtain a target marked speech recognition result;

[0089] 023: Perform natural language understanding on the target labeled speech recognition result to obtain a first understanding result.

[0090] In some embodiments, the language processing module is further configured to perform natural language recognition on the first voice request segment set to obtain a first recognition result, and to perform tagging processing on the first recognition result according to a first preset tag to obtain a target-tagged voice recognition result, and to perform natural language understanding on the target-tagged voice recognition result to obtain a first understanding result.

[0091] In some embodiments, the processor is further configured to perform natural language recognition on the first voice request segment set to obtain a first recognition result, and to perform tagging processing on the first recognition result according to a first preset tag to obtain a target-tagged voice recognition result, and to perform natural language understanding on the target-tagged voice recognition result to obtain a first understanding result.

[0092] Specifically, natural language recognition refers to the process of converting human natural language (such as English, Chinese, etc.) into a form that can be understood and processed by machines.

[0093] The first preset tag refers to a tag "wait" that is added before the natural language recognition result before the big language model performs natural language understanding processing on the natural language recognition result. This tag "wait" can guide the big language model to perform natural language understanding processing mode, that is, it tells the big language model that the next text can be processed by natural language understanding.

[0094] The server first performs natural language recognition on the first set of voice request segments to obtain a first recognition result. Then, the server performs labeling processing on the first natural language recognition result according to the first preset label to obtain a target labeled voice recognition result. Continuing with the above example, for example, at time T1, the target labeled language recognition result obtained by the server is "wait to open the window". It should be noted that the labeling process does not only label the voice request segments that can output application interfaces and application interface parameter filling results, but also labels the natural language recognition results obtained by performing natural language recognition processing on each received voice request segment. Continuing with the above example, at time T0, the target labeled language recognition result obtained by the server is "wait to open".

[0095] Subsequently, the server performs natural language understanding on the target tag speech recognition result to obtain a first understanding result. Continuing with the above example, at time T1, the server performs natural language understanding processing on "wait to open the car window" to obtain a natural language understanding result "WindowOpen, {device: car window}".

[0096] In this way, by marking the first natural language recognition result with the first preset tag, the large language model can accurately recognize the input natural language recognition result, accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0097] See also Figure 5 In some embodiments, the method further comprises:

[0098] 024: Mark the first understanding result according to the second preset mark to obtain a target marked speech understanding result.

[0099] In some implementations, the processing module is further configured to mark the first understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0100] In some implementations, the processor is further configured to mark the first understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0101] Specifically, the second preset tag refers to a tag waitEnd that will be added after the first understanding result is obtained after the large language model performs natural language understanding processing on the target tag speech recognition result and obtains the first understanding result. This tag waitEnd can guide the large language model to perform natural language understanding processing without being affected by the natural language understanding results obtained at the previous moment. In other words, the server will only perform natural language understanding processing based on the obtained natural language recognition results. Continuing with the above example, for example, at time T0, the large language model will receive the voice request segment set "open" and the natural language understanding result "wait open waitEnd" obtained at time T0. Due to the second preset tag waitEnd, the server will not perform natural language understanding processing on the natural language understanding result "wait open waitEnd".

[0102] The server performs labeling processing on the first understanding result according to the second preset label to obtain a target labeled speech understanding result.

[0103] In this way, by marking the natural language understanding results with the second preset mark, the large language model can optimize the natural language understanding process, ignore the natural language understanding results marked with the second preset mark, reduce the accumulation of errors in the conversation process, and thus improve the overall understanding accuracy. In addition, the amount of information that needs to be considered can be reduced so that the system can process subsequent inputs more quickly, thereby improving the overall response speed.

[0104] See also Figure 6 In some implementations, step 023 (performing natural language understanding on the target mark speech recognition result to obtain a first understanding result) includes:

[0105] 0231: Perform slot recognition on the target mark speech recognition result to obtain the slot recognition result;

[0106] 0232: Perform application program interface prediction on the target labeled speech recognition result to obtain a predicted application interface;

[0107] 0233: According to the slot recognition result and the prediction application interface, the prediction application interface is selected to execute the application program interface parameter filling to obtain the first natural language understanding result.

[0108] In some embodiments, the processing module is further configured to perform slot recognition on the target mark speech recognition result to obtain a slot recognition result. The processing module is further configured to perform application program interface prediction on the target mark speech recognition result to obtain a predicted application interface. The processing module is further configured to perform application program interface parameter filling based on the slot recognition result and the predicted application interface, and select the predicted application interface to obtain a first natural language understanding result.

[0109] In some embodiments, the processor is further configured to perform slot recognition on the target mark speech recognition result to obtain a slot recognition result, and perform application program interface prediction on the target mark speech recognition result to obtain a predicted application interface, and select the predicted application interface to perform application program interface parameter filling according to the slot recognition result and the predicted application interface to obtain a first natural language understanding result.

[0110] Specifically, slot recognition refers to extracting specific information fragments from the user's input, which are usually called "slots". Slots are usually key information necessary to complete a task or request, such as time, place, object, etc. Taking the user's voice request "What will the temperature be tomorrow" as an example, the slot information that can be obtained through slot recognition includes ["tomorrow" - date (Date)], that is, the slot information includes slot value and slot type, where "tomorrow" is the slot value and date (Date) is the slot type. Taking the user's voice request "Navigate to Zhongguancun" as an example, the slot information that can be obtained through slot recognition is ["Zhongguancun" - place name (Place)], where "Zhongguancun" is the slot value and the place name (Place) is the slot type.

[0111] The slot recognition result refers to the named entity obtained by performing slot recognition on the voice request, such as the slot information ["tomorrow" - date (Date)] and slot information ["Zhongguancun" - place name (Place)] mentioned above.

[0112] The server performs slot recognition on the target tag speech recognition result to obtain a slot recognition result. Continuing with the above example, at time T1, the server performs slot recognition on "open the car window" and obtains a slot recognition result "car window - device".

[0113] Next, the server performs an API prediction on the target tag speech recognition result to obtain a predicted API. Continuing with the above example, at time T1, the server performs an API prediction on the speech request segment set "open the car window" to obtain a predicted API "WindowOpen".

[0114] Finally, the server selects the predicted application interface to execute the application program interface parameter filling according to the slot recognition result and the predicted application interface, and obtains the first natural language understanding result.

[0115] It should be noted that the above processing will be performed for all voice request segments received at each moment. If the slot recognition result is not identified and / or the prediction application interface is not obtained, the large language model will output "Unclear". If the voice request segment has no practical meaning, the large language model will output "Noise". In some embodiments, the large language model will eliminate the voice request segment whose natural language understanding result is "Noise", and will no longer consider the voice request segment during subsequent natural language recognition processing.

[0116] In this way, through slot identification and API prediction, the system can understand the user's request more accurately and select the most appropriate API to perform the task, thereby reducing misunderstandings and errors and providing a good user experience.

[0117] See also Figure 7 In some implementations, step 023 (performing natural language understanding on the target mark speech recognition result to obtain a first understanding result) includes:

[0118] 0234: Based on the pre-trained large language model, perform natural language understanding on the target labeled speech recognition results to obtain the first understanding result.

[0119] In some embodiments, the processing module is further used to perform natural language understanding on the target labeled speech recognition result based on the pre-trained large language model to obtain a first understanding result.

[0120] In some embodiments, the processor is further used to perform natural language understanding on the target labeled speech recognition result based on the pre-trained large language model to obtain a first understanding result.

[0121] Specifically, in some embodiments, the large language model has been trained. In some embodiments, the training steps of the large language model are as follows: construct training data based on the collected voice request samples and the vehicle system function customization solution samples constructed according to the voice request samples. The sample data can come from the collected user logs, such as the user's history of successfully constructing the vehicle system function customization solution according to the voice request. Of course, training data can also be constructed as needed. For example, a set of training data can be: a voice request "turn on the air conditioner immediately", and the vehicle system function customization solution constructed according to the voice request is "AcOpen{device:air conditioner}".

[0122] Furthermore, in order to guide the large model to meet the task requirements, the prompt information template needs to be configured. Specifically, the prompt information template is used to guide the model to generate output, providing the background of the problem and relevant guidance. It usually includes a description and instructions of the problem to tell the model how to process the input data. The prompt information template can provide additional auxiliary information to facilitate the model to better understand and generate output. It can also include a specific format, style, theme that the model is required to follow, or instruct the model to pay attention to certain specific aspects and details. As a result, the model will understand the requirements of the task based on the content of the prompt information template, thereby generating answers and texts that meet expectations. Generally, the configuration process of the prompt information template may include clarifying the role that the model needs to play (optional), clarifying the input text content and structure, and clarifying the output text content and structure.

[0123] In an example, the content of the prompt message template is as follows:

[0124] "Suppose you are an intelligent cockpit dialogue system. You need to understand the recognized commands and output the corresponding API and Arguments. The input command fragments will be input multiple times. Please determine the current API and Argumengts based on the existing commands."

[0125] Based on the pre-trained large language model, the server performs natural language understanding on the target labeled speech recognition result to obtain a first natural language understanding result.

[0126] In this way, the accuracy and efficiency of natural language understanding can be improved by performing natural language understanding on the target labeled speech recognition results through a pre-trained large language model.

[0127] See also Figure 8 In some implementations, step 05 (completing voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request) includes:

[0128] 051: concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result;

[0129] 052: matching the spliced ​​recognition result and the third recognition result;

[0130] 053: Complete the voice interaction according to the result of the matching process.

[0131] In some embodiments, the processing module is used to perform splicing processing on the first recognition result and the second recognition result to obtain a spliced ​​recognition result, and to perform matching processing on the spliced ​​recognition result and the third recognition result, and to complete the voice interaction according to the result of the matching processing.

[0132] In some embodiments, the processor is further configured to concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result, and to match the concatenated recognition result and the third recognition result, and to complete the voice interaction according to the result of the matching process.

[0133] Specifically, the matching process refers to comparing the first natural language recognition result and the second natural language recognition result based on text alignment, semantic analysis and other technologies to verify the consistency of the first natural language recognition result and the second natural language recognition result.

[0134] See also Fig. 9 , the server concatenates the first recognition result and the second recognition result to obtain a concatenated recognition result. Continuing with the above example, at time T4, the large language model does not receive any new voice fragments within a certain period of time, confirming that the user's voice request has been received. If the received voice request is "open the car window and turn on the air conditioner", the first recognition result obtained is "open the car window", and the second recognition result is "and turn on the air conditioner", the first recognition result and the second recognition result are concatenated. If the received voice request is "open the car window, turn on the air conditioner, and play music", the first recognition result obtained is "open the car window", and the second recognition result is "turn on the air conditioner" and "and play music". In some embodiments, matching is performed through special tags "group" and "groupEnd", and the result obtained is the position corresponding to the natural language understanding result.

[0135] Next, the server matches the concatenated recognition result and the third recognition result. Continuing with the above example, if the received voice request is "open the car window and turn on the air conditioner", the server matches the concatenated first recognition result and the second recognition result to obtain "open the car window and turn on the air conditioner" and the third recognition result "open the car window and turn on the air conditioner" obtained by performing natural language recognition on the complete voice request.

[0136] Finally, the server completes the voice interaction based on the results of the matching process.

[0137] In this way, by splicing multiple recognition results obtained in the process of receiving voice requests and matching them with the third recognition result, the server can accurately understand the user's complex voice requests and effectively perform corresponding operations.

[0138] See also Fig.10 and Fig.11 In some implementations, step 053 (completing voice interaction according to the result of the matching process) includes:

[0139] 0531: when the spliced ​​recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result;

[0140] 0532: Send the target output result to the vehicle to complete the voice interaction; or

[0141] 0533: When the concatenated recognition result and the third recognition result do not match, natural language understanding is performed on the voice request to obtain a target output result;

[0142] 0534: Send the target output result to the vehicle to complete the voice interaction.

[0143] In some embodiments, the processing module is further configured to obtain a target output result according to the first understanding result and the second understanding result when the spliced ​​recognition result matches the third recognition result, and send the target output result to the vehicle to complete the voice interaction. Alternatively, the processing module is further configured to perform natural language understanding on the voice request to obtain a target output result when the spliced ​​recognition result does not match the third recognition result, and send the target output result to the vehicle to complete the voice interaction.

[0144] In some embodiments, the processor is further configured to obtain a target output result according to the first understanding result and the second understanding result when the spliced ​​recognition result matches the third recognition result, and send the target output result to the vehicle to complete the voice interaction. Alternatively, the processor is further configured to perform natural language understanding on the voice request to obtain a target output result when the spliced ​​recognition result does not match the third recognition result, and send the target output result to the vehicle to complete the voice interaction.

[0145] For details, please refer to Fig.10 , when the spliced ​​recognition result and the third recognition result match, the server obtains the target output result according to the first understanding result and the second understanding result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Continuing with the above example, it can be found that the third recognition result "open the window and turn on the air conditioner" is consistent with the spliced ​​recognition result "open the window and turn on the air conditioner", then according to the first understanding result "WindowOpen, {device: window}" and the second understanding result "AcOpen, {device: air conditioner}", the target output result "WindowOpen, {device: window}; AcOpen, {device: air conditioner}" is obtained.

[0146] In the case where the spliced ​​recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain the target output result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Continuing with the above example, if the spliced ​​recognition result is "open the car windows and turn on the air conditioner", then the spliced ​​recognition result "open the car windows and turn on the air conditioner" is inconsistent with the third recognition result "open the car windows and turn on the air conditioner". The large language model will perform natural language understanding on the second natural language recognition result to obtain the target output result. Subsequently, the target output result is sent to the vehicle to complete the voice interaction.

[0147] In this way, when processing complex voice commands, by flexibly handling the matching of the spliced ​​recognition results and the third recognition results, the system can more accurately understand and respond to the user's voice commands, and can also make effective corrections when understanding deviations occur, which helps to improve the overall performance of the voice interaction system and user experience.

[0148] See also Fig.12 In some embodiments, step 0531 (obtaining a target output result according to the first understanding result and the second understanding result) includes:

[0149] 05311: screening the first understanding result to obtain a first sub-goal output result, where the first sub-goal output result includes a first application interface and a first application program interface parameter filling result;

[0150] 05312: Screening the second understanding result to obtain a second sub-goal output result, where the second sub-goal output result includes a second application interface and a second application program interface parameter filling result;

[0151] 05313: According to the output result of the first sub-goal and the output result of the second sub-goal, the target output result is obtained.

[0152] In some embodiments, the processing module is used to filter the first understanding result to obtain a first sub-goal output result, the first sub-goal output result includes a first application interface and a result of filling the first application interface parameters. And filter the second understanding result to obtain a second sub-goal output result, the second sub-goal output result includes a second application interface and a result of filling the second application interface parameters. And according to the first sub-goal output result and the second sub-goal output result, obtain the target output result.

[0153] In some embodiments, the processor is further configured to filter the first understanding result to obtain a first sub-target output result, the first sub-target output result including a first application interface and a result of filling in the first application interface parameters. The processor is further configured to filter the second understanding result to obtain a second sub-target output result, the second sub-target output result including a second application interface and a result of filling in the second application interface parameters. The processor is further configured to obtain a target output result based on the first sub-target output result and the second sub-target output result.

[0154] Specifically, the screening process refers to matching the natural language recognition result corresponding to the natural language understanding result and the third recognition result. If the third recognition result includes the natural language recognition result, then the natural language understanding result is considered to be the sub-target output result.

[0155] The server filters and processes the first understanding result to obtain the first sub-target output result, which includes the first application interface and the first application interface parameter filling result. Continuing with the above example, if the first recognition result is "open the car window" and the third recognition result is "open the car window and turn on the air conditioner", then it is considered that the first understanding result corresponding to the first recognition result cannot be used as the first sub-target output result. If the first recognition result is "open the car window" and the third recognition result is "open the car window and turn on the air conditioner", then it is considered that the first understanding result corresponding to the first recognition result can be used as the first sub-target output result, that is, the obtained first application interface "WindowOpen" and the first application interface parameter filling result "{WindowOpen, device: car window}" are used as the first sub-target output result.

[0156] Next, the server screens the second understanding result to obtain the second sub-target output result, which includes the second application interface and the second application interface parameter filling result. Continuing with the above example, if the second recognition result is "and open the car window wide", and the third recognition result is "open the car window and turn on the air conditioner", then it is considered that the first understanding result corresponding to the second recognition result cannot be used as the second target output result. If the second recognition result is "open the car window" and the third recognition result is "open the car window and turn on the air conditioner", then it is considered that the second understanding result corresponding to the second recognition result can be used as the second sub-target output result, that is, the obtained second application interface "AcOpen" and the first application interface parameter filling result {"AcOpen, device: air conditioner}" are used as the second sub-target output result.

[0157] Finally, the server obtains the target output result according to the first sub-target output result and the second sub-target output result.

[0158] It should be noted that in some embodiments, each natural language understanding result will not be screened separately, that is, the natural language recognition result corresponding to each natural language understanding result will not be compared with the third recognition result, but the natural language recognition result obtained at the last moment will be directly compared with the third recognition result. When the target output result is finally output, either all natural language understanding results are sent to the vehicle, or none of the natural language understanding results are sent.

[0159] In this way, when processing complex voice requests, by screening and processing the understanding results in steps, the system can more accurately extract and integrate natural language understanding results, thereby generating more precise target output results, which helps to improve the understanding and response quality of the voice interaction system.

[0160] The present application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the one or more processors execute the method of the present application.

[0161] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium.

[0162] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0163] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0164] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receive voice request segments forwarded sequentially by the vehicle; Performing natural language processing on a first voice request segment set to obtain a first recognition result and a first understanding result, wherein the first voice request segment set includes the voice request segment received at a first moment; performing natural language processing on a second voice request segment set to obtain a second recognition result and a second understanding result, wherein the second voice request segment set includes the voice request segment received at a second time point subsequent to the first voice request segment set, and the second time point is later than the first time point; Performing natural language recognition on the received voice request to obtain a third recognition result, wherein the voice request is obtained by concatenating the voice request segments; The voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

2. The method according to claim 1, characterized in that The performing natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result includes: Performing natural language recognition on the first voice request segment set to obtain the first recognition result; According to a first preset mark, the first recognition result is marked to obtain a target marked speech recognition result; Natural language understanding is performed on the target mark speech recognition result to obtain the first understanding result.

3. The voice interaction method according to claim 2, characterized in that: The method further comprises: The first understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

4. The voice interaction method according to claim 2, characterized in that: The performing natural language understanding on the target mark speech recognition result to obtain the first understanding result includes: Performing slot recognition on the target mark speech recognition result to obtain a slot recognition result; Performing application program interface prediction on the target mark speech recognition result to obtain a predicted application interface; According to the slot identification result and the predicted application interface, the predicted application interface is selected to perform application program interface parameter filling to obtain the first understanding result.

5. The voice interaction method according to claim 2, characterized in that: The performing natural language understanding on the target mark speech recognition result to obtain the first understanding result includes: Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech recognition result to obtain the first understanding result.

6. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request includes: Concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result; Performing matching processing on the splicing recognition result and the third recognition result; The voice interaction is completed according to the result of the matching process.

7. The voice interaction method according to claim 6, characterized in that: The completing the voice interaction according to the result of the matching process includes: When the splicing recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result; Sending the target output result to the vehicle to complete the voice interaction; or In the case that the concatenated recognition result and the third recognition result do not match, performing natural language understanding on the voice request to obtain the target output result; The target output result is sent to the vehicle to complete the voice interaction.

8. The voice interaction method according to claim 7, characterized in that: The obtaining a target output result according to the first understanding result and the second understanding result includes: Screening the first understanding result to obtain a first sub-goal output result, wherein the first sub-goal output result includes a first application interface and a first application program interface parameter filling result; Screening the second understanding result to obtain a second sub-goal output result, wherein the second sub-goal output result includes a second application interface and a second application program interface parameter filling result; The target output result is obtained according to the first sub-target output result and the second sub-target output result.

9. A server, characterized in that: The server includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the voice interaction method described in any one of claims 1 to 8 is implemented.

10. A non-volatile computer-readable storage medium containing a computer program, characterized in that: When the computer program is executed by one or more processors, the voice interaction method described in any one of claims 1 to 8 is implemented.

Images