Voice interaction method, server and readable storage medium

Abstract

The invention discloses a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program. The method comprises the steps of receiving voice request fragments forwarded by a vehicle in sequence; and performing natural language recognition on all the currently received voice request segments to obtain a first natural language recognition result. And obtaining a first natural language understanding result according to the first natural language recognition result. And performing natural language recognition on a received voice request to obtain a second natural language recognition result, the voice request being obtained by splicing all voice request fragments. And completing voice interaction according to the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request. Thus, in the end-to-end architecture, through combination of the large language model and the streaming dialogue technology, the streaming natural language processing capability of the large language model is realized, the time required for waiting for the completion of the whole voice request is reduced, and the response time is shortened.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a voice interaction method, a server and a non-volatile computer-readable storage medium containing a computer program. Background Art

[0002] In the related art, the in-vehicle intelligent system usually performs human-vehicle interaction based on the automatic speech recognition results of the complete voice request. However, in this case, it is necessary to wait for the sound activity detection of the voice request to be completed before automatic speech recognition, and then natural language understanding, which takes a long time overall and has a poor user experience. Summary of the invention

[0003] The present application provides a voice interaction method, a server, and a non-volatile computer-readable storage medium containing a computer program.

[0004] The present application provides a voice interaction method, the method comprising:

[0005] Receive voice request segments forwarded sequentially by the vehicle;

[0006] Performing natural language recognition on all the currently received voice request segments to obtain a first natural language recognition result;

[0007] Obtaining a first natural language understanding result according to the first natural language recognition result;

[0008] Performing natural language recognition on the received voice request to obtain a second natural language recognition result, wherein the voice request is obtained by concatenating all the voice request segments;

[0009] The voice interaction is completed according to the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request.

[0010] In this way, the server receives the voice request fragments forwarded by the vehicle in sequence. Next, the server performs natural language recognition on all the voice request fragments currently received to obtain a first natural language recognition result. Then, the server obtains a first natural language understanding result based on the first natural language recognition result. The server then performs natural language recognition on the received voice request to obtain a second natural language recognition result, and the voice request is obtained by splicing all the voice request fragments. Finally, the server completes the voice interaction based on the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request. In this way, in the end-to-end architecture, the streaming natural language recognition and streaming natural language understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, reducing the resources and time required to wait for the entire voice request to be completed, thereby greatly shortening the server's response time. In addition, through staged natural language recognition and natural language understanding, the large language model can more accurately understand the user's intentions, thereby improving the efficiency and accuracy of voice interaction.

[0011] In some implementations, obtaining a first natural language understanding result according to the first natural language recognition result includes:

[0012] Marking the first natural language recognition result according to a first preset mark to obtain a target marked speech recognition result;

[0013] Natural language understanding is performed on the target mark speech recognition result to obtain the first natural language understanding result.

[0014] In this way, the server performs a tagging process on the first natural language recognition result according to the first preset tag to obtain a target-tagged speech recognition result. Next, the server performs natural language understanding on the target-tagged speech recognition result to obtain a first natural language understanding result. In this way, by using the first preset tag to tag the first natural language recognition result, the large language model can accurately recognize the input natural language recognition result and accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0015] In certain embodiments, the method further comprises:

[0016] The first natural language understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

[0017] In this way, the server marks the first natural language understanding result according to the second preset mark to obtain the target marked speech understanding result. In this way, by marking the natural language understanding result with the second preset mark, the large language model can optimize the natural language understanding process, ignore the natural language understanding results marked with the second preset mark, reduce the accumulation of errors in the dialogue process, and thus improve the overall understanding accuracy. In addition, the amount of information that needs to be considered can be reduced so that the system can process subsequent inputs more quickly, thereby improving the overall response speed.

[0018] In some implementations, performing natural language understanding on the target marked speech understanding result to obtain the first natural language understanding result includes:

[0019] Performing slot recognition on the target mark speech recognition result to obtain a slot recognition result;

[0020] Performing application program interface prediction on the target mark speech recognition result to obtain a predicted application interface;

[0021] According to the slot identification result and the predicted application interface, the predicted application interface is selected to perform application program interface parameter filling to obtain the first natural language understanding result.

[0022] In this way, the server performs slot recognition on the target tag speech recognition result to obtain a slot recognition result. Next, the server performs application program interface prediction on the target tag speech recognition result to obtain a predicted application interface. Finally, the server selects the predicted application interface to perform application program interface parameter filling based on the slot recognition result and the predicted application interface to obtain a first natural language understanding result. In this way, through slot recognition and application program interface prediction, the system can understand the user's request more accurately and select the most appropriate application interface to perform the task, thereby reducing misunderstandings and errors and providing a good user experience.

[0023] In some implementations, performing natural language understanding on the target marked speech understanding result to obtain the first natural language understanding result includes:

[0024] Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech understanding result to obtain the first natural language understanding result.

[0025] In this way, based on the pre-trained large language model, the server performs natural language understanding on the target tag speech recognition result to obtain a first natural language understanding result. In this way, by performing natural language understanding on the target tag speech recognition result through the pre-trained large language model, the accuracy and efficiency of natural language understanding can be improved.

[0026] In some embodiments, the performing natural language understanding on the target labeled speech understanding result based on the pre-trained large language model to obtain the first natural language understanding result includes:

[0027] Based on the pre-trained large language model, a predicted application interface and an application interface parameter filling result are obtained to obtain the first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface.

[0028] In this way, based on the pre-trained large language model, the server obtains the predicted application interface and the application interface parameter filling result to obtain the first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface. In this way, through the entity recognition technology, slot filling technology and application interface prediction technology provided by the large language model, the large language model recognizes and extracts the parameters in the target tag speech recognition result and fills them into the application interface parameters, so as to better understand the context of the conversation and provide a more coherent and relevant response.

[0029] In some embodiments, completing the voice interaction according to the first natural language recognition result, the second natural language recognition result, and the first natural language understanding result includes:

[0030] Performing matching processing on the first natural language recognition result and the second natural language recognition result;

[0031] When the first natural language recognition result matches the second natural language, the first natural language understanding result is sent to the vehicle to complete the voice interaction.

[0032] In this way, the server matches the first natural language recognition result with the second natural language recognition result. Then, when the first natural language recognition result and the second natural language match, the server sends the first natural language understanding result to the vehicle to complete the voice interaction. In this way, by matching the first natural language recognition result and the second natural language recognition result, it is ensured that the sent natural language understanding result is correct, thereby reducing the risk of misrecognition and improving the reliability of the system.

[0033] In some embodiments, completing the voice interaction according to the first natural language recognition result, the second natural language recognition result, and the first natural language understanding result includes:

[0034] When the first natural language recognition result and the second natural language recognition result do not match, performing natural language understanding on the second natural language recognition result to obtain a second natural language understanding result;

[0035] The second natural language understanding result is sent to the vehicle to complete the voice interaction.

[0036] In this way, when the first natural language recognition result and the second natural language recognition result do not match, the server performs natural language understanding on the second natural language recognition result to obtain the second natural language understanding result. Then, the server sends the second natural language understanding result to the vehicle to complete the voice interaction. In this way, when the natural language recognition results do not match, the server performs natural language understanding on the voice request, and the system can better handle errors and uncertainties, provide accurate responses, and thus improve the user experience.

[0037] An embodiment of the present application provides a server, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0038] An embodiment of the present application provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0039] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 It is one of the flowcharts of the voice interaction method of certain embodiments of the present application;

[0042] Figure 2 It is one of the flowcharts of processing voice requests in certain embodiments of the present application;

[0043] Figure 3 This is a second schematic diagram of a flow chart of processing a voice request in certain embodiments of the present application;

[0044] Figure 4 This is the second flow chart of the voice interaction method of certain implementation modes of the present application;

[0045] Figure 5 This is the third flow chart of the voice interaction method of certain implementation modes of the present application;

[0046] Figure 6This is a fourth flowchart of a voice interaction method according to certain embodiments of the present application;

[0047] Figure 7 This is a fifth flow chart of a voice interaction method according to certain embodiments of the present application;

[0048] Figure 8 This is the sixth flow chart of the voice interaction method of certain implementation modes of the present application;

[0049] Fig. 9 This is the seventh flow chart of the voice interaction method of certain implementation modes of the present application;

[0050] Fig.10 is a flowchart of a matching process in certain embodiments of the present application;

[0051] Fig.11 This is the eighth flow chart of the voice interaction method of certain embodiments of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0053] In related in-vehicle intelligent system technologies, the traditional voice interaction process is usually based on a complete voice request. This means that the system needs to first complete Voice Activity Detection (VAD) to determine the start and end of voice input. Once VAD detects the end of voice activity, the system will start the Automatic Speech Recognition (ASR) process to convert the voice signal into text. Subsequently, the system performs Natural Language Understanding (NLU) to parse the user's intentions.

[0054] However, there are some problems with this process. First, because ASR can only start after the entire voice request is completed, the overall processing time is long. This delay may affect the user experience, especially in driving environments that require quick responses. Second, this sequential processing method may cause users to feel unnatural or interrupted during the waiting process, affecting the smoothness of voice interaction.

[0055] Voice activity detection refers to a technique used to determine the presence of a speech signal. In speech processing and communication systems, VAD is used to identify the beginning and end of speech activity so that only the portions containing speech can be processed, ignoring periods of silence or background noise.

[0056] Based on the above questions, please refer to Figure 1 , the present application embodiment provides a voice interaction method, the method comprising:

[0057] 01: Receive the voice request segments forwarded by the vehicle in sequence;

[0058] 02: Perform natural language recognition on all currently received voice request segments to obtain a first natural language recognition result;

[0059] 03: Obtain a first natural language understanding result according to the first natural language recognition result;

[0060] 04: Perform natural language recognition on the received voice request to obtain a second natural language recognition result;

[0061] 05: Complete voice interaction based on the first natural language recognition result, the second natural language recognition result and the first natural language understanding result.

[0062] The embodiment of the present application also provides a server, including a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive voice request fragments forwarded sequentially by the vehicle. And natural language recognition is performed on all currently received voice request fragments to obtain a first natural language recognition result. And based on the first natural language recognition result, a first natural language understanding result is obtained. The processor is also used to perform natural language recognition on the received voice request to obtain a second natural language recognition result, and the voice request is obtained by splicing all voice request fragments. And according to the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request, the voice interaction is completed.

[0063] The embodiment of the present application also provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a language recognition module, a language understanding module and a processing module. The receiving module is used to receive voice request fragments forwarded sequentially by the vehicle. The language recognition module is used to perform natural language recognition on all currently received voice request fragments to obtain a first natural language recognition result. The language understanding module is used to obtain a first natural language understanding result based on the first natural language recognition result. The language recognition module is used to perform natural language recognition on the received voice request to obtain a second natural language recognition result, and the voice request is obtained by splicing all voice request fragments. The processing module is used to complete the voice interaction based on the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request.

[0064] Specifically, see Figure 2 In the related art, the process of human-computer interaction can be divided into five stages, including receiving user voice request, voice activity detection, automatic speech recognition, natural language understanding (NLU) and robot understanding instructions. Among them, the time required for receiving user voice request may be A1, the time required for voice activity detection may be A2, the time required for automatic speech recognition may be A3, the time required for natural language understanding may be A4 and the time required for robot understanding instructions may be A5. The time for the server to complete this human-computer interaction is A1+A2+A3+A4+A5.

[0065] Robot understanding instructions refers to the large language model receiving instructions (natural language understanding results). The instruction may be a piece of program code, and the robot that interacts with humans needs to understand this program code, which takes a certain amount of time.

[0066] Compared with the method in the related art of performing natural language processing on the complete voice request after receiving the complete voice request, the voice interaction method provided in the embodiment of the present application processes the voice request fragments in real time during the process of receiving the user's voice request, which will have a faster response speed and a better interactive experience. Figure 2As shown, it can be found that when the server receives the first voice request segment, it will perform natural speech recognition processing and natural language understanding processing. When the voice request is received, the natural language recognition processing and natural language understanding processing of each voice request segment are also completed. Subsequently, the server performs sound activity detection, and performs natural language recognition processing and result selection processing on the complete voice request. In this way, the time required for the server to receive the user's voice request is also A1, the time required for sound activity detection is also A2, the time required for natural language recognition processing on the complete voice request is also A3, and the time required for result selection processing may be A6. The time for the server to complete this human-computer interaction is A1+A2+A3+A6, and the response speed is faster.

[0067] Result selection processing refers to selecting the natural language understanding results obtained according to the voice request fragments through matching and screening processing, selecting the appropriate natural language understanding results and sending them to the vehicle to complete the voice interaction. At the same time, since the program code of each natural language understanding result obtained according to different voice request fragments has been understood by the robot, the response time is reduced.

[0068] A voice request segment refers to a voice segment obtained by intercepting the received voice request based on a certain interval when the server first receives the voice request forwarded by the vehicle. For example, if the voice request sent by the user is "open the car window", the server may intercept two voice request segments at different times. The voice request intercepted at the first time is "open", and the voice request intercepted at the second time is "car window".

[0069] All currently received voice request segments refer to the natural language recognition performed at a certain moment, when natural language recognition is performed at that moment and before that moment. For example, the voice request sent by the user is "open the car window", and the server may intercept two voice request segments at different times. The voice request intercepted at the first moment is "open", and the voice request intercepted at the second moment is "car window". When natural language recognition is performed at the second moment, it is necessary to perform natural language recognition processing on the "open" received at the first moment and the "car window" received at the second moment, that is, perform natural language recognition processing on "open the car window".

[0070] The received voice request refers to the situation where no voice request fragments forwarded by the vehicle are received within a certain period of time. The server determines that the user has completed his or her voice request and splices all the voice request fragments received at this time in the order in which they were received to obtain a spliced ​​body of voice request fragments.

[0071] See also Figure 3, the server can receive the voice request segments forwarded by the vehicle in sequence, that is, process the voice stream in real time, instead of waiting for the entire voice request to be completed. In other words, if the user's voice request is "help me open the window immediately", the server will receive multiple voice request segments, the voice request segment received at time T0 is "immediately", the voice request segment received at time T1 is "help me", the voice request segment received at time T2 is "open", and the voice request segment received at time T3 is "window".

[0072] Next, the server performs natural language recognition on all the voice request segments currently received to obtain a first natural language recognition result. That is to say, the first natural language recognition result can indicate multiple natural language recognition results, indicating different natural language recognition results at different times. Continuing with the above example, the voice request that should be processed at time T0 is "immediately", the voice request that should be processed at time T1 is "help me immediately", the voice request that should be processed at time T2 is "help me open it immediately", and the voice request that should be processed at time T3 is "help me open the window immediately". Of course, the natural language recognition results recognized at different times may not be completely correct, but the server can continuously correct the natural language recognition results of voice requests. For example, at time T0, the server may recognize the natural language recognition result as "Like", but at time T1, based on the most recently received voice request segment "help me", the natural language recognition result is updated to "help me immediately".

[0073] Then, the server obtains the first natural language understanding result based on the first natural language recognition result. That is to say, the natural language recognition result obtained at each moment will obtain a natural language understanding result. Continuing with the above example, at time T0, the natural language recognition result is "immediately", and the natural language understanding result is "Unclear". At time T3, the natural language recognition result is "Help me open the window immediately", and the natural language understanding result is "AcOpen{device: window}".

[0074] After a certain period of time, if no voice request fragments forwarded by the vehicle are received, the server determines that the user has completed his voice request. The server will perform natural language recognition on the received voice request and obtain a second natural language recognition result. In other words, there is only one second natural language recognition result, which is obtained by performing natural language recognition processing on the final voice request. Continuing with the above example, the second natural language recognition result is "Help me open the window immediately."

[0075] Finally, the server completes the voice interaction based on the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request.

[0076] In summary, in the voice interaction method and server provided in the embodiment of the present application, the server receives the voice request segments forwarded by the vehicle in sequence. Then, the server performs natural language recognition on all the voice request segments currently received to obtain a first natural language recognition result. Then, the server obtains a first natural language understanding result based on the first natural language recognition result. The server then performs natural language recognition on the received voice request to obtain a second natural language recognition result, and the voice request is obtained by splicing all the voice request segments. Finally, the server completes the voice interaction based on the first natural language recognition result, the second natural language recognition result, the first natural language understanding result and the voice request. In this way, in the end-to-end architecture, the streaming natural language recognition and streaming natural language understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, reducing the resources and time required to wait for the entire voice request to be completed, thereby greatly shortening the response time of the server. In addition, through the phased natural language recognition and natural language understanding, the large language model can more accurately understand the user's intentions, thereby improving the efficiency and accuracy of voice interaction.

[0077] See also Figure 4 In some implementations, step 03 (obtaining a first natural language understanding result based on the first natural language recognition result) includes:

[0078] 031: Marking the first natural language recognition result according to the first preset mark to obtain a target marked speech recognition result;

[0079] 032: Perform natural language understanding on the target labeled speech recognition result to obtain a first natural language understanding result.

[0080] In some embodiments, the processing module is further used to mark the first natural language recognition result according to the first preset mark to obtain the target marked speech recognition result, and perform natural language understanding on the target marked speech recognition result to obtain the first natural language understanding result.

[0081] In some embodiments, the processor is further configured to mark the first natural language recognition result according to the first preset mark to obtain a target marked speech recognition result, and perform natural language understanding on the target marked speech recognition result to obtain a first natural language understanding result.

[0082] Specifically, the first preset mark refers to a mark "wait" added before the natural language recognition result before the big language model performs natural language understanding processing on the natural language recognition result. This mark "wait" can guide the big language model to perform natural language understanding processing mode, that is, inform the big language model that the next text can be processed by natural language understanding.

[0083] The server performs tagging processing on the first natural language recognition result according to the first preset tag to obtain a target tag speech recognition result. Continuing with the above example, for example, at time T0, the target tag language recognition result obtained by the server is "wait immediately".

[0084] Next, the server performs natural language understanding on the target mark speech recognition result to obtain a first natural language understanding result. Continuing with the above example, at time T0, the server performs natural language understanding processing on "wait immediately" to obtain a natural language understanding result.

[0085] In this way, by marking the first natural language recognition result with the first preset tag, the large language model can accurately recognize the input natural language recognition result, accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0086] See also Figure 5 In some embodiments, the method further comprises:

[0087] 033: According to the second preset mark, the first natural language understanding result is marked to obtain a target marked speech understanding result.

[0088] In some embodiments, the processing module is further used to mark the first natural language understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0089] In some embodiments, the processor is further configured to mark the first natural language understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0090] Specifically, the second preset tag refers to a tag waitEnd that will be added after the first natural language understanding result is obtained after the large language model performs natural language understanding processing on the target tag speech recognition result. This tag waitEnd can guide the large language model to perform natural language understanding processing without being affected by the natural language understanding results obtained at the previous moment. In other words, the server will only perform natural language understanding processing based on the obtained natural language recognition results. Continuing with the above example, for example, at time T1, the large language model will receive the voice request fragments "immediately" and "help me", and the natural language understanding result "wait immediately waitEnd" obtained at time T0. Due to the second preset tag waitEnd, the server will not perform natural language understanding processing on the natural language understanding result "wait immediately waitEnd".

[0091] The server performs labeling processing on the first natural language understanding result according to the second preset label to obtain a target labeled speech understanding result.

[0092] In this way, by marking the natural language understanding results with the second preset mark, the large language model can optimize the natural language understanding process, ignore the natural language understanding results marked with the second preset mark, reduce the accumulation of errors in the conversation process, and thus improve the overall understanding accuracy. In addition, the amount of information that needs to be considered can be reduced so that the system can process subsequent inputs more quickly, thereby improving the overall response speed.

[0093] See also Figure 6 In some implementations, step 032 (performing natural language understanding on the target marked speech understanding result to obtain a first natural language understanding result) includes:

[0094] 0321: Perform slot recognition on the target mark speech recognition result to obtain the slot recognition result;

[0095] 0322: Predict the application program interface of the target labeled speech recognition result to obtain the predicted application interface;

[0096] 0323: According to the slot recognition result and the prediction application interface, the prediction application interface is selected to execute the application program interface parameter filling to obtain the first natural language understanding result.

[0097] In some embodiments, the processing module is further configured to perform slot recognition on the target mark speech recognition result to obtain a slot recognition result. The processing module is further configured to perform application program interface prediction on the target mark speech recognition result to obtain a predicted application interface. The processing module is further configured to perform application program interface parameter filling based on the slot recognition result and the predicted application interface, and select the predicted application interface to obtain a first natural language understanding result.

[0098] In some embodiments, the processor is further configured to perform slot recognition on the target mark speech recognition result to obtain a slot recognition result, and perform application program interface prediction on the target mark speech recognition result to obtain a predicted application interface, and select the predicted application interface to perform application program interface parameter filling according to the slot recognition result and the predicted application interface to obtain a first natural language understanding result.

[0099] Specifically, slot recognition refers to extracting specific information fragments from the user's input, which are usually called "slots". Slots are usually key information necessary to complete a task or request, such as time, place, object, etc. Taking the user's voice request "What will the temperature be tomorrow" as an example, the slot information that can be obtained through slot recognition includes ["tomorrow" - date (Date)], that is, the slot information includes slot value and slot type, where "tomorrow" is the slot value and date (Date) is the slot type. Taking the user's voice request "Navigate to Zhongguancun" as an example, the slot information that can be obtained through slot recognition is ["Zhongguancun" - place name (Place)], where "Zhongguancun" is the slot value and the place name (Place) is the slot type.

[0100] The slot recognition result refers to the named entity obtained by performing slot recognition on the voice request, such as the slot information ["tomorrow" - date (Date)] and slot information ["Zhongguancun" - place name (Place)] mentioned above.

[0101] The server performs slot recognition on the target tag speech recognition result to obtain a slot recognition result. Continuing with the above example, at time T3, the server performs slot recognition on "help me open the car window immediately" and obtains a slot recognition result "car window - device".

[0102] Next, the server performs an API prediction on the target tag speech recognition result to obtain a predicted API. Continuing with the above example, at time T3, the server performs an API prediction on "Help me open the window immediately" and obtains a slot recognition result "AcOpen".

[0103] Finally, the server selects the predicted application interface to execute the application program interface parameter filling according to the slot recognition result and the predicted application interface, and obtains the first natural language understanding result.

[0104] It should be noted that the above processing will be performed for all voice request segments received at each time. If no slot recognition result is identified and / or no prediction application interface is obtained, the large language model will output "Unclear".

[0105] In this way, through slot identification and API prediction, the system can understand the user's request more accurately and select the most appropriate API to perform the task, thereby reducing misunderstandings and errors and providing a good user experience.

[0106] See also Figure 7 In some implementations, step 032 (performing natural language understanding on the target marked speech understanding result to obtain a first natural language understanding result) includes:

[0107] 0324: Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech understanding result to obtain the first natural language understanding result.

[0108] In some embodiments, the processing module is further used to perform natural language understanding on the target labeled speech understanding result based on the pre-trained large language model to obtain a first natural language understanding result.

[0109] In some embodiments, the processor is further used to perform natural language understanding on the target labeled speech understanding result based on the pre-trained large language model to obtain a first natural language understanding result.

[0110] Specifically, in some embodiments, the large language model has been trained. In some embodiments, the training steps of the large language model are as follows: construct training data based on the collected voice request samples and the vehicle system function customization solution samples constructed according to the voice request samples. The sample data can come from the collected user logs, such as the user's history of successfully constructing the vehicle system function customization solution according to the voice request. Of course, the training data can also be constructed as needed. For example, a set of training data can be: a voice request "open the car window immediately", and the vehicle system function customization solution constructed according to the voice request is "AcOpen{device:car window}".

[0111] Furthermore, in order to guide the large model to meet the task requirements, the prompt information template needs to be configured. Specifically, the prompt information template is used to guide the model to generate output, providing the background of the problem and relevant guidance. It usually includes a description and instructions of the problem to tell the model how to process the input data. The prompt information template can provide additional auxiliary information to facilitate the model to better understand and generate output. It can also include a specific format, style, theme that the model is required to follow, or instruct the model to pay attention to certain specific aspects and details. As a result, the model will understand the requirements of the task based on the content of the prompt information template, thereby generating answers and texts that meet expectations. Generally, the configuration process of the prompt information template may include clarifying the role that the model needs to play (optional), clarifying the input text content and structure, and clarifying the output text content and structure.

[0112] In an example, the content of the prompt message template is as follows:

[0113] "Suppose you are an intelligent cockpit dialogue system. You need to understand the recognized commands and output the corresponding API and Arguments. The input command fragments will be input multiple times. Please determine the current API and Argumengts based on the existing commands."

[0114] Based on the pre-trained large language model, the server performs natural language understanding on the target labeled speech recognition result to obtain a first natural language understanding result.

[0115] In this way, the accuracy and efficiency of natural language understanding can be improved by performing natural language understanding on the target labeled speech recognition results through a pre-trained large language model.

[0116] See also Figure 8 In some embodiments, step 0324 (based on the pre-trained large language model, performing natural language understanding on the target labeled speech understanding result to obtain a first natural language understanding result) includes:

[0117] 03241: Based on the pre-trained large language model, a predicted application interface and an application interface parameter filling result are obtained to obtain a first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface.

[0118] In certain embodiments, the processing module is used to obtain a predicted application interface and an application interface parameter filling result based on a pre-trained large language model to obtain a first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface.

[0119] In certain embodiments, the processor is further used to obtain a predicted application interface and an application interface parameter filling result based on a pre-trained large language model to obtain a first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface.

[0120] Specifically, a pre-trained large language model is used to perform entity recognition processing, slot filling processing and application interface prediction processing on the target labeled speech understanding result to obtain the predicted application interface and application program interface parameter filling results to obtain the first natural language understanding result.

[0121] In this way, through the entity recognition technology, slot filling technology and application interface prediction technology provided by the large language model, the large language model recognizes and extracts the parameters in the target tag speech recognition results and fills them into the parameters of the application interface, so that it can better understand the context of the conversation and provide more coherent and relevant responses.

[0122] See also Fig. 9 In some embodiments, step 05 (completing voice interaction according to the first natural language recognition result, the second natural language recognition result and the first natural language understanding result) includes:

[0123] 051: Perform matching processing on the first natural language recognition result and the second natural language recognition result;

[0124] 052: When the first natural language recognition result matches the second natural language, the first natural language understanding result is sent to the vehicle to complete the voice interaction.

[0125] In some embodiments, the processing module is used to match the first natural language recognition result with the second natural language recognition result, and when the first natural language recognition result and the second natural language recognition result match, the first natural language understanding result is sent to the vehicle to complete the voice interaction.

[0126] In some embodiments, the processor is further configured to match the first natural language recognition result with the second natural language recognition result, and when the first natural language recognition result and the second natural language recognition result match, the first natural language understanding result is sent to the vehicle to complete the voice interaction.

[0127] Specifically, the matching process refers to comparing the first natural language recognition result and the second natural language recognition result based on text alignment, semantic analysis and other technologies to verify the consistency of the first natural language recognition result and the second natural language recognition result.

[0128] See also Fig.10 , the server matches the first natural language recognition result with the second natural language recognition result. Continuing with the above example, at time T3, the large language model does not receive a new voice segment within a certain period of time, confirming that the user's voice request has been received. Subsequently, the voice request is processed by natural language recognition to obtain the second natural language recognition result "Help me open the window immediately".

[0129] Subsequently, the second natural language recognition result "Help me open the car window immediately" is matched with the first natural language recognition result "Help me open the car window immediately" at the previous time T3 (that is, the time when the large language model determines that the user's voice request has been received).

[0130] Then, if the first natural language recognition result and the second natural language match, the server sends the first natural language understanding result to the vehicle to complete the voice interaction. Continuing with the above example, it can be found that the second natural language recognition result "Help me open the window immediately" is consistent with the first natural language recognition result "Help me open the window immediately" at the previous time T3, then the first natural language understanding result is considered to be the answer that the large language model needs to output, and the first natural language understanding result is sent to the vehicle to complete the voice interaction.

[0131] In this way, by matching the first natural language recognition result and the second natural language recognition result, it is ensured that the natural language understanding result sent down is correct, thereby reducing the risk of misrecognition and improving the reliability of the system.

[0132] See also Fig.11 In some embodiments, step 05 (completing voice interaction according to the first natural language recognition result, the second natural language recognition result and the first natural language understanding result) includes:

[0133] 053: When the first natural language recognition result and the second natural language recognition result do not match, performing natural language understanding on the second natural language recognition result to obtain a second natural language understanding result;

[0134] 054: Send the second natural language understanding result to the vehicle to complete the voice interaction.

[0135] In some embodiments, the processing module is used to perform natural language understanding on the second natural language recognition result to obtain a second natural language understanding result when the first natural language recognition result and the second natural language recognition result do not match. The processing module is used to send the second natural language understanding result to the vehicle to complete the voice interaction.

[0136] In some embodiments, the processor is further configured to perform natural language understanding on the second natural language recognition result to obtain a second natural language understanding result when the first natural language recognition result and the second natural language recognition result do not match, and send the second natural language understanding result to the vehicle to complete the voice interaction.

[0137] For details, please refer to Fig.10 , when the first natural language recognition result and the second natural language recognition result do not match, the server performs natural language understanding on the second natural language recognition result to obtain a second natural language understanding result. Continuing with the above example, if the first natural language recognition result obtained after natural language recognition processing of all the voice segments received at time T3 is "Lick, help me open the cabin", then it is inconsistent with the second natural language recognition result identified as "Help me open the window immediately", then the large language model will perform natural language understanding on the second natural language recognition result to obtain a second natural language understanding result. Subsequently, the second natural language understanding result is sent to the vehicle to complete the voice interaction.

[0138] In this way, when the natural language recognition results do not match, the server can better handle errors and uncertainties and provide accurate responses through natural language understanding of the voice request, thereby improving the user experience.

[0139] The present application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the one or more processors execute the method of the present application.

[0140] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution medium.

[0141] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0142] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0143] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: The method comprises: Receive voice request segments forwarded sequentially by the vehicle; Performing natural language recognition on all the currently received voice request segments to obtain a first natural language recognition result; Obtaining a first natural language understanding result according to the first natural language recognition result; Performing natural language recognition on the received voice request to obtain a second natural language recognition result, wherein the voice request is obtained by concatenating all the voice request segments; The voice interaction is completed according to the first natural language recognition result, the second natural language recognition result and the first natural language understanding result.

2. The voice interaction method according to claim 1, characterized in that: Obtaining a first natural language understanding result according to the first natural language recognition result includes: According to a first preset mark, the first natural language recognition result is marked to obtain a target marked speech recognition result; Natural language understanding is performed on the target mark speech recognition result to obtain the first natural language understanding result.

3. The voice interaction method according to claim 2, characterized in that: The method further comprises: The first natural language understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

4. The voice interaction method according to claim 2, characterized in that: The performing natural language understanding on the target mark speech recognition result to obtain the first natural language understanding result includes: Performing slot recognition on the target mark speech recognition result to obtain a slot recognition result; Performing application program interface prediction on the target mark speech recognition result to obtain a predicted application interface; According to the slot identification result and the predicted application interface, the predicted application interface is selected to perform application program interface parameter filling to obtain the first natural language understanding result.

5. The voice interaction method according to claim 2, characterized in that: The performing natural language understanding on the target mark speech recognition result to obtain the first natural language understanding result includes: Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech recognition result to obtain the first natural language understanding result.

6. The voice interaction method according to claim 5, characterized in that: The step of performing natural language understanding on the target labeled speech recognition result based on the pre-trained large language model to obtain the first natural language understanding result includes: Based on the pre-trained large language model, a predicted application interface and an application interface parameter filling result are obtained to obtain the first natural language understanding result, wherein the application interface parameter filling result is obtained based on the predicted application interface.

7. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the first natural language recognition result, the second natural language recognition result and the first natural language understanding result includes: Performing matching processing on the first natural language recognition result and the second natural language recognition result; When the first natural language recognition result matches the second natural language, the first natural language understanding result is sent to the vehicle to complete the voice interaction.

8. The voice interaction method according to claim 7, characterized in that: The completing the voice interaction according to the first natural language recognition result, the second natural language recognition result and the first natural language understanding result includes: When the first natural language recognition result and the second natural language recognition result do not match, performing natural language understanding on the second natural language recognition result to obtain a second natural language understanding result; The second natural language understanding result is sent to the vehicle to complete the voice interaction.

9. A server, characterized in that: The server includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the voice interaction method described in any one of claims 1 to 8 is implemented.

10. A non-volatile computer-readable storage medium containing a computer program, characterized in that: When the computer program is executed by one or more processors, the voice interaction method described in any one of claims 1 to 8 is implemented.

Images