Voice interaction method, server and computer readable storage medium

Abstract

The invention discloses a voice interaction method. The method comprises the following steps: receiving voice request fragments forwarded by a vehicle in sequence; based on a pre-configured external knowledge base, natural language processing is carried out on the first voice request fragment set, a first recognition result and a first understanding result are obtained, and the first voice request fragment set comprises voice request fragments received at the first moment. Based on the external knowledge base, natural language processing is carried out on a second voice request fragment set, a second recognition result and a second understanding result are obtained, and the second voice request fragment set comprises voice request fragments which are received at a second moment and follow the first voice request fragment set. And performing natural language recognition on the received voice request to obtain a third recognition result. And completing voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, the accuracy of the system in processing the composite sentences including the specific domain terms is improved.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a voice interaction method, a server, and a computer-readable storage medium. Background Art

[0002] In the prior art, in-vehicle intelligent systems can perform streaming natural language processing on user voice requests. However, when user voice requests contain business-specific knowledge or domain-specific terminology, the in-vehicle intelligent system cannot process the natural language of the user voice request, reducing the system's reasoning performance and affecting the user experience. Summary of the Invention

[0003] The present application provides a voice interaction method, a server, and a computer-readable storage medium.

[0004] The present application provides a voice interaction method, the method comprising:

[0005] receiving voice request segments forwarded sequentially by the vehicle, the voice request segments including at least a first voice request segment and a second voice request segment;

[0006] performing natural language processing on a first set of voice request segments based on a preconfigured external knowledge base to obtain a first recognition result and a first understanding result, the first set of voice request segments including the first voice request segments received within a first moment;

[0007] performing natural language processing on a second set of voice request segments based on the external knowledge base to obtain a second recognition result and a second understanding result, wherein the second set of voice request segments includes the second voice request segments received at a second time point subsequent to the first set of voice request segments, the second time point being later than the first time point;

[0008] performing natural language recognition on the received voice request to obtain a third recognition result, wherein the voice request is obtained by concatenating the voice request segments;

[0009] The voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0010] In this way, the server receives the voice request segments forwarded sequentially by the vehicle, and the voice request segments include at least the first voice request segment and the second voice request segment. Then, based on the pre-configured external knowledge base, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result. The first voice request segment set includes the first voice request segment received within the first moment. Then, based on the external knowledge base, the server performs natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result. The second voice request segment set includes the second voice request segment received within the second moment and subsequent to the first voice request segment set, and the second moment is later than the first moment. The server then performs natural language recognition on the received voice request to obtain a third recognition result. The voice request is obtained by splicing the voice request segments. Finally, the server completes the voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, in an end-to-end architecture, the large language model is combined with streaming conversation technology to achieve streaming natural language recognition and understanding capabilities. This reduces the resources and time required to complete the entire voice request, significantly shortening server response time. Furthermore, based on an external knowledge base, the large language model can acquire required domain knowledge in real time. Through phased natural language processing of different voice request segments, it accurately understands the user's intent. By combining the natural language understanding results from different phases, the system improves its accuracy when processing long or complex sentences containing business-specific knowledge or domain-specific terminology.

[0011] In some embodiments, performing natural language processing on the first voice request segment set based on a preconfigured external knowledge base to obtain a first recognition result and a first understanding result includes:

[0012] performing natural language recognition on the first voice request segment set to obtain a first recognition result;

[0013] Marking the first recognition result according to the first preset mark to obtain a target marked speech recognition result;

[0014] Based on the pre-trained large language model, natural language understanding is performed on the target markup speech recognition result according to the external knowledge base to obtain the first understanding result.

[0015] In this way, the server performs natural language recognition on the first set of voice request segments to obtain a first recognition result. Next, the server tags the first recognition result according to the first preset tag to obtain a target-tagged voice recognition result. Finally, based on the pre-trained large language model, the server performs natural language understanding on the target-tagged voice recognition result according to the external knowledge base to obtain a first understanding result. In this way, by tagging the first natural language recognition result with the first preset tag, the large language model can accurately identify the input natural language recognition result and accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0016] In some embodiments, performing natural language understanding on the target mark speech recognition result according to the external knowledge base to obtain the first understanding result includes:

[0017] Acquiring target external knowledge associated with the target mark speech recognition result from the external knowledge base according to the target mark speech recognition result;

[0018] The target external knowledge and the target mark speech recognition result are processed by natural language understanding to obtain the first understanding result.

[0019] In this way, the server retrieves target external knowledge associated with the target-tagged speech recognition result from an external knowledge base based on the target-tagged speech recognition result. The server then performs natural language understanding on the target external knowledge and the target-tagged speech recognition result to obtain a first understanding result. This allows the server to more accurately parse the target-tagged speech recognition result when processing voice requests containing business-specific knowledge or domain-specific terminology, by combining the external knowledge base with natural language understanding technology. This improves system intelligence and enhances the user experience.

[0020] In certain embodiments, the method further comprises:

[0021] The first understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

[0022] In this way, the server tags the first understanding result according to the second preset tag, obtaining the target-tagged speech understanding result. By tagging the natural language understanding results with the second preset tag, the large language model can optimize the natural language understanding process, ignoring natural language understanding results already tagged with the second preset tag, reducing the accumulation of errors during the conversation and thus improving overall understanding accuracy. Furthermore, the amount of information required for consideration is reduced, allowing the system to process subsequent input more quickly, thereby improving overall response speed.

[0023] In some embodiments, completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request includes:

[0024] Concatenating the first recognition result and the second recognition result to obtain a concatenated recognition result;

[0025] performing matching processing on the splicing recognition result and the third recognition result;

[0026] The voice interaction is completed according to the result of the matching process.

[0027] In this way, the server concatenates the first and second recognition results to obtain a concatenated recognition result. Next, the server matches the concatenated recognition result with the third recognition result. Finally, the server completes the voice interaction based on the matching results. In this way, by concatenating the multiple recognition results obtained during the voice request process and matching them with the third recognition result, the server can accurately understand the user's complex voice request and effectively execute the corresponding operation.

[0028] In some embodiments, completing the voice interaction according to the result of the matching process includes:

[0029] When the splicing recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result;

[0030] Sending the target output result to the vehicle to complete the voice interaction; or

[0031] If the concatenated recognition result and the third recognition result do not match, performing natural language understanding on the voice request to obtain the target output result;

[0032] The target output result is sent to the vehicle to complete the voice interaction.

[0033] In this way, when the spliced ​​recognition result and the third recognition result match, the server obtains the target output result based on the first understanding result and the second understanding result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Or, when the spliced ​​recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain the target output result. Then, the server sends the target output result to the vehicle to complete the voice interaction. In this way, when processing complex voice commands, by flexibly handling the matching of the spliced ​​recognition result and the third recognition result, the system can more accurately understand and respond to the user's voice commands, and can also make effective corrections when there is a deviation in understanding, which helps to improve the overall performance and user experience of the voice interaction system.

[0034] In some embodiments, obtaining a target output result based on the first understanding result and the second understanding result includes:

[0035] Filtering the first understanding result to obtain a first sub-goal output result, where the first sub-goal output result includes a first application interface and a first application program interface parameter filling result;

[0036] Filtering the second understanding result to obtain a second sub-goal output result, where the second sub-goal output result includes a second application interface and a second application program interface parameter filling result;

[0037] The target output result is obtained according to the first sub-target output result and the second sub-target output result.

[0038] In this way, the server filters and processes the first understanding result to obtain the first sub-target output result, which includes the first application interface and the first application interface parameter filling result. Next, the server filters and processes the second understanding result to obtain the second sub-target output result, which includes the second application interface and the second application interface parameter filling result. Finally, the server obtains the target output result based on the first sub-target output result and the second sub-target output result. In this way, when processing complex voice requests, by filtering and processing the understanding results in steps, the system can more accurately extract and integrate natural language understanding results, thereby generating more accurate target output results, which helps to improve the understanding and response quality of the voice interaction system.

[0039] In certain embodiments, the large language model is trained by the following steps:

[0040] The original large language model is trained based on the pre-built false call dataset to obtain an initial large language model;

[0041] Fine-tuning and optimizing the initial large language model to obtain the large language model.

[0042] First, the original large language model is trained using a pre-built false call dataset to obtain an initial large language model. Next, the initial large language model is fine-tuned and optimized to obtain a large language model. By constructing a false call dataset and performing fine-tuning and optimization, the false calls of the large language model when processing long or complex sentences containing business-specific knowledge or domain-specific terminology can be reduced, thereby improving the reliability of the large language model and the user experience.

[0043] An embodiment of the present application provides a server, which includes a processor and a memory. A computer program is stored in the memory. When the computer program is executed by the processor, the above method is implemented.

[0044] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the above method are implemented.

[0045] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 This is one of the flow charts of the voice interaction method of certain embodiments of the present application;

[0048] Figure 2 This is one of the flowcharts of processing a voice request in certain embodiments of the present application;

[0049] Figure 3 This is a second schematic diagram of a flow chart for processing a voice request in certain embodiments of the present application;

[0050] Figure 4 This is the second flow chart of the voice interaction method of certain embodiments of the present application;

[0051] Figure 5 This is the third flow chart of the voice interaction method of certain embodiments of the present application;

[0052] Figure 6 This is a fourth flow chart of a voice interaction method according to certain embodiments of the present application;

[0053] Figure 7This is the fifth flow chart of the voice interaction method of certain embodiments of the present application;

[0054] Figure 8 is a flowchart of a matching process in certain embodiments of the present application;

[0055] Figure 9 This is the seventh flowchart of the voice interaction method of certain embodiments of the present application;

[0056] Figure 10 This is the eighth flow chart of the voice interaction method of certain embodiments of the present application;

[0057] Figure 11 This is a ninth flowchart of a voice interaction method according to certain embodiments of the present application;

[0058] Figure 12 It is a schematic diagram of the training process of a large language model in certain embodiments of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and should not be understood as limiting the embodiments of the present application.

[0060] In related technologies, to improve response speed and user experience, in-vehicle intelligent systems can perform real-time natural language processing (NLP) while receiving user voice requests, namely during voice activity detection (VAD). This allows the in-vehicle intelligent system to begin understanding and responding to user requests before the user's voice command is fully completed, thereby improving the real-time and natural nature of the interaction.

[0061] However, when a user's voice request involves specialized business knowledge or domain-specific terminology, such as car repair, geographic information, or detailed information about a specific brand, the in-vehicle intelligent system may not accurately understand the user's intent. Because the in-vehicle intelligent system cannot acquire and accurately process this domain knowledge in real time, its reasoning performance may be affected, impacting the user experience.

[0062] For example, if a user asks for maintenance information about a specific car model and the in-vehicle intelligent system doesn't have the knowledge to know about the repairs, it might not provide accurate information or take longer to process the user's request. This delay or incorrect information can negatively impact the user experience, especially in situations where fast and accurate information is needed, such as emergency repairs or navigation.

[0063] Voice activity detection (VAD) refers to a technique used to determine the presence of speech signals. In speech processing and communication systems, VAD identifies the beginning and end of speech activity, allowing only the speech portion to be processed, ignoring periods of silence or background noise.

[0064] Based on the above questions, please refer to Figure 1 , an embodiment of the present application provides a voice interaction method, the method comprising:

[0065] 01: Receive the voice request segments forwarded by the vehicle in sequence;

[0066] 02: Based on a pre-configured external knowledge base, natural language processing is performed on the first voice request segment set to obtain a first recognition result and a first understanding result;

[0067] 03: Based on the external knowledge base, natural language processing is performed on the second voice request segment set to obtain a second recognition result and a second understanding result;

[0068] 04: Perform natural language recognition on the received voice request to obtain a third recognition result;

[0069] 05: Complete voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request.

[0070] The embodiment of the present application also provides a server, including a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive voice request segments forwarded sequentially by the vehicle. And based on a pre-configured external knowledge base, natural language processing is performed on the first set of voice request segments to obtain a first recognition result and a first understanding result. And based on the external knowledge base, natural language processing is performed on the second set of voice request segments to obtain a second recognition result and a second understanding result. The processor is also used to perform natural language recognition on the received voice request to obtain a third recognition result. And according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request, the voice interaction is completed.

[0071] The embodiment of the present application also provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a language processing module, a language recognition module and a processing module. The receiving module is used to receive voice request segments forwarded sequentially by the vehicle. The language processing module is used to perform natural language processing on a first set of voice request segments based on a pre-configured external knowledge base to obtain a first recognition result and a first understanding result. The language processing module is used to perform natural language processing on a second set of voice request segments based on an external knowledge base to obtain a second recognition result and a second understanding result. The language recognition module is used to perform natural language recognition on the received voice request to obtain a third recognition result. The processing module is used to complete the voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0072] Specifically, see Figure 2 In classic technology, the human-computer interaction process can be divided into five stages: receiving a user's voice request, voice activity detection, automatic speech recognition, natural language understanding (NLU), and the robot understanding the command. The time required to receive the user's voice request may be A1, the time required for voice activity detection may be A2, the time required for automatic speech recognition may be A3, the time required for natural language understanding may be A4, and the time required for the robot to understand the command may be A5. The time it takes for the server to complete this human-computer interaction is A1+A2+A3+A4+A5.

[0073] Robot understanding instructions refers to the process after a large language model receives an instruction (natural language understanding result). The instruction may be a piece of program code, and the robot interacting with humans needs to understand this piece of program code, which takes a certain amount of time.

[0074] However, this classic voice interaction technology, which is to perform natural language processing on the complete voice request after receiving it, has a slow system response time and a poor interactive experience. Based on the above problems, a voice interaction method that can improve the response speed is proposed in the related art, that is, in the process of receiving the user's voice request, the voice request fragment is processed in real time. Figure 2As shown, it can be seen that when the server receives the first voice request segment, it will perform natural speech recognition and natural language understanding processing. When the voice request is received, the natural language recognition and natural language understanding processing for each voice request segment are also completed. Subsequently, the server performs voice activity detection, natural language recognition processing for the complete voice request, and result selection processing. In this way, the time required for the server to receive the user's voice request is also A1, the time required for voice activity detection is also A2, the time required for natural language recognition processing for the complete voice request is also A3, and the time required for result selection processing may be A6. The time required for the server to complete this human-computer interaction is A1+A2+A3+A6, which is a faster response speed.

[0075] Result selection involves matching and filtering the natural language understanding results obtained based on the voice request fragments. The appropriate natural language understanding result is then sent to the vehicle to complete the voice interaction. Furthermore, because the program code for each natural language understanding result obtained based on a different voice request fragment is already understood by the robot, response time is reduced.

[0076] However, when faced with complex voice requests involving business-specific knowledge or domain-specific terminology, the methods provided in the above-mentioned related technologies have a reduced reasoning effect on the in-vehicle intelligent system and a degraded user experience due to the increased length of the voice requests that need to be inferred and the involvement of business-specific knowledge or domain-specific terminology. The voice interaction method provided in the embodiments of the present application is based on an external knowledge base and uses phased natural language recognition and natural language understanding of different sets of voice request fragments. The large language model can more accurately understand the user's intention and respond to the voice request based on the natural language understanding results of different phases, thereby improving the system's accuracy in processing long or complex sentences that include business-specific knowledge or domain-specific terminology.

[0077] A voice request segment is a segment of the voice request that is captured by the server at regular intervals when the server first receives the voice request forwarded by the vehicle. For example, if a user sends a voice request saying "Help me open the beautiful fantasy and close the car window immediately," the server may capture two voice request segments at different times: the voice request segment "Help me open" and the voice request segment "Beautiful fantasy."

[0078] A voice request segment set refers to a set of voice request segments that can output the application interface and the result of API parameter filling. For example, if the user inputs "Help me open the beautiful fantasy and close the car windows immediately", the voice request segment set is "Help me open the beautiful fantasy" and "and close the car windows immediately".

[0079] The first voice request segment set refers to the first voice request segment set that can output the application interface and the application interface parameter filling results, including all voice request segments received within the first moment. These voice request segments are collectively referred to as first voice request segments. The first moment is used to indicate the time when the application interface and the application interface parameter filling results can be output. In other words, if the user inputs "Help me open the beautiful fantasy and close the car window immediately", then the first voice request segment set is "Help me open the beautiful fantasy", consisting of the first voice request segment "Help me open" and the first voice request segment "Beautiful Fantasy".

[0080] The second voice request segment set can refer to the set of all second voice request segments received after the first voice request segment set, including all voice request segments received during the second moment. These voice request segments are collectively referred to as second voice request segments. The second moment indicates any moment after the first moment at which the application interface and API parameter filling results can be output. In other words, if the user inputs "Help me open the beautiful fantasy and close the car windows immediately," the second voice request segment set is "and immediately close the car windows," and the second voice request segment set includes the second voice request segment "and immediately" and the second voice request segment "close the car windows." If the user inputs "Close the car windows, help me open the beautiful fantasy, and play music immediately," the second voice request segment set is "Help me open the beautiful fantasy" and "and play music immediately." "Help me open the beautiful fantasy" is composed of the second voice request segment "Help me open" and the second voice request segment "Beautiful Fantasy," and "and play music immediately" is composed of the second voice request segment "and immediately" and the second voice request segment "play music." It should be noted that “and play music immediately” can be called the third voice request segment set in terms of time sequence.

[0081] An external knowledge base refers to a database or knowledge management system that stores rich information and can provide additional background knowledge, business-specific knowledge, professional terminology, etc. to support the natural language understanding process. When the server confirms that all voice request fragments received at a certain moment can output application interfaces and application interface parameter filling results, the server can retrieve and utilize relevant information from the external knowledge base through retrieval enhancement generation technology (Reinforcement Learning based Automatic Curricula Generation, RAG). In particular, when processing complex voice request fragments that include business-specific knowledge or specific domain terminology, the server may not be able to accurately understand the user's intentions due to the lack of relevant knowledge reserves. However, through RAG technology, the server can automatically retrieve and utilize relevant information from the external knowledge base to solve the problem that the business-specific knowledge or specific domain terminology in complex voice request fragments does not understand the large language model, thereby providing accurate and rich answers, enabling the large language model to better understand and respond to user requests.

[0082] See also Figure 3 , the server can receive the voice request segments forwarded by the vehicle in sequence, that is, process the voice stream in real time, rather than waiting for the entire voice request to be completed before processing. The following example illustrates the voice interaction method of the embodiment of the present application. If the user's voice request is "Help me open the beautiful fantasy and close the car window immediately", the server will receive multiple voice request segments. The voice request segment received at time T0 is "Help me open", the voice request segment received at time T1 is "Beautiful Fantasy", the voice request segment received at time T2 is "And immediately", and the voice request segment received at time T3 is "Close the car window".

[0083] Then, based on the pre-configured external knowledge base, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result. The first voice request segment set includes all the first voice request segments received within the first moment. Continuing with the above example, the server will perform natural language processing on the received voice request segments, that is, at time T0, the server will perform natural language processing on the voice request segment "Help me open", and obtain a natural language recognition result of "Help me open" and a natural language understanding result of "Unclear". It can be found that no application interface and application interface parameter filling result are output at time T0, so the voice request segment set at time T0 is not the first voice request segment set. At time T1, the server will perform natural language processing on the voice request segment set of the voice request segment "Help me open" and the voice request segment "Beautiful Fantasy". It can be seen that the voice request segments at times T0 and T1 can output application interfaces and API parameter-filled results. At this time, the server performs natural language processing on the first set of voice request segments based on a preconfigured external knowledge base, resulting in a natural language recognition result of "Help me open the beautiful fantasy world" and a natural language understanding result of "AcOpen, {target_function: beautiful fantasy world}". Since the application interface and API parameter-filled results were output at time T1, and this voice request segment set is the first in this voice request to output an application interface and API parameter-filled results, the voice request segment set at time T1 is the first set of voice request segments, the natural language recognition result "Help me open the beautiful fantasy world" is the first recognition result, and the natural language understanding result "AcOpen, {target_function: beautiful fantasy world}" is the first understanding result.

[0084] Then, based on the external knowledge base, the server performs natural language processing on the second set of voice request segments to obtain a second recognition result and a second understanding result. The second set of voice request segments includes all second voice request segments received at the second moment that follow the first set of voice request segments, and the second moment is later than the first moment. Continuing with the above example, after obtaining the first set of voice request segments, the server no longer needs to consider the voice request segments before that moment, that is, at moment T2, and no longer needs to perform natural language processing on the voice request segments received at moments T0 and T1. Then, the set of voice request segments that need to be processed by natural language at moment T2 is "and immediately", and the natural language recognition result "and immediately" and the natural language understanding result "Noise" are obtained. In this way, the second set of voice request segments is the set of voice request segments at moment T3, and the second recognition result obtained is "and immediately close the car window", and the second understanding result is "WindowClosed, {device: car window}".

[0085] If no voice request fragments are forwarded by the vehicle within a certain period of time, the server determines that the user has completed their voice request. The server then performs natural language recognition on the received voice request, obtaining a third recognition result. The voice request is concatenated from all the voice request fragments. In other words, there is only one third natural language recognition result, which is obtained by performing natural language recognition processing on the final voice request. Continuing with the above example, the second natural language recognition result is "Help me open the beautiful fantasy and close the car windows immediately."

[0086] Finally, the server completes the voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0087] In summary, in the voice interaction method and server provided in the embodiment of the present application, the server receives voice request segments forwarded sequentially by the vehicle, and the voice request segments include at least a first voice request segment and a second voice request segment. Then, based on a pre-configured external knowledge base, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result. The first voice request segment set includes the first voice request segment received within the first moment. Then, based on the external knowledge base, the server performs natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result. The second voice request segment set includes the second voice request segment received within the second moment and subsequent to the first voice request segment set, and the second moment is later than the first moment. The server then performs natural language recognition on the received voice request to obtain a third recognition result. The voice request is obtained by splicing the voice request segments. Finally, the server completes the voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, in an end-to-end architecture, the large language model is combined with streaming conversation technology to achieve streaming natural language recognition and understanding capabilities. This reduces the resources and time required to complete the entire voice request, significantly shortening server response time. Furthermore, based on an external knowledge base, the large language model can acquire required domain knowledge in real time. Through phased natural language processing of different voice request segments, it accurately understands the user's intent. By combining the natural language understanding results from different phases, the system improves its accuracy when processing long or complex sentences containing business-specific knowledge or domain-specific terminology.

[0088] See also Figure 4 In some embodiments, step 02 (performing natural language processing on the first set of voice request segments based on a preconfigured external knowledge base to obtain a first recognition result and a first understanding result) includes:

[0089] 021: Perform natural language recognition on the first voice request segment set to obtain a first recognition result;

[0090] 022: Mark the first recognition result according to the first preset mark to obtain a target marked speech recognition result;

[0091] 023: Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech recognition results according to the external knowledge base to obtain the first understanding result.

[0092] In certain embodiments, the language processing module is further configured to perform natural language recognition on the first set of voice request segments to obtain a first recognition result, label the first recognition result according to a first preset label to obtain a target labeled voice recognition result, and perform natural language understanding on the target labeled voice recognition result based on a pre-trained large language model and an external knowledge base to obtain a first understanding result.

[0093] In certain embodiments, the processor is further configured to perform natural language recognition on the first set of voice request segments to obtain a first recognition result, label the first recognition result according to a first preset label to obtain a target labeled voice recognition result, and perform natural language understanding on the target labeled voice recognition result based on a pre-trained large language model according to an external knowledge base to obtain a first understanding result.

[0094] Specifically, natural language recognition refers to the process of converting human natural language (such as English, Chinese, etc.) into a form that can be understood and processed by computers.

[0095] The first preset tag is a "wait" tag added to the natural language recognition results before the large language model performs natural language understanding processing. This "wait" tag guides the large language model into natural language understanding processing mode, informing the large language model that the next section of text is ready for natural language understanding processing.

[0096] Please refer to Figure 3, the server first performs natural language recognition on the first set of voice request segments to obtain a first recognition result. Then, the server performs labeling processing on the first natural language recognition result according to the first preset label to obtain a target label voice recognition result. Continuing with the above example, for example, at time T1, the target labeling language recognition result obtained by the server is "wait help me open the beautiful fantasy world". It should be noted that the labeling processing does not only label the voice request segments that can output application interfaces and application interface parameter filling results, but also labels the natural language recognition results obtained by performing natural language recognition processing on each received voice request segment. Continuing with the above example, at time T0, the target labeling language recognition result obtained by the server is "wait help me open the beautiful fantasy world".

[0097] Subsequently, based on the pre-trained large language model, the server performs natural language understanding on the target labeled speech recognition results according to the external knowledge base, obtaining the first understanding result. Continuing with the above example, at time T1, based on the pre-trained large language model, the server performs natural language understanding processing on the sentence "wait help me open the beautiful fantasy world" according to the external knowledge base, obtaining the natural language understanding result "AcOpen, {target_function: beautiful fantasy world}".

[0098] In this way, by marking the first natural language recognition result with the first preset tag, the large language model can accurately recognize the input natural language recognition result, accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0099] See also Figure 5 In some embodiments, step 023 (performing natural language understanding on the target mark speech recognition result according to the external knowledge base to obtain a first understanding result) includes:

[0100] 0231: acquiring target external knowledge associated with the target labeled speech recognition result from an external knowledge base according to the target labeled speech recognition result;

[0101] 0232: Perform natural language understanding processing on the target external knowledge and the target labeled speech recognition results to obtain a first understanding result.

[0102] In certain embodiments, the language processing module is further configured to obtain target external knowledge associated with the target-marked speech recognition result from an external knowledge base based on the target-marked speech recognition result, and perform natural language understanding processing on the target external knowledge and the target-marked speech recognition result to obtain a first understanding result.

[0103] In some embodiments, the processor is further configured to obtain target external knowledge associated with the target-marked speech recognition result from an external knowledge base based on the target-marked speech recognition result, and perform natural language understanding processing on the target external knowledge and the target-marked speech recognition result to obtain a first understanding result.

[0104] Specifically, after obtaining the target-marked speech recognition result, the server retrieves the target external knowledge associated with the target-marked speech recognition result from the external knowledge base based on the target-marked speech recognition result. Continuing with the above example, at time T1, after obtaining the target-marked language recognition result "wait help me open the beautiful fantasy world," the server automatically retrieves data related to the target-marked language recognition result "wait help me open the beautiful fantasy world" from the external knowledge base using RAG technology based on the pre-trained large language model, obtaining the target external knowledge "Beautiful Fantasy World is a type of air conditioning fragrance in vehicles."

[0105] Next, the server performs natural language understanding on the target external knowledge and the target-labeled speech recognition results, producing a first understanding result. Continuing with the previous example, the server inputs the target external knowledge "Qilihuanjing is a type of air conditioning fragrance in vehicles" and the target-labeled speech recognition result "wait, help me open Qilihuanjing" into the large language model. The large language model then performs natural language understanding on the target external knowledge and the target-labeled speech recognition results, producing the first understanding result "AcOpen, {target_function: Qilihuanjing}."

[0106] In some embodiments, natural language understanding can be performed on the target external knowledge and the target labeled speech recognition results based on a pre-trained large language model to obtain a first understanding result. Natural language understanding of the target labeled speech recognition results can be performed on the pre-trained large language model to improve the accuracy and efficiency of natural language understanding.

[0107] The training steps for the large language model are as follows: Training data is constructed based on the collected voice request samples and the in-vehicle system function customization solution samples constructed based on the voice request samples. Sample data can come from collected user logs, such as the user's history of successfully constructing in-vehicle system function customization solutions based on voice requests. Of course, training data can also be constructed as needed. For example, a set of training data may be: a voice request "Help me open the beautiful fantasy world immediately", and the in-vehicle system function customization solution constructed based on this voice request (i.e., the natural language understanding result) is "AcOpen, {target_function: beautiful fantasy world}".

[0108] Furthermore, in order to guide the large model to meet the task requirements, the prompt information template needs to be configured. Specifically, the prompt information template is used to guide the model to generate output, providing the background of the problem and relevant guidance. It usually includes a description and instructions of the problem to tell the model how to process the input data. The prompt information template can provide additional auxiliary information to facilitate the model to better understand and generate output. It can also include a specific format, style, theme that the model is required to follow, or instruct the model to pay attention to certain specific aspects and details. In this way, the model will understand the requirements of the task based on the content of the prompt information template, thereby generating answers and text that meet expectations. Generally, the configuration process of the prompt information template can include clarifying the role that the model needs to play (optional), clarifying the input text content and structure, and immediately clarifying the output text content and structure.

[0109] In an example, the content of the prompt message template is as follows:

[0110] "Suppose you are an intelligent cockpit dialogue system that needs to understand the recognized commands and output the corresponding API and arguments. The input command fragments will be entered multiple times. Please determine the current API and arguments based on the existing commands."

[0111] In certain embodiments, the server first performs slot identification on the target external knowledge and target tag speech recognition results to obtain a slot identification result. Next, the server performs API prediction on the target external knowledge and target tag speech recognition results to obtain a predicted API. Finally, based on the slot identification results and the predicted API, the server selects the predicted API and performs API parameter filling, obtaining a first natural language understanding result. Continuing with the above example, at time T1, the server performs slot identification on the first voice request segment set "Help me open the beautiful fantasy world" and obtains the slot identification result "Air conditioning fragrance - control function (target_function)". Furthermore, the server performs API prediction on the voice request segment set "Help me open the beautiful fantasy world" and obtains the predicted API "AcOpen". Finally, based on the slot identification results and the predicted API, the server selects the predicted API and performs API parameter filling, obtaining a first natural language understanding result "AcOpen, {target_function: beautiful fantasy world}". Through slot identification and API prediction, the system can more accurately understand the user's request and select the most appropriate API to perform the task, thereby reducing misunderstandings and errors and providing a good user experience.

[0112] In this way, when processing voice requests containing business-specific knowledge or domain-specific terminology, by combining external knowledge bases and natural language understanding technology, the server can more accurately parse the target tag voice recognition results, thereby improving system intelligence and enhancing the user experience.

[0113] See also Figure 6 In certain embodiments, the method further comprises:

[0114] 024: Mark the first understanding result according to the second preset mark to obtain a target marked speech understanding result.

[0115] In some embodiments, the processing module is further configured to mark the first understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0116] In some embodiments, the processor is further configured to mark the first understanding result according to a second preset mark to obtain a target marked speech understanding result.

[0117] Specifically, the second preset tag refers to a tag waitEnd that will be added after the first understanding result is obtained after the large language model performs natural language understanding processing on the target tag speech recognition result. This tag waitEnd can guide the large language model to perform natural language understanding processing without being affected by the natural language understanding results obtained at the previous moment. In other words, the server will only perform natural language understanding processing on the obtained natural language recognition results. Continuing with the above example, for example, at time T1, the large language model will process the voice request segment set "help me open" at time T0 and the voice request segment "Beautiful Fantasy" received at time T1, as well as the natural language understanding result "wait help me open waitEnd" obtained at time T0. Due to the existence of the second preset tag waitEnd, at time T1, the server will not perform natural language understanding processing on the natural language understanding result "wait Unclear waitEnd".

[0118] Please refer to Figure 3 The server marks the first understanding result according to the second preset mark to obtain the target marked speech understanding result.

[0119] By marking the natural language understanding results with the second preset marker, the large language model can optimize the natural language understanding process, ignoring natural language understanding results marked with the second preset marker, reducing the accumulation of errors during the conversation and thus improving overall understanding accuracy. Furthermore, the amount of information required for consideration is reduced, allowing the system to process subsequent input more quickly, thereby improving overall response speed.

[0120] See also Figure 7 In some embodiments, step 05 (completing voice interaction based on the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request) includes:

[0121] 051: Concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result;

[0122] 052: matching the spliced ​​recognition result and the third recognition result;

[0123] 053: Complete the voice interaction based on the matching results.

[0124] In some embodiments, the processing module is configured to concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result, perform matching processing on the concatenated recognition result and the third recognition result, and complete the voice interaction based on the matching processing result.

[0125] In some embodiments, the processor is further configured to concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result, match the concatenated recognition result with the third recognition result, and complete the voice interaction based on the matching result.

[0126] Specifically, the matching process refers to comparing the spliced ​​recognition result and the third recognition result based on technologies such as text alignment and semantic analysis to verify the consistency of the spliced ​​recognition result and the third recognition result.

[0127] See also Figure 8 , the server concatenates the first recognition result and the second recognition result to obtain a concatenated recognition result. Continuing with the above example, after time T3, the large language model does not receive any new voice clips within a certain period of time, confirming that the user's voice request has been received. If the received voice request is "Help me open the beautiful fantasy and close the car windows immediately", the first recognition result is "Help me open the beautiful fantasy", and the second recognition result is "and close the car windows immediately", the first recognition result and the second recognition result are concatenated, and the concatenated recognition result is "Help me open the beautiful fantasy and close the car windows immediately". If the received voice request is "Close the car windows, help me open the beautiful fantasy, and play music immediately", the first recognition result is "Close the car windows", the second recognition results are "Help me open the beautiful fantasy" and "and play music immediately", and the concatenated recognition result is "Close the car windows, help me open the beautiful fantasy, and play music immediately".

[0128] Next, the server matches the concatenated recognition result with the third recognition result. Continuing with the above example, if the received voice request is "Help me open the beautiful fantasy and close the car windows immediately," the server matches the concatenated first and second recognition results ("Help me open the beautiful fantasy and close the car windows immediately") with the third recognition result ("Help me open the beautiful fantasy and close the car windows immediately") obtained by performing natural language recognition on the complete voice request. In some embodiments, the matching process is performed using the special tags "group" and "groupEnd," resulting in a result corresponding to the location of the natural language understanding result.

[0129] Finally, the server completes the voice interaction based on the results of the matching process.

[0130] In this way, by splicing multiple recognition results obtained in the process of receiving voice requests and matching them with the third recognition result, the server can accurately understand the user's complex voice requests and effectively perform corresponding operations.

[0131] See also Figure 9 and Figure 10 In some embodiments, step 053 (completing voice interaction based on the result of the matching process) includes:

[0132] 0531: When the spliced ​​recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result;

[0133] 0532: Send the target output result to the vehicle to complete the voice interaction; or

[0134] 0533: When the concatenated recognition result and the third recognition result do not match, perform natural language understanding on the voice request to obtain the target output result;

[0135] 0534: Send the target output result to the vehicle to complete the voice interaction.

[0136] In certain embodiments, the processing module is further configured to, if the concatenated recognition result and the third recognition result match, obtain a target output result based on the first understanding result and the second understanding result, and transmit the target output result to the vehicle to complete the voice interaction. Alternatively, if the concatenated recognition result and the third recognition result do not match, the processing module is further configured to perform natural language understanding on the voice request to obtain a target output result, and transmit the target output result to the vehicle to complete the voice interaction.

[0137] In certain embodiments, the processor is further configured to, if the concatenated recognition result and the third recognition result match, obtain a target output result based on the first understanding result and the second understanding result, and transmit the target output result to the vehicle to complete the voice interaction. Alternatively, if the concatenated recognition result and the third recognition result do not match, the processor is further configured to perform natural language understanding on the voice request to obtain a target output result, and transmit the target output result to the vehicle to complete the voice interaction.

[0138] For details, please refer to Figure 9 When the spliced ​​recognition result and the third recognition result match, the server obtains the target output result based on the first understanding result and the second understanding result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Continuing with the above example, it can be found that the third recognition result "Help me open the beautiful fantasy and close the window immediately" is consistent with the spliced ​​recognition result "Help me open the beautiful fantasy and close the window immediately". Based on the first understanding result "AcOpen, {target_function: beautiful fantasy}" and the second understanding result "WindowClosed, {device: window}", the target output result "AcOpen, {target_function: beautiful fantasy}, WindowClosed, {device: window}" is obtained.

[0139] Please refer to Figure 10 If the spliced ​​recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain the target output result. The server then sends the target output result to the vehicle to complete the voice interaction. Continuing with the above example, if the spliced ​​recognition result is "Help me open the beautiful fantasy and close the windows immediately", then the spliced ​​recognition result "Help me open the beautiful fantasy and close the windows immediately" is inconsistent with the third recognition result "Help me open the beautiful fantasy and close the windows immediately". The large language model will directly perform natural language understanding on the third recognition result to obtain the target output result. The target output result is then sent to the vehicle to complete the voice interaction.

[0140] In this way, when processing complex voice commands, by flexibly handling the matching of spliced ​​recognition results and third recognition results, the system can more accurately understand and respond to the user's voice commands, and can also make effective corrections when understanding deviations occur, which helps to improve the overall performance of the voice interaction system and user experience.

[0141] See also Figure 11 In some embodiments, step 0531 (obtaining a target output result based on the first understanding result and the second understanding result) includes:

[0142] 05311: Filtering the first understanding result to obtain a first sub-goal output result, where the first sub-goal output result includes a first application interface and a first application interface parameter filling result;

[0143] 05312: Filter the second understanding result to obtain a second sub-goal output result, where the second sub-goal output result includes a second application interface and a second application interface parameter filling result;

[0144] 05313: According to the output result of the first sub-goal and the output result of the second sub-goal, the target output result is obtained.

[0145] In certain embodiments, the processing module is configured to filter the first understanding result to obtain a first sub-goal output result, the first sub-goal output result including the first application interface and a result of populating the first application interface parameters; filter the second understanding result to obtain a second sub-goal output result, the second sub-goal output result including the second application interface and a result of populating the second application interface parameters; and obtain a target output result based on the first sub-goal output result and the second sub-goal output result.

[0146] In certain embodiments, the processor is further configured to filter the first understanding result to obtain a first sub-goal output result, the first sub-goal output result including the first application interface and a result of populating the first application interface parameters; filter the second understanding result to obtain a second sub-goal output result, the second sub-goal output result including the second application interface and a result of populating the second application interface parameters; and obtain a target output result based on the first sub-goal output result and the second sub-goal output result.

[0147] Specifically, the screening process refers to matching the natural language recognition result corresponding to the natural language understanding result and the third recognition result. If the third recognition result includes the natural language recognition result, then the natural language understanding result is considered to be the sub-target output result.

[0148] The server filters and processes the first understanding result to obtain the first sub-target output result, which includes the first application interface and the first application interface parameter filling result. Continuing with the above example, if the first recognition result is "Help me open the beautiful fantasy world" and the third recognition result is "Help me open the beautiful fantasy world and close the car window immediately", then it is considered that the first understanding result corresponding to the first recognition result cannot be used as the first sub-target output result. If the first recognition result is "Help me open the beautiful fantasy world" and the third recognition result is "Help me open the beautiful fantasy world and close the car window immediately", then it is considered that the first understanding result corresponding to the first recognition result can be used as the first sub-target output result, that is, the obtained first application interface "AcOpen" and the first application interface parameter filling result "{AcOpen, target_function: beautiful fantasy world}" are used as the first sub-target output result.

[0149] Next, the server filters and processes the second understanding result to obtain the second sub-target output result, which includes the second application interface and the second application interface parameter filling result. Continuing with the above example, if the second recognition result is "and close the cabin immediately" and the third recognition result is "help me open the beautiful fantasy and close the window immediately", then it is considered that the first understanding result corresponding to the second recognition result cannot be used as the second target output result. If the second recognition result is "and close the window immediately" and the third recognition result is "help me open the beautiful fantasy and close the window immediately", then it is considered that the second understanding result corresponding to the second recognition result can be used as the second sub-target output result, that is, the obtained second application interface "WindowClosed" and the first application interface parameter filling result {"WindowClosed, device: window}" are used as the second sub-target output result.

[0150] Finally, the server obtains the target output result based on the first sub-goal output result and the second sub-goal output result. In some embodiments, the first sub-goal output result and the second sub-goal output result are concatenated to obtain the target output result.

[0151] It should be noted that in some embodiments, each natural language understanding result is not individually screened. That is, the natural language recognition result corresponding to each natural language understanding result is not individually compared with the third recognition result. Instead, the last natural language recognition result obtained is directly compared with the third recognition result. When the target output result is finally output, either all natural language understanding results are sent to the vehicle, or no natural language understanding results are sent.

[0152] In this way, when processing complex voice requests, by screening and processing the understanding results in steps, the system can more accurately extract and integrate natural language understanding results, thereby generating more precise target output results, which helps to improve the understanding and response quality of the voice interaction system.

[0153] See also Figure 12 In some embodiments, the large language model is trained by the following steps:

[0154] 06: Train the original large language model based on the pre-built false call dataset to obtain the initial large language model;

[0155] 07: Fine-tune and optimize the initial large language model to obtain a large language model.

[0156] Specifically, a false call in a large language model refers to a situation in which the model mistakenly predicts a negative class (i.e., a class that should not be recognized or selected) as a positive class (i.e., a class that should be recognized or selected) in a classification problem. In fields such as natural language processing and natural language recognition, a false call usually refers to a large language model incorrectly identifying or responding to irrelevant or incorrect input.

[0157] A false call dataset refers to a pre-constructed dataset that includes training data that may cause the model to generate false calls in specific tasks. These training data are usually obtained by collecting and analyzing errors that occur in actual applications, or they can be obtained through simulation or design by relevant technical personnel. The purpose of the false call dataset is to train and evaluate models to improve their accuracy and reduce false calls. For example, in an in-vehicle dialogue system, the user's voice commands may be misrecognized. The user's voice request is "Help me open the beautiful fantasy world", but the in-vehicle dialogue system mistakenly recognizes it as "Help me open the beautiful environment". The natural language understanding result obtained by the in-vehicle dialogue system may then be "Find a place with a beautiful environment and navigate."

[0158] Fine-tuning optimization refers to optimizing the initial large language model using supervised fine-tuning (SFT) and diverse prompting optimization (DPO). Supervised fine-tuning (SFT) can further train a pre-trained language model using labeled data for a specific task. The steps are as follows: First, the pre-trained model is fine-tuned on a task-specific dataset. Second, the model's parameters are updated through optimization algorithms such as backpropagation and gradient descent to improve performance on the specific task. Diverse prompting optimization (DPO) guides the model to produce higher-quality outputs using diverse prompts. The steps are as follows: First, diverse prompts are designed, which can be extracted from different perspectives or contexts. Second, these prompts are used to generate multiple outputs. Third, the best output is selected based on specific evaluation criteria (such as accuracy, diversity, or relevance).

[0159] First, the original large language model is trained using the false call dataset to obtain an initial large language model. The initial large language model can perform natural language processing on input voice requests and has a certain ability to prevent hallucinations or false calls.

[0160] Next, the initial large language model is fine-tuned and optimized, that is, labeled data for a specific task is used to further train the model to improve its performance on that task. This can help the large language model better adapt to the context and requirements of the specific task, thereby providing more accurate and richer answers.

[0161] In this way, by constructing a false-call dataset and performing fine-tuning and optimization, we can reduce the false-call situations caused by the large language model when processing long or complex sentences that include business-specific knowledge or domain-specific terminology, thereby improving the reliability of the large language model and user experience.

[0162] The present application also provides a computer-readable storage medium. When the computer program is executed by one or more processors, the one or more processors execute the method of the present application.

[0163] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file, or some intermediate form. Computer-readable storage media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media.

[0164] In the description of this specification, the descriptions with reference to the terms "particularly", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0165] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0166] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: The method comprises: receiving voice request segments forwarded sequentially by the vehicle, the voice request segments including at least a first voice request segment and a second voice request segment; Based on a preconfigured external knowledge base, natural language processing is performed on a first voice request segment set to obtain a first recognition result and a first understanding result, wherein the first voice request segment set includes the first voice request segment received within a first time; Based on the external knowledge base, natural language processing is performed on a second voice request segment set to obtain a second recognition result and a second understanding result, wherein the second voice request segment set includes the second voice request segment received at a second time point and subsequent to the first voice request segment set, and the second time point is later than the first time point; Performing natural language recognition on the received voice request to obtain a third recognition result, wherein the voice request is obtained by concatenating the voice request segments; The voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

2. The method according to claim 1, characterized in that The performing natural language processing on the first voice request segment set based on the pre-configured external knowledge base to obtain a first recognition result and a first understanding result includes: Performing natural language recognition on the first voice request segment set to obtain the first recognition result; According to a first preset mark, the first recognition result is marked to obtain a target marked speech recognition result; Based on the pre-trained large language model, natural language understanding is performed on the target labeled speech recognition result according to the external knowledge base to obtain the first understanding result.

3. The voice interaction method according to claim 2, characterized in that: The performing natural language understanding on the target mark speech recognition result according to the external knowledge base to obtain the first understanding result includes: Acquiring target external knowledge associated with the target marked speech recognition result from the external knowledge base according to the target marked speech recognition result; The target external knowledge and the target marked speech recognition result are processed by natural language understanding to obtain the first understanding result.

4. The voice interaction method according to claim 2, characterized in that: The method further comprises: The first understanding result is marked according to the second preset mark to obtain a target marked speech understanding result.

5. The voice interaction method according to claim 1, characterized in that: The completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request includes: Concatenate the first recognition result and the second recognition result to obtain a concatenated recognition result; Performing matching processing on the splicing recognition result and the third recognition result; The voice interaction is completed according to the result of the matching process.

6. The voice interaction method according to claim 5, characterized in that: The completing the voice interaction according to the result of the matching process includes: When the splicing recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result; Sending the target output result to the vehicle to complete the voice interaction; or In the case that the concatenated recognition result and the third recognition result do not match, performing natural language understanding on the voice request to obtain the target output result; The target output result is sent to the vehicle to complete the voice interaction.

7. The voice interaction method according to claim 6, characterized in that: The obtaining a target output result according to the first understanding result and the second understanding result includes: Screening the first understanding result to obtain a first sub-goal output result, wherein the first sub-goal output result includes a first application interface and a first application program interface parameter filling result; Screening the second understanding result to obtain a second sub-goal output result, wherein the second sub-goal output result includes a second application interface and a second application program interface parameter filling result; The target output result is obtained according to the first sub-target output result and the second sub-target output result.

8. The voice interaction method according to claim 2, characterized in that: The large language model is trained by the following steps: The original large language model is trained according to the pre-constructed false call data set to obtain an initial large language model; The initial large language model is fine-tuned and optimized to obtain the large language model.

9. A server, characterized in that: The server includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the voice interaction method described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the voice interaction method as described in any one of claims 1 to 8 are implemented.

Images