Voice interaction method, server and readable storage medium

Abstract

The application discloses a voice interaction method, comprising: receiving a voice request fragment forwarded by a vehicle in sequence. A first set of voice request fragments is subjected to natural language processing to obtain a first recognition result and a first understanding result, the first set of voice request fragments including a voice request fragment received at a first time. A second set of voice request fragments is subjected to natural language processing to obtain a second recognition result and a second understanding result, the second set of voice request fragments including a voice request fragment received at a second time and succeeding the first set of voice request fragments, the second time being later than the first time. A received voice request is subjected to natural language recognition to obtain a third recognition result, the voice request being obtained by splicing the voice request fragments. The voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, the accuracy of the system in processing complex sentences is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program. BACKGROUND

[0002] In related technologies, a vehicle-mounted intelligent system can perform natural language processing on a user voice request in the process of receiving the user voice request, so that the system can quickly respond to the user voice request. However, when the user voice request is complex, the inference length increases, and the inference effect of the vehicle-mounted intelligent system decreases, which affects the user experience. SUMMARY

[0003] The present application provides a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program.

[0004] The present application provides a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program.

[0005] receiving a voice request segment forwarded by a vehicle in sequence;

[0006] performing natural language processing on a first voice request segment set to obtain a first recognition result and a first understanding result, the first voice request segment set including the voice request segment received at a first time;

[0007] performing natural language processing on a second voice request segment set to obtain a second recognition result and a second understanding result, the second voice request segment set including the voice request segment received at a second time after the first voice request segment set, the second time being later than the first time;

[0008] performing natural language recognition on the received voice request to obtain a third recognition result, the voice request being obtained by splicing the voice request segments;

[0009] completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0010] Thus, the server receives the voice request segments forwarded by the vehicle in sequence. Then, the server performs natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result, the first voice request segment set including the voice request segment received at the first time. Then, the server performs natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result, the second voice request segment set including the voice request segment received at the second time, which is later than the first time, and continuing the first voice request segment set. The server further performs natural language recognition on the received voice request to obtain a third recognition result, the voice request being obtained by splicing the voice request segments. Finally, the server completes voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request. In this way, in the end-to-end architecture, the streaming natural language recognition and understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, which reduces the resources and time required for waiting for the entire voice request to be completed, thereby greatly shortening the response time of the server. Moreover, through the phased natural language recognition and understanding of different voice request segment sets, the large language model can accurately understand the user's intent and combine the natural language understanding results of different stages, thereby improving the accuracy of the system in processing long sentences or complex sentences.

[0011] In some embodiments, the natural language processing on the first voice request segment set to obtain the first recognition result and the first understanding result comprises:

[0012] performing natural language recognition on the first voice request segment set to obtain the first recognition result;

[0013] performing mark processing on the first recognition result according to a first preset mark to obtain a target mark voice recognition result;

[0014] performing natural language understanding on the target mark voice recognition result to obtain the first understanding result.

[0015] Thus, the server performs natural language recognition on the first voice request segment set to obtain the first recognition result. Then, the server performs mark processing on the first recognition result according to a first preset mark to obtain a target mark voice recognition result. Finally, the server performs natural language understanding on the target mark voice recognition result to obtain the first understanding result. In this way, by using the first preset mark to perform mark processing on the first natural language recognition result, the large language model can accurately recognize the input natural language recognition result and accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0016] In some embodiments, the method further comprises:

[0017] According to the second preset mark, the first understanding result is marked to obtain a target marked speech understanding result.

[0018] In this way, the server marks the first understanding result according to the second preset mark to obtain a target marked speech understanding result. In this way, by using the second preset mark to mark the natural language understanding result, the large language model can optimize the natural language understanding process, ignore the natural language understanding result that has been marked with the second preset mark, reduce the accumulation of errors in the dialogue process, and thus improve the overall understanding accuracy. Moreover, the amount of information that needs to be considered can be reduced, so that the system can process subsequent inputs faster, thereby improving the overall response speed.

[0019] In some embodiments, the natural language understanding of the target marked speech recognition result to obtain the first understanding result comprises:

[0020] Slot recognition is performed on the target marked speech recognition result to obtain a slot recognition result.

[0021] Application program interface prediction is performed on the target marked speech recognition result to obtain a predicted application interface.

[0022] According to the slot recognition result and the predicted application interface, the predicted application interface is selected to perform application program interface parameter filling to obtain the first understanding result.

[0023] In this way, the server performs slot recognition on the target marked speech understanding result to obtain a slot recognition result. Then, the server performs application program interface prediction on the target marked speech understanding result to obtain a predicted application interface. Finally, according to the slot recognition result and the predicted application interface, the server selects the predicted application interface to perform application program interface parameter filling to obtain the first understanding result. In this way, through slot recognition and application program interface prediction, the system can more accurately understand the user's request and select the most suitable application interface to perform the task, thereby reducing misunderstanding and errors and providing a good user experience.

[0024] In some embodiments, the natural language understanding of the target marked speech recognition result to obtain the first understanding result comprises:

[0025] Based on a pre-trained large language model, the target marked speech recognition result is subjected to natural language understanding to obtain the first understanding result.

[0026] Thus, based on the pre-trained large language model, the server performs natural language understanding on the target labeled speech understanding result to obtain a first understanding result. In this way, the pre-trained large language model is used to perform natural language understanding on the target labeled speech recognition result, which can improve the accuracy and efficiency of natural language understanding.

[0027] In some embodiments, the voice interaction is completed according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request.

[0028] The first recognition result and the second recognition result are spliced to obtain a spliced recognition result.

[0029] The spliced recognition result and the third recognition result are matched.

[0030] The voice interaction is completed according to the matching result.

[0031] Thus, the server splices the first recognition result and the second recognition result to obtain a spliced recognition result. Then, the server matches the spliced recognition result and the third recognition result. Finally, the server completes the voice interaction according to the matching result. In this way, by splicing multiple recognition results obtained during the reception of the voice request and matching them with the third recognition result, the server can accurately understand the complex voice request of the user and effectively perform the corresponding operation.

[0032] In some embodiments, the voice interaction is completed according to the matching result, including:

[0033] In the case where the spliced recognition result and the third recognition result match, the target output result is obtained according to the first understanding result and the second understanding result.

[0034] The target output result is sent to the vehicle to complete the voice interaction; or

[0035] In the case where the spliced recognition result and the third recognition result do not match, the voice request is subjected to natural language understanding to obtain the target output result.

[0036] The target output result is sent to the vehicle to complete the voice interaction.

[0037] Thus, the server obtains the target output result according to the first understanding result and the second understanding result in a case where the spliced recognition result and the third recognition result match. Then, the server issues the target output result to the vehicle to complete the voice interaction. Or, in a case where the spliced recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain the target output result. Then, the server issues the target output result to the vehicle to complete the voice interaction. In this way, in processing a complex voice instruction, the system can more accurately understand and respond to the voice instruction of the user by flexibly processing the matching case of the spliced recognition result and the third recognition result, and can effectively correct the understanding deviation, which helps to improve the overall performance and user experience of the voice interaction system.

[0038] In some embodiments, the obtaining the target output result according to the first understanding result and the second understanding result comprises:

[0039] performing screening processing on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result;

[0040] performing screening processing on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result;

[0041] obtaining the target output result according to the first sub-target output result and the second sub-target output result.

[0042] Thus, the server performs screening processing on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result. Then, the server performs screening processing on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result. Finally, the server obtains the target output result according to the first sub-target output result and the second sub-target output result. In this way, in processing a complex voice request, the system can more accurately extract and integrate the natural language understanding result by performing step-by-step screening and processing on the understanding result, so as to generate a more accurate target output result, which helps to improve the understanding and response quality of the voice interaction system.

[0043] The embodiments of the present application provide a server, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the voice interaction method described above is implemented.

[0044] The embodiment of the present application provides a nonvolatile computer readable storage medium containing a computer program, which, when executed by one or more processors, implements the voice interaction method.

[0045] Additional aspects and advantages of the embodiments of the present application will be in part apparent and in part pointed out below. BRIEF DESCRIPTION OF DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings in which:

[0047] Figure 1 is one of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0048] Figure 2 is one of the flow diagrams of the voice request processing according to some embodiments of the present application;

[0049] Figure 3 is another of the flow diagrams of the voice request processing according to some embodiments of the present application;

[0050] Figure 4 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0051] Figure 5 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0052] Figure 6 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0053] Figure 7 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0054] Figure 8 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0055] Figure 9 is the flow diagram of the matching processing according to some embodiments of the present application;

[0056] Figure 10 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0057] Figure 11 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0058] Figure 12Fig. 9 is a ninth schematic diagram of a flow of a voice interaction method according to some embodiments of the present application. DETAILED DESCRIPTION

[0059] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and letters throughout the several views denote the same or

[0060] In the related art, in order to improve the response speed and user experience, the system can perform natural language processing (NLP) in real time during the process of receiving the user voice request, that is, during the voice activity detection (VAD) process. In this way, the system is allowed to start understanding and responding to the user's request before the user voice instruction is completely finished, thereby realizing faster interaction.

[0061] However, when the user voice request becomes more complex, for example, contains multiple sub-instructions or needs to understand long sentences, the inference length of the system will increase. In this case, the inference effect of the in-vehicle intelligent system may decrease, because the system needs more time and computing resources to process and understand the complex voice request. This delay and performance decline may affect the user experience, because the user may need to wait longer to get a response, or the system may not accurately understand the user's intention.

[0062] Voice activity detection refers to a technology for determining the presence of a speech signal. In speech processing and communication systems, the role of VAD is to identify the start and end of speech activity, so that only the part containing speech can be processed, and the silent period or background noise can be ignored.

[0063] Based on the above problems, please refer to Figure 1 The embodiments of the present application provide a voice interaction method, the method comprising:

[0064] 01: receiving the voice request segments forwarded by the vehicle in sequence;

[0065] 02: performing natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result;

[0066] 03: performing natural language processing on the second voice request segment set to obtain a second recognition result and a second understanding result;

[0067] 04: performing natural language recognition on the received voice request to obtain a third recognition result;

[0068] 05: completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0069] The embodiment of the application further provides a server comprising a memory and a processor. The voice interaction method of the embodiment of the application can be implemented by the server of the embodiment of the application. Specifically, the memory stores a computer program, and the processor is configured to receive the voice request fragments forwarded by the vehicle in sequence, perform natural language processing on the first set of voice request fragments to obtain the first recognition result and the first understanding result, perform natural language processing on the second set of voice request fragments to obtain the second recognition result and the second understanding result, perform natural language recognition on the received voice request to obtain the third recognition result, and complete the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0070] The embodiment of the application further provides a voice interaction device. The voice interaction method of the embodiment of the application can be implemented by the voice interaction device of the embodiment of the application. Specifically, the voice interaction device comprises a receiving module, a language processing module, a language recognition module and a processing module. The receiving module is configured to receive the voice request fragments forwarded by the vehicle in sequence. The language processing module is configured to perform natural language processing on the first set of voice request fragments to obtain the first recognition result and the first understanding result. The language processing module is configured to perform natural language processing on the second set of voice request fragments to obtain the second recognition result and the second understanding result. The language recognition module is configured to perform natural language recognition on the received voice request to obtain the third recognition result. The processing module is configured to complete the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request.

[0071] Specifically, refer to Figure 2 In the classic technology, the process of human-computer interaction can be divided into five stages, including receiving a user voice request, sound activity detection, automatic speech recognition, natural language understanding (NLU) and robot understanding instruction. Among them, the time required for receiving a user voice request can be A1, the time required for sound activity detection can be A2, the time required for automatic speech recognition can be A3, the time required for natural language understanding can be A4, and the time required for robot understanding instruction can be A5. The time for the server to complete this human-computer interaction is A1+A2+A3+A4+A5.

[0072] Robot understanding instruction refers to that after a large language model obtains an instruction (natural language understanding result), the instruction can be a program code, and the robot interacting with the person needs to understand the program code, which needs a certain time.

[0073] However, such a classic voice interaction technology, that is, after receiving a complete voice request, the complete voice request is processed by natural language processing, the system response time is slow, and the interaction experience is poor. Based on the above problems, a voice interaction method capable of improving the response speed is given in the related art, that is, the voice request fragments are processed in real time during the process of receiving the user voice request. As shown in Figure 2 It can be found that the server performs natural speech recognition processing and natural language understanding processing when receiving the first voice request fragment, and the natural language recognition processing and natural language understanding processing of each voice request fragment are also completed when the voice request is received. Subsequently, the server performs sound activity detection, and performs natural language recognition processing and result selection processing on the complete voice request. In this way, the time required for the server to receive the user voice request is A1, the time required for the sound activity detection is A2, the time required for the natural language recognition processing on the complete voice request is A3, and the time required for the result selection processing can be A6. The time for the server to complete this human-computer interaction is A1+A2+A3+A6, and the response speed is faster.

[0074] The result selection processing refers to selecting the natural language understanding result obtained according to the voice request fragment by matching and filtering processing, selecting the appropriate natural language understanding result and issuing it to the vehicle to complete the voice interaction. At the same time, since the program code of each natural language understanding result obtained according to different voice request fragments has been understood by the robot, the response time is reduced.

[0075] Compared with the related art, when facing a complex voice request, the length of the voice request that needs to be inferred increases, resulting in a decline in the inference effect of the vehicle-mounted intelligent system and a decline in user experience. The voice interaction method provided in the embodiments of the present application can more accurately understand the user's intention through the natural language recognition and natural language understanding of different voice request fragment sets in stages, and reply to the voice request in combination with the natural language understanding results of different stages, thereby improving the accuracy of the system in processing long sentences or complex sentences.

[0076] The voice request segment refers to a voice segment obtained by the server intercepting the received voice request based on a certain interval length when the server first receives the voice request forwarded by the vehicle. For example, if the voice request issued by the user is "open the window and open the air conditioner", the server may intercept two voice request segments at different times, the first voice request segment is "open", and the second voice request segment is "window".

[0077] The voice request segment set refers to a voice request segment set capable of outputting the application interface and the application program interface parameter filling result. That is, if the user inputs "open the window and open the air conditioner", the voice request segment set is "open the window" and "open the window".

[0078] The first voice request segment set refers to the first voice request segment set capable of outputting the application interface and the application program interface parameter filling result, including all voice request segments received within the first time. The first time is used to indicate the time capable of outputting the application interface and the application program interface parameter filling result. That is, if the user inputs "open the window and open the air conditioner", the first voice request segment set is "open the window", which is composed of the first voice request segment and the second voice request segment.

[0079] The second voice request segment set can refer to all voice request segment sets received after the first voice request segment set, which is a large concept, including all voice request segments received within the second time. The second time is used to indicate any time capable of outputting the application interface and the application program interface parameter filling result after the first time. That is, if the user inputs "open the window and open the air conditioner", the second voice request segment set is "open the air conditioner", which is composed of the fourth voice request segment and the fifth voice request segment. If the user inputs "open the window, open the air conditioner, and play music", the second voice request segment set is "open the air conditioner" and "play music", "open the air conditioner" is composed of the third voice request segment and the fourth voice request segment, and "play music" is composed of the sixth voice request segment and the seventh voice request segment. It should be noted that "play music" can be referred to as the third voice request segment set in terms of time sequence.

[0080] Please refer to Figure 3The server can receive the voice request fragments forwarded by the vehicle in sequence, i.e., process the voice stream in real time, instead of waiting for the entire voice request to end before processing. The following describes the voice interaction method of the embodiment of the application with an example. If the voice request given by the user is "open the window and open the air conditioner", the server will receive multiple voice request fragments. The voice request fragment received at T0 is "open", the voice request fragment received at T1 is "window", the voice request fragment received at T2 is "and", the voice request fragment received at T3 is "open", and the voice request fragment received at T4 is "air conditioner".

[0081] Next, the server performs natural language processing on the first voice request fragment set to obtain a first recognition result and a first understanding result. The first voice request fragment set includes all the voice request fragments received at the first time. Continuing the above example, the server will perform natural language processing on the received voice request fragments. At T0, the server will perform natural language processing on the voice request fragment "open" to obtain a natural language recognition result of "open" and a natural language understanding result of "Unclear". It can be found that no application interface and application program interface parameter filling result are output at T0, so the voice request fragment set at T0 is not the first voice request fragment set. At T1, the server will perform natural language processing on the voice request fragment set of the voice request fragment "open" and the voice request fragment "window" to obtain a natural language recognition result of "open the window" and a natural language understanding result of "WindowOpen, {device: window}". It can be found that the application interface and application program interface parameter filling result are output at T1, and this voice request fragment set is the first voice request fragment set in this voice request that outputs the application interface and application program interface parameter filling result. Therefore, the voice request fragment set at T1 is the first voice request fragment set, the natural language recognition result "open the window" is the first recognition result, and the natural language understanding result "WindowOpen, {device: window}" is the first understanding result.

[0082] Then, the server performs natural language processing on the second set of voice request segments to obtain a second recognition result and a second understanding result. The second set of voice request segments includes all voice request segments received at a second time point after the first set of voice request segments. In the above example, after obtaining the first set of voice request segments, the server does not need to consider voice request segments received before the first time point, i.e., at the T2 time point, the server does not need to perform natural language processing on voice request segments received at the T0 time point and the T1 time point. Therefore, the set of voice request segments that need to be processed at the T2 time point is "and", and the natural language recognition result and the natural language understanding result are "and" and "AcOpen, {device: air conditioner}", respectively. In this way, the second set of voice request segments is the set of voice request segments received at the T4 time point, the second recognition result is "open the window and open the air conditioner", and the second understanding result is "AcOpen, {device: air conditioner}".

[0083] After a certain period of time, the server determines that the user has completed the voice request explanation because no voice request segments forwarded by the vehicle are received. The server performs natural language recognition on the received voice request to obtain a third recognition result. The voice request is obtained by splicing all voice request segments. In other words, the third natural language recognition result is obtained by performing natural language recognition on the final voice request. In the above example, the second natural language recognition result is "open the window and open the air conditioner".

[0084] Finally, the server completes the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request.

[0085] In summary, the voice interaction method and the server provided by the embodiments of the present application are as follows: the server receives voice request segments forwarded by a vehicle in sequence. Then, the server performs natural language processing on a first voice request segment set to obtain a first recognition result and a first understanding result, the first voice request segment set including voice request segments received at a first time. Then, the server performs natural language processing on a second voice request segment set to obtain a second recognition result and a second understanding result, the second voice request segment set including voice request segments received at a second time, the voice request segments being continuous to the first voice request segment set, and the second time being later than the first time. The server further performs natural language recognition on a received voice request to obtain a third recognition result, the voice request being obtained by splicing the voice request segments. Finally, the server completes voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result and the voice request. In this way, in an end-to-end architecture, the streaming natural language recognition and understanding capabilities of the large language model are realized by combining the large language model with the streaming dialogue technology, the resources and time required for waiting for the completion of the entire voice request are reduced, and thus the response time of the server is greatly shortened. Moreover, through the natural language recognition and understanding of different voice request segment sets in different stages, the large language model can accurately understand the user's intention, and the accuracy of the system in processing long sentences or complex sentences is improved by combining the natural language understanding results of different stages.

[0086] Referring to Figure 4 In some embodiments, step 02 (performing natural language processing on the first voice request segment set to obtain a first recognition result and a first understanding result) includes:

[0087] 021: performing natural language recognition on the first voice request segment set to obtain a first recognition result;

[0088] 022: performing marking processing on the first recognition result according to a first preset marking to obtain a target marked voice recognition result;

[0089] 023: performing natural language understanding on the target marked voice recognition result to obtain a first understanding result.

[0090] In some embodiments, the language processing module is further configured to perform natural language recognition on the first voice request segment set to obtain a first recognition result, perform marking processing on the first recognition result according to a first preset marking to obtain a target marked voice recognition result, and perform natural language understanding on the target marked voice recognition result to obtain a first understanding result.

[0091] In some embodiments, the processor is further configured to perform natural language recognition on the first set of voice request segments to obtain a first recognition result, and perform marking processing on the first recognition result according to a first preset marker to obtain a target marked voice recognition result, and perform natural language understanding on the target marked voice recognition result to obtain a first understanding result.

[0092] Specifically, natural language recognition refers to the process of converting human natural language (such as English, Chinese, etc.) into a form that machines can understand and process.

[0093] The first preset marker refers to a marker "wait" added in front of the natural language recognition result before the large language model performs natural language understanding processing on the natural language recognition result. This marker "wait" can guide the large language model to enter the natural language understanding processing mode, i.e., to inform the large language model that the next text can be processed for natural language understanding.

[0094] The server first performs natural language recognition on the first set of voice request segments to obtain a first recognition result. Then, the server performs marking processing on the first natural language recognition result according to a first preset marker to obtain a target marked voice recognition result. Continuing the above example, at T1, the target marked language recognition result obtained by the server is "wait open the window". It should be noted that the marking processing is not only performed on the voice request segments that can output application interface and application program interface parameter filling results, but also performed on the natural language recognition results obtained by performing natural language recognition on each voice request segment received. Continuing the above example, at T0, the target marked language recognition result obtained by the server is "wait open".

[0095] Subsequently, the server performs natural language understanding on the target marked voice recognition result to obtain a first understanding result. Continuing the above example, at T1, the server performs natural language understanding processing on "wait open the window" to obtain a natural language understanding result "WindowOpen, {device: window}".

[0096] In this way, by using the first preset marker to perform marking processing on the first natural language recognition result, the large language model can accurately recognize the input natural language recognition result and accurately respond to the user's request, thereby improving the accuracy of natural language understanding and providing a good user experience.

[0097] Please refer to Figure 5 In some embodiments, the method further comprises:

[0098] 024: performing marking processing on the first understanding result according to a second preset marker to obtain a target marked voice understanding result.

[0099] In some embodiments, the processing module is further configured to perform mark processing on the first understanding result according to a second preset mark to obtain a target mark speech understanding result.

[0100] In some embodiments, the processor is further configured to perform mark processing on the first understanding result according to a second preset mark to obtain a target mark speech understanding result.

[0101] Specifically, the second preset mark refers to a mark waitEnd added after the first understanding result after the large language model performs natural language understanding processing on the target mark speech recognition result to obtain the first understanding result. This mark waitEnd can guide the large language model to perform natural language understanding processing without being affected by the natural language understanding result obtained at the previous time. That is, the server will only perform natural language understanding processing according to the obtained natural language recognition result. Continuing the above example, for example, at T0, the large language model will receive the speech request segment set "open" and the natural language understanding result "wait open waitEnd" obtained at T0. Due to the second preset mark waitEnd, the server will not perform natural language understanding processing on the natural language understanding result "wait open waitEnd".

[0102] The server performs mark processing on the first understanding result according to a second preset mark to obtain a target mark speech understanding result.

[0103] In this way, by using the second preset mark to perform mark processing on the natural language understanding result, the large language model can optimize the natural language understanding process, ignore the natural language understanding result marked with the second preset mark, reduce the accumulation of errors in the conversation process, and thus improve the overall understanding accuracy. Moreover, the amount of information that needs to be considered can be reduced, so that the system can process subsequent inputs faster, thereby improving the overall response speed.

[0104] Please refer to Figure 6 In some embodiments, step 023 (performing natural language understanding on the target mark speech recognition result to obtain a first understanding result) comprises:

[0105] 0231: performing slot identification on the target mark speech recognition result to obtain a slot identification result;

[0106] 0232: performing application program interface prediction on the target mark speech recognition result to obtain a predicted application program interface;

[0107] 0233: selecting a predicted application program interface to perform application program interface parameter filling according to the slot identification result and the predicted application program interface, to obtain a first natural language understanding result.

[0108] In certain embodiments, the processing module is further configured to perform slot identification on the target mark speech recognition result to obtain a slot identification result. The processing module is further configured to perform application program interface (API) prediction on the target mark speech recognition result to obtain a predicted application program interface (API). The processing module is further configured to select the predicted application program interface based on the slot identification result and the predicted application program interface, and execute API parameter filling to obtain a first natural language understanding result.

[0109] In certain embodiments, the processor is further configured to perform slot identification on the target mark speech recognition result to obtain a slot identification result, perform application program interface prediction on the target mark speech recognition result to obtain a predicted application program interface, and select the predicted application program interface based on the slot identification result and the predicted application program interface to perform application program interface parameter filling to obtain a first natural language understanding result.

[0110] Specifically, slot recognition refers to extracting specific information fragments from user input, which are usually called "slots". Slots are usually key information required to complete a task or request, such as time, place, object, etc. Taking the user voice request "What will the temperature be tomorrow" as an example, the slot information that can be obtained through slot recognition includes ["tomorrow" - date (Date)], that is, the slot information includes the slot value and the slot type, where "tomorrow" is the slot value and the date (Date) is the slot type. Taking the user voice request "Navigate to Zhongguancun" as an example, the slot information that can be obtained through slot recognition is ["Zhongguancun" - place name (Place)], where "Zhongguancun" is the slot value and the place name (Place) is the slot type.

[0111] The slot recognition result refers to the named entity obtained by slot recognition of the voice request, such as the slot information ["tomorrow" - Date] and the slot information ["Zhongguancun" - Place] mentioned above.

[0112] The server performs slot recognition on the target tag speech recognition result to obtain a slot recognition result. Continuing with the above example, at time T1, the server performs slot recognition on "open the car window" and obtains a slot recognition result "car window - device".

[0113] Next, the server performs API prediction on the target tag speech recognition result to obtain a predicted API. Continuing with the above example, at time T1, the server performs API prediction on the speech request segment set "open the car window" to obtain a predicted API "WindowOpen".

[0114] Finally, the server performs application interface parameter filling based on the slot recognition result and the predicted application interface to obtain a first natural language understanding result.

[0115] Need to be explained, for each time received all voice request fragments, will be the above-mentioned processing. If the slot recognition result is not recognized and / or no predicted application interface is obtained, the large language model will output "Unclear". If the voice request fragment has no practical meaning, the large language model will output "Noise", in some embodiments, the large language model will exclude the voice request fragment whose natural language understanding result is "Noise", and in subsequent natural language recognition processing, the voice request fragment will not be considered.

[0116] In this way, through slot recognition and application interface prediction, the system can more accurately understand the user's request and select the most suitable application interface to execute the task, thereby reducing misunderstanding and errors and providing good user experience.

[0117] Please refer to Figure 7 In some embodiments, step 023 (performing natural language understanding on the target labeled voice recognition result to obtain a first understanding result) comprises:

[0118] 0234: Based on the pre-trained large language model, performing natural language understanding on the target labeled voice recognition result to obtain a first understanding result.

[0119] In some embodiments, the processing module is further configured to perform natural language understanding on the target labeled voice recognition result based on the pre-trained large language model to obtain a first understanding result.

[0120] In some embodiments, the processor is further configured to perform natural language understanding on the target labeled voice recognition result based on the pre-trained large language model to obtain a first understanding result.

[0121] Specifically, in some embodiments, the large language model has been trained. In some embodiments, the training steps of the large language model are as follows: training data is constructed according to the collected voice request samples and the vehicle system function customization scheme samples constructed according to the voice request samples. The sample data can be derived from the collected user logs, such as the record of the user's history of successfully constructing a vehicle system function customization scheme according to a voice request. Of course, the training data can also be constructed as needed. For example, a set of training data can be: the voice request "open the air conditioner immediately", and the vehicle system function customization scheme constructed according to the voice request is "AcOpen{device: air conditioner}".

[0122] Further, to guide the large model to meet the task requirements, prompt information template configuration is needed. Specifically, the prompt information template is used to guide the model to generate output, providing the background and relevant guidance of the question. It usually includes the description and explanation of the question, which tells the model how to process the input data. The prompt information template can provide additional auxiliary information to help the model better understand and generate output. It can also include specific formats, styles, topics that the model needs to follow, or indicate that the model should focus on certain specific aspects and details. In this way, the model will understand the requirements of the task according to the content of the prompt information template, and generate answers and texts that meet the expectations. Generally, the configuration process of the prompt information template can include explicitly stating the role the model needs to play (optional), explicitly stating the text content and structure of the input, and explicitly stating the text content and structure of the output.

[0123] In one example, the content of the prompt information template is as follows:

[0124] "Suppose you are an intelligent cockpit dialogue system that needs to understand the recognized instructions and output the corresponding API and Arguments. The input instruction fragments will be input in multiple times. Please determine the current API and Arguments based on the existing instructions."

[0125] Based on the pre-trained large language model, the server performs natural language understanding on the target marked speech recognition result to obtain a first natural language understanding result.

[0126] In this way, the pre-trained large language model can improve the accuracy and efficiency of natural language understanding.

[0127] See Figure 8 In some embodiments, step 05 (completing voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request) includes:

[0128] 051: Splicing the first recognition result and the second recognition result to obtain a spliced recognition result;

[0129] 052: Matching the spliced recognition result and the third recognition result;

[0130] 053: Completing voice interaction according to the matching result.

[0131] In some embodiments, the processing module is configured to splice the first recognition result and the second recognition result to obtain a spliced recognition result, match the spliced recognition result and the third recognition result, and complete voice interaction according to the matching result.

[0132] In some embodiments, the processor is further configured to perform stitching processing on the first recognition result and the second recognition result to obtain a stitched recognition result, and perform matching processing on the stitched recognition result and the third recognition result, and complete the voice interaction according to a result of the matching processing.

[0133] Specifically, the matching processing refers to comparing the first natural language recognition result and the second natural language recognition result based on text alignment, semantic analysis and the like, to verify consistency of the first natural language recognition result and the second natural language recognition result.

[0134] Referring to Figure 9 , the server performs stitching processing on the first recognition result and the second recognition result to obtain a stitched recognition result. Continuing with the above example, at T4, the large language model does not receive a new voice segment within a certain time, and confirms that the voice request of the user this time is received. If the received voice request is "open the window and open the air conditioner", the first recognition result obtained is "open the window", and the second recognition result is "and open the air conditioner", and the first recognition result and the second recognition result are stitched. If the received voice request is "open the window, open the air conditioner, and play music", then the first recognition result obtained is "open the window", and the second recognition result is "open the air conditioner" and "and play music". In some embodiments, the matching processing is performed through special marks "group" and "groupEnd", and the result is the position corresponding to the natural language understanding result.

[0135] Then, the server performs matching processing on the stitched recognition result and the third recognition result. Continuing with the above example, if the received voice request is "open the window and open the air conditioner", the server performs stitching processing on the first recognition result and the second recognition result to obtain "open the window and open the air conditioner", and performs matching processing on the third recognition result "open the window and open the air conditioner" obtained by performing natural language recognition processing on the complete voice request.

[0136] Finally, the server completes the voice interaction according to a result of the matching processing.

[0137] In this way, by stitching the multiple recognition results obtained in the process of receiving the voice request, and performing matching processing on the third recognition result, the server can accurately understand the complex voice request of the user, and effectively perform the corresponding operation.

[0138] Referring to Figure 10 and Figure 11 In some embodiments, step 053 (completing the voice interaction according to the result of the matching processing) comprises:

[0139] 0531: In a case where the spliced recognition result matches the third recognition result, the target output result is obtained according to the first understanding result and the second understanding result;

[0140] 0532: The target output result is sent to the vehicle to complete the voice interaction; or

[0141] 0533: In a case where the spliced recognition result does not match the third recognition result, the target output result is obtained by performing natural language understanding on the voice request; and

[0142] 0534: The target output result is sent to the vehicle to complete the voice interaction.

[0143] In some embodiments, the processing module is further configured to, in a case where the spliced recognition result matches the third recognition result, obtain the target output result according to the first understanding result and the second understanding result, and send the target output result to the vehicle to complete the voice interaction. Alternatively, the processing module is further configured to, in a case where the spliced recognition result does not match the third recognition result, obtain the target output result by performing natural language understanding on the voice request, and send the target output result to the vehicle to complete the voice interaction.

[0144] In some embodiments, the processor is further configured to, in a case where the spliced recognition result matches the third recognition result, obtain the target output result according to the first understanding result and the second understanding result, and send the target output result to the vehicle to complete the voice interaction. Alternatively, the processor is further configured to, in a case where the spliced recognition result does not match the third recognition result, obtain the target output result by performing natural language understanding on the voice request, and send the target output result to the vehicle to complete the voice interaction.

[0145] Specifically, please refer to Figure 10 , in a case where the spliced recognition result matches the third recognition result, the target output result is obtained according to the first understanding result and the second understanding result. Then, the server sends the target output result to the vehicle to complete the voice interaction. Following the above example, it can be found that the third recognition result "open the window and turn on the air conditioner" is consistent with the spliced recognition result "open the window and turn on the air conditioner", and then the target output result "WindowOpen, {device: window}; AcOpen, {device: air conditioner}" is obtained according to the first understanding result "WindowOpen, {device: window}" and the second understanding result "AcOpen, {device: air conditioner}".

[0146] In the case that the spliced recognition result and the third recognition result do not match, the server performs natural language understanding on the voice request to obtain a target output result. Then, the server issues the target output result to the vehicle to complete the voice interaction. Continuing the above example, if the spliced recognition result is "open the window and open the air conditioner", the spliced recognition result "open the window and open the air conditioner" is inconsistent with the third recognition result "open the window and open the air conditioner", and the large language model will perform natural language understanding on the second natural language recognition result to obtain a target output result. Then, the target output result is issued to the vehicle to complete the voice interaction.

[0147] In this way, when processing complex voice instructions, by flexibly processing the matching conditions of the spliced recognition result and the third recognition result, the system can more accurately understand and respond to the user's voice instructions, and can also effectively correct the understanding deviation, which helps to improve the overall performance and user experience of the voice interaction system.

[0148] Please refer to Figure 12 In some embodiments, step 0531 (obtaining a target output result according to the first understanding result and the second understanding result) comprises:

[0149] 05311: performing screening processing on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result;

[0150] 05312: performing screening processing on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result;

[0151] 05313: obtaining a target output result according to the first sub-target output result and the second sub-target output result.

[0152] In some embodiments, the processing module is configured to perform screening processing on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result; perform screening processing on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result; and obtain a target output result according to the first sub-target output result and the second sub-target output result.

[0153] In some embodiments, the processor is further configured to perform a screening process on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result; perform a screening process on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result; and obtain a target output result according to the first sub-target output result and the second sub-target output result.

[0154] Specifically, the screening process refers to performing a matching process according to the natural language recognition result corresponding to the natural language understanding result and the third recognition result, and if the third recognition result comprises the natural language recognition result, it is considered that the natural language understanding result is a sub-target output result.

[0155] The server performs a screening process on the first understanding result to obtain a first sub-target output result, the first sub-target output result comprising a first application interface and a first application program interface parameter filling result. Continuing the above example, if the first recognition result is "open the car window", and the third recognition result is "open the car window and open the air conditioner", it is considered that the first understanding result corresponding to the first recognition result cannot be used as the first sub-target output result. If the first recognition result is "open the car window", and the third recognition result is "open the car window and open the air conditioner", it is considered that the first understanding result corresponding to the first recognition result can be used as the first sub-target output result, i.e., the obtained first application interface "WindowOpen" and the first application program interface parameter filling result "{WindowOpen, device: car window}" are used as the first sub-target output result.

[0156] Then, the server performs a screening process on the second understanding result to obtain a second sub-target output result, the second sub-target output result comprising a second application interface and a second application program interface parameter filling result. Continuing the above example, if the second recognition result is "and open the car window", and the third recognition result is "open the car window and open the air conditioner", it is considered that the first understanding result corresponding to the second recognition result cannot be used as the second target output result. If the second recognition result is "open the car window", and the third recognition result is "open the car window and open the air conditioner", it is considered that the second understanding result corresponding to the second recognition result can be used as the second sub-target output result, i.e., the obtained second application interface "AcOpen" and the first application program interface parameter filling result "{AcOpen, device: air conditioner}" are used as the second sub-target output result.

[0157] Finally, the server obtains a target output result according to the first sub-target output result and the second sub-target output result.

[0158] It should be noted that in some embodiments, the screening process is not performed on each natural language understanding result respectively, that is, the natural language recognition result corresponding to each natural language understanding result is not compared with the third recognition result respectively, but the natural language recognition result obtained at the last moment is compared with the third recognition result directly. When the target output result is finally output, all the natural language understanding results are delivered to the vehicle or none of the natural language understanding results is delivered.

[0159] In this way, when processing a complex voice request, by performing step-by-step screening and processing on the understanding results, the system can more accurately extract and integrate the natural language understanding results, thereby generating more accurate target output results, which helps to improve the understanding and response quality of the voice interaction system.

[0160] The present application also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, the one or more processors perform the method of the present application.

[0161] It can be understood that the computer program includes computer program code. The computer program code can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable storage medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), and software distribution medium, etc.

[0162] In the description of the present application, the description of the terms "specifically", "further", "particularly", "understandably" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0163] Any processes or methods described in the flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) of the process, and / or that the various processes described herein can be understood as representing executable instructions, code segments, or portions of code which include one or more steps for implementing the functions (or steps) of the processes, and that the various processes described herein can be implemented with or without the use of hardware, software, firmware, or any combination thereof.

[0164] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above-described embodiments are exemplary only, and should not be understood as limiting the present application, and that changes, modifications, substitutions, and variations of the above-described embodiments can be made by those skilled in the art within the scope of the present application.

Claims

1. A voice interaction method, characterized in that, Comprising: receiving a voice request fragment forwarded by a vehicle in sequence; performing natural language processing on a first voice request fragment set to obtain a first recognition result and a first understanding result, the first voice request fragment set including the voice request fragment received at a first time; performing natural language processing on a second voice request fragment set to obtain a second recognition result and a second understanding result, the second voice request fragment set including the voice request fragment received at a second time subsequent to the first voice request fragment set, the second time being later than the first time; performing natural language recognition on a received voice request to obtain a third recognition result, the voice request being concatenated from the voice request fragments; completing the voice interaction according to the first recognition result, the second recognition result, the third recognition result, the first understanding result, the second understanding result, and the voice request; the method further comprising: performing natural language processing on the first understanding result according to a second preset mark to obtain a target marked voice understanding result. the performing natural language understanding on the target marked voice recognition result to obtain the first understanding result, comprising: performing slot recognition on the target marked voice recognition result to obtain a slot recognition result; 2. The voice interaction method of claim 1, wherein, performing application program interface prediction on the target marked voice recognition result to obtain a predicted application interface; selecting the predicted application interface to perform application program interface parameter filling according to the slot recognition result and the predicted application interface, to obtain the first understanding result.

3. The voice interaction method of claim 1, wherein, the performing natural language understanding on the target marked voice recognition result to obtain the first understanding result, comprising: performing natural language understanding on the target marked voice recognition result based on a pre-trained large language model to obtain the first understanding result. the completing the voice interaction according to the matching processing result, comprising: in a case where the concatenated recognition result and the third recognition result match, obtaining a target output result according to the first understanding result and the second understanding result; 4. The voice interaction method of claim 1, wherein, issuing the target output result to the vehicle to complete the voice interaction; or ​ 5. The voice interaction method of claim 1, wherein, ​ ​ ​ ​ 6. The voice interaction method of claim 5, wherein, ​ ​ ​ In a case where the splicing recognition result and the third recognition result do not match, performing natural language understanding on the voice request to obtain the target output result; The target output result is sent to the vehicle to complete voice interaction.

7. The voice interaction method of claim 6, wherein, The target output result is obtained according to the first understanding result and the second understanding result, including: The first understanding result is filtered to obtain a first sub-target output result, and the first sub-target output result includes a first application interface and a first application program interface parameter filling result; The second understanding result is filtered to obtain a second sub-target output result, and the second sub-target output result includes a second application interface and a second application program interface parameter filling result; The target output result is obtained according to the first sub-target output result and the second sub-target output result.

8. A server, characterized by The server includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, the voice interaction method of any one of claims 1-7 is implemented.

9. A non-transitory computer readable storage medium embodying a computer program, comprising: When the computer program is executed by one or more processors, the voice interaction method of any one of claims 1-7 is implemented.

Images