Voice interaction method, server and computer readable storage medium

Abstract

The invention discloses a voice interaction method, a server and a computer readable storage medium, and the method comprises the steps: receiving a first voice request, forwarded by a vehicle, of a current conversation round, and carrying out the natural language understanding of the first voice request and a second voice request according to the first and second voice requests and a natural language understanding result of the second voice request; and performing application program interface prediction and interface parameter filling on the first voice request, and sending an execution result of interface parameter filling to the vehicle to complete voice interaction. Therefore, the voice request processing of the current dialogue round is executed based on the voice request of the previous dialogue round and the natural language understanding result, the voice interaction based on the context is realized, the result credibility of the first voice request is guaranteed, the reliable execution of the voice interaction is guaranteed, and the voice interaction experience is improved. The user experience of the voice interaction function is guaranteed, and the application effect of the voice interaction function in a complex multi-round dialogue scene is improved.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, a server, and a computer-readable storage medium. Background Art

[0002] As users gradually get used to the in-vehicle voice interaction function, they may control the vehicle through short voice commands during the voice interaction process, such as "adjust to 28 degrees" instead of "set the air conditioning temperature to 28 degrees Celsius". However, since such short voice commands can point to a variety of in-vehicle functions, it is difficult to identify the user's intention, and thus difficult to be accurately understood and executed. Summary of the invention

[0003] The present application provides a voice interaction method, a server and a computer-readable storage medium.

[0004] A voice interaction method provided in an embodiment of the present application includes:

[0005] Receiving a first voice request of a current conversation round forwarded by a vehicle;

[0006] According to the first voice request, the second voice request and the natural language understanding result of the second voice request, performing application program interface prediction and interface parameter filling for the first voice request, wherein the second voice request is a voice request of a previous dialogue round of the current dialogue round;

[0007] The execution result of filling the interface parameters is sent to the vehicle to complete the voice interaction.

[0008] In the voice interaction method provided in the implementation mode of the present application, the server can receive the first voice request of the current conversation round forwarded by the vehicle, and perform application program interface prediction and interface parameter filling for the first voice request based on the first voice request, the second voice request and the natural language understanding result of the second voice request, and send the execution result of the interface parameter filling to the vehicle, so that the vehicle can perform corresponding operations according to the execution result, thereby completing the voice interaction with the user.

[0009] Thus, in the implementation mode of the present application, the processing of the first voice request of the current dialogue round can be performed based on the second voice request of the previous dialogue round and the natural language understanding result of the second voice request, thereby realizing context-based voice interaction, and further guaranteeing the credibility of the result predicted by the application program of the first voice request and the execution result of the parameter filling to a certain extent, thereby guaranteeing the reliable execution of voice interaction and the user experience of the voice interaction function. In addition, when the voice request of the current dialogue round is relatively short, the voice request and natural language understanding result of the previous dialogue round can be tracked to perform the user intention determination and natural language understanding corresponding to the voice request of the current dialogue round, which guarantees the reliable processing of the voice request of the current dialogue round to a certain extent and improves the applicability of the voice interaction function in complex multi-round dialogue scenarios.

[0010] In certain embodiments of the present application, the performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request, and the natural language understanding result of the second voice request includes:

[0011] Performing concatenation processing on the first voice request, the second voice request, and the natural language understanding result;

[0012] The application program interface prediction and the interface parameter filling are performed on the first voice request according to the result of the splicing processing.

[0013] Thus, in the implementation mode of the present application, the first voice request, the second voice request and the natural language understanding result of the second voice request can be concatenated, and the application program interface prediction and interface parameter filling of the first voice request can be performed based on the concatenated result.

[0014] In certain embodiments of the present application, the performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request, and the natural language understanding result of the second voice request includes:

[0015] Configure the dialog turn label of each parameter in the result of the splicing process.

[0016] Thus, in an embodiment of the present application, after completing the first voice request, the second voice request, and the natural language understanding results of the second voice request and performing splicing processing to obtain the splicing processing result, the conversation turn labels of the parameters in the splicing processing result can be configured, so that the parameters belonging to the current conversation turn and the parameters belonging to the previous conversation turn in the splicing processing result can be distinguished based on the conversation turn labels.

[0017] In certain embodiments of the present application, the method further comprises:

[0018] Performing slot identification on the first voice request;

[0019] The concatenating the first voice request, the second voice request, and the natural language understanding result includes:

[0020] The first voice request, the result of the slot recognition, the second voice request, and the natural language understanding result are concatenated to obtain the concatenated result.

[0021] In this way, in the implementation mode of the present application, slot recognition can be performed on the first voice request, and the first voice request, the result of the slot recognition of the first voice request, and the natural language understanding result of the second voice request can be spliced ​​to obtain the splicing result.

[0022] In certain embodiments of the present application, the performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request, and the natural language understanding result of the second voice request includes:

[0023] Configure the position label of each parameter in the result of the splicing process.

[0024] Thus, in an embodiment of the present application, parameter filling processing can be performed on the result of the application interface prediction of the first voice request according to the result of the slot identification of the first voice request and the interface filling parameters of the application interface of the second voice request.

[0025] In certain embodiments of the present application, the natural language understanding result of the second voice request includes an application prediction corresponding to the second voice request and a target parameter for filling in the application prediction.

[0026] In this way, in an embodiment of the present application, after completing the first voice request, the second voice request, and the natural language understanding result of the second voice request for splicing processing to obtain the splicing processing result, the position labels of each parameter in the splicing processing result can be configured, so that each parameter in the splicing processing result can be distinguished and associated based on the position labels, thereby improving the accuracy of application interface prediction and parameter filling to a certain extent.

[0027] In certain embodiments of the present application, the method further comprises:

[0028] Determine, based on the first voice request, the second voice request, and the natural language understanding result, conversation association attribute information of the first voice request relative to the second voice request, wherein the conversation association attribute information is used to indicate whether the first voice request and the second voice request are associated.

[0029] Thus, in the implementation mode of the present application, the conversation-related attribute information of the first voice request relative to the second voice request can be determined according to the first voice request, the second voice request and the natural language understanding result, thereby determining the relationship between the current round of conversation and the previous round of conversation.

[0030] In certain embodiments of the present application, the step of sending the execution result of filling the interface parameters to the vehicle to complete the voice interaction includes:

[0031] The execution result and the dialogue-related attribute information are sent to the vehicle to complete the voice interaction.

[0032] In this way, in an embodiment of the present application, the execution results of the filling process of the application interface parameters and the dialogue-related attribute information of the first voice request relative to the second voice request can be sent to the vehicle, so that the vehicle can perform corresponding operations based on the execution results and the dialogue-related attribute information.

[0033] An embodiment of the present application provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0034] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0035] The server and computer-readable storage medium provided in the embodiment of the present application can make the processing of the first voice request of the current dialogue round based on the second voice request of the previous dialogue round and the natural language understanding result of the second voice request, realize context-based voice interaction, and then to a certain extent guarantee the credibility of the result predicted by the application program of the first voice request and the execution result of the parameter filling, thereby guaranteeing the reliable execution of voice interaction and the user's experience of the voice interaction function. In addition, when the voice request of the current dialogue round is relatively short, the voice request and natural language understanding result of the previous dialogue round can be tracked to perform the user intention determination and natural language understanding corresponding to the voice request of the current dialogue round, which to a certain extent guarantees the reliable processing of the voice request of the current dialogue round and improves the applicability of the voice interaction function in complex multi-round dialogue scenarios.

[0036] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0038] Figure 1 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0039] Figure 2 A schematic diagram of application scenarios in certain embodiments of the present application;

[0040] Figure 3 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0041] Figure 4 A schematic diagram of application scenarios in certain embodiments of the present application;

[0042] Figure 5 A schematic diagram of the result of the splicing process in some embodiments of the present application;

[0043] Figure 6 A schematic diagram of the result of the splicing process in some embodiments of the present application;

[0044] Figure 7 A schematic diagram of application scenarios in certain embodiments of the present application;

[0045] Figure 8 A schematic diagram of the result of the splicing process in some embodiments of the present application;

[0046] Fig. 9 A schematic diagram of the result of the splicing process in some embodiments of the present application;

[0047] Fig.10 A schematic diagram of an application scenario in certain embodiments of the present application. DETAILED DESCRIPTION

[0048] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0049] As users become more familiar with the in-vehicle voice interaction function, they expect to have long and coherent multi-round conversations with the vehicle so that the vehicle can complete complex command execution or question responses. During the multi-round conversation, the voice commands expressed by the user may be simplified as the number of conversation rounds increases.

[0050] For example, in a multi-round conversation about vehicle air conditioning, at the beginning of the conversation, the user may express his or her intention by expressing a relatively complete voice command such as "help me turn on the air conditioner" or "turn on the air conditioner". As the conversation progresses, the user may gradually omit the components in the voice command. For example, the user may express a voice command such as "set to 28 degrees" that lacks both the subject and object components, rather than expressing a relatively complete voice command such as "set the air conditioner temperature to 18 degrees Celsius".

[0051] It is understandable that when a user expresses a short voice instruction with some sentence components omitted, it is usually because these omitted sentence components have been explicitly or implicitly stated in the previous round of conversation. For example, in the above example, the user has clearly defined the executed object (or object) as "air conditioner" at the beginning of the conversation, so the user defaults to the executed object (or object) as "air conditioner" in subsequent conversations, so the executed objects "air conditioner" and "air conditioner temperature" are omitted in "set the air conditioner to 28 degrees" or "set the air conditioner temperature to 28 degrees Celsius", thereby expressing a short voice instruction such as "set to 28 degrees".

[0052] It can also be understood that the above-mentioned short voice commands can point to a variety of in-vehicle functions. For example, the above-mentioned "set to 28 degrees" can point to "seat heating temperature set to 28 degrees" and "air conditioning temperature set to 28 degrees". Therefore, there is an unclear direction and unclear user intention. Therefore, when directly performing natural language understanding on the above-mentioned short voice commands, the dialogue system (or natural language processing system) may not be able to accurately understand such short voice commands, resulting in difficulty for the vehicle to correctly respond to the user's voice commands, or, understanding the wrong results and causing the vehicle to perform incorrect operations.

[0053] Therefore, the dialogue system can perform natural language understanding and processing for each round of dialogue based on the context information of each round of dialogue. It can be understood that for the in-vehicle voice interaction function, or in other words, for the dialogue system, as the number of dialogues between the user and the vehicle continues to increase and the voice interaction continues to deepen, the context information of each round of dialogue increases with the increase of the vehicle's dialogue rounds. Furthermore, in order to ensure the consistency of the previous dialogue scene and the smoothness of the front and rear wheel dialogues, the dialogue system needs to accurately understand and save the previous information.

[0054] In traditional solutions, a context state is usually maintained to track the progress of the conversation, but it is easy to fail when it comes to complex conversation scenarios, and the conversation system is required to be able to parse and associate various information points in multiple rounds of conversations, which places high demands on the system. At the same time, as the number of conversation rounds increases, the difficulty and complexity of processing the context information of each round of conversation also increases, resulting in higher hardware computing power and storage requirements.

[0055] Moreover, this traditional solution based on contextual state has difficulty distinguishing contextual information from different rounds, causing the dialogue system to easily forget previous interactions or be unable to interpret new user input in the correct context.

[0056] Based on the above problems that may be encountered, please refer to Figure 1 , the present application embodiment provides a voice interaction method, including:

[0057] 01: Receive the first voice request of the current conversation round forwarded by the vehicle;

[0058] 02: According to the first voice request, the second voice request, and the natural language understanding result of the second voice request, perform application program interface prediction and interface parameter filling for the first voice request, wherein the second voice request is a voice request of a previous dialogue round of the current dialogue round;

[0059] 03: Send the execution result of the interface parameter filling to the vehicle to complete the voice interaction.

[0060] The embodiment of the present application provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a processing module and an interaction module. Among them, the receiving module is used to receive the first voice request of the current dialogue round forwarded by the vehicle. The processing module is used to perform application program interface prediction and interface parameter filling for the first voice request according to the first voice request, the second voice request and the natural language understanding result of the second voice request, wherein the second voice request is the voice request of the previous dialogue round of the current dialogue round. The interaction module sends the execution result of the interface parameter filling to the vehicle to complete the voice interaction.

[0061] The embodiment of the present application also provides a server, which includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive a first voice request of the current dialogue round forwarded by the vehicle, and to perform application program interface prediction and interface parameter filling for the first voice request based on the first voice request, the second voice request and the natural language understanding result of the second voice request, wherein the second voice request is a voice request of the previous dialogue round of the current dialogue round, and to send the execution result of the interface parameter filling to the vehicle to complete the voice interaction.

[0062] Specifically, in an embodiment of the present application, when a user conducts multiple rounds of conversations with a vehicle in order for the vehicle to perform corresponding actions, the vehicle may send each round of conversation expressed by the user to the server. Therefore, for the first voice request expressed by the user in the current round of conversation, the vehicle may forward or report the first voice request to the server.

[0063] When the server receives a first voice request for a current conversation round forwarded by the vehicle, it can obtain a second voice request for a previous conversation round before the current conversation round, which was last forwarded by the vehicle, and obtain a natural language understanding result of the second voice request.

[0064] Furthermore, the server can predict the application program interface (API) of the first voice request based on the first voice request, the second voice request, and the natural language understanding result of the second voice request, so as to predict the application program interface used by the vehicle to implement the first voice request (or the user's intention). At the same time, since the call of the application program interface usually requires the corresponding input parameters, the server can also perform parameter filling processing on the result of the application program interface prediction to obtain the result of parameter filling.

[0065] Furthermore, the server can send the result of parameter filling to the vehicle so that the vehicle can perform corresponding operations according to the result of parameter filling, meet the user's intention and complete voice interaction with the user.

[0066] To more clearly illustrate the implementation of this application, please refer to Figure 2 , Figure 2 Schematic diagram of application scenarios in certain embodiments of the present application. That is, in one example, let the first voice request be recorded as now_query, and let now_query be "set to 28 degrees", and let the second voice request be recorded as last_query, and let last_query be "turn on the air conditioner", then:

[0067] After receiving now_query, the server reads last_query of the previous dialogue round of the current dialogue round and the natural language understanding result of last_query based on a preset history tracker (History Tracker) for storing voice requests and natural language understanding results of voice requests in each dialogue round.

[0068] Next, based on the pre-trained natural language processing model and according to the natural language understanding results of now_query, last_query and last_query, the server performs application program interface prediction and parameter filling for now_query to determine that the API corresponding to now_query is AcSet, and the device parameter of AcSet is filled in with "air conditioning" and the value parameter is filled in with "twenty-eight degrees". The execution result of the application program interface parameter filling is represented in a preset format, and the execution result is: [{"apiName":"AcSet","arguments":[{"name":"device","type":"string","value":" Air conditioning},{"name":"value","type":"string","value":"Twenty-eight degrees"}]}].

[0069] Finally, the server can send the execution result [{"apiName":"AcSet","arguments":[{"name":"device","type":"string","value":"Air Conditioner"},{"name":"value","type":"string","value":"Twenty-eight degrees"}]}] to the vehicle. The vehicle can parse the received execution result to parse it into a corresponding control instruction and execute it, so that the temperature of the air conditioner is set to twenty-eight degrees in response to the user voice request and the server.

[0070] It can be understood that in the implementation manner of the present application, the natural language understanding result of the first voice request may include the application prediction result corresponding to the first voice request and the execution result of the application interface parameter filling.

[0071] Therefore, in certain embodiments of the present application, the natural language understanding result of the second voice request includes the application prediction corresponding to the second voice request and the target parameter filled in the application prediction. For example, in one example, the second voice request last_query is "turn on the air conditioner", and the natural language understanding result of the second voice request last_query may include {last_api:AcOpen,last_arguments:[{"device":"air conditioner"}]}, that is, the API prediction corresponding to "turn on the air conditioner" is AcOpen, the parameters of AcOpen include device, and the target parameter filled in device is "air conditioner".

[0072] It can also be understood that historical turn information (i.e., the second voice request and the natural language understanding result of the second voice request) is critical to maintaining the continuity and context relevance of the dialogue, and it can help the dialogue system adjust the current response according to the previous context information. Therefore, the server of the embodiment of the present application can read the pre-stored second voice request and the natural language understanding result of the second voice request of the previous dialogue turn to perform natural language processing of the first voice request of the current dialogue turn.

[0073] Also, it can be understood that the implementation methods of the present application can predict the application interface of the voice request and execute the application interface parameter filling based on the pre-trained natural language processing model, and the natural language processing model adopted in the implementation methods of the present application is content that can be set according to actual conditions.

[0074] For example, in certain embodiments of the present application, the natural language processing model used in the embodiments of the present application includes a pre-trained generative model that can perform application program interface prediction and application program interface parameter filling for the first voice request sample based on the received first voice request sample, the second voice request sample, and the second voice request sample.

[0075] Furthermore, in certain embodiments of the present application, the generative model is obtained based on the pre-trained weights of BERT (Bidirectional Encoder Representations from Transformers).

[0076] Optionally, in certain embodiments of the present application, the vehicle may also broadcast a specific voice message such as “OK, executed” to prompt the user when executing the corresponding instruction according to the execution result.

[0077] Optionally, in certain embodiments of the present application, the vehicle may also send corresponding reply information to the server when executing the corresponding instruction according to the execution result, so that the server can determine the execution status of the instruction on the vehicle side.

[0078] In summary, in the implementation of the present application, the processing of the first voice request of the current dialogue round can be performed based on the second voice request of the previous dialogue round and the natural language understanding result of the second voice request, thereby realizing context-based voice interaction, and further guaranteeing the credibility of the result of the application prediction of the first voice request and the execution result of the parameter filling to a certain extent, thereby guaranteeing the reliable execution of the voice interaction and the user experience of the voice interaction function. In addition, when the voice request of the current dialogue round is relatively short, the voice request and natural language understanding result of the previous dialogue round can be tracked to perform the user intention determination and natural language understanding corresponding to the voice request of the current dialogue round, which guarantees the reliable processing of the voice request of the current dialogue round to a certain extent and improves the applicability of the voice interaction function in complex multi-round dialogue scenarios.

[0079] See also Figure 3 In certain embodiments of the present application, step 02 includes:

[0080] 020: performing splicing processing on the first voice request, the second voice request, and the natural language understanding result;

[0081] 021: According to the result of the splicing processing, perform application program interface prediction and interface parameter filling for the first voice request.

[0082] The processing module of the implementation mode of the present application is also used to perform splicing processing based on the first voice request, the second voice request, and the natural language understanding result, and to perform application program interface prediction and interface parameter filling for the first voice request based on the result of the splicing processing.

[0083] The processor of the embodiment of the present application is also used to perform splicing processing based on the first voice request, the second voice request, and the natural language understanding result, and to perform application program interface prediction and interface parameter filling for the first voice request based on the result of the splicing processing.

[0084] Specifically, in an implementation manner of the present application, the server may concatenate the first voice request, the second voice request, and the natural language understanding results of the second voice request, and then call the corresponding tools, programs or natural language processing models to predict the application program interface and fill in the interface parameters of the first voice request through the concatenated results.

[0085] Optional, see Figure 4 and Figure 5 , Figure 4 This is a schematic diagram of application scenarios in certain embodiments of the present application. Figure 5Schematic diagram of the result of splicing processing in some embodiments of the present application. That is, in some embodiments of the present application, the server can perform splicing processing on the first voice request (i.e., "set to 28 degrees"), the second voice request ("turn on the air conditioner"), and the natural language understanding result of the second voice request (i.e., API prediction "AcOpen" and filling the target parameter "air conditioner" of "AcOpen") to form a result similar to Figure 5 After the result of the splicing processing shown, input this result into a pre-trained natural language processing model, so that the natural language processing model predicts the API of the first voice request and fills the parameters of the API.

[0086] Among them, it can be understood that Figure 5 In the result of the splicing processing shown, the "result of the splicing processing" is a serialized vector, so CLS represents the starting position of the serialized vector. And it should be noted that Figure 5 In the result of the splicing processing shown, <value> Indicates the "starting position of the target parameter of the second voice request" in the serialized vector,< / value> Represents the end position of the target parameter of the second voice request in the serialized vector.

[0087] It can also be understood that CLS, <value> and< / value> Can be used to enable the natural language processing model to determine the meaning and boundaries of each data in the input serialized vector, so that the model can reliably predict the API of the first voice request and fill the parameters of the API according to each data in the serialized vector accurately.

[0088] And it can also be understood that in the embodiments of the present application, before the server splices the first voice request, the second voice request, and the natural language understanding result of the second voice request to form a splicing processing result as shown in Figure 5 Before, the server can perform tokenization processing on the first voice request, the second voice request, and the natural language understanding result of the second voice request to convert the first voice request, the second voice request, and the natural language understanding result of the second voice request into semantic units (tokens), so that when performing splicing processing subsequently, the first voice request, the second voice request, and the natural language understanding result of the second voice request can be spliced into a serialized vector.

[0089] It can be understood that in the splicing processing result as shown in Figure 5 Each piece of data in the box can be understood as a semantic unit, such as "AcOpen" and "open" are both semantic units.

[0090] Optionally, in certain embodiments of the present application, the server has completed tokenization processing on the second voice request and the natural language understanding result of the second voice request. Then, the server may only tokenize the first voice request, and then concatenate the first voice request after tokenization processing with the second voice request and the natural language understanding result of the second voice request, thereby forming a Figure 5 The stitching result is shown.

[0091] Optionally, in certain embodiments of the present application, after receiving a voice request forwarded by a vehicle, the server may perform the above-mentioned word segmentation processing on the voice request to obtain all semantic units contained in the voice request, and store all semantic units contained in the voice request. In addition, after predicting the application program interface of the voice request and completing the parameter filling processing of the application program interface, the server may also store the application program interface prediction of the voice request and the filled target parameters.

[0092] Furthermore, when the server reads the speech request of the previous dialogue round before the current dialogue round and its natural language understanding result, it can read multiple semantic units of the speech request. Therefore, when performing the splicing process later, the speech request of the current dialogue round, the speech request of the previous dialogue round, and the natural language understanding result of the speech request of the previous dialogue round can be directly spliced ​​to form a similar Figure 5 The stitching result is shown.

[0093] Thus, in the implementation mode of the present application, the first voice request, the second voice request and the natural language understanding result of the second voice request can be concatenated, and the application program interface prediction and interface parameter filling of the first voice request can be performed based on the concatenated result.

[0094] In certain embodiments of the present application, step 02 includes:

[0095] Configure the conversation turn label for each parameter in the result of the splicing process.

[0096] The processing module of the implementation mode of the present application is also used to configure the dialogue turn label of each parameter in the result of the splicing processing.

[0097] The processor of the embodiment of the present application is also used to configure the dialogue turn label of each parameter in the result of the splicing processing.

[0098] Specifically, in order to be able to distinguish the parameters of different conversation turns in the result of the splicing process, the implementation method of the present application can also configure a conversation turn label for each parameter in the result of the splicing process.

[0099] For example, see Figure 6 , Figure 6 Schematic diagram of the result of splicing processing in some embodiments of the present application. That is, the dialog turn label of the parameters of the current dialog turn (such as the semantic units 'set', 'for', 'two', 'ten', 'eight' and 'degree') is configured to 0, and the parameters of the previous dialog turn (such as the semantic units 'AcOpen', 'open', 'open', 'empty', 'adjust', ' <value> ', 'empty', 'adjust' and '< / value> ') is configured with a dialog turn label of 1.

[0100] Optionally, in certain embodiments of the present application, the conversation turn label is represented as a preset turn-embedding vector.

[0101] It is understandable that based on Figure 2 or Figure 4 The method shown is that the pre-trained natural language processing model is used to Figure 6 In the case where the result of the splicing processing is inferred to perform application interface prediction and application interface parameter filling for the first voice request, since the "result of the splicing processing" is configured with a dialogue turn label, the natural language processing model can recognize the connection and difference between the current dialogue turn and the previous dialogue turn after receiving the "result of the splicing processing".

[0102] Thus, in an embodiment of the present application, after completing the first voice request, the second voice request, and the natural language understanding results of the second voice request and performing splicing processing to obtain the splicing processing result, the conversation turn labels of the parameters in the splicing processing result can be configured, so that the parameters belonging to the current conversation turn and the parameters belonging to the previous conversation turn in the splicing processing result can be distinguished based on the conversation turn labels.

[0103] Moreover, when performing application program interface prediction and parameter filling based on the results of the splicing processing, it is possible to confirm the connection and difference between the current dialogue round and the previous dialogue round, thereby improving the accuracy of application program interface prediction and parameter filling to a certain extent.

[0104] In some embodiments of the present application, the voice interaction method further includes:

[0105] Perform slot identification on the first voice request.

[0106] Furthermore, the above step 021 includes:

[0107] The first voice request, the result of slot recognition, the second voice request, and the result of natural language understanding are concatenated to obtain a concatenated result.

[0108] The voice interaction device of the embodiment of the present application further includes a slot recognition module. The slot recognition module is used to perform slot recognition on the first voice request. The processing module of the embodiment of the present application is also used to perform splicing processing on the first voice request, the result of the slot recognition, the second voice request, and the result of the natural language understanding to obtain a splicing processing result.

[0109] The processor of the embodiment of the present application is also used to perform slot recognition on the first voice request, and to perform splicing processing on the first voice request, the result of the slot recognition, the second voice request, and the natural language understanding result to obtain a splicing processing result.

[0110] It is understandable that slot information (or named entities) can be used to understand user voice requests. Also, in the in-vehicle voice interaction scenario, the dialogue system needs to quickly and accurately extract vehicle control related entities in the user request, such as temperature, volume, navigation destination, etc.

[0111] Therefore, in the implementation mode of the present application, the server may also perform slot identification on the first voice request before predicting the API of the first voice request to obtain the corresponding slot identification result. For example, in one example, the first voice request is "set to twenty-eight degrees", then the slot identification result of the first voice request may include [{"value":"twenty-eight degrees","pos":[3,6]}]. Among them, "pos":[3,6] means that the sequence numbers of the semantic units in the entity "twenty-eight degrees" in "set to twenty-eight degrees" are 3, 4, 5 and 6.

[0112] It can also be understood that, when the slot recognition result of the first voice request is obtained, the server can concatenate the first voice request, the slot recognition result of the first voice request, and the natural language understanding result of the second voice request to obtain a concatenated result. Figure 7 and Figure 8 , Figure 7 This is a schematic diagram of application scenarios in certain embodiments of the present application. Figure 8 It is a schematic diagram of the result of the stitching process in certain embodiments of the present application.

[0113] That is, if Figure 7 As shown, the server of the embodiment of the present application can perform named entity recognition on the first voice request through a pre-trained named entity recognition model to determine the named entities of each slot type in the first voice request, thereby obtaining the slot recognition result. For example, for the first voice request of "set to twenty-eight degrees", the slot recognition result may include [{"value":"twenty-eight degrees","pos":[3,6]}].

[0114] Furthermore, the server of the embodiment of the present application may, when obtaining the result of slot recognition of the first voice request, concatenate the first voice request, the result of slot recognition of the first voice request, and the natural language understanding result of the second voice request, so that Figure 7 The natural language processing model in can be based on Figure 8 The splicing result shown is used to perform API prediction and API parameter filling for the first voice request.

[0115] Furthermore, it is understandable that Figure 8 The second row of numbers in can be understood as dialogue turn labels, which are used to indicate the dialogue turn to which each parameter (or semantic unit) in the first row belongs.

[0116] For example, the dialog turn label is 0, indicating that the above parameters belong to the current dialog turn, such as semantic units such as 'set', 'for', 'two', 'ten', 'eight' and 'degree' belonging to the first voice request of the current dialog turn.

[0117] The dialog turn label is 1, indicating that the parameters above belong to the previous dialog turn, such as AcOpen', 'open', 'open', 'empty', 'adjust', ' <value> ', 'empty', 'adjust' and '< / value> ' and other semantic units belong to the second voice request of the previous dialogue turn.

[0118] In this way, in the implementation mode of the present application, slot recognition can be performed on the first voice request, and the first voice request, the result of the slot recognition of the first voice request, and the natural language understanding result of the second voice request can be spliced ​​to obtain the splicing result.

[0119] Optionally, in an embodiment of the present application, the natural language understanding result of the second voice request includes an interface filling parameter of the second voice request, and step 03 includes:

[0120] According to the slot identification result and the interface filling parameter, the parameter filling process is performed on the result of the application interface prediction to obtain the execution result of the parameter filling process.

[0121] The processing module of the implementation mode of the present application is also used to perform parameter filling processing on the result of application interface prediction according to the result of slot identification and interface filling parameters, so as to obtain the execution result of the parameter filling processing.

[0122] The processor of the embodiment of the present application is also used to perform parameter filling processing on the result of application interface prediction according to the result of slot identification and interface filling parameters, so as to obtain the execution result of the parameter filling processing.

[0123] Specifically, in the implementation manner of the present application, the server may perform parameter filling of the API of the first voice request according to the result of the slot identification of the first voice request.

[0124] For example, the API prediction for the first voice request is AcSet, the slot recognition result of the first voice request includes [{"value":"twenty-eight degrees"}], the API for the second voice request is AcOpen, and the interface filling parameters of AcOpen include [{"device":"air conditioner"}]. Then, the device parameter of AcSet can be filled in with "air conditioner" and the value parameter of AcSet can be filled in with "twenty-eight degrees".

[0125] Thus, in an embodiment of the present application, parameter filling processing can be performed on the result of the application interface prediction of the first voice request according to the result of the slot identification of the first voice request and the interface filling parameters of the application interface of the second voice request.

[0126] In certain embodiments of the present application, step 02 includes:

[0127] Configure the position label of each parameter in the result of the splicing process.

[0128] The processing module of the embodiment of the present application is also used to configure the position label of each parameter in the result of the splicing processing.

[0129] The processor of the embodiment of the present application is also used to configure a position label of each parameter in the result of the splicing process.

[0130] Specifically, in the implementation manner of the present application, the server may also configure a position tag for each parameter in the result of the splicing process, so that each parameter in the result of the splicing process can be associated and distinguished based on the position tag.

[0131] Optionally, in certain embodiments of the present application, the server may set a position code in the result of the splicing process to implement the configuration of the position tag.

[0132] To more clearly illustrate the implementation of this application, please refer to Fig. 9 , Fig. 9 Schematic diagram of the result of the splicing process in some embodiments of the present application. That is, when the semantic units in the first voice request and the semantic units in the second voice request are spliced ​​and the CLS mark is set, the splicing result can be a sequence of eleven semantic units, namely, 'CLS', 'set', 'for', 'two', 'ten', 'eight', 'degree', 'hit', 'open', 'space' and 'adjust', which are arranged and combined in sequence.

[0133] Further, since the slot recognition result of the first voice request includes [{"value": "Twenty-eight degrees"}] and the natural language understanding result of the second voice request includes [{"device": "Air conditioner"}], and the two entities "Twenty-eight degrees" and "Air conditioner" have appeared in the "sequence formed by sequentially sorting and combining eleven semantic units", the parameters corresponding to the slot type of value (i.e., " <value> and< / value> ") and the parameters corresponding to the slot type of device (i.e., " <device> and< / device> ") can be set in this sequence according to the positions of "Twenty-eight degrees" and "Air conditioner" in the sequence, and the position labels corresponding to these parameters are configured. That is, <value> Configuration is 3.< / value> is configured as 6 to indicate that the semantic units corresponding to the entity of the slot type of value are at positions 3 and 6 at the beginning and end of the sequence.

[0134] Similarly, <device> Configuration is 10,< / device> is configured as 11 to indicate that the semantic units corresponding to the entity of the slot type of device are at positions 10 and 11 at the beginning and end of the sequence.

[0135] It can be understood that, compared with the method of directly splicing the slot recognition result of the first voice request and the natural language understanding result of the second voice request, or rather, compared with Figure 5 、 6 and the splicing processing results shown in 8, Fig. 9 contains more concise information.

[0136] Moreover, in the case of inferring the splicing processing result through a natural language processing model (such as Figure 2 、 4 、7), based on the splicing processing result shown in Fig. 9 , the natural language processing model can capture and understand the association between the "slot recognition result of the first voice request" and the "first voice request", and the association between the "natural speech understanding result of the second voice request" and the "second voice request", thereby improving the accuracy of inference to a certain extent.

[0137] Thus, in the embodiment of the present application, when the splicing processing is completed on the first voice request, the second voice request and the natural language understanding result of the second voice request to obtain the splicing processing result, the position labels of each parameter in the splicing processing result can be configured so that each parameter in the splicing processing result can be distinguished and associated based on the position labels, and further, the accuracy of application program interface prediction and parameter filling can be improved to a certain extent.

[0138] In some embodiments of the present application, the voice interaction method further includes:

[0139] According to the first voice request, the second voice request and the natural language understanding result, the conversation correlation attribute information of the first voice request relative to the second voice request is determined, wherein the conversation correlation attribute information is used to indicate whether the first voice request and the second voice request are correlated.

[0140] The voice interaction device of the embodiment of the present application further includes an attribute information determination module. The attribute information determination module is used to determine the dialogue association attribute information of the first voice request relative to the second voice request based on the first voice request, the second voice request and the natural language understanding result, wherein the dialogue association attribute information is used to indicate whether the first voice request and the second voice request are associated.

[0141] The processor of the embodiment of the present application is also used to determine the dialogue association attribute information of the first voice request relative to the second voice request based on the first voice request, the second voice request and the natural language understanding result, wherein the dialogue association attribute information is used to indicate whether the first voice request and the second voice request are associated.

[0142] Specifically, in the implementation manner of the present application, the server can also determine the conversation-related attribute information of the first voice request relative to the second voice request based on the first voice request, the second voice request and the natural language understanding result, that is, determine whether the current round of conversation is semantically continuous with the previous round of conversation.

[0143] Optionally, to more clearly illustrate the implementation of this application, please refer to Fig.10 , Fig.10 Schematic diagram of application scenarios in certain embodiments of the present application. That is, when the server performs reasoning based on the first voice request, the second voice request and the natural language understanding result based on the pre-trained natural language processing model, or when receiving the result spliced ​​by the first voice request, the slot recognition result of the first voice request, the second voice request and the natural language understanding result, the natural language processing model can output the dialogue association attribute information (corresponding to Fig.10 The inheritedLabel in the natural language processing model is used, and the hidden state of the natural language processing model can be used for API prediction and API parameter filling of the first voice request.

[0144] Furthermore, when inheritLabel is 1, it indicates that the first voice request and the second voice request are semantically continuous. Correspondingly, when inheritLabel is 0, it indicates that the first voice request and the second voice request are not semantically continuous.

[0145] Thus, in the implementation mode of the present application, the conversation-related attribute information of the first voice request relative to the second voice request can be determined according to the first voice request, the second voice request and the natural language understanding result, thereby determining the relationship between the current round of conversation and the previous round of conversation.

[0146] In certain embodiments of the present application, step 03 includes:

[0147] Send the execution results and dialogue-related attribute information to the vehicle to complete the voice interaction.

[0148] The interaction module of the implementation mode of the present application is also used to send execution results and dialogue-related attribute information to the vehicle to complete voice interaction.

[0149] The processor of the implementation mode of the present application is also used to send execution results and dialogue-related attribute information to the vehicle to complete the voice interaction.

[0150] It can be understood that when the first voice request and the second voice request are semantically continuous, it indicates that the relevant processing of the first voice request can continue to use the information of the historical conversation turn. Correspondingly, when the first voice request and the second voice request are not semantically continuous, it indicates that the relevant processing of the first voice request can ignore the information of the historical conversation turn.

[0151] Therefore, the server of the implementation mode of the present application can also send the dialogue association attribute information to the vehicle, so that the vehicle can determine whether the relevant processing of the first voice request can continue to use the historical round information based on the received dialogue association attribute information.

[0152] To more clearly illustrate the implementation method of this application, please refer to Fig.10 That is, in the implementation manner of the present application, the server may send [{"apiName":"AcSet","arguments":[{"name":"device","type":"string","value":"Air conditioning"},{"name":"value","type":"string","value":"Twenty-eight degrees"}],"inheritLabel":1}] output by the natural language processing model to the vehicle, so that the vehicle performs the relevant processing of the first voice request according to the value of the inheritLabel label (1).

[0153] It should also be noted that in the implementation mode of the present application, "the related processing of the first voice request" can be understood as the vehicle's TTS (Text To Speech) prompt. For example, if the user says "turn on the air conditioner" and then says "set to 28 degrees", then the vehicle confirms that "turn on the air conditioner" and "set to 28 degrees" are semantically continuous based on the label inheritLabel, and can therefore feedback "a response that is also semantically continuous" to the user, thereby achieving a natural conversation flow.

[0154] In this way, in an embodiment of the present application, the execution results of the filling process of the application interface parameters and the dialogue-related attribute information of the first voice request relative to the second voice request can be sent to the vehicle, so that the vehicle can perform corresponding operations based on the execution results and the dialogue-related attribute information.

[0155] Optionally, in the implementation manner of the present application, Figure 2 , 4 , 7 and 10, the process of acquiring the natural language processing model may include: fine-tuning the generative model using the pre-trained weights of BERT.

[0156] It is understandable that fine-tuning of the generative model can effectively improve the system's natural language understanding ability. Moreover, when combined with pre-trained models such as BERT for fine-tuning training, the performance of the model can be greatly optimized.

[0157] The embodiment of the present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned voice interaction method when the computer program is executed by one or more processors.

[0158] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0159] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0160] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receiving a first voice request of a current conversation round forwarded by a vehicle; According to the first voice request, the second voice request and the natural language understanding result of the second voice request, performing application program interface prediction and interface parameter filling for the first voice request, wherein the second voice request is a voice request of a previous dialogue round of the current dialogue round; The execution result of filling the interface parameters is sent to the vehicle to complete the voice interaction.

2. The method according to claim 1, characterized in that The performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request and the natural language understanding result of the second voice request includes: Performing concatenation processing on the first voice request, the second voice request, and the natural language understanding result; The application program interface prediction and the interface parameter filling are performed on the first voice request according to the result of the splicing processing.

3. The method according to claim 2, characterized in that The performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request and the natural language understanding result of the second voice request includes: Configure the dialog turn label of each parameter in the result of the splicing process.

4. The method according to claim 2, characterized in that: The method further comprises: Performing slot identification on the first voice request; The concatenating the first voice request, the second voice request, and the natural language understanding result includes: The first voice request, the result of the slot recognition, the second voice request, and the natural language understanding result are concatenated to obtain the concatenated result.

5. The method according to claim 4, characterized in that The performing application program interface prediction and interface parameter filling on the first voice request according to the first voice request, the second voice request and the natural language understanding result of the second voice request includes: Configure the position label of each parameter in the result of the splicing process.

6. The method according to claim 1, characterized in that The natural language understanding result of the second voice request includes an application prediction corresponding to the second voice request and a target parameter for filling in the application prediction.

7. The method according to claim 1, characterized in that The method further comprises: Determine, based on the first voice request, the second voice request, and the natural language understanding result, conversation association attribute information of the first voice request relative to the second voice request, wherein the conversation association attribute information is used to indicate whether the first voice request and the second voice request are associated.

8. The method according to claim 7, characterized in that The step of sending the execution result of filling the interface parameters to the vehicle to complete the voice interaction includes: The execution result and the dialogue-related attribute information are sent to the vehicle to complete the voice interaction.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images