Voice interaction method, vehicle and computer-readable storage medium

Abstract

The present application discloses a voice interaction method and a vehicle computer-readable storage medium, the method comprising: obtaining a voice request, performing slot recognition application program interface prediction on the voice request, obtaining a slot recognition result application program interface prediction result, calculating the correlation between the application program interface prediction result and the slot recognition result, filling application program interface parameters on the voice request according to the application program interface prediction result, the slot recognition result and the correlation, obtaining the execution result of the parameter filling, and controlling the vehicle to complete the voice interaction according to the execution result. In this way, the present application enables the parameters of the application program interface to be filled based on the correlation between the application program interface and the slot recognition result, the credibility of the execution result of the parameter filling is guaranteed, and when multiple application program interfaces are predicted, the parameters of each application program interface can be filled based on the correlation with the slot recognition result, so that each target indicated by the voice request can be achieved.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, a vehicle and a computer-readable storage medium. Background Art

[0002] As users gradually get used to performing operations through voice interaction systems, they tend to speak multiple commands at once and expect the system to process them all. However, for such voice information containing multiple commands, traditional natural language understanding systems can usually only recognize one command in the voice information and ignore other commands, which seriously affects the system's ability to understand complex commands and the accuracy of its response. Summary of the invention

[0003] The present application provides a voice interaction method, a vehicle and a computer-readable storage medium.

[0004] A voice interaction method provided in an embodiment of the present application includes:

[0005] Get voice request;

[0006] Performing slot recognition and application program interface prediction on the voice request to obtain a slot recognition result and an application program interface prediction result;

[0007] Calculating the correlation between the application program interface prediction result and the slot identification result;

[0008] According to the application program interface prediction result, the slot recognition result and the correlation degree, filling application program interface parameters for the voice request to obtain an execution result of the application program interface parameter filling;

[0009] The vehicle is controlled according to the execution result to complete the voice interaction.

[0010] In the voice interaction method provided in the embodiment of the present application, the vehicle can obtain a voice request, and perform slot identification and application program interface prediction on the voice request, thereby obtaining a slot identification result and an application program interface prediction result, and calculating the correlation between the application program interface prediction result and the slot identification result, and then fill in application program interface parameters for the voice request based on the application program interface prediction result, the slot identification result, and the correlation between the application program interface prediction result and the slot identification result, thereby obtaining an execution result of the application program interface parameter filling, and controlling the vehicle based on the execution result to complete the voice interaction.

[0011] Thus, in the embodiment of the present application, when the vehicle obtains the API prediction result and the slot identification result of the voice request, it can calculate the correlation between the API prediction result and the slot identification result, so that the parameters of the API can be filled based on the correlation between the API and the slot identification result, thus ensuring the reliable filling of the API parameters to a certain extent, and the credibility of the execution result of the API parameter filling can be guaranteed. Also, when the API prediction result includes multiple APIs, the parameters of each API can be determined based on the slot identification result and the correlation between each API and the slot identification result, so that the parameters of each API can be reliably filled, thereby enabling the vehicle to reliably achieve each goal indicated by the voice request, and the correct understanding and accurate response of the voice request can be guaranteed.

[0012] In certain embodiments of the present application, the calculating the correlation between the application program interface prediction result and the slot identification result includes:

[0013] The attention scores of the application program interface prediction result and the slot identification result are calculated to determine the degree of correlation between the application program interface prediction result and the slot identification result.

[0014] In this way, in an embodiment of the present application, the vehicle can calculate the attention scores of the application interface prediction results and the slot identification results based on the setting of the attention mechanism, thereby determining the degree of correlation between the application interface prediction results and the slot identification results, so that the correlation performance between the application interface prediction results and the slot identification results can be reliably determined.

[0015] In certain embodiments of the present application, the slot recognition result includes a slot type, an entity of the slot type, and a position of the entity in the voice request, and the calculating the correlation between the application program interface prediction result and the slot recognition result includes:

[0016] The correlation between the application program interface prediction result and the entity of the slot type is calculated based on the position.

[0017] In this way, in the implementation mode of the present application, the vehicle can calculate the correlation between the application interface prediction result and the slot type entity based on the slot type in the slot information, the slot type entity and the position of the entity in the voice request, which ensures the reliable calculation of the correlation to a certain extent and realizes the indexing of the slot type entity based on the position.

[0018] In certain embodiments of the present application, controlling the vehicle according to the execution result to complete the voice interaction includes:

[0019] A control instruction is generated according to the execution result to control the vehicle to execute a corresponding function and complete the voice interaction.

[0020] Thus, in the implementation mode of the present application, the vehicle can complete the parameter filling of the application result, thereby obtaining the execution result of the parameter filling, and can generate a control instruction through the execution result to execute the corresponding function.

[0021] In certain embodiments of the present application, controlling the vehicle according to the execution result to complete the voice interaction includes:

[0022] The execution result is filled into a target field of the output content, wherein the target field includes a first target field and a second target field, wherein the first target field is configured to fill in the application program interface, and the second target field is configured to fill in the execution result filled with the application program interface parameters.

[0023] Thus, in the embodiment of the present application, when the vehicle fills in the parameters of the application program interface to obtain the execution result of the parameter filling, the vehicle fills in the first target field and the second target field using the execution result to form output content.

[0024] In certain embodiments of the present application, controlling the vehicle according to the execution result to complete the voice interaction includes:

[0025] Parsing the filling content of the target field in the output content;

[0026] An execution instruction corresponding to the voice request is generated according to the filled content and sent to the vehicle to complete the voice interaction.

[0027] In this way, in the implementation mode of the present application, the vehicle can generate control instructions corresponding to the voice request according to the filled content, and execute corresponding functions through the control instructions to complete the voice interaction.

[0028] In certain embodiments of the present application, the performing slot identification and application program interface prediction on the voice request to obtain a slot identification result and an application program interface prediction result includes:

[0029] Based on the pre-trained generative model, an application program interface prediction is performed on the voice request.

[0030] Thus, in the implementation mode of the present application, the vehicle can predict the application program interface based on the pre-trained generative model, so the credibility of the predicted application program interface can be guaranteed to a certain extent.

[0031] In certain embodiments of the present application, the performing slot identification and application program interface prediction on the voice request to obtain a slot identification result and an application program interface prediction result includes:

[0032] Based on the pre-trained recognition model and pre-configured dictionary data, slot recognition is performed on the voice request.

[0033] In this way, the implementation method of the present application can perform slot recognition of voice requests based on dictionary data and recognition models, so the credibility of the slot recognition results can be guaranteed to a certain extent.

[0034] An embodiment of the present application provides a vehicle, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0035] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0036] The vehicle and computer-readable storage medium provided by the embodiment of the present application can calculate the correlation between the API prediction result and the slot identification result when the API prediction result and the slot identification result of the voice request are obtained, so that the parameters of the API can be filled based on the correlation between the API and the slot identification result, thus ensuring the reliable filling of the API parameters to a certain extent, and the credibility of the execution result of the API parameter filling can be guaranteed. And, when the API prediction result includes multiple APIs, the parameters of each API can be determined based on the slot identification result and the correlation between each API and the slot identification result, so that the parameters of each API can be reliably filled, thereby enabling the vehicle to reliably achieve each goal indicated by the voice request, and the correct understanding and accurate response of the voice request can be guaranteed.

[0037] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0039] Figure 1 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0040] Figure 2 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0041] Figure 3 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0042] Figure 4 This is a flowchart of the voice interaction method in certain implementation modes of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0044] With the popularization of in-vehicle voice interaction functions, users can control vehicles through voice commands, such as opening and closing windows, turning off air conditioning, etc. Further, when users are accustomed to controlling vehicles through voice, they may expect the vehicle to perform multiple operations at one time, and thus speak "voice commands containing multiple intentions" to control the vehicle. For example, a user may speak a voice command of "turn on the air conditioning and navigate to the mall" to start the vehicle's air conditioning and display "navigation page to the mall".

[0045] However, traditional natural language processing solutions only support the recognition and processing of single voice commands, and it is difficult to normally understand and analyze a single voice command containing multiple intentions. For example, for the above-mentioned voice command "turn on the air conditioner and navigate to the mall", the vehicle's voice interaction system may recognize the voice command as "turn on the air conditioner" or "navigate to the mall", or even the wrong voice command as "turn on 'air conditioner and navigate to the mall'".

[0046] Therefore, since traditional natural language processing solutions only support the recognition and processing of a single voice command, it is difficult for users to control the vehicle through "voice commands containing multiple intentions". They still need to speak multiple voice commands based on the form of one intention corresponding to one sentence to control the vehicle to perform multiple operations, such as speaking the two voice commands of "turn on the air conditioner" and "navigate to the mall" to start the vehicle's air conditioner and display "navigation page to the mall".

[0047] Accordingly, in order to avoid the above-mentioned situation where "the user needs to say multiple voice commands to control the vehicle to perform multiple operations", or in other words, to correctly understand and analyze a single sentence containing multiple intents (i.e., multi-intent understanding), some natural language processing solutions are based on the pipeline design method, which performs sentence segmentation (Sentence Segmentation) on a single command to detect the clauses corresponding to each intent in a single voice command, and then process each independent intent according to each clause, and then identify and process it through the natural language understanding (NLU) mechanism to obtain the final result.

[0048] However, the above pipeline-based natural language processing solution has multiple problems, such as the need to maintain and update each module and / or service in the pipeline. In addition, when the pipeline is long, the processing flow of the entire solution is lengthy, and the large number of modules and / or services leads to higher subsequent maintenance costs.

[0049] In addition, there is an error accumulation situation in the pipeline-based natural language processing solution. That is, because the output of the previous module is the input of the next module, if there is an error in the output of the previous module, the next module will produce an output with a larger error based on the "input with error", resulting in the final output may have an unpredictable difference / error from the correct result.

[0050] Moreover, when the pipeline is long and therefore contains modules and / or services, the entire processing flow may not be completed within the scheduled time due to processing timeout of a single module / service, thereby affecting the stability of the system and user experience.

[0051] Based on the above problems that may be encountered, please refer to Figure 1 , the present application embodiment provides a voice interaction method, including:

[0052] 01: Get voice request;

[0053] 02: Perform slot recognition and application program interface prediction on the voice request to obtain slot recognition results and application program interface prediction results;

[0054] 03: Calculate the correlation between the API prediction results and the slot identification results;

[0055] 04: According to the API prediction result, the slot recognition result and the correlation degree, the API parameters are filled in the voice request to obtain the execution result of the API parameter filling;

[0056] 05: Control the vehicle according to the execution result to complete the voice interaction.

[0057] The embodiment of the present application provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes an acquisition module, an identification module, a calculation module, a filling module and an interaction module. Among them, the acquisition module is used to obtain a voice request. The identification module is used to perform slot identification and application program interface prediction on the voice request to obtain the slot identification result and the application program interface prediction result. The calculation module is used to calculate the correlation between the application program interface prediction result and the slot identification result. The filling module is used to fill the application program interface parameters of the voice request according to the application program interface prediction result, the slot identification result and the correlation degree, and obtain the execution result of the application program interface parameter filling. The interaction module is used to control the vehicle according to the execution result to complete the voice interaction.

[0058] The embodiment of the present application also provides a vehicle, which includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the vehicle of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain a voice request, and to perform slot identification and application program interface prediction on the voice request, obtain slot identification results and application program interface prediction results, and to calculate the correlation between the application program interface prediction results and the slot identification results, and to fill the application program interface parameters of the voice request according to the application program interface prediction results, slot identification results and correlation, obtain the execution result of the application program interface parameter filling, and to control the vehicle according to the execution result to complete the voice interaction.

[0059] Specifically, in the implementation mode of the present application, when the user wants to control the vehicle to perform a specific action such as using the vehicle air conditioning and the in-vehicle navigation function, the user can say a corresponding sentence such as "turn on the air conditioning and navigate to the mall", thereby triggering a voice request. When the vehicle obtains the user's voice request, the voice request can be slot identified and the application interface (API) predicted, that is, each type of named entity in the voice request can be identified, and the application interface used to implement the voice request can be predicted, thereby obtaining a slot identification result and an application interface prediction result.

[0060] For the slot recognition result and the application interface prediction result, the vehicle can determine the correlation between the information in the slot recognition result and the predicted application interface, that is, calculate the correlation between the application interface prediction result and the slot recognition result, and then fill the application interface parameters of the voice request according to the application interface prediction result, the slot recognition result and the correlation, and obtain the execution result of the application interface parameter filling. The vehicle can perform corresponding control operations based on the execution result, such as starting the vehicle air conditioner and displaying "navigation page to the mall" on the display screen, thereby completing the voice interaction with the user.

[0061] It should be noted that the voice request in the implementation mode of the present application can be understood as a sentence or voice "used to make the vehicle perform a certain action", such as "turn on the air conditioner", "navigate to a restaurant" and "close the car window", etc., which only include sentences or voices with a single intention.

[0062] Furthermore, in the implementation manner of the present application, the voice request can also be understood as a sentence or voice "used to make the vehicle perform multiple actions", and the sentence is difficult to be divided into sentences through voice activity detection (VAD). For example, when the user quickly says "turn on the air conditioner and navigate to the mall", it is difficult to divide "turn on the air conditioner and navigate to the mall" into 'turn on the air conditioner' and 'navigate to the mall' through voice activity detection. Similarly, it can also be "close the windows and turn off the air conditioner", which is difficult to obtain 'close the windows' and 'turn off the air conditioner' through voice activity detection, and it can also be "turn on the air conditioner and cool the windows", which is difficult to obtain 'turn on the air conditioner and cool the windows' through voice activity detection.

[0063] It can be understood that both of the above two voice requests can be applied and processed by the implementation methods of the present application. Therefore, in order to avoid repetition, the following description will only take the above “the sentence or voice for causing the vehicle to perform multiple actions” as an example.

[0064] For example, when the voice request triggered by the user is "turn on the air conditioner and navigate to the mall", the vehicle can perform slot recognition and application program interface prediction on the voice request, and the slot recognition result slots obtained may include: ["name":"device","pos":[3,4],"rawvalue":"air conditioner","name":"poi_name","pos":[8,9],"rawvalue":"restaurant"]. And, the obtained application program interface prediction result API may include "AC_Open" and "NavigationSearchPoi". In addition, it can be understood that the slot recognition result slots includes two types of entities, namely "'air conditioner' of 'device' type" and "'restaurant' of 'poi_name' type".

[0065] Furthermore, when calling an application program interface, corresponding incoming parameters are usually used in conjunction. For example, when calling the "AC_Open" interface, it is also necessary to set the execution object corresponding to the "AC_Open", or in other words, set the incoming parameters of the "AC_Open" interface (or function), and then perform the corresponding operation through the incoming parameters and the application program interface.

[0066] Based on this, the vehicle of the implementation mode of the present application can determine the correlation between the application interface prediction result API and the slot identification result slots based on the above-mentioned slot identification result slots and the above-mentioned application interface prediction result API, so as to determine the filling parameters of the application interface prediction result API, such as based on the correlation between the above-mentioned ["name":"device","pos":[3,4],"rawvalue":"air conditioning","name":"poi_name","pos":[8,9],"raw value":"restaurant"] and "AC_Open" and "NavigationSearchPoi", parameter filling processing is performed on "AC_Open" and "NavigationSearchPoi" respectively, and the execution result of the application interface parameter filling obtained may include: "apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"air conditioning"}],"apiName":"NavigationSearchPoi","arguments":[{"name":"poi_name","value":"restaurant"}].

[0067] It is understood that the specific calculation method of the correlation degree can be set according to the actual situation. For example, taking the above slots and "AC_Open" as an example, let the vector corresponding to "AC_Open" be V AC , the vector corresponding to "'air conditioner' of type 'device'" in slots is V 1 , slots "'poi_name' type 'restaurant'" is V 2 , the vehicle can calculate V based on the pre-set weights for each slot type. AC Respectively with V 1 、V 2 The weighted values ​​of are obtained to obtain the relevance of "AC_Open" to "'air conditioner' of 'device' type" and "'restaurant' of 'poi_name' type".

[0068] Furthermore, when the weighted value or correlation between "AC_Open" and "'air conditioning' of 'device' type" is higher than the correlation between "AC_Open" and "'restaurant' of 'poi_name' type", the vehicle can complete the parameter filling of "AC_Open" based on "'air conditioning' of 'device' type" to obtain "apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"air conditioning"}].

[0069] Therefore, the vehicle can complete the call of the application interface to perform corresponding actions based on the execution results of the above-mentioned application interface parameters. For example, based on the "execution result of the application interface parameters filled in" corresponding to "turn on the air conditioner and navigate to the mall", the vehicle's air conditioner is controlled to start, and the display screen displays "navigation page to the mall".

[0070] In summary, in the implementation mode of the present application, the vehicle can calculate the correlation between the API prediction result and the slot identification result when obtaining the API prediction result and the slot identification result of the voice request, so that the parameters of the API can be filled based on the correlation between the API and the slot identification result, thus ensuring the reliable filling of the API parameters to a certain extent, and the credibility of the execution result of the API parameter filling can be guaranteed. And, when the API prediction result includes multiple APIs, the parameters of each API can be determined based on the slot identification result and the correlation between each API and the slot identification result, so that the parameters of each API can be reliably filled, and then the vehicle can reliably achieve each goal indicated by the voice request, and the correct understanding and accurate response of the voice request can be guaranteed.

[0071] In certain embodiments of the present application, step 03 includes:

[0072] The attention scores of the application interface prediction results and the slot recognition results are calculated to determine the correlation between the application interface prediction results and the slot recognition results.

[0073] The calculation module of the implementation mode of the present application is also used to calculate the attention scores of the application interface prediction results and the slot recognition results to determine the degree of correlation between the application interface prediction results and the slot recognition results.

[0074] The processor of the embodiment of the present application is also used to calculate the attention score of the application interface prediction result and the slot identification result to determine the correlation degree between the application interface prediction result and the slot identification result.

[0075] Specifically, the implementation method of the present application is to reliably predict the parameter filling value of the application interface prediction result, and thus adopts an attention mechanism to determine the correlation between the application interface prediction result and the slot recognition result.

[0076] For example, taking the aforementioned slot recognition result slots and application prediction result API as an example, that is, taking ["name":"device","pos":[3,4],"rawvalue":"Air Conditioner","name":"poi_name","pos":[8,9],"raw value":"Restaurant"] and "AC_Open","NavigationSearchPoi" as examples, the vehicle can calculate the first attention score grade between "AC_Open" and "'air conditioner' of type 'device'". 11, and calculate the second attention score grade of "AC_Open" and "'restaurant' of type 'poi_name'" 12 Similarly, the vehicle can also calculate the first attention score grade of "NavigationSearchPoi" and "'air conditioning' of type 'device'" 21 , and calculate the second attention score grade of "NavigationSearchPoi" and "'restaurant' of type 'poi_name'" 22 .

[0077] Furthermore, for the first attention score grade of “AC_Open” 11 and the second attention score grade 12 , in the first attention score grade 11 Greater than the second attention score grade 12 In this case, the vehicle can fill in the parameters of "AC_Open" according to ""name":"device","pos":[3,4],"rawvalue":"Air conditioning"", and the execution result of the parameter filling may include: "apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"Air conditioning"}].

[0078] Similarly, for the first attention score grade of “NavigationSearchPoi” 21 and the second attention score grade 22 , in the second attention score grade 21 Greater than the first attention score grade 22 In this case, the vehicle can fill in the parameters of "AC_Open" according to ""name":"poi_name","pos":[8,9],"raw value":"restaurant"]", and the execution result of the parameter filling may include: "apiName":"NavigationSearchPoi","arguments":[{"name":"poi_name","value":"restaurant"}.

[0079] It can be understood that the attention score can be understood to a certain extent as a weighted value of the application prediction result and the slot recognition result, which is used to characterize the degree of correlation between the application prediction result and the slot recognition result.

[0080] Optionally, in certain embodiments of the present application, the attention score of the application prediction result and the slot recognition result can be regarded as the correlation between the application prediction result and the slot recognition result. Therefore, the higher the attention score of the application prediction result and the slot recognition result, the higher the correlation between the application prediction result and the slot recognition result. 11 Higher than grade 12 , and grade 11 The higher the value, the higher the grade. 21 The corresponding "'device' type 'air conditioner'", or in other words, according to grade 21 Corresponding to “”name”:”device”,”pos”:[3,4],”rawvalue”:”Air Conditioner””, the execution result obtained by filling in the parameters of “AC_Open” is more credible, and vice versa.

[0081] In this way, in an embodiment of the present application, the vehicle can calculate the attention scores of the application interface prediction results and the slot identification results based on the setting of the attention mechanism, thereby determining the degree of correlation between the application interface prediction results and the slot identification results, so that the correlation performance between the application interface prediction results and the slot identification results can be reliably determined.

[0082] In some embodiments of the present application, the slot recognition result includes the slot type, the entity of the slot type, and the position of the entity in the voice request, and step 03 includes:

[0083] The location-based calculation API predicts how relevant the result is to the entity of the slot type.

[0084] The calculation module of the implementation mode of the present application is also used to calculate the correlation between the prediction result of the application program interface and the entity of the slot type based on the position.

[0085] The processor of the embodiment of the present application is also used to calculate the degree of relevance between the application program interface prediction result and the entity of the slot type based on the location.

[0086] Specifically, in an embodiment of the present application, when one or more application programming interfaces are predicted, the vehicle can determine the degree of correlation between each predicted application programming interface and the entity of each slot type based on the position of the entity of each slot type in the slot recognition result in the voice request.

[0087] For example, taking the above slot recognition result slots and the application program interface prediction result API as an example, that is, taking ["name":"device","pos":[3,4],"rawvalue":"Air Conditioning","name":"poi_name","pos":[8,9],"raw value":"Restaurant"] and "AC_Open","NavigationSearchPoi" as examples, the ""name":"device"" in the slot recognition result slots can be understood as: the slot type named device.

[0088] “”pos”:[3,4]” can be understood as: the slot type entity of device is composed of the 3rd to 4th semantic units (tokens) in the voice request (i.e. “turn on the air conditioner and navigate to the mall”).

[0089] ""raw value":"restaurant"]" can be understood as: the entity of the slot type device includes the "air conditioner" in the voice request (that is, "turn on the air conditioner and navigate to the mall").

[0090] Similarly, “”name”:”poi_name”,”pos”:[8,9],”raw value”:”restaurant”]” can be understood as: for the slot type named device, the entity of this type is “restaurant”, and “restaurant” is composed of the 8th to 9th semantic units (tokens) in the voice request (i.e. “turn on the air conditioner and navigate to the mall”).

[0091] To explain the implementation of the present application more clearly, please refer to Table 1, which can be used to characterize the positions of semantic units of various slot types in the voice request "turn on the air conditioner and navigate to the mall."

[0092] Table 1

[0093]

[0094] It is understandable that in Table 1, the first row <device> and< / device> Together with 3 and 4 in the second row, it indicates that the entity of the slot type device is composed of the 3rd to 4th semantic units (tokens) in the voice request (i.e., "Turn on the air conditioner and navigate to the mall"). At the same time, in Table 1,<pos_name> and< / pos_name> Together with 8 and 9 in the second line, it indicates that the entity of the slot type pos_name is composed of the 8th to 9th semantic units (tokens) in the voice request (i.e., "Turn on the air conditioner and navigate to the mall").

[0095] Based on this, the vehicle in the implementation mode of the present application can calculate the correlation between the "predicted application interface" and the "entity of each slot type" pointed to by the "position of each slot type entity (or semantic unit) in the voice request" in the slot identification information, that is, obtain the correlation between the slot identification result and the application prediction result.

[0096] Optionally, in certain embodiments of the present application, when the interface interface in the “predicted application program interface” t An entity of the type R in "Entities of each slot type" t The degree of rela t , higher than interface t R in "Entities of each slot type" t When the degree of relevance to other entities outside the interface t The parameter filling process can be based on R t Finish.

[0097] For example, when the correlation between the application interface "AC_Open" and "device type air conditioner" is higher than the correlation between the application interface "AC_Open" and "pos_name type restaurant", "AC_Open" can be filled based on ""name":"device","pos":[3,4],"rawvalue":"air conditioner"", and the execution result of "apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"air conditioner"}] can be obtained.

[0098] In addition, it is understood that before the vehicle performs natural language processing on the voice request, it can divide the voice request to determine the various semantic units included in the voice request, or in other words, it can perform tokenization processing on the voice request to determine the various semantic units included in the voice request. For example, for the voice request of "turn on the air conditioner and navigate to the mall", the semantic units obtained by tokenizing the voice request and arranged in order can be ['turn on', 'open', 'air', 'adjust', 'guide', 'navigation', 'go', 'shopping', 'mall'], that is, 9 semantic units.

[0099] Furthermore, it is understandable that in different language environments, the specific setting of semantic units may be different. For example, in the above example, for the Chinese voice request "Turn on the air conditioner and navigate to the mall", the semantic unit can be understood as a single Chinese character. For voice requests in other languages, it needs to be set according to the actual situation. For example, for the English voice request "Turn on the air conditioner", the semantic unit of the voice request can be ['Turn', 'on', 'the', 'air', 'conditioner'], that is, 5 semantic units / English words.

[0100] In this way, in the implementation mode of the present application, the vehicle can calculate the correlation between the application interface prediction result and the slot type entity based on the slot type in the slot information, the slot type entity and the position of the entity in the voice request, which ensures the reliable calculation of the correlation to a certain extent and realizes the indexing of the slot type entity based on the position.

[0101] In certain embodiments of the present application, step 05 includes:

[0102] Generate control instructions based on the execution results to control the vehicle to perform corresponding functions and complete voice interaction.

[0103] The interaction module of the implementation mode of the present application is also used to generate control instructions according to the execution results to control the vehicle to execute corresponding functions and complete voice interaction.

[0104] The processor of the implementation mode of the present application is also used to generate control instructions according to the execution results to control the vehicle to execute corresponding functions and complete voice interaction.

[0105] Specifically, in the implementation manner of the present application, the vehicle can complete the filling of the parameters of the application program interface and obtain the execution result of the parameter filling, and then generate a control instruction based on the execution result and execute the corresponding function through the control instruction.

[0106] For example, taking the above voice request "Turn on the air conditioner and navigate to the restaurant" and the above execution result apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"Air conditioner"}],"apiName":"NavigationSearchPoi","arguments":[{"name":"poi_name","value":"Restaurant"}] as an example, the vehicle can generate corresponding control instructions to control the start of the vehicle's air conditioner, and determine and display the driving route to the restaurant through the on-board navigation function.

[0107] In one example, the vehicle may generate control instructions of "using 'air conditioning' as an input parameter to call interface 'AcOpen'" and "using restaurant as an input parameter to call interface NavigationSearchPoi", thereby controlling the start-up of the vehicle's air conditioning and determining and displaying the driving route to the restaurant through the vehicle's navigation function.

[0108] Thus, in the implementation mode of the present application, the vehicle can complete the parameter filling of the application result, thereby obtaining the execution result of the parameter filling, and can generate a control instruction through the execution result to execute the corresponding function.

[0109] In certain embodiments of the present application, step 05 includes:

[0110] Fill the execution result into the target field of the output content, the target field includes a first target field and a second target field, the first target field is configured to fill in the application interface, and the second target field is configured to fill in the execution result filled with the application interface parameters.

[0111] The interactive module of the implementation mode of the present application is also used to fill the execution result into the target field of the output content, the target field includes a first target field and a second target field, the first target field is configured to fill in the application interface, and the second target field is configured to fill in the execution result filled with the application interface parameters.

[0112] The processor of the embodiment of the present application is also used to fill the execution result into the target field of the output content, the target field includes a first target field and a second target field, the first target field is configured to fill in the application interface, and the second target field is configured to fill in the execution result filled with the application interface parameters.

[0113] Specifically, it is understood that when a certain function of a vehicle is executed through a control instruction (or a computer instruction), it is usually necessary to specify an execution object and execution parameters. Among them, the execution object can be understood as the application program interface in the implementation of the present application. The execution parameters can be understood as the input parameters of the application program interface, which can indicate the execution effect, execution method, execution degree, etc. of the application program interface.

[0114] Based on this, the implementation methods of the present application can construct output content for outputting control instructions through "a first target field for filling in the application interface" and "a second target field for filling in the (incoming) parameters of the application interface".

[0115] For example, taking the above voice request "Turn on the air conditioner and navigate to a restaurant" and the above execution apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"Air conditioner"}],"apiName":"NavigationSearchPoi","arguments":[{"name":"poi_name","value":"Restaurant"}] as an example, the vehicle can fill in {'ACTIONS':[X1],'ARGUMENTS':[X2]} based on the execution result, such as filling AcOpen into [X1], and filling device and air conditioner into [X2]. The input content obtained may include: {'ACTIONS':'ACOPEN','ARGUMENTS':['DEVICE','Air conditioner']}. It can be understood that X1 can be understood as the first target field, and X2 can be understood as the second target field.

[0116] Similarly, NavigationSearchPoi may be filled into [X1], and poi_name and air conditioner may be filled into [X2], and the input content obtained thereby may include: {'ACTIONS':'NAVIGATIONSEARCHPOI','ARGUMENTS':['POI_NAME','restaurant']}.

[0117] Thus, in the embodiment of the present application, when the vehicle fills in the parameters of the application program interface to obtain the execution result of the parameter filling, the vehicle fills in the first target field and the second target field using the execution result to form output content.

[0118] See also Figure 2 In certain embodiments of the present application, step 05 includes:

[0119] 050: Parse the filling content of the target field in the output content;

[0120] 051: Generate an execution instruction corresponding to the voice request based on the filled content and send it to the vehicle to complete the voice interaction.

[0121] The interaction module of the implementation mode of the present application is also used to parse the filled content of the target field in the output content, and to generate an execution instruction corresponding to the voice request according to the filled content and send it to the vehicle to complete the voice interaction.

[0122] The processor of the embodiment of the present application is also used to parse the filled content of the target field in the output content, and to generate an execution instruction corresponding to the voice request based on the filled content and send it to the vehicle to complete the voice interaction.

[0123] Specifically, in an embodiment of the present application, the vehicle may generate a control instruction for implementing the voice request based on the filled content of the target field in the filling result.

[0124] For example, taking the above {'ACTIONS':'ACOPEN','ARGUMENTS':['DEVICE','Air-conditioning']} as an example, the vehicle can generate a corresponding control instruction based on 'ACOPEN' and ['DEVICE','Air-conditioning'], and control the vehicle to start according to the control instruction.

[0125] Similarly, taking the above {'ACTIONS':'NAVIGATIONSEARCHPOI','ARGUMENTS':['POI_NAME','restaurant']} as an example, the vehicle can determine the driving route to the restaurant through the on-board navigation function according to 'NAVIGATIONSEARCHPOI' and ['POI_NAME','restaurant'] and display it on the vehicle display.

[0126] In this way, in the implementation mode of the present application, the vehicle can generate control instructions corresponding to the voice request according to the filled content, and execute corresponding functions through the control instructions to complete the voice interaction.

[0127] In certain embodiments of the present application, step 02 includes:

[0128] Make API predictions for voice requests based on a pre-trained generative model.

[0129] The recognition module of the implementation mode of the present application is also used to perform application program interface prediction on the voice request based on the pre-trained generative model.

[0130] The processor of the embodiment of the present application is also used to perform application program interface prediction on voice requests based on a pre-trained generative model.

[0131] Specifically, in order to ensure reliable prediction of the application program interface, the implementation mode of the present application can perform application program interface prediction on voice requests based on a pre-trained generative model.

[0132] Specifically, in order to ensure the prediction accuracy of the generative model, the implementation method of the present application can fine-tune the pre-set generative model based on the pre-trained weights of BERT (Bidirectional Encoder Representations from Transformers) so that the generative model after fine-tuning training can be suitable for application interface prediction tasks.

[0133] It should also be noted that the slot identification information in the implementation of the present application can be determined based on other algorithms or models, and the vehicle in the implementation of the present application can splice the slot identification information and the voice request to form corresponding splicing information and then input the splicing information into the generative model. The generative model can predict the application program interface for implementing the voice request based on the splicing information.

[0134] Furthermore, in certain embodiments of the present application, after obtaining the above-mentioned splicing information, the generative model can predict the application program interface used to implement the voice request based on the splicing information to obtain the application program interface prediction result. In addition, when the application program interface prediction result is obtained, the generative model can also calculate the correlation between the application program interface prediction result and the "slot identification information in the splicing information", and perform parameter filling processing of the application program interface according to the correlation.

[0135] For example, taking the above-mentioned AcOpen, NavigationSearchPoi and slots as an example, and letting the voice request "turn on the air conditioner and navigate to the restaurant" be the query, and letting the "splicing information formed by splicing query and slots" be "query+slots", then: when the generative model receives query+slots, the generative model predicts the application program interface based on query+slots to obtain two application program interfaces, AcOpen and NavigationSearchPoi.

[0136] After that, the generative model can calculate the correlation between AcOpen and query+slots, and then determine that AcOpen can be filled with arguments based on the device type entity in the query (i.e., air conditioner). Similarly, the generative model can calculate the correlation between NavigationSearchPoi and query+slots, and then determine that AcOpen can be filled with arguments based on the poi_name type entity in the query (i.e., restaurant).

[0137] Furthermore, in certain embodiments of the present application, a code or module for implementing an attention mechanism is provided in the generative model, and thus: the generative model can pay attention to AcOpen and query+slot to obtain a corresponding attention score, that is, the degree of relevance; similarly, the generative model can pay attention to NavigationSearchPoi and query+slot to obtain a corresponding attention score.

[0138] Thus, in the implementation mode of the present application, the vehicle can predict the application program interface based on the pre-trained generative model, so the credibility of the predicted application program interface can be guaranteed to a certain extent.

[0139] In certain embodiments of the present application, step 02 includes:

[0140] Perform slot recognition on voice requests based on the pre-trained recognition model and pre-configured dictionary data.

[0141] The recognition module of the implementation mode of the present application is also used to perform slot recognition on the voice request based on the pre-trained recognition model and pre-configured dictionary data.

[0142] The processor of the embodiment of the present application is also used to perform slot recognition on the voice request based on the pre-trained recognition model and pre-configured dictionary data.

[0143] It is understandable that when using the NER (Named Entity Recognition) model to perform slot recognition for voice requests, the corpus information used to train the NER model may be outdated or cover fewer fields. When the NER model is trained again using currently popular corpus information or corpus information from multiple fields, corresponding maintenance and update costs will have to be faced.

[0144] Based on this, in order to ensure the credibility of the slot recognition results, the vehicle in the implementation mode of the present application can also assist in the recognition of named entities and slots of voice requests based on a pre-configured external dictionary (i.e., dictionary data). That is, when performing slot recognition of voice requests through the NER model, it is also possible to assist in the recognition of named entities and slots with the help of corpus information such as currently popular playlists, singer names, song vocabulary, and navigation location names maintained by the external dictionary, and then output the slot recognition results of the voice request. That is, after the vehicle can input the voice request into the recognition model (i.e., the NER model), the recognition model can perform slot recognition on the voice request based on the external dictionary, and then output the slot recognition results of the voice request.

[0145] In this way, the implementation method of the present application can perform slot recognition of voice requests based on dictionary data and recognition models, so the credibility of the slot recognition results can be guaranteed to a certain extent.

[0146] Optional, see Figure 3 and Figure 4 , Figure 3 and Figure 4 are all flowcharts of the interaction method in certain embodiments of the present application. Specifically, Figure 3 As shown, let the voice request of "turn on the air conditioner and navigate to the restaurant" be a query, then: when the vehicle obtains the query, the vehicle can perform corresponding query operations through an external dictionary (dictionary data), thereby inputting the query and the query results into the recognition model (or, NER model) together.

[0147] Subsequently, the recognition model can perform slot recognition on the query based on the query and the query result to output the slot recognition results slots, that is, ["name":"device","pos":[3,4],"rawvalue":"air conditioner","name":"poi_name","pos":[8,9],"raw value":"restaurant"].

[0148] Afterwards, the vehicle can concatenate query and slots to form a corresponding concatenation result query+slots, and then input the concatenation result into the generative model. The specific content of query+slots can be found in Table 1.

[0149] like Figure 4 As shown, the generative model can predict the API of query based on the concatenation information query+slot to obtain API1 and API2, namely AcOpen and NavigationSearchPoi.

[0150] Then, for AcOpen and NavigationSearchPoi, the generative model can pay attention to AcOpen and query+slot based on the entities in the query pointed to by the position coordinates in the splicing information query+slot (i.e., [3,4] and [8,9] in the slot), and when it is determined that the "attention of AcOpen and device type entities" is higher than the "attention of AcOpen and pos_name type entities", the parameter filling of AcOpen is completed using "name":"device","pos":[3,4],"rawvalue":"air conditioner".

[0151] Similarly, the generative model pays attention to NavigationSearchPoi and query+slot based on the entities in the query pointed to by the position coordinates in the concatenated information query+slot (i.e., [3,4] and [8,9] in the slot), and when it is determined that the "attention of NavigationSearchPoi and device-type entities" is lower than the "attention of NavigationSearchPoi and pos_name-type entities", the parameters of NavigationSearchPoi are filled in using "name":"poi_name","pos":[8,9],"raw value":"restaurant".

[0152] Thus, the generative model completes the parameter filling of NavigationSearchPoi and AcOpen respectively, and the output execution result of the parameter filling can be: apiName":"AcOpen","arguments":[{"name":"device","type":"string","value":"air conditioner"}],"apiName":"NavigationSearchPoi","arguments":[{"name":"poi_name","value":"restaurant"}].

[0153] Ultimately, the vehicle can perform analysis and related processing based on the execution results to generate corresponding control instructions, and execute corresponding functions according to the control instructions, such as controlling the start-up of the vehicle's air conditioning, and controlling the display screen to display the "navigation page to the mall", thereby completing voice interaction with the user.

[0154] It can be understood that compared with the natural language processing solution implemented based on pipeline, the implementation method of this application is based on Figure 3 , 4 The natural language processing solution shown implements end-to-end natural language processing. At the same time, based on the generative model, the corresponding natural language understanding results (i.e., "execution results") are directly generated from the voice request triggered by the user, thereby reducing the difficulty of understanding voice commands with complex intentions to a certain extent.

[0155] In addition, because the reasoning of natural language understanding results is completed based on a single generative model, it can avoid the problems of complex system, high maintenance cost, unstable service, etc. in pipeline solutions to a certain extent, optimize the architecture of natural language processing system, improve the coordination efficiency between modules, and facilitate expansion and maintenance. It can also reduce service timeouts and failures to a certain extent, and ensure the user's voice interaction experience.

[0156] Optionally, in order to accurately locate entities of different slot types in the query, the vehicle can perform position encoding operations on the semantic units in the query based on a pre-set program or tool. Specifically, the vehicle can construct special tokens based on {"device":"Air-conditioning"} and {"poi_name":"restaurant"} output by the NER model to record the starting and ending positions of entities of different slot types, such as through <device> Points to the starting position of the entity of device type, through< / device> Represents the end position of the device.

[0157] Furthermore, the vehicle can also correspond the slot position encoding to the words in the query. For example, the semantic units constituting "air conditioning" are located at the 3rd and 4th positions in the query, and the semantic units of "restaurant" are located at the 8th and 9th positions in the query. Therefore, the vehicle can encode the corresponding position in the slot, thereby obtaining the encoding result shown in Table 1.

[0158] In addition, it should be noted that the voice interaction method provided in the embodiment of the present application can be applied to a server that has established a communication connection with the vehicle in addition to being applied to the vehicle. It can be understood that when the voice interaction provided in the embodiment of the present application is applied to the server, the server can execute the above steps 02 to 04 based on the voice request forwarded by the vehicle, and then when the execution result of the parameter filling of the application is obtained, the execution result can be sent to the vehicle for the vehicle to perform the corresponding function, or the corresponding vehicle control instruction can be generated according to the execution result, and then the vehicle control instruction can be sent to the vehicle to control the vehicle to perform the corresponding function.

[0159] The present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned voice interaction method when the computer program is executed by one or more processors.

[0160] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0161] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0162] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Get voice request; Performing slot recognition and application program interface prediction on the voice request to obtain a slot recognition result and an application program interface prediction result; Calculating the correlation between the application program interface prediction result and the slot identification result; According to the application program interface prediction result, the slot identification result and the correlation degree, filling application program interface parameters for the voice request to obtain an execution result of the application program interface parameter filling; Controlling the vehicle according to the execution result to complete the voice interaction; The calculating the correlation between the application program interface prediction result and the slot identification result includes: In the case where the application program interface prediction results include multiple ones and the slot identification results include multiple ones, the correlation degree between each of the application program interface prediction results and each of the slot identification results is calculated.

2. The method according to claim 1, characterized in that The calculating the correlation between the application program interface prediction result and the slot identification result includes: The attention scores of the application program interface prediction result and the slot identification result are calculated to determine the degree of correlation between the application program interface prediction result and the slot identification result.

3. The method according to claim 1, characterized in that The slot recognition result includes a slot type, an entity of the slot type, and a position of the entity in the voice request, and the calculating the correlation between the application program interface prediction result and the slot recognition result includes: The correlation between the application program interface prediction result and the entity of the slot type is calculated based on the position.

4. The method according to claim 1, characterized in that: The controlling the vehicle according to the execution result to complete the voice interaction includes: A control instruction is generated according to the execution result to control the vehicle to execute a corresponding function and complete the voice interaction.

5. The method according to claim 4, characterized in that The controlling the vehicle according to the execution result to complete the voice interaction includes: The execution result is filled into a target field of the output content, wherein the target field includes a first target field and a second target field, wherein the first target field is configured to fill in the application program interface, and the second target field is configured to fill in the execution result filled with the application program interface parameters.

6. The method according to claim 5, characterized in that The controlling the vehicle according to the execution result to complete the voice interaction includes: Parsing the filling content of the target field in the output content; An execution instruction corresponding to the voice request is generated according to the filled content and sent to the vehicle to complete the voice interaction.

7. The method according to claim 1, characterized in that The performing slot identification and application program interface prediction on the voice request to obtain a slot identification result and an application program interface prediction result includes: Based on the pre-trained generative model, an application program interface prediction is performed on the voice request.

8. The method according to claim 1, characterized in that The performing slot identification and application program interface prediction on the voice request to obtain a slot identification result and an application program interface prediction result includes: Based on the pre-trained recognition model and pre-configured dictionary data, slot recognition is performed on the voice request.

9. A vehicle, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images