Voice interaction method, device, server and computer readable storage medium

Abstract

The application discloses a voice interaction method, comprising: receiving a voice request forwarded by a vehicle; encoding the voice request according to a preset model to obtain an encoded sequence matrix for slot recognition; dividing the encoded sequence matrix; performing slot recognition on the voice request according to the divided encoded sequence matrix; performing application program interface prediction on the voice request; selecting a predicted application program interface to perform application program interface parameter filling according to the result of slot recognition and the predicted application program interface; and outputting an execution result to the vehicle to complete voice interaction. The application encodes the voice request, performs slot recognition according to the divided encoded sequence matrix, performs application program interface parameter filling according to the result of slot recognition, and finally outputs an execution result and sends it to the vehicle, effectively improving the accuracy of recognizing slots composed of discontinuous words in the voice request and improving the voice interaction experience of users.

Description

Technical Field

[0001] This invention relates to the field of vehicle voice technology, and in particular to a voice interaction method, a voice interaction device, a server, and a computer-readable storage medium. Background Technology

[0002] Current dialogue systems utilize natural language generation modules to parse user statements into machine-understandable semantic tags. A dialogue state tracking module maintains an internal dialogue state as a compact representation of the entire dialogue history. Based on this state, a dialogue strategy module selects appropriate dialogue actions, and finally, the natural language generation module converts these actions into natural language responses. However, in real-world dialogue scenarios, user voice requests may not accurately match the required slot information. For example, if the slot to be identified is discontinuous in the user's voice request, the recognition results in related technologies may be incorrect, failing to extract the expected slot result. This leads to a lack of fluency in voice interaction in in-vehicle environments, making it difficult to meet the vehicle control requirements of in-vehicle scenarios. Summary of the Invention

[0003] This application provides a voice interaction method, a voice interaction device, a server, and a computer-readable storage medium.

[0004] The voice interaction method of this application includes:

[0005] Receive voice requests forwarded by vehicles;

[0006] The voice request is encoded according to a preset model to obtain an encoding sequence matrix for slot identification.

[0007] The encoded sequence matrix is ​​divided into parts;

[0008] Slot identification is performed on the voice request based on the encoded sequence matrix after the partitioning process.

[0009] The voice request is predicted using the application programming interface (API).

[0010] Based on the slot identification results and the predicted application interface, the predicted application interface is selected to perform application interface parameter filling, and the execution result is sent to the vehicle to complete the voice interaction.

[0011] Thus, the voice interaction method of this application can encode voice requests to obtain an encoding sequence matrix, and then perform slot recognition on the voice requests based on the divided encoding sequence matrix. Finally, it can fill in the application interface with parameters based on the slot recognition results and the predicted slot recognition results, and finally output the execution result and send it to the vehicle to complete the voice interaction. In this application embodiment, the encoding sequence matrix obtained by encoding voice requests and the slot recognition based on it can effectively improve the accuracy of recognizing slots composed of discontinuous words in user voice requests, thereby improving the user's voice interaction experience.

[0012] In some implementations, encoding the voice request according to a preset model to obtain an encoded sequence matrix for slot identification includes:

[0013] The voice request is processed by text sequence encoding to obtain a first encoding vector;

[0014] The first encoding vector is input into the pre-trained model to obtain the output matrix corresponding to each encoding in the text sequence;

[0015] The encoded sequence matrix is ​​obtained based on the output matrix and the preset model.

[0016] In this way, the voice request can be encoded according to the preset model to obtain the encoded sequence matrix used for slot recognition.

[0017] In some implementations, obtaining the encoded sequence matrix based on the output matrix and the preset model includes:

[0018] Extract the head matrix corresponding to the first code and the tail matrix corresponding to the last code from the output matrix;

[0019] The head matrix and the tail matrix are encoded according to the preset model to obtain the encoded sequence matrix.

[0020] In this way, the matrix corresponding to the first and last codes can be extracted from the output matrix, and the extracted matrix can be encoded to obtain the encoding sequence matrix, so as to identify the slot of the voice request.

[0021] In some embodiments, the partitioning process of the encoded sequence matrix includes:

[0022] The encoded sequence matrix is ​​divided along its diagonal to obtain a first sub-matrix and a second sub-matrix.

[0023] The voice request is slot-identified based on the encoded sequence matrix after the partitioning process.

[0024] Slot identification is performed on the voice request based on the first sub-matrix and the second sub-matrix.

[0025] In this way, the encoded sequence matrix can be divided, and the resulting sub-matrix can be used to perform slot recognition on the voice request, making the slot recognition result more accurate.

[0026] In some implementations, the step of slot identification of the voice request based on the first sub-matrix and the second sub-matrix includes:

[0027] Based on the first sub-matrix, determine the target coded sub-sequence for slot identification;

[0028] Based on the target encoded sub-sequence and the second sub-matrix, slot identification is performed on the voice request.

[0029] In this way, the target encoded subsequence can be obtained from the first submatrix, and combined with the second submatrix to complete the slot identification process, making the identification results of slots composed of discontinuous words more accurate.

[0030] In some implementations, determining the target coded subsequence for slot identification based on the first sub-matrix includes:

[0031] Identify all semantic vectors in the voice request;

[0032] According to the preset slot value table, mark the head and tail positions of the target slot value in the first sub-matrix, wherein the target slot value is determined according to the correspondence between the semantic vector and the slot value table;

[0033] The target encoded subsequence in which the target slot value is located is determined based on the head and tail position identifiers.

[0034] In this way, by using all the identified semantic vectors and slot value table, the target encoded subsequence where the target slot value is located in the first sub-matrix can be identified by the head and tail position identifiers, so as to determine the target slot value in the future, complete the identification process of slots composed of discontinuous words, improve the accuracy of the slot identification process and the fluency of the voice interaction process.

[0035] In some implementations, the step of performing slot identification on the voice request based on the target encoded sub-sequence and the second sub-matrix includes:

[0036] In the second sub-matrix, adjacency relationships of each code in the target coded sub-sequence are marked according to the target slot value;

[0037] The adjacent relationship identifier is used to concatenate the codes that have the adjacent relationship to obtain the target slot value, so as to obtain the result of slot recognition for the voice request.

[0038] In this way, the adjacency relationship of each code in the target coded subsequence can be identified by the second submatrix, and the codes with adjacency relationships can be concatenated to obtain the target slot value, thus completing the recognition process of slots composed of discontinuous words, improving the accuracy of slot recognition and the fluency of the voice interaction process.

[0039] In some implementations, the step of selecting the predicted application interface (API) based on the slot identification result and the predicted API, performing API parameter filling, and outputting the execution result to the vehicle to complete the voice interaction includes:

[0040] The target parameters for slot filling are determined based on the voice request, the slot recognition result, the predicted application interface, and the predicted application interface type.

[0041] Based on the slot identification result and the target parameter, the predicted application interface is selected to perform application interface parameter filling, and the execution result is output and sent to the vehicle to complete the voice interaction.

[0042] Thus, the implementation method of this application can select the predicted application interface to perform application interface parameter filling based on the slot identification result and target parameters, and directly output the execution result to the vehicle to complete the voice interaction, which can reduce the latency of the vehicle system and improve the response speed to user commands.

[0043] The voice interaction device according to the embodiments of this application includes:

[0044] The receiving module is used to receive voice requests forwarded by the vehicle;

[0045] The encoding module is used to encode the voice request according to a preset model to obtain an encoding sequence matrix for slot identification.

[0046] The processing module is used to partition the encoded sequence matrix;

[0047] The slot identification module identifies the slot of the voice request based on the encoded sequence matrix after the partitioning process.

[0048] The interface prediction module performs application interface prediction on the voice request;

[0049] The parameter filling module is used to select the predicted application interface based on the slot identification result and the predicted application interface, perform application interface parameter filling, and output the execution result to the vehicle to complete the voice interaction.

[0050] The server according to the embodiments of this application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the above-described method.

[0051] The computer-readable storage medium of this application embodiment stores a computer program that, when executed by one or more processors, implements the voice interaction method described in any of the above embodiments.

[0052] Additional aspects and advantages of embodiments of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0053] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0054] Figure 1 This is a schematic diagram of the structure of a dialogue system in related technologies;

[0055] Figure 2 This is a schematic diagram of the structure of a dialogue system with an end-to-end architecture according to an embodiment of this application;

[0056] Figure 3 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the modules of the voice interaction method according to the embodiments of this application;

[0058] Figure 5 This is a schematic diagram of the encoding sequence matrix of the voice interaction method according to the embodiments of this application;

[0059] Figure 6 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0060] Figure 7 This is a schematic diagram of the processing flow of the voice interaction method according to the embodiments of this application;

[0061] Figure 8 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0062] Figure 9 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0063] Figure 10 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0064] Figure 11 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0065] Figure 12 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0066] Figure 13 This is a flowchart illustrating the voice interaction method according to an embodiment of this application;

[0067] Figure 14 This is a schematic diagram of the encoding sequence matrix of the voice interaction method according to the embodiments of this application. Detailed Implementation

[0068] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present invention, and should not be construed as limiting the embodiments of the present invention.

[0069] Please see Figure 1 Traditional in-vehicle voice architectures are based on a modular strategy, employing a division of labor among components to achieve the entire dialogue process, such as natural language understanding, state tracking, dialogue strategy, and natural language generation. These components are either primarily manually created according to rules or generated by training models on supervised datasets. Training each component requires a large amount of labeled data, which is often very expensive, limiting the system's scalability. Furthermore, traditional in-vehicle voice systems rely heavily on rules and business logic to ensure accuracy and stability, further limiting their scale and functionality.

[0070] From the perspective of the overall dialogue processing chain, after the traditional in-vehicle voice architecture receives user input, it needs to perform natural language understanding, that is, perform domain classification, intent recognition and slot recognition. Then, in the dialogue management module, the dialogue state and dialogue strategy are combined to select and execute the application programming interface (API) that meets the user input requirements, and the system output of the interaction with the user is returned through the natural language generation module.

[0071] In view of this, please refer to Figure 2The dialogue system based on an end-to-end architecture according to the embodiments of this application includes three core algorithm modules: a slot recognition module for extracting slot information from the user's voice input; an action prediction (AP) module for predicting the application interface corresponding to the user's input to achieve the user's current goal; and an argument filling (AF) module for identifying the parameters in the application interface obtained in the previous step corresponding to the slot information in the user input.

[0072] The slot recognition module is used to obtain the slot information of the action execution subject that needs to be called in the application interface. The action prediction module determines whether the application interface called by the user voice input is correct. The parameter filling module selects which vehicle parts to use as parameters for the application interface to be executed.

[0073] However, for diverse voice requests from users, the slot recognition process may fail to identify slots composed of discontinuous characters in the voice request, resulting in accuracy issues. For example, regarding the target slot "refrigerator lock," users may not directly say standard phrases like "open / close the refrigerator lock," but rather "unlock the refrigerator." However, because related technologies cannot skip the "unlock" step to accurately identify the slot value "refrigerator lock," slot recognition may fail or fail to identify, thus the extracted slot may not meet the requirements. In related technologies, to meet users' recognition needs for such expressions, "unlock the refrigerator" is usually normalized to "refrigerator lock" during the normalization stage. However, this method increases business logic, development costs, and maintenance costs.

[0074] Based on the issues mentioned above, please refer to Figure 3 This application provides a voice interaction method. The voice interaction method includes:

[0075] 01: Receive voice requests forwarded by vehicles;

[0076] 02: Encode the voice request according to the preset model to obtain the encoding sequence matrix used for slot recognition;

[0077] 03: Divide the encoded sequence matrix;

[0078] 04: Slot identification of voice requests based on the segmented encoded sequence matrix;

[0079] 05: Perform application programming interface (API) prediction on voice requests;

[0080] 06: Based on the slot recognition results and the predicted application interface, select the predicted application interface to perform application interface parameter filling, and output the execution result to the vehicle to complete the voice interaction.

[0081] Please see Figure 4 This application provides a voice interaction device 100. The voice interaction method of this application can be implemented by the voice interaction device 100. Specifically, the voice interaction device 100 includes a receiving module 101, an encoding module 102, a processing module 103, a slot recognition module 104, an interface prediction module 105, and a parameter filling module 106. The receiving module 101 receives voice requests forwarded by a vehicle. The encoding module 102 encodes the voice request according to a preset model to obtain an encoding sequence matrix for slot recognition. The processing module 103 divides the encoding sequence matrix. The slot recognition module 104 performs slot recognition on the voice request based on the divided encoding sequence matrix. The interface prediction module 105 predicts the application programming interface (API) for the voice request. The parameter filling module 106 selects the predicted API based on the slot recognition result and the predicted API, performs API parameter filling, and outputs the execution result to the vehicle to complete the voice interaction.

[0082] This application also provides a server. The server includes a processor and a memory, and the memory stores a computer program. The processor is used to receive voice requests forwarded by a vehicle, encode the voice requests according to a preset model to obtain an encoding sequence matrix for slot identification, divide the encoding sequence matrix, perform slot identification on the voice requests based on the divided encoding sequence matrix, predict the application programming interface (API) for the voice requests, select the predicted API based on the slot identification result and the predicted API, perform API parameter filling, and output the execution result to the vehicle to complete the voice interaction.

[0083] Specifically, first, the server receives user voice requests forwarded by vehicles and encodes the voice requests according to a preset model. After encoding, slot information and other information within the user's voice request statement can be distinguished. This encoding process yields a slot identification encoding sequence matrix. In addition to the text content, the encoding sequence matrix includes marker characters such as "[CLS]" and "[SEP]". The "[CLS]" character is the starting marker for the text. For multiple consecutive voice requests, a "[SEP]" identifier is added between each voice request to separate the two sentences. For example... Figure 5 The encoded sequence matrix corresponding to the voice request "unlock the refrigerator".

[0084] After encoding the voice request to obtain the corresponding encoded sequence, the encoded matrix sequence needs to be divided to locate continuous or discontinuous slot information within the voice request. In some implementations, the encoded sequence matrix can be divided into an upper triangular matrix and a lower triangular matrix along the diagonal. Figure 5 The image shows the result of dividing the encoded sequence matrix corresponding to the voice request "unlock the refrigerator" diagonally. Different implementations may use different methods for dividing the encoded sequence matrix; the specific method is not limited here.

[0085] Specifically, when the slot information to be identified is discontinuous in the user's voice request, slot identification can be performed based on the information in the pre-divided encoded sequence matrix. When the encoded sequence is divided into upper and lower triangular matrices along the diagonal, each element in the lower triangular encoded sequence matrix can be viewed as a vector representation of a sub-text sequence within the voice request. Slot identification can then be performed based on the vector representation of the sub-text sequence. For example... Figure 5 As shown, for the voice request "unlock the refrigerator", the system can identify the two discontinuous slots of "refrigerator" and "lock" in the voice request, and finally obtain the slot identification result as "refrigerator lock". It can be understood that the slot identification result of this voice request is the same as the slot identification result of similar voice requests such as "open the refrigerator lock" that include the complete target slot.

[0086] To address the issue of excessively high labor and data costs caused by the need to design separate slot recognition models for each vertical domain in slot recognition, the slot recognition scheme of this application adopts an end-to-end architecture, does not distinguish between vertical domains, and does not require training models within each vertical domain.

[0087] After slot recognition is completed, the application programming interface (API) prediction for the user's voice request can be performed based on the slot recognition results. First, the required API for the voice request can be predicted using the Action Prediction (AP) module, based on the slot recognition results. For example, the API prediction for the user's voice request "Play song A" yields API 1 for playing music. The API prediction for the user's voice request "Navigate to destination A" yields API 2 for navigation.

[0088] In addition, the Argument Filling (AF) module can fill the parameters in the application interface by selecting the slot recognition results, and finally output the execution results to the vehicle to complete the voice interaction.

[0089] The end-to-end architecture of this application can simplify the intermediate modules of traditional dialogue system architectures, such as natural language understanding modules, dialogue management modules, vehicle command generation modules, and natural language generation modules, reduce the calls to multiple models in different vertical domains, reduce the latency of the vehicle system, and improve the response speed to user commands.

[0090] In summary, the voice interaction method of this application encodes voice requests to obtain an encoding sequence matrix, performs slot recognition on the voice request based on the processed encoding sequence matrix, and finally fills in the application interface with parameters based on the slot recognition results and predictions. The final execution result is then output and sent to the vehicle, completing the voice interaction. In this application, the encoding sequence matrix obtained by encoding the voice request and performing slot recognition effectively improves the accuracy of recognizing slots composed of discontinuous words in the user's voice request, enhancing the user's voice interaction experience. Furthermore, it eliminates the need for special processing of target slots during the normalization stage, effectively saving development and maintenance costs.

[0091] Please see Figure 6 In some implementations, step 02 includes:

[0092] 021: Perform text sequence encoding on the voice request to obtain the first encoding vector;

[0093] 022: Input the first encoding vector into the pre-trained model to obtain the output matrix corresponding to each encoding in the text sequence;

[0094] 023: Obtain the encoding sequence matrix based on the output matrix and the preset model.

[0095] In some implementations, the encoding module 102 is used to perform text sequence encoding processing on the voice request to obtain a first encoding vector; input the first encoding vector into a pre-trained model to obtain an output matrix corresponding to each encoding in the text sequence, and obtain an encoding sequence matrix based on the output matrix and the preset model.

[0096] In some implementations, the processor is used to perform text sequence encoding on the voice request to obtain a first encoding vector; input the first encoding vector into a pre-trained model to obtain an output matrix corresponding to each encoding in the text sequence, and obtain an encoding sequence matrix based on the output matrix and the preset model.

[0097] Specifically, after receiving a user's voice request forwarded by the vehicle, the server first needs to perform text sequence encoding on the voice request to obtain the character sequence corresponding to the voice request, called the first encoding vector. In one example, the user's voice request is "unlock the refrigerator," such as... Figure 7 As shown, the voice request is processed by text sequence encoding to obtain the character sequence "Token" corresponding to the voice request, which is the first encoding vector, and its content is "[CLS], put, ice, box, unlock, lock, [SEP]".

[0098] After performing text sequence encoding to obtain the first encoded vector, this first encoded vector needs to be input into the pre-trained model. For example... Figure 7 As shown, BERT can be selected as the pre-trained model, but the specific model selection is not limited here. After pre-training, an output matrix corresponding to each code in the text sequence is obtained. That is, the output matrix includes information about all text sequence codes of the voice request.

[0099] Finally, the encoded sequence matrix can be obtained based on the output matrix and the preset model, such as... Figure 5 and Figure 7 As shown. The preset model can be Biaffine, but the specific model is not limited here. After the output matrix is ​​processed by the preset model, the encoded sequence matrix is ​​obtained. The information in the encoded sequence matrix is ​​the basis for slot recognition of the user's voice request.

[0100] In this way, the voice request can be encoded according to the preset model to obtain the encoded sequence matrix used for slot recognition.

[0101] Please see Figure 8 In some implementations, step 023 includes:

[0102] 0231: Extract the head matrix corresponding to the first code and the tail matrix corresponding to the last code from the output matrix;

[0103] 0232: The head matrix and tail matrix are encoded according to the preset model to obtain the encoded sequence matrix.

[0104] In some implementations, the encoding module 102 is used to extract the head matrix corresponding to the first encoding and the tail matrix corresponding to the last encoding from the output matrix, and to encode the head matrix and the tail matrix according to a preset model to obtain an encoding sequence matrix.

[0105] In some implementations, the processor is used to extract the head matrix corresponding to the first code and the tail matrix corresponding to the last code from the output matrix, and to encode the head matrix and the tail matrix according to a preset model to obtain an encoded sequence matrix.

[0106] Specifically, the first encoding vector obtained by encoding the speech request into a text sequence can be pre-trained to obtain an output matrix. The output matrix contains the information of each encoding in the first encoding vector. It is necessary to extract the head matrix corresponding to the first encoding and the tail matrix corresponding to the last encoding in the output matrix. The head matrix and the tail matrix can relatively completely display the relationship between different slots in the sentence. For example, for the speech request "Unlock the refrigerator", the corresponding first encoding vector is "[CLS], unlock, the, refrigerator, [SEP]". Therefore, the size of the head matrix corresponding to the first encoding "[CLS]" is [batch_size, seq_len, hidden], and it can contain the first complete slot value "refrigerator" in the sequential encoding. Similarly, the size of the head matrix corresponding to the last encoding "[SEP]" is [batch_size, hidden, seq_len], and it can contain the last complete slot value "Unlock the refrigerator" in the sequential encoding.

[0107] The encoding process can introduce a "Biaffine matrix" with a size of [hidden, hidden, num_label]. According to a preset model, such as the Biaffine model, the head matrix and the tail matrix are encoded to obtain an encoded sequence matrix. The encoded matrix contains the slot information of the speech request, and its shape is [batch_size, seq_len, seq_len, num_label].

[0108] In this way, the matrices corresponding to the first encoding and the last encoding in the output matrix can be extracted, and the extracted matrices are encoded to obtain an encoded sequence matrix for slot recognition of the speech request.

[0109] Please refer to Figure 9 In some embodiments, step 03 includes:

[0110] 031: Divide the encoded sequence matrix along the diagonal of the encoded sequence matrix to obtain a first sub-matrix and a second sub-matrix;

[0111] 032: Perform slot recognition on the speech request according to the encoded sequence matrix after the division process;

[0112] 033: Perform slot recognition on the speech request according to the first sub-matrix and the second sub-matrix.

[0113] In some embodiments, the processing module 103 is configured to partition the encoded sequence matrix along the diagonal of the encoded sequence matrix to obtain a first sub-matrix and a second sub-matrix, and perform slot recognition on the voice request according to the partitioned encoded sequence matrix, and perform slot recognition on the voice request according to the first sub-matrix and the second sub-matrix.

[0114] In some embodiments, the processor is configured to partition the encoded sequence matrix along the diagonal of the encoded sequence matrix to obtain a first sub-matrix and a second sub-matrix, and perform slot recognition on the voice request according to the partitioned encoded sequence matrix, and perform slot recognition on the voice request according to the first sub-matrix and the second sub-matrix.

[0115] Specifically, the slot information of the voice request can be determined according to the information in the encoded sequence matrix with the shape of [batch_size, seq_len, seq_len, num_label].

[0116] In one example, when the user issues a voice request "Unlock the refrigerator", the corresponding first encoded vector is "[CLS], put, ice, box, unlock, [SEP]", and the schematic diagram of the encoded matrix sequence corresponding to this voice request as shown in Figure 5 According to the diagonal in the encoded sequence matrix, the encoded sequence matrix can be partitioned into a first sub-matrix, that is, a lower triangular matrix, and a second sub-matrix, that is, an upper triangular matrix.

[0117] According to Figure 5 the information shown in the first sub-matrix in , in the voice request "Unlock the refrigerator", there is a slot value "refrigerator" starting with "ice" and ending with "box", and a single-word slot value "lock".

[0118] In this way, the encoded sequence matrix can be partitioned, and slot recognition is performed on the voice request through the sub-matrices obtained by the partition, making the result of slot recognition more accurate.

[0119] Please refer to Figure 10 , in some embodiments, step 033 includes:

[0120] 0331: Determine the target encoded subsequence for slot recognition according to the first sub-matrix;

[0121] 0332: Perform slot recognition on the voice request according to the target encoded subsequence and the second sub-matrix.

[0122] In some embodiments, the processing module 103 is configured to determine the target encoded subsequence for slot recognition according to the first sub-matrix, and perform slot recognition on the voice request according to the target encoded subsequence and the second sub-matrix.

[0123] In some implementations, the processor is configured to determine a target coded subsequence for slot identification based on a first submatrix, and to perform slot identification on the voice request based on the target coded subsequence and a second submatrix.

[0124] Specifically, the first submatrix is ​​a lower triangular matrix obtained by dividing the encoding sequence matrix along its diagonal. Specific identifiers can be used within this lower triangular matrix to identify the target encoded subsequence for slot identification. The target encoded subsequence is the encoded subsequence containing the slot identification result within the encoding sequence matrix. For example, in the voice request "unlock the refrigerator," the slot identification result is "refrigerator lock." Based on the first submatrix, specific identifiers can be used within the encoding sequence matrix to represent the target encoded subsequence that begins with "ice" and ends with "lock."

[0125] Furthermore, after determining the target encoded sub-sequence for slot recognition based on the first sub-matrix, i.e., the lower triangular matrix, the second sub-matrix, i.e., the upper triangular matrix, can be combined to further perform slot recognition on the voice request. During slot recognition using the upper triangular matrix, specific markers are used to indicate whether two characters have a semantic connection. For example, in the voice request "unlock the refrigerator," the characters "ice" and "box," and "box" and "lock," have a connection, while the other two characters do not have the connection included in the target slot. It should be noted that the connection here includes not only the sequential adjacency of two characters but also semantic continuity. For example, after recognizing the slot "unlock the refrigerator" in the sentence, the slot can be transformed to obtain the slot "refrigerator lock."

[0126] In this way, the target encoded subsequence can be obtained from the first submatrix, and combined with the second submatrix to complete the slot identification process, making the identification results of slots composed of discontinuous words more accurate.

[0127] Please see Figure 11 In some implementations, step 0331 includes:

[0128] 03311: Recognize all semantic vectors in a voice request;

[0129] 03312: According to the preset slot value table, mark the beginning and end positions of the target slot value in the first sub-matrix;

[0130] 03313: Determine the target encoding subsequence where the target slot value is located based on the head and tail position identifiers.

[0131] In some implementations, the processing module 103 is used to identify all semantic vectors in the voice request, and to mark the beginning and end positions of the target slot value in the first sub-matrix according to a preset slot value table. The target slot value is determined based on the correspondence between the semantic vector and the slot value table, and the target encoded sub-sequence where the target slot value is located is determined based on the beginning and end position marks.

[0132] In some implementations, the processor is used to identify all semantic vectors in the voice request, and to mark the beginning and end positions of the target slot value in the first sub-matrix according to a preset slot value table, wherein the target slot value is determined according to the correspondence between the semantic vector and the slot value table, and the target encoded subsequence in which the target slot value is located is determined according to the beginning and end position marks.

[0133] Specifically, in the first submatrix obtained after dividing the encoded sequence matrix, the semantic vector corresponding to the target encoded subsequence, starting with the character corresponding to column "j" and ending with the character corresponding to row "i", can be denoted as coordinates (i,j). The target encoded subsequence may contain specific slot values.

[0134] In one example, a user makes a voice request to "unlock the refrigerator," and receives the following response: Figure 5 The diagram shows the encoding matrix sequence corresponding to the voice request. The diagram contains 7×7 (49 in total) semantic vectors corresponding to the target encoding sub-sequences. The start and end positions of the target slot values ​​in the first sub-matrix can be marked according to a preset slot value table. This preset slot value table can include all possible slot values ​​in the current vehicle system. The target slot value can be determined based on the correspondence between the semantic vector and the slot value table, and can serve as the final result of slot recognition. For example, in the current voice request "unlock the refrigerator," (5,2) represents the target encoding sub-sequence "unlock the refrigerator" starting with 2 and ending with 5. The start and end position markers can be set at position (5,2) in the first sub-matrix. The marker can be THW (Tail-Head-Word). Finally, the sub-sequence corresponding to the position of this start and end position marker can be determined as the target encoding sub-sequence; that is, the target slot value can be obtained from this target encoding sub-sequence. The text corresponding to the remaining semantic vectors does not contain actual semantics and therefore does not correspond to a specific target encoding sub-sequence.

[0135] In this way, by using all the identified semantic vectors and slot value table, the target encoded subsequence where the target slot value is located in the first sub-matrix can be identified by the head and tail position identifiers, so as to determine the target slot value in the future, complete the identification process of slots composed of discontinuous words, improve the accuracy of the slot identification process and the fluency of the voice interaction process.

[0136] Please see Figure 12 In some implementations, step 0332 includes:

[0137] 03321: In the second sub-matrix, based on the target slot value, mark the adjacency relationship identifiers of each code in the target coded sub-sequence;

[0138] 03322: Based on the adjacent relationship identifier, the codes that have adjacent relationships are concatenated to obtain the target slot value, so as to obtain the result of slot recognition for voice requests.

[0139] In some implementations, the processing module 103 is used to identify all semantic vectors in the voice request, and to mark the beginning and end positions of the target slot value in the first sub-matrix according to a preset slot value table. The target slot value is determined based on the correspondence between the semantic vector and the slot value table, and the target encoded sub-sequence containing the target slot value is determined based on the beginning and end position marks.

[0140] In some implementations, the processor is used to mark the adjacency identifiers of each code in the target coded subsequence according to the target slot value in the second submatrix, and to concatenate the codes that have adjacency relationships according to the adjacency identifiers to obtain the target slot value, so as to obtain the result of slot recognition for voice request.

[0141] Specifically, in the second sub-matrix obtained after dividing the encoded sequence matrix, the adjacency markers of each code in the corresponding target encoded sub-sequence can be marked according to the target slot value. At this point, the position (k, l) of the adjacency marker indicates whether there is an adjacency relationship between the character in row "k" and the character in column "l" of the encoded sequence matrix. When there is an adjacency relationship between the character in row "k" and the character in column "l" of the encoded sequence matrix, the corresponding adjacency marker can be assigned a value.

[0142] The adjacency identifier may include the NNW (Next-Neighboring-Word) identifier, and the specific identifier used is not limited here. The NNW is ultimately mapped to a binary vector. A value of 0 or 1 indicates an "adjacency" relationship, with 1 representing "adjacency" and 0 corresponding to "non-adjacency." When the adjacency identifier is 1, it means that the related characters must be connected in the slot recognition result. The assignment of the adjacency identifier is not directly related to whether the corresponding characters are connected in the speech request; that is, two characters with an "adjacency" relationship indicated by the adjacency identifier may not be continuous in the user's speech request.

[0143] Furthermore, adjacent characters can be concatenated based on their adjacent relationship identifiers. Specifically, the values ​​of the adjacent relationship identifiers, representing two characters that are "adjacent," are concatenated to obtain the slot identification result. In the aforementioned example, the slot identification result for the voice request "unlock the refrigerator" is found in "unlock the refrigerator," with the two adjacent character pairs being "ice-box" and "box-lock." These can be concatenated to obtain the target slot value "refrigerator lock," completing the slot identification process.

[0144] In this way, the adjacency relationship of each code in the target coded subsequence can be identified by the second submatrix, and the codes with adjacency relationships can be concatenated to obtain the target slot value, thus completing the recognition process of slots composed of discontinuous words, improving the accuracy of slot recognition and the fluency of the voice interaction process.

[0145] Please see Figure 13 In some implementations, step 05 includes:

[0146] 051: Determine the target parameters for slot filling based on the voice request, the slot recognition result, the predicted application programming interface (API), and the predicted API type;

[0147] 052: Based on the slot recognition results and target parameters, select the predicted application interface to perform application interface parameter filling, and output the execution results to the vehicle to complete the voice interaction.

[0148] In some implementations, the interface prediction module 105 is used to determine the target parameters for slot filling based on the voice request, the slot recognition result, the predicted application interface, and the predicted application interface type; and to select the predicted application interface to perform application interface parameter filling based on the slot recognition result and the target parameters, and output the execution result to the vehicle to complete the voice interaction.

[0149] In some implementations, the processor is used to determine the target parameters for slot filling based on the voice request, the slot recognition result, the predicted application interface, and the predicted application interface type, and to select the predicted application interface to perform application interface parameter filling based on the slot recognition result and the target parameters, and to output the execution result to the vehicle to complete the voice interaction.

[0150] Specifically, the target parameters for slot filling can be determined based on the user's voice request, the slot recognition result, and the predicted application programming interface (API) and interface type. The target parameter is the slot name corresponding to the slot recognition result. Finally, based on the slot recognition result and the target parameter, the predicted API can be selected, the target parameter filling can be executed, and the output execution result can be sent to the vehicle to complete the voice interaction.

[0151] For example, when a user makes a voice request to "unlock the refrigerator", the identified slot value includes "refrigerator lock". After filling the "refrigerator lock" value from the slot identification result into the corresponding application interface, the output execution result, which is the corresponding "unlock the refrigerator" control command, is sent to the vehicle. The vehicle can then execute the action of unlocking the refrigerator, thus completing the voice interaction process.

[0152] For example, when a user makes a voice request to "turn off the music and navigation pages", the identified slot values ​​include "music page" and "navigation page". After filling the "music page" and "navigation page" values ​​from the slot identification results into the corresponding application interface, the output execution results, namely the corresponding "turn off music page" control command and "turn off navigation page" control command, are sent to the vehicle. The vehicle can then perform the action of turning off the corresponding pages of the in-vehicle system, thus completing the voice interaction process.

[0153] Thus, the implementation method of this application can select the predicted application interface to perform application interface parameter filling based on the slot identification result and target parameters, and directly output the execution result to the vehicle to complete the voice interaction, which can reduce the latency of the vehicle system and improve the response speed to user commands.

[0154] The following example, using a complete scenario, provides supplementary explanations for recognizing slots composed of non-contiguous characters in a voice request. For example... Figure 14 The image shows the encoded sequence matrix of the voice request "Close Music and Navigation Page". In the first sub-matrix, the lower triangular matrix, the target encoded sub-sequence "Music and Navigation Page" corresponds to the target slot value of semantic vector (9,3). In the second sub-matrix, the upper triangular matrix, the adjacent relationship identifiers (3,4), (4,8), and (8,9) are all 1, indicating three sets of adjacent characters: "Music-Music", "Music-Page", and "Page-Face". Finally, the slot value is "Music Page", resulting in the final slot recognition result.

[0155] The computer-readable storage medium of this application embodiment stores a computer program that, when executed by one or more processors, implements the above-described method.

[0156] In the description of this specification, the terms "foreshadowing," "specifically," "particularly," "furthermore," etc., refer to specific features, structures, materials, or characteristics described in connection with embodiments or examples that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0157] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of executable request code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0158] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A voice interaction method, characterized in that, include: Receive voice requests forwarded by vehicles; The voice request is encoded according to a preset model to obtain an encoding sequence matrix for slot identification. The encoded sequence matrix is ​​divided along its diagonal to obtain a first sub-matrix and a second sub-matrix. Based on the first sub-matrix, determine the target coded sub-sequence for slot identification; Based on the target encoded subsequence and the second submatrix, slot identification is performed on the voice request to identify slots composed of discontinuous words in the voice request; The voice request is predicted using the application programming interface (API). Based on the slot identification results and the predicted application interface, the predicted application interface is selected to perform application interface parameter filling, and the execution result is output and sent to the vehicle to complete the voice interaction.

2. The voice interaction method according to claim 1, characterized in that, The step of encoding the voice request according to a preset model to obtain an encoding sequence matrix for slot identification includes: The voice request is processed by text sequence encoding to obtain a first encoding vector; The first encoding vector is input into the pre-trained model to obtain the output matrix corresponding to each encoding in the text sequence; The encoded sequence matrix is ​​obtained based on the output matrix and the preset model.

3. The voice interaction method according to claim 2, characterized in that, The step of obtaining the encoded sequence matrix based on the output matrix and the preset model includes: Extract the head matrix corresponding to the first code and the tail matrix corresponding to the last code from the output matrix; The head matrix and the tail matrix are encoded according to the preset model to obtain the encoded sequence matrix.

4. The voice interaction method according to claim 1, characterized in that, The step of determining the target encoded sub-sequence for slot identification based on the first sub-matrix includes: Identify all semantic vectors in the voice request; According to the preset slot value table, mark the head and tail positions of the target slot value in the first sub-matrix, wherein the target slot value is determined according to the correspondence between the semantic vector and the slot value table; The target encoded subsequence in which the target slot value is located is determined based on the head and tail position identifiers.

5. The voice interaction method according to claim 4, characterized in that, The step of performing slot identification on the voice request based on the target encoded sub-sequence and the second sub-matrix includes: In the second sub-matrix, adjacency relationships of each code in the target coded sub-sequence are marked according to the target slot value; The adjacent relationship identifier is used to concatenate the codes that have the adjacent relationship to obtain the target slot value, so as to obtain the result of slot recognition for the voice request.

6. The voice interaction method according to claim 1, characterized in that, The step of selecting the predicted application interface based on the slot recognition result and the predicted application interface, performing application interface parameter filling, and outputting the execution result to the vehicle to complete the voice interaction includes: The target parameters for slot filling are determined based on the voice request, the slot recognition result, the predicted application interface, and the predicted application interface type. Based on the slot identification result and the target parameter, the predicted application interface is selected to perform application interface parameter filling, and the execution result is output and sent to the vehicle to complete the voice interaction.

7. A voice interaction device, characterized in that, include: The receiving module is used to receive voice requests forwarded by the vehicle; The encoding module is used to encode the voice request according to a preset model to obtain an encoding sequence matrix for slot identification. The processing module is used to divide the encoding sequence matrix along the diagonal of the encoding sequence matrix to obtain a first sub-matrix and a second sub-matrix, and to determine a target encoding sub-sequence for slot identification based on the first sub-matrix. The slot identification module is used to identify slots in the voice request based on the target encoded sub-sequence and the second sub-matrix, so as to identify slots composed of discontinuous words in the voice request; The interface prediction module performs application interface prediction on the voice request; The parameter filling module is used to select the predicted application interface based on the slot identification result and the predicted application interface, perform application interface parameter filling, and output the execution result to the vehicle to complete the voice interaction.

8. A server, characterized in that, The server includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the voice interaction method according to any one of claims 1-6.

9. A non-volatile computer-readable storage medium containing a computer program, characterized in that, When the computer program is executed by one or more processors, it implements the voice interaction method according to any one of claims 1-6.

Images