Vehicle voice interaction method, server and storage medium

Abstract

This invention discloses a vehicle voice interaction method, server, and storage medium. The method includes: receiving user voice data sent by a vehicle to perform real-time voice recognition to obtain user voice requests; if the real-time recognized user voice request is an intermediate-state voice request, performing completion prediction on the intermediate-state voice request and adding request header information to obtain an intermediate-state predicted voice request; preprocessing the intermediate-state predicted voice request to obtain an intermediate-state preprocessing result; storing the intermediate-state preprocessing result according to the request header information; and if the real-time recognized user voice request is a final-state voice request and matches the intermediate-state predicted voice request, completing the voice interaction according to the corresponding intermediate-state preprocessing result. This invention distinguishes between intermediate-state predicted voice requests and final voice requests by adding request header information, and distinguishes the nth intermediate-state predicted voice request, achieving end-to-end service recognition of intermediate-state voice requests.

Description

Technical Field

[0001] This invention relates to the field of vehicle voice interaction technology, and in particular to a vehicle voice interaction method, server, and storage medium. Background Technology

[0002] In voice interaction systems, the algorithmic ability to predict the final voice request result in advance by taking advantage of the streaming frame-by-frame transmission characteristics of automatic speech recognition technology is relatively mature. For example, the algorithm can predict the intermediate state request "open" input by the user to obtain the intermediate state predicted voice request result "open the car window", and the intermediate state predicted voice request result "open the car window" can be used as the predicted final voice request result in advance.

[0003] In the automotive field, if the predictive capabilities of streaming automatic speech recognition algorithms are applied to engineering implementation, there will be some benefits in terms of voice interaction latency. However, this will also bring a series of problems to the engineering implementation, such as how to notify the end-to-end service whether the voice request being processed is an intermediate voice request, the nth intermediate voice request, or the final voice request. Summary of the Invention

[0004] This invention provides a vehicle voice interaction method, a server, and a storage medium.

[0005] This invention provides a vehicle voice interaction method. The method includes: receiving user voice data sent by a vehicle and performing real-time voice recognition to obtain a user voice request; if the real-time recognized user voice request is an intermediate-state voice request, performing completion prediction on the intermediate-state voice request and adding request header information to obtain an intermediate-state predicted voice request; preprocessing the intermediate-state predicted voice request to obtain an intermediate-state preprocessing result; storing the intermediate-state preprocessing result according to the request header information; and if the real-time recognized user voice request is a final-state voice request and matches the intermediate-state predicted voice request, completing the voice interaction according to the corresponding intermediate-state preprocessing result.

[0006] Thus, the vehicle voice interaction method of the present invention distinguishes between intermediate predicted voice requests and final voice requests by adding request header information, and can distinguish in detail which intermediate predicted voice request is the nth intermediate predicted voice request, ensuring that the algorithm based on streaming automatic speech recognition predicts the subsequent processing of intermediate voice requests, and realizing full-link service recognition of intermediate voice requests.

[0007] Before the step of preprocessing the intermediate-state predicted voice request to obtain the intermediate-state preprocessing result, the vehicle voice interaction method includes: intercepting the intermediate-state predicted voice request to obtain state information of the request header information and information identifiers indicating the order of the intermediate-state predicted voice requests; and adding the state information and the information identifiers to a thread variable.

[0008] Thus, this invention introduces an interceptor SDK to intercept intermediate-state predictive speech requests, obtain the status information of the request header and the information identifier indicating the order of intermediate-state predictive speech requests, and add the status information and information identifier to a thread variable so that in subsequent processing, the request header information of the thread variable can be read to determine whether the current request is an intermediate-state predictive speech request or a final-state speech request.

[0009] The step of storing the intermediate state preprocessing result according to the request header information includes: determining the order of the corresponding intermediate state predicted speech requests according to the information identifier; generating an initial intermediate state key-value pair according to the information identifier and the intermediate state preprocessing result; if the information identifier successfully matches the preset key value prefix, modifying the key value of the initial intermediate state key-value pair according to the order of the intermediate state predicted speech requests to obtain a target intermediate state key-value pair; and storing the target intermediate state key-value pair in a KV database.

[0010] Thus, the implementation method of the present invention configures a preset key value prefix for the key value that the business needs to modify, and performs automatic and imperceptible intermediate state modification processing on the key value according to the prefix matching method, and stores the target intermediate state key-value pair in the KV database.

[0011] The step of modifying the Key value of the intermediate state key-value pair according to the order of the intermediate state predicted speech requests when the information identifier successfully matches the preset Key value prefix includes: adding a suffix to the Key value of the initial intermediate state key-value pair according to the order of the intermediate state predicted speech requests to obtain the Key value of the target intermediate state key-value pair when the information identifier successfully matches the preset Key value prefix.

[0012] Thus, the vehicle voice interaction method of the present invention performs automatic and imperceptible modification of the intermediate state of the Key value to be modified according to the prefix matching method, and then adds a suffix to the Key value of the initial intermediate state key value pair according to the order of the intermediate state predicted voice requests to obtain the Key value of the target intermediate state key value pair, thereby completing the storage of intermediate state data in the KV database.

[0013] The step of storing the intermediate preprocessing result according to the request header information includes: if the information identifier fails to match the preset key value prefix, directly storing the initial intermediate key-value pair.

[0014] In this way, if the information identifier fails to match the preset key value prefix, the initial intermediate key-value pair can be stored in the KV database.

[0015] After the step of storing the target intermediate state key-value pair into the KV database, the vehicle voice interaction method includes: generating a modification record key-value pair based on the information identifier, the order of the intermediate state predicted voice requests, and the Key value of the initial intermediate state key-value pair; and storing the modification record key-value pair into the KV database.

[0016] Thus, by storing the modification records in the KV database, it is beneficial that when the vehicle voice interaction method of the present invention applies the data backtracking mechanism, it can trigger the backtracking of the intermediate state data stored in the current service for renaming or deletion operations.

[0017] The step of completing voice interaction based on the corresponding intermediate state preprocessing result when the user voice request identified in real time is a final state voice request and matches the intermediate state predicted voice request includes: when all the intermediate state predicted voice requests have been preprocessed and the final state voice request matches the intermediate state predicted voice request, publishing a Kafka message to update the KV database based on the matched intermediate state predicted voice request; and completing voice interaction based on the intermediate state preprocessing result corresponding to the updated KV database.

[0018] Thus, the vehicle voice interaction method of the present invention can publish Kafka messages based on the matched intermediate state predicted voice requests to update the KV database when all intermediate state predicted voice requests have been preprocessed and the final state voice request matches the intermediate state predicted voice request.

[0019] The step of updating the KV database by publishing a Kafka message based on the matched intermediate-state predicted speech requests, after all the intermediate-state predicted speech requests have been preprocessed and the final-state speech request matches the intermediate-state predicted speech request, includes: determining the corresponding modification record key-value pair based on the Kafka message, wherein the Kafka message includes matching status information, the information identifier of the matched predicted speech request, and the order of the matched predicted speech requests; modifying the Key value of the corresponding target intermediate-state key-value pair to the Key value of the initial intermediate-state key-value pair based on the determined modification record key-value pair; and deleting the target intermediate-state key-value pair that does not correspond to the order of the matched predicted speech requests based on the information identifier of the matched predicted speech request.

[0020] Thus, the vehicle voice interaction method of the present invention can realize the storage history of intermediate state data and delete or rename the corresponding data.

[0021] The step of completing voice interaction based on the corresponding intermediate state preprocessing result when the user voice request identified in real time is a final state voice request and matches the intermediate state predicted voice request includes: sending the corresponding intermediate state preprocessing result to the vehicle client to complete voice interaction when the user voice request identified in real time is a final state voice request and matches the intermediate state predicted voice request.

[0022] Thus, when the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the corresponding intermediate state preprocessing result is sent to the vehicle client to complete the voice interaction. This can generate the time delay benefit of the round-trip time for the central control to call downstream services, achieving a preemptive effect in voice interaction.

[0023] The step of performing completion prediction and adding request header information to the intermediate voice request to obtain an intermediate predicted voice request when the user voice request is identified as an intermediate voice request in real time includes: when the user voice request is identified as an intermediate voice request in real time, if the intermediate voice request has not requested speech completion, then performing completion prediction on the intermediate voice request and adding request header information to obtain an intermediate predicted voice request.

[0024] Thus, the vehicle voice interaction method of the present invention can, when the user voice request is identified in real time as an intermediate voice request, if the intermediate voice request has not requested speech completion, perform completion prediction on the intermediate voice request and add request header information to obtain the intermediate predicted voice request.

[0025] The step of preprocessing the intermediate-state predicted speech request to obtain an intermediate-state preprocessing result includes: when the intermediate-state predicted speech request obtained in real time is different from the intermediate-state predicted speech request that has been preprocessed, preprocessing the intermediate-state predicted speech request obtained in real time to obtain the corresponding intermediate-state preprocessing result.

[0026] In this way, the repeated preprocessing of the same intermediate state prediction speech request can be avoided to obtain the same intermediate state preprocessing result, thus improving the efficiency of voice interaction.

[0027] The step of preprocessing the intermediate state prediction request to obtain the intermediate state preprocessing result includes: in the case that the final state speech request is not recognized, preprocessing the intermediate state prediction speech request to obtain the intermediate state preprocessing result.

[0028] Thus, if the central control unit does not recognize the final state voice request, the intermediate state request is preprocessed to obtain the intermediate state preprocessing result, which can lay the foundation for creating latency benefits later.

[0029] The present invention also provides a server. The server includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the vehicle voice interaction method described in any of the above embodiments is implemented.

[0030] Thus, the server of the present invention uses the above-mentioned vehicle voice interaction method to distinguish between intermediate predicted voice requests and final voice requests by adding request header information, and can distinguish in detail which intermediate predicted voice request is the nth intermediate predicted voice request, ensuring that the algorithm based on streaming automatic speech recognition predicts the subsequent processing of intermediate voice requests, and realizing full-link service recognition of intermediate voice requests.

[0031] The present invention also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, it implements the vehicle voice interaction method described in any of the above embodiments.

[0032] Thus, the storage medium of the present invention, by applying the above-mentioned vehicle voice interaction method, distinguishes between intermediate predicted voice requests and final voice requests by adding request header information, and can distinguish in detail which intermediate predicted voice request is the nth intermediate predicted voice request, ensuring that the algorithm based on streaming automatic speech recognition predicts the subsequent processing of intermediate voice requests, and realizing full-link service recognition of intermediate voice requests.

[0033] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0034] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0035] Figure 1 This is one of the flowcharts illustrating the vehicle voice interaction method of the present invention;

[0036] Figure 2 This is a schematic diagram of the algorithm prediction architecture of the vehicle voice interaction method of the present invention based on streaming automatic speech recognition service;

[0037] Figure 3 This is the second flowchart of the vehicle voice interaction method of the present invention;

[0038] Figure 4 This is the third flowchart of the vehicle voice interaction method of the present invention;

[0039] Figure 5 This is the fourth flowchart of the vehicle voice interaction method of the present invention;

[0040] Figure 6 This is the fifth flowchart of the vehicle voice interaction method of the present invention;

[0041] Figure 7 This is the sixth flowchart of the vehicle voice interaction method of the present invention;

[0042] Figure 8 This is the seventh flowchart of the vehicle voice interaction method of the present invention;

[0043] Figure 9 This is the eighth flowchart of the vehicle voice interaction method of the present invention;

[0044] Figure 10 This is the ninth flowchart of the vehicle voice interaction method of the present invention;

[0045] Figure 11 This is one of the schematic diagrams of the algorithm prediction process of the server side of the vehicle voice interaction method of the present invention based on streaming automatic speech recognition service;

[0046] Figure 12 This is the second schematic diagram of the algorithm prediction process of the server side of the vehicle voice interaction method of the present invention based on streaming automatic speech recognition service;

[0047] Figure 13 This is the tenth flowchart of the vehicle voice interaction method of the present invention;

[0048] Figure 14This is the eleventh flowchart of the vehicle voice interaction method of the present invention;

[0049] Figure 15 This is the twelfth flowchart of the vehicle voice interaction method of the present invention. Detailed Implementation

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present invention, and should not be construed as limiting the embodiments of the present invention.

[0051] Please see Figure 1 This invention provides a vehicle voice interaction method. The vehicle voice interaction method includes:

[0052] 01: Receive user voice data sent by the vehicle to perform real-time voice recognition to obtain user voice requests;

[0053] 03: When the user voice request identified in real time is an intermediate voice request, the intermediate voice request is completed and predicted, and request header information is added to obtain the intermediate predicted voice request;

[0054] 05: Preprocess the intermediate state predicted speech request to obtain the intermediate state preprocessing result;

[0055] 07: Store the intermediate preprocessing results based on the request header information;

[0056] 09: If the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the voice interaction is completed based on the corresponding intermediate state preprocessing result.

[0057] The present invention also provides a server. The server includes a processor and a memory, the memory storing a computer program. The processor is configured to: receive user voice data sent by a vehicle to perform real-time speech recognition to obtain user voice requests; if the real-time recognized user voice request is an intermediate-state voice request, perform completion prediction on the intermediate-state voice request and add request header information to obtain an intermediate-state predicted voice request; preprocess the intermediate-state predicted voice request to obtain an intermediate-state preprocessing result; store the intermediate-state preprocessing result according to the request header information; and if the real-time recognized user voice request is a final-state voice request and matches the intermediate-state predicted voice request, complete the voice interaction according to the corresponding intermediate-state preprocessing result.

[0058] This application presents an algorithm for predicting the final voice request result in advance based on the characteristics of streaming frame-by-frame transmission using automatic speech recognition technology. The overall process is as follows: Figure 2 As shown.

[0059] Please see Figure 2 In the intermediate processing flow, the server first obtains the user's voice data sent in real time from the vehicle's central control screen in the form of streaming audio, and then performs automatic voice recognition in real time to obtain the user's voice request.

[0060] Then, the server's automatic speech recognition service can call the third-party service provider's interface (DUI) to return an intermediate speech request. At this point, the server can send the recognized user speech request to the streaming completion algorithm service in real time for prediction and completion. If the user speech request is an intermediate speech request, the intermediate speech request is completed and predicted, and then request header information is added to obtain the intermediate preprocessing result, i.e., the intermediate predicted speech request. For example, if the recognized intermediate speech request is "open the car", the intermediate predicted speech request is "open the car window".

[0061] Furthermore, the server's automatic speech recognition service sends the intermediate-state speech prediction request to the central control service, requesting the central control service to preprocess the intermediate-state speech prediction request. After receiving the intermediate-state speech prediction request, the central control service requests the downstream service, and then obtains and stores the preprocessing results of the downstream service.

[0062] For the final state processing flow, after the in-vehicle central control screen obtains all the audio of the user's voice data, it sends it to the server. The server's automatic speech recognition service can call the third-party service provider's interface (DUI) to return the final state voice request. Then, the server's automatic speech recognition service sends the final state voice request to the central control service. The central control service determines whether each intermediate state predicted voice request matches the final state voice request. If it matches, it reads the preprocessing result corresponding to the matched intermediate state voice request from the stored data and finally sends the result to the in-vehicle central control screen; if it does not match, it requests the downstream service to preprocess the final state voice request, waits for the final result to be returned, and then sends the result to the in-vehicle central control screen.

[0063] It is understandable that when the intermediate-state predicted voice request hits the final-state voice request, the pre-processed warning result is directly issued, without the need to preprocess the final-state voice request. This reduces the round-trip time for the central control to call downstream services, shortens the response time of the in-vehicle central control screen, achieves a preemptive effect in voice interaction, and improves the user experience.

[0064] In this application, the request header information is newly added to the request header. The request header information includes two fields: "isTempState" and "tempStateMsgId".

[0065] The request header information is used to identify which intermediate state prediction speech request result it is. As shown in Table 1 below, if the request header information contains the "isTempState" field, it indicates that the speech request is an intermediate state speech request. When the request header information contains the "isTempState" field, and the status information in the request header information is "isTempState=true", and the information identifier in the request header information is "tempStateMsgId=62f59c20cd32385-0", it indicates that it is the first intermediate state prediction speech request.

[0066] If the request header contains the "isTempState" field, it indicates that the voice request is an intermediate-state voice request. If the request header contains the "isTempState" field, and the status information in the request header is "isTempState=true", and the information identifier in the request header is "tempStateMsgId=62f59c20cd32385-1", it indicates that it is a second intermediate-state predicted voice request.

[0067] If these two fields are not present in the request header of a voice request, it indicates that this request is a final voice request.

[0068] Table 1

[0069] Request header combination explanation isTempState=true tempStateMsgId=62f59c20cd32385-0 First intermediate state prediction speech request tempStateMsgId=62f59c20cd32385-1 Second intermediate state prediction speech request

[0070] After receiving an intermediate-state predicted speech request, the request can be preprocessed to obtain an intermediate-state preprocessing result. The server can request a central control service, which in turn calls a downstream service to preprocess the intermediate-state predicted speech request to obtain the intermediate-state preprocessing result.

[0071] Then, the central control service can store the intermediate preprocessing results based on the request header information.

[0072] Finally, if the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the voice interaction is completed based on the corresponding intermediate state preprocessing result.

[0073] For example, if the real-time recognized user voice request is the final state voice request without request header information, which is "open the car window and the car door", and the intermediate state voice request is "open the car window and", and the intermediate state predicted voice request is also "open the car window and the car door", then the voice interaction can be completed based on the intermediate state preprocessing result corresponding to the intermediate state predicted voice request "open the car window and the car door".

[0074] Thus, the vehicle voice interaction method of the present invention distinguishes between intermediate predicted voice requests and final voice requests by adding request header information, and can distinguish in detail which intermediate predicted voice request is the nth intermediate predicted voice request, ensuring that the algorithm based on streaming automatic speech recognition predicts the subsequent processing of intermediate voice requests, and realizing full-link service recognition of intermediate voice requests.

[0075] Specifically, please refer to Figure 3 Before step 05, the vehicle voice interaction method includes:

[0076] 041: Intercept intermediate state predictive speech requests to obtain status information of the request header and information identifiers indicating the order of intermediate state predictive speech requests;

[0077] 042: Add status information and information identifier to the thread variable.

[0078] The processor intercepts intermediate-state predictive speech requests to obtain status information of the request header and information identifiers indicating the order of intermediate-state predictive speech requests; it then adds the status information and information identifiers to thread variables.

[0079] When the server receives an intermediate-state speech prediction request (HTTP request) from the speech completion client, the request is intercepted by the server's interceptor: a filter in the Software Development Kit (SDK). The server retrieves the request header's status information and an identifier indicating the order of the intermediate-state speech prediction requests, adding these to a thread variable (ThreadLocal) before executing subsequent business logic. In later processing, the request header information in the thread variable can be read to determine whether the request is an intermediate-state speech prediction request or a final-state speech request.

[0080] For example, if the request header of the first intermediate-state predictive speech request contains the status information "isTempState=true" and the information identifier indicating the order of the intermediate-state predictive speech requests is "tempStateMsgId=62f59c20cd32385-0", then both the status information and the information identifier can be added to the thread variable. The request header information for other orders of intermediate-state predictive speech requests can be added to the thread variable in the same way.

[0081] Thus, this invention introduces an SDK to intercept intermediate-state predictive speech requests, obtain the status information of the request header and the information identifier indicating the order of intermediate-state predictive speech requests, and add the status information and information identifier to a thread variable so that in subsequent processing, the request header information of the thread variable can be read to determine whether the current request is an intermediate-state predictive speech request or a final-state speech request.

[0082] Furthermore, before embedding intermediate state prediction capabilities into the end-to-end service, the end-to-end service involves multiple key-value database read / write operations for final state requests. After embedding intermediate state prediction capabilities, it is necessary to add and store relevant data for intermediate state predicted voice requests.

[0083] Considering the current problem of how to automatically and seamlessly modify key values ​​and store intermediate state predictive voice request data when storing KV (Key-Value) databases normally on servers, this invention continues to use an interceptor SDK to intercept intermediate state predictive voice requests, thereby seamlessly modifying key values ​​and storing intermediate state predictive voice request data.

[0084] Specifically, please refer to Figure 4 Step 07 includes:

[0085] 071: Determine the order of the corresponding intermediate state predicted speech requests based on the information identifier;

[0086] 072: Generate initial intermediate state key-value pairs based on the information identifier and the intermediate state preprocessing results;

[0087] 073: If the information identifier successfully matches the preset key value prefix, modify the key value of the initial intermediate state key-value pair according to the order of the intermediate state predicted voice requests to obtain the target intermediate state key-value pair;

[0088] 074: Store the target intermediate key-value pair into the KV database.

[0089] The processor is used to determine the order of the corresponding intermediate state predicted speech requests based on the information identifier; generate an initial intermediate state key-value pair based on the information identifier and the intermediate state preprocessing result; if the information identifier successfully matches the preset key value prefix, modify the key value of the initial intermediate state key-value pair according to the order of the intermediate state predicted speech requests to obtain the target intermediate state key-value pair; and store the target intermediate state key-value pair in the KV database.

[0090] First, the order of the corresponding intermediate state predicted speech requests is determined based on the information identifiers.

[0091] Then, initial intermediate key-value pairs are generated based on the information identifier and the intermediate preprocessing result. For example, the server can configure a preset key prefix of "abc" for each service. If, after the intermediate preprocessing returns the preprocessing result, it is necessary to store the information identifier of the voice request as "abcd" and the data of the preprocessing result as "test", then initial intermediate key-value pairs can be generated based on the information identifier and the intermediate preprocessing result.<K:abcd,V:test> This facilitates storage in a key-value database.

[0092] Next, for the generated initial intermediate key-value pair, if the key value "abcd" successfully matches the preset key value prefix "abc", the key value of the initial intermediate key-value pair is modified according to the order of the intermediate predicted speech requests to obtain the target intermediate key-value pair.

[0093] For example, if the order of intermediate state prediction speech requests is the first intermediate state prediction speech request, the Key value of the initial intermediate state key-value pair can be modified to obtain the target intermediate state key-value pair for the first intermediate state prediction speech request as "".<K:abcd-0,V:test> "If the order of intermediate state prediction speech requests is the second intermediate state prediction speech request, the Key value of the initial intermediate state key-value pair can be modified to obtain the target intermediate state key-value pair for the second intermediate state prediction speech request as "<K:abcd-1,V:test> ".

[0094] Finally, the target intermediate key-value pairs are stored in the KV database.

[0095] Thus, the implementation method of the present invention configures a preset key value prefix for the key value that the business needs to modify, and performs automatic and imperceptible intermediate state modification processing on the key value according to the prefix matching method, and stores the target intermediate state key-value pair in the KV database.

[0096] Please see Figure 5 Step 073 includes:

[0097] 0731: If the information identifier successfully matches the preset key value prefix, add a suffix to the key value of the initial intermediate key value pair according to the order of the intermediate state predicted voice requests to obtain the key value of the target intermediate key value pair.

[0098] The processor is used to add a suffix to the key value of the initial intermediate key-value pair to obtain the key value of the target intermediate key-value pair when the information identifier successfully matches the preset key value prefix.

[0099] For example, the initial intermediate key-value pairs are<K:abcd,V:test> When the order of intermediate state prediction speech requests is the first intermediate state prediction speech request, a suffix "-0" can be added to the key value of the initial intermediate state key-value pair to obtain the target intermediate state key-value pair for the first intermediate state prediction speech request.<K:abcd-0,V:test> "If the order of intermediate state prediction speech requests is the second intermediate state prediction speech request, a suffix "-1" can be added to the Key value of the initial intermediate state key-value pair to obtain the target intermediate state key-value pair for the second intermediate state prediction speech request."<K:abcd-1,V:test> ".

[0100] Thus, the vehicle voice interaction method of the present invention performs automatic and imperceptible modification of the intermediate state of the Key value to be modified according to the prefix matching method, and then adds a suffix to the Key value of the initial intermediate state key value pair according to the order of the intermediate state predicted voice requests to obtain the Key value of the target intermediate state key value pair, thereby completing the storage of intermediate state data in the KV database.

[0101] Please see Figure 6 Step 07 includes:

[0102] 075: If the information identifier fails to match the preset key value prefix, directly store the initial intermediate key-value pair.

[0103] The processor is used to directly store the initial intermediate key-value pair when the information identifier fails to match the preset key value prefix.

[0104] In addition, when the target intermediate key-value pair needs to be stored in the final state voice request, it will be intercepted by the SDK and it will be recognized that the information identifier of the final state voice request fails to match the preset key value prefix abc. At this time, the key value of the initial intermediate key-value pair will not be modified for storage, that is, the initial intermediate key-value pair will be stored directly.

[0105] In this way, if the information identifier fails to match the preset key value prefix, the initial intermediate key-value pair can be stored in the KV database.

[0106] In addition, considering that processing intermediate state predicted speech requests in advance will generate dirty data in the KV database that does not match the final state speech request, it is necessary to backtrack and delete the dirty data after receiving the final state speech request.

[0107] In this invention, the intermediate state preprocessing result of the corresponding request for the hit intermediate state predicted speech request is renamed as the final state preprocessing result, and the relevant data of the intermediate state preprocessing result of the corresponding request for the miss intermediate state predicted speech request is deleted, thereby storing the intermediate state preprocessing result according to the request header information.

[0108] Please see Figure 7 After step 074, the vehicle voice interaction method includes:

[0109] 076: Generate modification record key-value pairs based on the information identifier, the order of intermediate state predicted voice requests, and the key value of the initial intermediate state key-value pair;

[0110] 077: Store the modified record key-value pairs in the KV database.

[0111] The processor generates modified record key-value pairs based on the information identifier, the order of intermediate predicted voice requests, and the key value of the initial intermediate key-value pair; and stores the modified record key-value pairs in the KV database.

[0112] After modifying the intermediate state key, the modified record needs to be stored in the KV database. This record is stored in the KV database as a key-value pair. The structure of this modified record key-value pair is a hash structure, specifically including: the first-level key value is the information identifier of the intermediate state predicted speech request corresponding to the current data, the second-level hash key value is the order of the intermediate state predicted speech request, and the value is the key value in the initial intermediate state key-value pair before the modification.

[0113] Thus, by storing the modification records in the KV database, it is beneficial for the vehicle voice interaction method of the present invention to trigger the renaming or deletion operation of the intermediate state data stored in the current service when applying the data backtracking mechanism.

[0114] Furthermore, upon receiving a final-state speech request, since many intermediate-state data are generated during the processing of intermediate-state predicted speech requests, it is necessary to rename the intermediate-state data corresponding to the intermediate-state predicted speech requests that hit the final-state speech request to final-state data, and delete the intermediate-state data corresponding to the intermediate-state predicted speech requests that do not hit the final-state speech request in order to update the KV database. This enables backtracking and deletion of dirty data generated in the KV database during the processing of intermediate-state predicted speech requests after receiving a final-state speech request.

[0115] Specifically, please refer to Figure 8 Step 09 includes:

[0116] 091: After all intermediate state predicted speech requests have been preprocessed and the final state speech request matches the intermediate state predicted speech request, publish a Kafka message based on the matched intermediate state predicted speech request to update the KV database.

[0117] 092: Complete the voice interaction based on the intermediate state preprocessing results corresponding to the updated KV database.

[0118] The processor is used to publish Kafka messages to update the KV database based on the matched intermediate state predicted speech requests, provided that all intermediate state predicted speech requests have been preprocessed and the final state speech request matches the intermediate state predicted speech request; and to complete the voice interaction based on the intermediate state preprocessing results corresponding to the updated KV database.

[0119] When the central control service in the server receives the final state voice request, it can determine whether multiple intermediate state predicted voice requests match the final state voice request, thus determining whether voice interaction can be completed based on the preprocessing results of the intermediate state predicted voice requests.

[0120] If the final state voice request matches the intermediate state predicted voice request, it is determined whether all intermediate state predicted voice requests have been preprocessed. If all intermediate state predicted voice requests have been preprocessed, the server will publish a Kafka message. Upon receiving the Kafka message, the SDK performs post-processing of the data, thereby updating the KV database to rename or delete data within the KV database. Then, the voice interaction is completed based on the intermediate state preprocessing results corresponding to the updated KV database. Specifically, if the central control service determines that all intermediate state preprocessing results have been returned, it determines whether all intermediate state predicted voice requests have been preprocessed; otherwise, all intermediate state predicted voice requests have not been preprocessed.

[0121] It should be noted that if the final state voice request matches the intermediate state predicted voice request, but the intermediate state predicted voice request has not been fully preprocessed, the central control service can wait for all intermediate state preprocessing results to be returned before publishing the Kafka message.

[0122] Thus, the vehicle voice interaction method of the present invention can publish Kafka messages based on the matched intermediate state predicted voice requests to update the KV database when all intermediate state predicted voice requests have been preprocessed and the final state voice request matches the intermediate state predicted voice request.

[0123] Please see Figure 9 Step 091 includes:

[0124] 0911: Determine the corresponding modification record key-value pair based on the Kafka message. The Kafka message includes matching status information, the information identifier of the matched predicted speech request, and the order of the matched predicted speech requests.

[0125] 0912: Based on the determined modification record key-value pairs, modify the key value of the corresponding target intermediate state key-value pair to the key value of the initial intermediate state key-value pair;

[0126] 0913: Delete target intermediate key-value pairs that do not correspond to the order of the matched predicted speech requests, based on the information identifier of the matched predicted speech requests.

[0127] The processor determines the corresponding modified record key-value pair based on the Kafka message, which includes matching status information, the information identifier of the matched predicted speech request, and the order of the matched predicted speech requests. Based on the determined modified record key-value pair, the processor modifies the key value of the corresponding target intermediate key-value pair to the key value of the initial intermediate key-value pair. Based on the information identifier of the matched predicted speech request, the processor deletes the target intermediate key-value pair that does not correspond to the order of the matched predicted speech requests.

[0128] When the central control system in the service determines that the data rollback is triggered, it will publish a Kafka message. The corresponding modification record key-value pair is determined based on the Kafka message. The Kafka message includes matching status information, the information identifier of the matched predicted speech request, and the order of the matched predicted speech requests. That is, the Kafka message body includes three fields: isHit, msgId, and hitNumber.

[0129] The modification record key-value pairs record the key value of the initial intermediate key-value pair before the corresponding target intermediate key-value pair is modified. Based on the determined modification record key-value pairs, the key value of the corresponding target intermediate key-value pair is modified to the key value of the initial intermediate key-value pair, thus changing the target intermediate key-value pair to its corresponding initial intermediate key-value pair. Then, based on the matched predicted speech request information identifier (msgId), target intermediate key-value pairs that do not correspond to the order (hitNumber) of the matched predicted speech requests are deleted. Understandably, the entire service chain in the server can trace back the storage history of intermediate data and delete or rename the corresponding data.

[0130] Thus, the vehicle voice interaction method of the present invention can realize the storage history of intermediate state data and delete or rename the corresponding data.

[0131] Please see Figure 10 Step 09 includes:

[0132] 093: If the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the corresponding intermediate state preprocessing result is sent to the vehicle client to complete the voice interaction.

[0133] The processor is used to send the corresponding intermediate state preprocessing result to the vehicle client to complete the voice interaction when the user voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request.

[0134] When the user's voice request is identified as the final state voice request in real time, and the intermediate state predicted voice request hits the final state voice request, the corresponding intermediate state preprocessing result can be directly sent to the vehicle client to complete the voice interaction, thereby achieving the effect of getting ahead of the voice interaction.

[0135] In other words, when the central control unit receives the final state voice request, and the intermediate state preprocessing result of the intermediate state predicted voice request preprocessing that matches the final state voice request is also returned, a latency benefit can be generated. The latency benefit is as follows: Figure 11 The diagram shows the round-trip time for the central control unit to call downstream services.

[0136] Thus, when the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the corresponding intermediate state preprocessing result is sent to the vehicle client to complete the voice interaction. This can generate the time delay benefit of the round-trip time for the central control to call downstream services, achieving a preemptive effect in voice interaction.

[0137] Furthermore, when the central control unit receives the final state voice request, the intermediate state predicted voice request matching that final state voice request has not yet returned. The latency benefit at this point is the time it takes for the central control unit to receive the final state voice request minus the time it takes for the central control unit to receive the intermediate state predicted voice request matching that final state voice request. Figure 12 As shown.

[0138] Preprocessing intermediate state predicted speech requests in advance will increase traffic. This application controls traffic on the server side based on rules through traffic control logic.

[0139] Specifically, in step 03, after receiving the user's voice data from the vehicle's central control screen, the service performs automatic voice recognition to obtain the user's voice request. It can determine whether the user's voice request is the final state user voice request. If the user's voice request is the final state voice request, it requests the central control service to perform preprocessing and sends the result.

[0140] If the user's voice request is not the final state user voice request, then it is determined whether the completion prediction switch is turned on. If the completion prediction switch is not turned on, it means that the user has not enabled completion prediction, and there is no need to perform completion prediction on the user's voice request. The subsequent processing can be carried out after the final state voice request is received.

[0141] If the completion prediction switch is turned on, it is determined whether the vehicle meets the vehicle model requirements. If the vehicle does not meet the vehicle model requirements, it means that the current vehicle does not support the algorithm for predicting the final voice request result in advance based on the characteristics of streaming frame-by-frame transmission of automatic speech recognition technology in this application, and no processing is required.

[0142] If the vehicle meets the vehicle type requirements, it is determined whether the voice state is active or listening. If the voice state is not active or listening, no further processing is required.

[0143] If the voice status is active or listening, then determine whether the user's voice request is in the starting voice zone or the voice zone of the previous valid voice request. If the user's voice request is not in the starting voice zone or the voice zone of the previous valid voice request, then no further processing is required.

[0144] If the user's voice request is in the starting voice region or the voice region of the previous valid voice request, then it is necessary to determine whether the current intermediate voice request has requested speech completion. If the current intermediate voice request has already requested speech completion, then it is not necessary to repeat the prediction for the current intermediate voice request.

[0145] If the current intermediate speech request has not requested speech completion, then the intermediate speech request can be sent to the speech completion prediction service for speech completion. In other words, please refer to [link to relevant documentation]. Figure 13 Step 03 specifically includes:

[0146] 031: If the user voice request identified in real time is an intermediate voice request, and if the intermediate voice request has not requested speech completion, then the intermediate voice request is completed and predicted, and request header information is added to obtain the intermediate predicted voice request.

[0147] The processor is used to perform completion prediction on the intermediate voice request and add request header information to obtain the intermediate predicted voice request if the user voice request identified in real time is an intermediate voice request and the intermediate voice request has not requested speech completion.

[0148] For example, if the user's voice request identified in real time is "open the car" as an intermediate voice request, and if this intermediate voice request has not requested utterance completion, then utterance completion can be requested. The intermediate voice request can be completed and predicted, and request header information can be added to obtain the intermediate predicted voice request, such as "open the car window".

[0149] Thus, the vehicle voice interaction method of the present invention can, when the user voice request is identified in real time as an intermediate voice request, if the intermediate voice request has not requested speech completion, perform completion prediction on the intermediate voice request and add request header information to obtain the intermediate predicted voice request.

[0150] For the intermediate predicted speech request obtained after the prediction is completed, in step 05, it is determined whether the server's automatic speech recognition service has already sent an intermediate predicted speech request that is the same as the intermediate predicted speech request obtained in real time to the central control service for preprocessing. That is, it is determined whether the intermediate predicted speech request obtained in real time is the same as the intermediate predicted speech request that has been preprocessed. If the intermediate predicted speech request obtained in real time is the same as the intermediate predicted speech request that has been preprocessed, then it is not necessary to send the current intermediate predicted speech request obtained in real time to the central control service for preprocessing.

[0151] If the intermediate predicted speech request obtained in real-time differs from the pre-processed intermediate predicted speech request, the real-time predicted intermediate predicted speech request is sent to the central control service for pre-processing. In other words, please refer to [link to relevant documentation]. Figure 14 Step 05 includes:

[0152] 051: When the intermediate predicted speech request obtained in real time is different from the intermediate predicted speech request that has been preprocessed, the intermediate predicted speech request obtained in real time is preprocessed to obtain the corresponding intermediate preprocessing result.

[0153] The processor is used to preprocess the intermediate prediction request obtained in real time to obtain an intermediate preprocessing result when the intermediate prediction speech request obtained in real time is different from the intermediate prediction speech request that has been preprocessed.

[0154] If the intermediate state voice request "open" and the intermediate state voice request "open car" both result in the intermediate state predicted voice request "open car door", then if the two predicted intermediate state predicted voice requests "open car door" are sent to the central control for preprocessing, then preprocessing needs to be repeated twice, but the intermediate state preprocessing results are the same, which will affect the efficiency of voice interaction.

[0155] Therefore, when it is determined that the intermediate predicted speech request obtained in real time is different from the intermediate predicted speech request that has been preprocessed, preprocessing the intermediate predicted speech request obtained in real time to obtain the corresponding intermediate preprocessing result can effectively avoid repeatedly preprocessing the same intermediate predicted speech request to obtain the same intermediate preprocessing result, thereby improving the efficiency of voice interaction.

[0156] In addition, before sending the intermediate-state predicted speech request obtained in real time to the central control service for preprocessing, it can be determined whether the server's automatic speech recognition service has recognized the final-state speech request. If the final-state speech request has been recognized, the final-state speech request has been sent to the central control service or can be sent to the central control service, so that preprocessing can be performed according to the final-state speech request, without having to send the intermediate-state predicted speech request to the central control service again.

[0157] If the server's automatic speech recognition service fails to recognize the final state speech request, it sends the predicted intermediate state prediction request to the central control service for preprocessing, thereby obtaining the intermediate state preprocessing result. In other words, please refer to... Figure 15 Step 05 includes:

[0158] 052: In the absence of recognition of the final state speech request, the intermediate state predicted speech request is preprocessed to obtain the intermediate state preprocessing result.

[0159] The processor is used to preprocess intermediate state predicted speech requests to obtain intermediate state preprocessing results when the final state speech request is not recognized.

[0160] like Figure 11 As shown, when the central control unit fails to recognize the final state voice request, the intermediate state request is preprocessed to obtain the intermediate state preprocessing result, which can lay the foundation for creating latency benefits in the future.

[0161] When the central control unit recognizes the final-state voice request, it can search among the multiple intermediate-state preprocessing results obtained in advance to find one that matches the final-state voice request, thereby generating a latency benefit. The latency benefit is as follows: Figure 11 The diagram shows the round-trip time for the central control unit to call downstream services.

[0162] Thus, if the central control unit does not recognize the final state voice request, the intermediate state request is preprocessed to obtain the intermediate state preprocessing result, which can lay the foundation for creating latency benefits later.

[0163] This application uses traffic control logic to control traffic on the server side based on rules, avoiding resource consumption caused by processing voice requests that do not meet the conditions. At the same time, it reduces the repetitive processing of the same content in each processing stage, reduces the traffic of processing intermediate state prediction voice requests, and improves processing efficiency.

[0164] The present invention also provides a non-volatile computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, it implements the interaction method described in any of the above embodiments.

[0165] For example, when a computer program is executed by a processor, it implements the following steps for interactive methods:

[0166] 01: Receive user voice data sent by the vehicle to perform real-time voice recognition to obtain user voice requests;

[0167] 03: When the user voice request identified in real time is an intermediate voice request, the intermediate voice request is completed and predicted, and request header information is added to obtain the intermediate predicted voice request;

[0168] 05: Preprocess the intermediate state predicted speech request to obtain the intermediate state preprocessing result;

[0169] 07: Store the intermediate preprocessing results based on the request header information;

[0170] 09: If the user's voice request identified in real time is the final state voice request and matches the intermediate state predicted voice request, the voice interaction is completed based on the corresponding intermediate state preprocessing result.

[0171] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0172] The computer-readable storage medium of the present invention applies the above-described vehicle voice interaction method by adding request header information to distinguish between intermediate predicted voice requests and final voice requests, and can distinguish in detail which intermediate predicted voice request is the nth intermediate predicted voice request, thereby improving the accuracy of algorithm prediction based on streaming automatic speech recognition.

Claims

1. A vehicle voice interaction method, characterized in that, include: Receive user voice data sent by the vehicle to perform real-time voice recognition to obtain user voice requests; If the user's voice request is identified as an intermediate voice request in real time, the intermediate voice request is completed and predicted, and a request header information is added to the request header of the intermediate voice request to obtain an intermediate predicted voice request. The request header information is not added to the request header of the final voice request. Intercept the intermediate-state predicted speech request to obtain the status information of the request header information and the information identifier indicating the order of the intermediate-state predicted speech requests; add the status information and the information identifier to a thread variable; The intermediate state predicted speech request is preprocessed to obtain the intermediate state preprocessing result; The intermediate preprocessing result is stored according to the request header information; The step of storing the intermediate preprocessing result according to the request header information includes: The order of the corresponding intermediate state predicted speech requests is determined based on the information identifier; An initial intermediate state key-value pair is generated based on the information identifier and the intermediate state preprocessing result; If the information identifier successfully matches the preset key value prefix, the key value of the initial intermediate key value pair is modified according to the order of the intermediate predicted voice requests to obtain the target intermediate key value pair; Store the target intermediate key-value pairs into a KV database; A modification record key-value pair is generated based on the information identifier, the order of the intermediate state predicted speech requests, and the Key value of the initial intermediate state key-value pair; The modified record key-value pairs are stored in the KV database; If the user's voice request is identified in real time as the final state voice request and matches the intermediate state predicted voice request, the voice interaction is completed according to the corresponding intermediate state preprocessing result. When the user's voice request identified in real time is a final-state voice request and matches the intermediate-state predicted voice request, the voice interaction is completed based on the corresponding intermediate-state preprocessing result, including: After all the intermediate state predicted speech requests have been preprocessed and the final state speech request matches the intermediate state predicted speech request, a Kafka message is published based on the matched intermediate state predicted speech request to update the KV database. Voice interaction is completed based on the intermediate preprocessing results corresponding to the updated KV database.

2. The vehicle voice interaction method according to claim 1, characterized in that, When the information identifier successfully matches the preset key value prefix, modifying the key value of the intermediate state key-value pair according to the order of the intermediate state predicted speech requests includes: If the information identifier successfully matches the preset key value prefix, the key value of the target intermediate key value pair is obtained by adding a suffix to the key value corresponding to the initial intermediate key value according to the order of the intermediate predicted speech requests.

3. The vehicle voice interaction method according to claim 1, characterized in that, The step of storing the intermediate preprocessing result according to the request header information includes: If the information identifier fails to match the preset key value prefix, the initial intermediate key-value pair is directly stored.

4. The vehicle voice interaction method according to claim 1, characterized in that, The step of publishing a Kafka message to update the KV database based on the matched intermediate-state predicted speech request after all intermediate-state predicted speech requests have been preprocessed and the final-state speech request matches the intermediate-state predicted speech request includes: The corresponding modification record key-value pair is determined based on the Kafka message, which includes matching status information, the information identifier of the matched predicted speech request, and the order of the matched predicted speech requests. Based on the determined modified record key-value pair, the key value of the corresponding target intermediate state key-value pair is modified to the key value of the initial intermediate state key-value pair; Based on the information identifier of the matched predicted speech request, delete the target intermediate key-value pairs that do not correspond to the order of the matched predicted speech requests.

5. The vehicle voice interaction method according to claim 1, characterized in that, When the user's voice request identified in real time is a final-state voice request and matches the intermediate-state predicted voice request, the voice interaction is completed based on the corresponding intermediate-state preprocessing result, including: If the user's voice request is identified in real time as a final state voice request and matches the intermediate state predicted voice request, the corresponding intermediate state preprocessing result is sent to the vehicle client to complete the voice interaction.

6. The vehicle voice interaction method according to claim 1, characterized in that, When the user's voice request is identified as an intermediate voice request in real time, the intermediate voice request is completed and predicted, and request header information is added to obtain an intermediate predicted voice request, including: If the user's voice request is identified as an intermediate voice request in real time, and if the intermediate voice request has not requested speech completion, then the intermediate voice request is subjected to completion prediction and request header information is added to obtain an intermediate predicted voice request.

7. The vehicle voice interaction method according to claim 1, characterized in that, The process of preprocessing the intermediate-state predicted speech request to obtain the intermediate-state preprocessing result includes: If the intermediate predicted speech request obtained in real time is different from the intermediate predicted speech request that has been preprocessed, the intermediate predicted speech request obtained in real time is preprocessed to obtain the corresponding intermediate preprocessing result.

8. The vehicle voice interaction method according to claim 1, characterized in that, The process of preprocessing the intermediate-state predicted speech request to obtain the intermediate-state preprocessing result includes: If the final state speech request is not identified, the intermediate state predicted speech request is preprocessed to obtain the intermediate state preprocessing result.

9. A server, characterized in that, The server includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the vehicle voice interaction method according to any one of claims 1-8.

10. A non-volatile computer-readable storage medium containing a computer program, characterized in that, When the computer program is executed by one or more processors, it implements the vehicle voice interaction method according to any one of claims 1-8.

Images