Voice interaction method, server and computer-readable storage medium

Abstract

The present application discloses a voice interaction method, a server and a computer-readable storage medium, the method comprising: receiving a current voice request forwarded by a vehicle, determining a candidate voice request according to the current voice request and a voice request database, and determining target guidance information according to a large language model, the current voice request and the candidate voice request when the vehicle control instruction corresponding to the current voice request cannot be determined, and feeding back the target guidance information to the user to guide the user to complete the voice interaction. In this way, the server of the present application can determine the target guidance information that is used to guide the user to adjust the voice request and is semantically related to the voice request according to the voice request determination, the candidate voice request and the large language model when the vehicle control instruction corresponding to the voice request cannot be determined, so that the user can adjust the current voice request based on the target guidance information to complete the voice interaction, and the user's experience is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, a server, and a computer-readable storage medium. Background Art

[0002] When the user first starts using the vehicle, he or she may express some vague and incomplete voice commands because he or she is unfamiliar with the vehicle and its voice interaction functions. When the vehicle processes these voice commands, it has difficulty understanding the actual meaning of the voice commands and is unable to execute the corresponding functions, resulting in voice interaction failure. Summary of the invention

[0003] The present application provides a voice interaction method, a server and a computer-readable storage medium.

[0004] The present application provides a voice interaction method, including:

[0005] Receive the current voice request forwarded by the vehicle;

[0006] Determining candidate voice requests based on the current voice request and a pre-built voice request database;

[0007] In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding a user to adjust the current voice request is determined according to the large language model, the current voice request and the candidate voice request, wherein the target guidance information is semantically related to the current voice request, the large language model is pre-trained and can determine guidance information prediction according to the voice request sample;

[0008] The target guidance information is fed back to guide the user to complete the voice interaction.

[0009] In the voice interaction method provided in the embodiment of the present application, the server can receive the current voice request forwarded by the vehicle, and determine the candidate voice request based on the current voice request and a pre-built voice request database, and when the vehicle control instruction corresponding to the current voice request cannot be determined, the server can determine the target guidance information that is used to guide the user to adjust the current voice request and is semantically related to the current voice request based on the pre-trained large language model, the current voice request and the candidate voice request, and feed back the target guidance information to the user to guide the user to complete the voice interaction.

[0010] Thus, in the implementation mode of the present application, the server can determine the target guidance information for guiding the user to adjust the current voice request when the vehicle control instruction corresponding to the current voice request cannot be determined, and feed back the target guidance information to the user, so that the user can adjust the current voice request to complete the voice interaction through the guidance of the target guidance information when the user is unclear about how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of the vehicle and the vehicle voice interaction function. The implementation mode of the present application can determine the candidate voice request according to the current voice request and the pre-built voice request database, and determine the target guidance information semantically related to the current voice request according to the large language model, the candidate voice request and the current voice request, so that the determination of the target guidance information can refer to the candidate voice request and can be semantically related to the current voice request, thereby enabling the target guidance information to match the user's current usage needs, and the credibility of the target guidance information can be guaranteed.

[0011] In certain embodiments of the present application, determining a candidate voice request based on the current voice request and a pre-built voice request database includes:

[0012] According to the current voice request and the voice request database, the candidate voice request in the voice request database that is semantically related to the current voice request is determined.

[0013] In this way, in an embodiment of the present application, the server can determine the candidate voice requests that are semantically related to the current voice request in the voice request data, so that when the target guidance information is subsequently determined, the target guidance information related to the current voice request semantics can be determined through the candidate voice requests that are semantically related to the current voice request, thereby ensuring the credibility of the target guidance information to a certain extent.

[0014] In certain embodiments of the present application, the voice request database includes multiple voice requests and an index of each voice request, the index is determined by encoding the voice request, and the determining of the candidate voice request based on the current voice request and the pre-built voice request database includes:

[0015] Performing the encoding process on the current voice request to determine an encoding result of the current voice request;

[0016] Determining, according to the encoding result and the voice request database, a target index in the voice request database that is semantically related to the encoding result;

[0017] The candidate voice request is determined according to the target index.

[0018] Thus, in the implementation manner of the present application, the server can determine the target index semantically related to the encoding result through the encoding result of the current voice request and the index of the voice request in the voice request database, and determine the candidate voice request semantically related to the current voice request through the target index, thereby reducing the computing resources required to be called by the server to determine the candidate voice request to a certain extent, and improving the efficiency of the server in determining the candidate voice request to a certain extent.

[0019] In certain embodiments of the present application, determining, according to the encoding result and the voice request database, a target index in the voice request database that is semantically related to the encoding result includes:

[0020] Calculating the similarity between the encoding result and the index in the voice request database;

[0021] The target index is determined according to the similarity.

[0022] Thus, in the implementation mode of the present application, the server can determine the target index in the voice request database based on calculating the encoding result of the current voice request and the similarity between the index of the voice request in the voice request database, and based on the similarity between the encoding result and the index, thereby ensuring the credibility of the target index to a certain extent.

[0023] In certain embodiments of the present application, determining the target index according to the similarity includes:

[0024] A preset number of target indexes in the voice request database are determined according to the similarity.

[0025] Thus, in the implementation mode of the present application, the server can determine a preset number of target indexes in the voice request database based on the similarity between the encoding result of the current voice request and the index of the voice request in the voice request database, and then perform subsequent guidance information inference work through the preset number of voice requests pointed to by the preset number of target indexes, thereby ensuring the credibility of the target guidance information to a certain extent.

[0026] In certain embodiments of the present application, when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request includes:

[0027] In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, the target guidance information including the target voice request is determined according to the large language model, the current voice request and the candidate voice request, and the current voice request is semantically related to the target voice request.

[0028] In this way, in an embodiment of the present application, the server can determine the target guidance information including the target voice request based on the large language model, and then after feeding back the target guidance information to the user, the user can adjust the current voice request based on the target voice request in the target guidance information that is semantically related to the current voice request to complete the voice interaction, thereby ensuring the user's experience of the vehicle's voice interaction function and the vehicle.

[0029] In certain embodiments of the present application, when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request includes:

[0030] When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request and the large language model, the target guidance information is determined based on the large language model, the current voice request and the candidate voice requests.

[0031] In this way, in the implementation mode of the present application, the server can confirm the vehicle control instructions of the current voice request through the large language model, and when the vehicle control instructions of the current voice request cannot be predicted by the large language model, the target guidance information is determined based on the large language model, the current voice request and the candidate voice request. Therefore, both the vehicle control instruction prediction and the target guidance information prediction can be achieved based on the large language model, avoiding the need to set up a separate module for predicting vehicle control instruction prediction and a separate module for predicting target guidance information in the server, thereby reducing the load on the server.

[0032] In certain embodiments of the present application, when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request and the large language model, determining the target guidance information according to the large language model, the current voice request and the candidate voice request includes:

[0033] When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and a first sub-prompt information template of a pre-configured prompt information template, the target guidance information is determined based on the large language model, the current voice request, the candidate voice requests and a second sub-prompt information template of the prompt information template.

[0034] Thus, in the implementation mode of the present application, the large language model can confirm the correspondence between the current voice request and the vehicle control instruction based on the first sub-prompt information template and the second sub-prompt information template in the prompt information template, and when it is confirmed that the current voice request fails to correspond to the vehicle control instruction, confirm the target guidance information corresponding to the current voice request, thereby realizing reasoning of the large language model based on the prompt information template, and thus the credibility of the model reasoning result can be guaranteed.

[0035] An embodiment of the present application provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0036] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0037] The server and computer-readable storage medium provided by the embodiments of the present application can determine the target guidance information for guiding the user to adjust the current voice request when the vehicle control instruction corresponding to the current voice request cannot be determined, and the target guidance information is fed back to the user, so that the user can adjust the current voice request to complete the voice interaction through the guidance of the target guidance information when it is unclear how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of the vehicle and the vehicle voice interaction function. The embodiments of the present application can determine the candidate voice request according to the current voice request and the pre-built voice request database, and determine the target guidance information semantically related to the current voice request according to the large language model, the candidate voice request and the current voice request, so that the determination of the target guidance information can refer to the candidate voice request and can be semantically related to the current voice request, thereby enabling the target guidance information to match the user's current usage needs, and the credibility of the target guidance information can be guaranteed.

[0038] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0040] Figure 1 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0041] Figure 2 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0042] Figure 3 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0043] Figure 4 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0044] Figure 5 This is a flowchart of the voice interaction method in certain implementation modes of the present application. DETAILED DESCRIPTION

[0045] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0046] When users first use a vehicle or its voice interaction functions, they are relatively unfamiliar with the objects and functions in the vehicle that can be controlled by voice. Therefore, they may express some voice commands with ambiguous meanings, confusing sentence components or missing components, such as "adjust the air conditioner" or "turn on the temperature adjuster".

[0047] It is understandable that these voice commands have one or more problems such as unclear intention, missing components (or ambiguity), incorrect collocation of sentence components, etc. For example, it can be considered that the collocation relationship between 'adjust' and 'air conditioner' in "adjust air conditioner" is inappropriate, and it can also be considered that the intention corresponding to "adjust air conditioner" includes increasing and decreasing the cooling temperature / heating temperature of the air conditioner, switching the operating mode of the air conditioner, etc., so the intention of "adjust air conditioner" is unclear.

[0048] Therefore, when processing these voice commands, it is difficult to determine the specific vehicle control instructions used to implement or complete these voice commands, which in turn causes the failure of the voice interaction with the user.

[0049] Further, to improve this situation, some solutions are to play specific information to teach users how to use specific functions of the vehicle through timed dynamic advertising examples. However, this solution is difficult to meet the current usage needs of users.

[0050] In other solutions, the user is requested to clarify the voice commands, or in other words, when the user expresses a voice command with ambiguous meanings or incomplete sentence components, the user is asked a question in return. For example, when the user's voice command is "adjust the air conditioner", the vehicle can play a voice message saying "We didn't understand what you meant, please speak more clearly" to request the user to clarify the true intention of the voice command.

[0051] However, when requesting clarification from the user, the interaction with the user is usually based on fixed sentences, such as the above-mentioned "I don't understand what you mean, please speak more clearly". In this case, if the user is using the vehicle and the voice interaction function installed in the vehicle for the first time or has just started to use it, and the user is relatively unfamiliar with the objects and functions in the vehicle that can be controlled by voice, then when the user faces the vehicle's counter-questions (or clarification requests), the user may not know how to clarify, or in other words, do not know "how to express the voice command so that the vehicle can perform the corresponding function", resulting in the user having to have multiple rounds of dialogue and clarification with the vehicle before the user can control the vehicle through the correct (or the vehicle can understand) voice command.

[0052] Based on the above problems you may encounter, please refer to Figure 1 , the present application embodiment provides a voice interaction method, including:

[0053] 01: Receive the current voice request forwarded by the vehicle;

[0054] 02: Determine candidate voice requests based on the current voice request and a pre-built voice request database;

[0055] 03: When a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding the user to adjust the current voice request is determined according to the large language model, the current voice request and the candidate voice requests, the target guidance information is semantically related to the current voice request, the large language model is pre-trained, and the guidance information prediction can be determined according to the voice request sample;

[0056] 04: Feedback target guidance information to guide users to complete voice interaction.

[0057] The embodiment of the present application provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a voice request determination module, a guidance information determination module and an interaction module. Among them, the receiving module is used to receive the current voice request forwarded by the vehicle. The voice request determination module is used to determine the candidate voice request based on the current voice request and a pre-built voice request database. The guidance information determination is used to determine the target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request. The target guidance information is semantically related to the current voice request, the large language model is pre-trained, and the guidance information prediction can be determined based on the voice request sample. The interaction module is used to feedback the target guidance information to guide the user to complete the voice interaction.

[0058] The embodiment of the present application also provides a server, which includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain the current voice request forwarded by the receiving vehicle, and to determine the candidate voice request based on the current voice request and a pre-built voice request database, and to determine the target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request. The target guidance information is semantically related to the current voice request, the large language model is pre-trained, and can determine the guidance information prediction based on the voice request sample, and is used to feedback the target guidance information to guide the user to complete the voice interaction.

[0059] Specifically, in the implementation of the present application, when the user wants to control the vehicle to perform a specific operation through voice control, and thus triggers the corresponding current voice request at the current moment, the vehicle can receive the current voice request and forward the current voice request to the server. Correspondingly, the server can receive the current voice request forwarded by the vehicle, and determine the vehicle control instruction corresponding to the current voice request according to the algorithm or model implemented by the preset code.

[0060] Furthermore, the server may determine one or more candidate voice requests based on the current voice request and a pre-built voice request database.

[0061] Furthermore, when the current voice request has one or more problems such as unclear intention, missing components (or ambiguity), incorrect collocation of sentence components, etc., and thus has semantic defects, resulting in the server being unable to determine the vehicle control instructions corresponding to the current voice request, the server may input the current voice request and the candidate voice request into a pre-trained large language model (Large Language Model, LLM) to determine through the large language model the target guidance information that can be used to guide the user to adjust the current voice request.

[0062] Furthermore, when the server determines the target guidance information through the large language model, the target guidance information can be sent to the vehicle, so that the target guidance information can be fed back to the user through the vehicle, guiding the user to adjust the triggered voice request (i.e., the current voice request), such as adjusting it so that the vehicle can determine a valid voice request corresponding to the vehicle control command, and enabling the vehicle to complete the interaction with the user through the valid voice request.

[0063] For example, if the user wants to adjust the air volume of the vehicle's air conditioner by voice at the current moment, but the current voice request triggered by the user is "adjust the air conditioner", the server can determine one or more candidate voice requests based on the current voice request and a pre-built voice request database, such as "raise the air conditioning temperature", "adjust the air volume to three", "turn on the air conditioner", "turn off the air conditioner" and "adjust the headlights".

[0064] Furthermore, since the server is unable to determine the vehicle control instruction corresponding to the current voice request, the current voice request and the candidate voice request are input into the large language model together. Furthermore, the pre-trained large language model can generate target guidance information that is semantically related to the current voice request and can be used to guide the user to adjust the current voice request ("adjust the air conditioner") based on the current voice request ("adjust the air conditioner") and the received candidate voice requests (a combination of one or more of "increase the air conditioner temperature", "adjust the air conditioner volume to three", "turn on the air conditioner", "turn off the air conditioner" and "adjust the headlights", etc.), such as: "I'm sorry, I don't understand what you mean. If you want to adjust the temperature, you can say: increase the air conditioner temperature, adjust the air conditioner volume to three, and turn on the air conditioner."

[0065] It is understandable that in the embodiments of the present application, a voice request may be understood as a sentence (or query) with ambiguous semantics, confusing or missing sentence components, such as "adjust the air conditioner", "turn on 'the temperature adjuster'", etc.

[0066] It can also be understood that there is no semantic defect in the voice request. Therefore, the server of the implementation mode of the present application can send the vehicle control instruction to the vehicle when the vehicle control instruction corresponding to the voice request is determined, so that the vehicle performs the corresponding function according to the vehicle control instruction, thereby realizing the voice request triggered by the user.

[0067] It should also be noted that the voice request database in the embodiment of the present application stores a plurality of standard voice requests. It is understandable that the standard voice request can be understood as a "voice request that can correspond to a vehicle control instruction", such as "increase the air conditioning temperature", "adjust the air conditioning volume to three", "turn on the air conditioning", "turn off the air conditioning", "turn on rapid heating" and "turn on rapid cooling".

[0068] Furthermore, the server of the embodiment of the present application can call the pre-trained large language model to receive the input of the current voice request and the candidate voice request, and output the guidance information prediction result corresponding to the current voice request. It can be understood that the large language model of the embodiment of the present application can infer the corresponding guidance information prediction result according to the voice request sample during and after the training.

[0069] In addition, the large language model in the implementation manner of the present application can generate target guidance information that is semantically related to the current voice request, such as generating "I'm sorry, I didn't understand what you meant. If you want to adjust the temperature, you can say: increase the air conditioning temperature, adjust the air volume to three, and turn on the air conditioning" corresponding to "adjust the air conditioner."

[0070] Furthermore, it can be understood that after the server generates target guidance information through the pre-trained large language model, the target guidance information can be sent to the vehicle. When the vehicle receives the target guidance information, it can play the target guidance information based on the vehicle display component or the sound playback component to guide the user to adjust or clarify the current voice request previously expressed, thereby adjusting the current voice request that cannot determine the vehicle control command to a valid voice request that can determine the vehicle control command, and allowing the vehicle to execute the vehicle control command corresponding to the valid voice request, and finally completing the voice interaction.

[0071] Optionally, in certain embodiments of the present application, the server may also send the target guidance information to the terminal of the user to which the vehicle belongs, so that the user can adjust the current voice request through the target guidance information played by the terminal.

[0072] In summary, in the implementation mode of the present application, the server can determine the target guidance information for guiding the user to adjust the current voice request when the vehicle control instruction corresponding to the current voice request cannot be determined, and feed back the target guidance information to the user, so that the user can adjust the current voice request to complete the voice interaction through the guidance of the target guidance information when the user is unclear about how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of the vehicle and the vehicle voice interaction function. The implementation mode of the present application can determine the candidate voice request according to the current voice request and the pre-built voice request database, and determine the target guidance information semantically related to the current voice request according to the large language model, the candidate voice request and the current voice request, so that the determination of the target guidance information can refer to the candidate voice request and can be semantically related to the current voice request, thereby enabling the target guidance information to match the user's current usage needs, and the credibility of the target guidance information can be guaranteed.

[0073] Moreover, since the target guidance information is determined based on a large language model, the credibility of the target guidance information can be guaranteed.

[0074] In certain embodiments of the present application, step 02 includes:

[0075] According to the current voice request and the voice request database, a candidate voice request in the voice request database that is semantically related to the current voice request is determined.

[0076] The voice request determination module of the implementation manner of the present application is also used to determine, based on the current voice request and the voice request database, candidate voice requests in the voice request database that are semantically related to the current voice request.

[0077] The processor of the embodiment of the present application is also used to determine, based on the current voice request and the voice request database, a candidate voice request in the voice request database that is semantically related to the current voice request.

[0078] Specifically, in order to ensure the reliability and effectiveness of the target guidance information, the server of the implementation mode of the present application can also search the voice request database to search for voice requests in the voice request database that are semantically related to the current voice request, that is, candidate voice requests.

[0079] Therefore, when the large language model generates target guidance information through candidate voice requests, it can refer to the candidate voice requests with the semantics of the current voice request to generate target guidance information related to the semantics of the current voice request.

[0080] Optionally, in certain embodiments of the present application, the server may calculate the similarity (such as cosine similarity) between the current voice request and each voice request in the voice request database, and extract one or more voice requests with the highest similarity in the voice request database, thereby obtaining candidate voice requests.

[0081] In this way, in an embodiment of the present application, the server can determine the candidate voice requests that are semantically related to the current voice request in the voice request data, so that when the target guidance information is subsequently determined, the target guidance information related to the current voice request semantics can be determined through the candidate voice requests that are semantically related to the current voice request, thereby ensuring the credibility of the target guidance information to a certain extent.

[0082] See also Figure 2 In some embodiments of the present application, the voice request database includes multiple voice requests and an index of each voice request, and the index is determined by encoding the voice request, and step 02 includes:

[0083] 020: Encode the current voice request and determine the encoding result of the current voice request;

[0084] 021: Determine, according to the encoding result and the voice request database, a target index in the voice request database that is semantically related to the encoding result;

[0085] 022: Determine candidate voice requests according to the target index.

[0086] The voice request determination module of the implementation mode of the present application is also used to encode the current voice request, determine the encoding result of the current voice request, and determine the target index in the voice request database that is semantically related to the encoding result based on the encoding result and the voice request database, and determine the candidate voice request based on the target index.

[0087] The processor of the embodiment of the present application is also used to encode the current voice request, determine the encoding result of the current voice request, and determine the target index in the voice request database that is semantically related to the encoding result based on the encoding result and the voice request database, and determine the candidate voice request based on the target index.

[0088] Specifically, in order to reduce the computing resources required to determine candidate voice requests, the voice request database of the implementation mode of the present application can store and query voice requests based on key-value pairs.

[0089] That is, in the process of storing the voice request in the voice request database, the server may encode the voice request to obtain an index of the voice request, and store the voice request and the index of the voice request in the voice request database.

[0090] Furthermore, in the process of determining the candidate voice requests in the voice request database through the current voice request, the server may perform the above-mentioned encoding processing on the current voice request so that the encoded current voice request can be located in the same data space (such as vector space) as the index of each voice request in the voice request database.

[0091] Furthermore, based on the encoding result of the current voice request and the index of each voice request in the voice request database, the server can determine the target index in the voice request database that is semantically related to the encoding result, and finally determine the voice request pointed to by the target index as the candidate voice request.

[0092] It can be understood that compared with the method of "searching the voice request database for candidate voice requests that are semantically related to the current voice request", the method of "searching the voice request database for indexes that are semantically related to the encoding results of the current voice request" requires fewer computing resources and has higher execution efficiency.

[0093] Optionally, since the voice request in the embodiments of the present application can be understood as information such as sentences or texts used to control the vehicle to perform specific functions, in certain embodiments of the present application, the index of the voice request in the database can be understood as the sentence embedding of the voice request. Furthermore, in the process of storing the voice request in the voice request database, the server can encode (embedding) the voice request (sentence) to obtain the sentence embedding of the voice request, that is, the index of the voice request.

[0094] Optionally, in certain embodiments of the present application, the server may determine multiple voice requests that can correspond to vehicle control instructions and cover all positive natural language understanding ranges based on information such as the actual functional scope of the vehicle, high-frequency usage of users, and user logs, and store these multiple voice requests in a voice request database.

[0095] Thus, in the implementation manner of the present application, the server can determine the target index semantically related to the encoding result through the encoding result of the current voice request and the index of the voice request in the voice request database, and determine the candidate voice request semantically related to the current voice request through the target index, thereby reducing the computing resources required to be called by the server to determine the candidate voice request to a certain extent, and improving the efficiency of the server in determining the candidate voice request to a certain extent.

[0096] See also Figure 3 In some implementations of the present application, step 021 includes:

[0097] 0210: Calculate the similarity between the encoding result and the index in the voice request database;

[0098] 0211: Determine the target index based on similarity.

[0099] The voice request determination module of the implementation mode of the present application is also used to calculate the similarity between the encoding result and the index in the voice request database, and to determine the target index according to the similarity.

[0100] The processor of the embodiment of the present application is also used to calculate the similarity between the encoding result and the index in the voice request database, and to determine the target index according to the similarity.

[0101] Specifically, in order to ensure reliable determination of the target index, the server of the implementation mode of the present application can calculate the encoding result of the current voice request and the similarity between it and the index of each voice request in the voice request database, and determine the target index among all indexes based on the similarity between the encoding result and each index.

[0102] Optionally, in certain embodiments of the present application, “the similarity between ‘the encoding result of the current voice request’ and ‘the index of the voice request’” includes “the cosine similarity between ‘the encoding result of the current voice request’ and ‘the index of the voice request’”.

[0103] Thus, in the implementation mode of the present application, the server can determine the target index in the voice request database based on calculating the encoding result of the current voice request and the similarity between the index of the voice request in the voice request database, and based on the similarity between the encoding result and the index, thereby ensuring the credibility of the target index to a certain extent.

[0104] In some embodiments of the present application, step 0211 includes:

[0105] A preset number of target indexes in the voice request database are determined according to the similarity.

[0106] The voice request determination module of the implementation manner of the present application is also used to determine a preset number of target indexes in the voice request database according to the similarity.

[0107] The processor of the embodiment of the present application is also used to determine a preset number of target indexes in the voice request database according to the similarity.

[0108] Specifically, in order to enable the large language model to reliably complete the inference of target guidance information, the server of the implementation mode of the present application can determine a preset number of target indexes from the voice request database according to the "similarity between 'the encoding result of the current voice request' and 'the index of the voice request'", and input the preset number of voice requests pointed to by the preset number of target indexes, or the preset number of candidate voice requests pointed to by the preset number of target indexes, together with the current voice request into the large language model, so that the large language model can complete the inference of the target guidance information according to the current voice request and the preset number of candidate voice requests.

[0109] Optionally, in certain embodiments of the present application, the preset number is a positive integer greater than or equal to 2, such as 5.

[0110] Optionally, in certain embodiments of the present application, the server may index a preset number of target indexes based on the similarity between 'the encoding result of the current voice request' and 'the index of each voice request in the voice request database' with the highest similarity.

[0111] Thus, in the implementation mode of the present application, the server can determine a preset number of target indexes in the voice request database based on the similarity between the encoding result of the current voice request and the index of the voice request in the voice request database, and then perform subsequent guidance information inference work through the preset number of voice requests pointed to by the preset number of target indexes, thereby ensuring the credibility of the target guidance information to a certain extent.

[0112] In certain embodiments of the present application, step 03 includes:

[0113] When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, target guidance information including a target voice request is determined based on the large language model, the current voice request and the candidate voice requests, and the current voice request is semantically related to the target voice request.

[0114] The guidance information determination module of the implementation mode of the present application is also used to determine the target guidance information including the target voice request based on the large language model, the current voice request and the candidate voice requests when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, and the current voice request is semantically related to the target voice request.

[0115] The processor of the embodiment of the present application is also used to determine target guidance information including a target voice request based on a large language model, the current voice request and candidate voice requests when a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, and the current voice request is semantically related to the target voice request.

[0116] Specifically, the implementation method of the present application is to ensure the guiding effect of the target guidance information on the user, so that the large language model determines the target voice request that is semantically related to the current voice request based on the current voice request and the candidate voice requests, and then forms the target guidance information based on the target voice request, so that the user can complete the adjustment of the current voice request through the target voice request and the target guidance information.

[0117] Exemplarily, when the current voice request is "adjust the air conditioner" and the candidate voice requests are "raise the air conditioner temperature", "adjust the air volume to three", "turn on the air conditioner", "turn off the air conditioner" and "adjust the headlights", the server can determine the target voice requests that are semantically related to the current voice request based on the current voice request and the candidate voice requests, such as "raise the air conditioner temperature", "adjust the air volume to three" and "turn on the air conditioner".

[0118] Further, when it is determined that there is a target voice request, the server generates target guidance information according to the target voice request, such as: "I'm sorry, I don't understand what you mean. If you want to adjust the temperature, you can say: turn up the air conditioner temperature, adjust the air volume to three, and turn on the air conditioner."

[0119] It can be understood that the number of target voice requests in the implementation manner of the present application can be one or more.

[0120] Furthermore, when the number of target voice requests is only one, the target voice request may be a candidate voice request, or may be a voice request generated by a large language model according to the current voice request and semantically related to the current voice request.

[0121] And, let the target voice request be query tar 、Candidate voice request is query can , and let the voice request generated by the large language model according to the current voice request and semantically related to the current voice request be the query gan , then when the number of target voice requests is multiple, query tar It can be understood as (query can ∪query gan ), namely, query can With query gan part of the union of .

[0122] Optionally, in some embodiments of the present application, the large language model can confirm whether the candidate voice request is similar when receiving the current voice request and the candidate voice request. Or, confirm whether the candidate voice request can be output as the target voice request. Further, when the large language model confirms that the candidate voice request is similar to the current voice request, the large language model can output the candidate voice request, and thus the candidate voice request can be the target voice request.

[0123] Furthermore, when the large language model confirms that the candidate voice request is not similar to the current voice request, the large language model generates and outputs a voice request based on the knowledge it has learned. Thus, the target voice request can also be a voice request generated by the large language model according to the current voice request and semantically related to the current voice request.

[0124] Or, when the large language model needs to output n target voice requests, and the large language model confirms that only m (m < n) candidate voice requests among multiple candidate voice requests are similar to the current voice request. Or, only m (m < n) candidate voice requests can be output as the target voice request. Then the large language model can generate and output (n - m) voice requests semantically related to the current voice request according to the current voice request, thereby completing the output of n target voice requests.

[0125] In this way, in the embodiments of the present application, the server can determine the target guidance information including the target voice request based on the large language model. Further, after the target guidance information is fed back to the user, the user can adjust the current voice request based on the target voice request semantically related to the current voice request in the target guidance information to complete the voice interaction. Therefore, the user's experience of the vehicle voice interaction function and the vehicle can be guaranteed.

[0126] In some embodiments of the present application, step 03 includes:

[0127] In the case where the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request and the large language model, determine the target guidance information according to the large language model, the current voice request and the candidate voice request.

[0128] The guidance information determination module in the embodiments of the present application is further configured to determine the target guidance information according to the large language model, the current voice request and the candidate voice request in the case where the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request and the large language model.

[0129] The processor of the embodiment of the present application is also used to determine target guidance information based on the large language model, the current voice request and candidate voice requests when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request and the large language model.

[0130] Specifically, in order to reduce the number of module deployments and make full use of the training parameters in the large language model, the large language model in the implementation mode of the present application also learns the knowledge of "outputting vehicle control instructions corresponding to the voice request based on the received voice request input" during the training process. Therefore, the server in the implementation mode of the present application can call the large language model to predict the vehicle control instructions for the current voice request.

[0131] Furthermore, if the large language model can predict the corresponding vehicle control instructions based on the current voice request, the server can feed back the vehicle control instructions to the vehicle so that the vehicle performs the corresponding operation according to the vehicle control instructions.

[0132] Furthermore, if the current voice request has semantic defects, resulting in the large language model being unable to confirm the vehicle control instructions corresponding to the current voice request, the large language model will also predict the guidance information based on the current voice request and the candidate voice requests, thereby obtaining the target guidance information, and feeding back the target guidance information to the user to guide the user to adjust the current voice request.

[0133] In this way, in the implementation mode of the present application, the server can confirm the vehicle control instructions of the current voice request through the large language model, and when the vehicle control instructions of the current voice request cannot be predicted by the large language model, the target guidance information is determined based on the large language model, the current voice request and the candidate voice request. Therefore, both the vehicle control instruction prediction and the target guidance information prediction can be achieved based on the large language model, avoiding the need to set up a separate module for predicting vehicle control instruction prediction and a separate module for predicting target guidance information in the server, thereby reducing the load on the server.

[0134] In certain embodiments of the present application, when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request and the large language model, the step of determining the target guidance information according to the large language model, the current voice request and the candidate voice request includes:

[0135] When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-prompt information template of the pre-configured prompt information template, the target guidance information is determined based on the large language model, the current voice request, the candidate voice requests and the second sub-prompt information template of the prompt information template.

[0136] The guidance information determination module of the implementation mode of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the candidate voice requests and the second sub-prompt information template of the prompt information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-prompt information template of the pre-configured prompt information template.

[0137] The processor of the embodiment of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the candidate voice request and the second sub-prompt information template of the prompt information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-prompt information template of the pre-configured prompt information template.

[0138] Specifically, in order to ensure the credibility of the reasoning results of the large language model in the implementation mode of the present application, the server of the implementation mode of the present application may use a prompt as a part of the input information of the large language model. That is, in the process of the large language model predicting the vehicle control instructions based on the current voice request, and in the process of the large language model predicting the target guidance information based on the current voice request and the candidate voice request, the server may input the prompt information template into the large language model, so that the large language model can complete the prediction of the vehicle control instructions and the target guidance information according to the reasoning method and natural language understanding method indicated by the prompt information template.

[0139] Furthermore, the prompt information template of the embodiment of the present application can be divided into two parts, one part can be understood as a first sub-prompt information template for prompting the large language model to identify "whether the current voice request can correspond to the vehicle control instruction". The other part can be understood as a second sub-prompt information template for prompting the large language model to generate "target guidance information that is semantically related to the current voice request and is used to guide the user to adjust the current voice request".

[0140] In one example, a prompt information template including a first sub-prompt information template and a second sub-prompt information template includes: "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information according to the instructions provided by the user, and extract the key information in the instructions for API search related information. The returned results include API name: api, key information: arguments. Step 2: If the API is unclear, recommend related annotation instructions to the user. Similar annotation instructions are: increase the air conditioning temperature, adjust the air conditioning volume to three, turn on the air conditioning, turn off the air conditioning, and adjust the headlights."

[0141] It should be noted that in the example of the above prompt information template, the first sub-prompt information template can be: "In the first step, according to the instructions provided by the user, select an API that can be executed or provide information, and extract key information in the instructions for API search related information. The returned results include API name: api, key information: arguments."

[0142] And, in the example of the above prompt information template, the second sub-prompt information template may be: "Step 2: If the API is unclear, recommend relevant annotation instructions to the user."

[0143] It should also be noted that in the example of the above prompt information template, "similar labeled instructions: increase the air conditioning temperature, adjust the air conditioning volume to three, turn on the air conditioning, turn off the air conditioning, adjust the headlights" can be understood as a candidate voice request in the implementation manner of the present application.

[0144] Therefore, in certain embodiments of the present application, the server may combine the current voice request, the candidate voice request and the pre-configured prompt information template to form a complete prompt, and the server may input the prompt into the large language model so that the large language model completes the vehicle control command prediction and the target guidance information prediction based on the prompt.

[0145] In addition, it can be understood that in the example of the above prompt information template, "user-provided instructions" can be understood as the current voice request forwarded by the vehicle. And, in the example of the above prompt information template, "if the API is unclear" can be understood as "failed to determine the vehicle control instructions corresponding to the current voice request".

[0146] Thus, in the implementation mode of the present application, the large language model can confirm the correspondence between the current voice request and the vehicle control instruction based on the first sub-prompt information template and the second sub-prompt information template in the prompt information template, and when it is confirmed that the current voice request fails to correspond to the vehicle control instruction, confirm the target guidance information corresponding to the current voice request, thereby realizing reasoning of the large language model based on the prompt information template, and thus the credibility of the model reasoning result can be guaranteed.

[0147] Optionally, in certain embodiments of the present application, specific reference may be made to 4 and Figure 5 , Figure 4 and Figure 5 are all flowcharts of the voice interaction method in certain embodiments of the present application. Figure 4 As shown, when the current voice request triggered by the user at the current moment is "adjust the air conditioner", and the vehicle obtains the current voice request, the current voice request is forwarded or reported to the server.

[0148] When the server receives the current voice request ("adjust the air conditioner") forwarded by the vehicle, Figure 5 As shown, the server may perform encoding processing on the current voice request to obtain an encoding result embedding of the current voice request.

[0149] Subsequently, the server can calculate the similarity between "the encoding result embedding of the current voice request" and "the embeddings of each voice request in the voice request database" based on the encoding result embedding of the current voice request and the embeddings of each voice request in the pre-built voice request database, and determine the top five embeddings in the voice request database that are most similar to "the encoding result embedding of the current voice request" based on the similarity, and then recall the voice requests indicated by the five embeddings from the voice request database, namely {1. Increase the air conditioning temperature; 2. Adjust the air conditioning volume to three; 3. Turn on the air conditioning; 4. Turn off the air conditioning; 5. Adjust the headlights}.

[0150] Then, if Figure 4 As shown, the server can splice the five candidate voice requests pointed to by the five embeddings with the pre-configured prompt information template to form spliced ​​information, namely: "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information based on the instructions provided by the user, and extract the key information in the instructions for API search for relevant information. The returned results include, API name: api, key information: arguments. Step 2: If the API is unclear, recommend related labeled instructions to the user. Similar labeled instructions are: increase the air conditioning temperature, adjust the air conditioning volume to three, turn on the air conditioning, turn off the air conditioning, and adjust the headlights."

[0151] Then, the server can input the spliced ​​information together with the above-mentioned current voice request ("adjust the air conditioner") into the large language model, so that the large language model can predict the vehicle control instructions and target guidance information.

[0152] Furthermore, due to the semantic defects of the current voice request ("adjust the air conditioner"), the large language model failed to determine the corresponding vehicle control instructions based on the current voice request ("adjust the air conditioner") and the "First step, based on the instructions provided by the user, select the API that can be executed or provide information, and extract the key information in the instructions for API search for relevant information. The returned results include, API name: api, key information: arguments" in the spliced ​​information.

[0153] Furthermore, the large language model can determine the target guidance information containing the target voice request based on the current voice request ("adjust the air conditioner") and the "Step 2: If the API is unclear, recommend relevant annotation instructions to the user. Similar annotation instructions are: increase the air conditioning temperature, adjust the air conditioning volume to three, turn on the air conditioning, turn off the air conditioning, and adjust the headlights" in the spliced ​​information, that is, "Sorry, I don't understand what you mean. If you want to adjust the temperature, you can say: increase the air conditioning temperature, adjust the air conditioning volume to three, and turn on the air conditioning."

[0154] It is understandable that in a solution where the guidance information is determined entirely through a neural network model (such as a large language model) without considering candidate voice requests, since the training process of the neural network model is usually carried out through pre-collected corpus data, or in other words, through static knowledge, once the neural network model is implemented, once new and updated functional points appear, if the neural network model is retrained so that the model can align with the new and updated functional points, there is a problem of high training costs.

[0155] However, if you choose to fine-tune the neural network model (Fine Turning), such as supervised fine-tuning (SFT), after fine-tuning training, the neural network model may still reason according to previously learned knowledge, and may infer answers that are inconsistent with the "newly added and updated functional points", that is, there is model hallucination.

[0156] It can also be understood that the implementation mode of the present application is based on a pre-constructed external database, namely, a voice request database, and combined with a similarity recall method, candidate voice requests that are semantically similar to the current voice request are recalled from the voice request database, so that the pre-trained large language model can determine the recommended voice request (i.e., the target voice request) corresponding to the current voice request based on the semantics of the candidate voice request and its own learned knowledge, and form target guidance information through the recommended voice request, so that the credibility and guiding effect of the target guidance information can be guaranteed.

[0157] At the same time, the implementation methods of the present application can also update the voice requests in the voice request data when new function points are added and updated, so that the large language model can complete the reasoning work based on the external knowledge provided by the voice request data, thereby avoiding the need to immediately update the large language model when new function points are added and updated. The performance of the large language model can be aligned with the newly added knowledge (i.e., the newly added and updated function points) through the voice request data, thereby meeting user needs in real time and efficiently.

[0158] The present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned voice interaction method when the computer program is executed by one or more processors.

[0159] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0160] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0161] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receive the current voice request forwarded by the vehicle; Based on the current voice request and a pre-built voice request database, determining a candidate voice request in the voice request database that is semantically related to the current voice request; In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding a user to adjust the current voice request is determined according to the large language model, the current voice request and the candidate voice request, wherein the target guidance information is semantically related to the current voice request, the large language model is pre-trained and can determine guidance information prediction according to the voice request sample; The target guidance information is fed back to guide the user to complete the voice interaction.

2. The method according to claim 1, characterized in that: The voice request database includes a plurality of voice requests and an index of each voice request, wherein the index is determined by encoding the voice request, and determining the candidate voice request based on the current voice request and the pre-built voice request database includes: Performing the encoding process on the current voice request to determine an encoding result of the current voice request; Determining, according to the encoding result and the voice request database, a target index in the voice request database that is semantically related to the encoding result; The candidate voice request is determined according to the target index.

3. The method according to claim 2, characterized in that The determining, according to the encoding result and the voice request database, a target index in the voice request database that is semantically related to the encoding result includes: Calculating the similarity between the encoding result and the index in the voice request database; The target index is determined according to the similarity.

4. The method according to claim 3, characterized in that The determining the target index according to the similarity comprises: A preset number of target indexes in the voice request database are determined according to the similarity.

5. The method according to claim 1, characterized in that: The method further comprises: determining target guidance information for guiding a user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, including: When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the target guidance information including the target voice request is determined based on the large language model, the current voice request and the candidate voice requests, and the current voice request is semantically related to the target voice request.

6. The method according to claim 1, characterized in that The method further comprises: determining target guidance information for guiding a user to adjust the current voice request according to the large language model, the current voice request and the candidate voice request when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, including: When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request and the large language model, the target guidance information is determined based on the large language model, the current voice request and the candidate voice requests.

7. The method according to claim 6, characterized in that The method of determining the target guidance information according to the large language model, the current voice request and the candidate voice request when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request and the large language model includes: When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and a first sub-prompt information template of a pre-configured prompt information template, the target guidance information is determined based on the large language model, the current voice request, the candidate voice requests and a second sub-prompt information template of the prompt information template.

8. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.

Images