Voice interaction method, server and computer-readable storage medium

Abstract

The present application discloses a voice interaction method, a server and a computer-readable storage medium, the method comprising: receiving a current voice request forwarded by a vehicle, and in the case where a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining target guidance information and feeding back the target guidance information to guide the user to complete the voice interaction according to a large language model, the current voice request and the target language style information. In this way, the server of the present application can determine the target guidance information for guiding the user to adjust the voice request and matching the target language style according to the current voice request, the target language style information and the large language model when the vehicle control instruction corresponding to the voice request cannot be determined, and guide the user to adjust the voice request to complete the voice interaction through the target guidance information, so that the user's experience of the vehicle voice interaction function and the vehicle can be guaranteed.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, a server, and a computer-readable storage medium. Background Art

[0002] When the user first starts using the vehicle, he or she may express some vague and incomplete voice commands because he or she is unfamiliar with the vehicle and its voice interaction functions. When the vehicle processes these voice commands, it has difficulty understanding the actual meaning of the voice commands and is unable to execute the corresponding functions, resulting in voice interaction failure. Summary of the invention

[0003] The present application provides a voice interaction method, a server and a computer-readable storage medium.

[0004] A voice interaction method provided in an embodiment of the present application includes:

[0005] Receive the current voice request forwarded by the vehicle;

[0006] In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding the user to adjust the current voice request is determined according to the large language model, the current voice request and the acquired preset target language style information, the target guidance information matches the target language style information, the large language model is pre-trained, and can generate guidance information prediction results matching each of the language style information according to the voice request sample and the multiple language style information;

[0007] The target guidance information is fed back to guide the user to complete the voice interaction.

[0008] In the voice interaction method provided in the embodiment of the present application, the server can receive the current voice request forwarded by the vehicle, and when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, the server can determine the target guidance information that is used to guide the user to adjust the current voice request and matches the target guidance information based on the pre-trained large language model, the current voice request and the acquired pre-set target language style information, and feed back the target guidance information to guide the user to complete the voice interaction.

[0009] Thus, in the implementation mode of the present application, the server can determine the target guidance information for guiding the user to adjust the voice request when the corresponding vehicle control instruction cannot be determined through the current voice request, and feedback the target guidance information to the user, so that the user can adjust the current voice request based on the guidance of the target guidance information to complete the voice interaction when the user is unclear about how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of the vehicle and the vehicle voice interaction function. The implementation mode of the present application can determine the target guidance information that matches the target language style information based on the current voice request, the large language model and the target voice style information, and then when the target language style information is set based on the user's personal language preference, the target guidance information fed back by the server can meet the user's personal language preference, thereby further improving the user's experience of the vehicle and the vehicle voice interaction function.

[0010] In certain embodiments of the present application, the step of training the large language model includes:

[0011] Determining a plurality of candidate guidance information according to the voice request sample and a predetermined reference model;

[0012] Determine a ranking result of the plurality of candidate guide information corresponding to each type of the language style information;

[0013] According to the ranking results corresponding to each type of language style information, the reference model is trained to obtain the large language model.

[0014] Thus, in an embodiment of the present application, the server may determine multiple candidate guidance information corresponding to a voice request sample based on a reference model, and after determining that there are multiple candidate guidance information corresponding to the sorting results of each language style information, perform training on the reference model based on the sorting results of each language style information, so that the reference model can learn the association between each language style information and each candidate guidance information corresponding to the language request sample, thereby ensuring to a certain extent the reliability of the large language model in determining the target guidance information that matches the target language style information based on the target language style information and the voice request.

[0015] In certain embodiments of the present application, the step of training the reference model according to the ranking result corresponding to each type of language style information to obtain the large language model includes:

[0016] The reference model is trained according to the language style information, the sorting result and a pre-configured first prompt information template to obtain the large language model.

[0017] Thus, in the implementation mode of the present application, the server can train the reference model according to each type of language style information and the sorting result corresponding to each type of language style information, as well as the pre-configured first prompt information template, so that the training effect of the reference model is guaranteed to a certain extent.

[0018] In certain embodiments of the present application, the step of training the reference model according to the ranking result corresponding to each type of language style information to obtain the large language model includes:

[0019] Determining first guidance information and second guidance information corresponding to each type of language style information according to the sorting result corresponding to each type of language style information;

[0020] The reference model is trained according to the first guide information and the second guide information to obtain the large language model.

[0021] Thus, in an embodiment of the present application, the server can determine the first guidance information and the second guidance information corresponding to each language style information from a variety of candidate guidance information determined by the reference model according to the voice request sample based on the sorting results corresponding to each language style information, so that the reference model can learn the difference between the first guidance information and the second guidance information corresponding to each language style information to a certain extent, thereby reliably completing the prediction of the target guidance information after the training is completed.

[0022] In some embodiments of the present application, the reference model includes a target model and a copy model of the target model, and the training of the reference model according to the first guidance information and the second guidance information to obtain the large language model includes:

[0023] Determine, according to the target model, the replica model, the language style information and the voice request sample, a first probability that the target model predicts the first guidance information and the second guidance information corresponding to the language style information, and a second probability that the replica model predicts the first guidance information and the second guidance information corresponding to the language style information;

[0024] According to the similarity between the first probability and the second probability, the target model is trained to obtain the large language model.

[0025] In this way, in an embodiment of the present application, the target model can be trained based on the first probability of the target model predicting the first guidance information and the second guidance information, and the second probability of the copy model predicting the first guidance information and the second guidance information, so that the training of the target model can be carried out with reference to the copy model, thereby reducing the possibility of model performance degradation due to training to a certain extent, and the training effect of the target model can be guaranteed.

[0026] In some embodiments of the present application, the first probability includes a first sub-probability corresponding to the first guide information and a second sub-probability corresponding to the second guide information, the second probability includes a third sub-probability corresponding to the first guide information and a fourth sub-probability corresponding to the second guide information, and training the target model to obtain the large language model according to the similarity between the first probability and the second probability includes:

[0027] According to the similarity between the first sub-probability and the third sub-probability, and the similarity between the second sub-probability and the fourth sub-probability, the target model is trained to obtain the large language model.

[0028] Thus, in the implementation mode of the present application, the server can perform training of the target model based on the similarity between the first sub-probability and the third sub-probability, as well as the similarity between the second sub-probability and the fourth sub-probability, with reference to the prediction results of the copy model. This can reduce the possibility of performance degradation of the target model due to training to a certain extent, and the training effect of the target model can be guaranteed.

[0029] In certain embodiments of the present application, when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the acquired preset target language style information includes:

[0030] In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, the target guidance information is determined according to the large language model, the current voice request, the target language style information and a pre-configured second prompt information template.

[0031] Thus, in the implementation mode of the present application, the large language model can perform natural language understanding and processing on the current voice request and the target language style information based on the instructions of the prompt information template, thereby generating target guidance information corresponding to the current voice request and matching the target language style information, thereby ensuring the prediction accuracy of the large language model to a certain extent.

[0032] In certain embodiments of the present application, the second prompt information template includes a first sub-information template and a second sub-information template, and when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining the target guidance information according to the large language model, the current voice request, the target language style information and the pre-configured second prompt information template includes:

[0033] When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-information template, the target guidance information is determined based on the large language model, the current voice request, the prompt information template, the target language style information and the second sub-information template.

[0034] Thus, in the implementation mode of the present application, the large language model can confirm the correspondence between the current voice request and the vehicle control instruction based on the first sub-prompt information template and the second sub-prompt information template in the prompt information template, and when it is confirmed that the current voice request fails to correspond to the vehicle control instruction, confirm the target guidance information corresponding to the current voice request, thereby ensuring the accuracy of the target guidance information inferred by the large language model.

[0035] An embodiment of the present application provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned voice interaction method is implemented.

[0036] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned voice interaction method is implemented.

[0037] The server and computer-readable storage medium provided by the embodiments of the present application can determine the target guidance information for guiding the user to adjust the voice request when the corresponding vehicle control instruction cannot be determined through the current voice request, and feedback the target guidance information to the user, so that the user can adjust the current voice request based on the guidance of the target guidance information to complete the voice interaction when the user is unclear about how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of using the vehicle and the vehicle voice interaction function. The embodiments of the present application can determine the target guidance information that matches the target language style information based on the current voice request, the large language model and the target voice style information, and then when the target language style information is set based on the user's personal language preference, the target guidance information fed back by the server can meet the user's personal language preference, thereby further improving the user's experience of using the vehicle and the vehicle voice interaction function.

[0038] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0040] Figure 1 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0041] Figure 2 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0042] Figure 3 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0043] Figure 4 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0044] Figure 5 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0045] Figure 6 A schematic diagram of application scenarios in certain embodiments of the present application;

[0046] Figure 7 A schematic diagram of application scenarios in certain embodiments of the present application;

[0047] Figure 8 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0048] Fig. 9 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0049] Fig.10 A schematic diagram of a large language model training process in certain implementations of the present application;

[0050] Fig.11 A schematic diagram of a large language model training process in certain implementations of the present application;

[0051] Fig.12 A flowchart of a voice interaction method in certain implementation modes of the present application;

[0052] Fig.13 This is a flowchart of the voice interaction method in certain implementation modes of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0054] When users first use a vehicle or its voice interaction functions, they are relatively unfamiliar with the objects and functions in the vehicle that can be controlled by voice. Therefore, they may express some voice commands with ambiguous meanings, confusing sentence components or missing components, such as "adjust the air conditioner" or "turn on the temperature adjuster".

[0055] It is understandable that these voice commands have one or more problems such as unclear intention, missing components (or ambiguity), incorrect collocation of sentence components, etc. For example, it can be considered that the collocation relationship between 'adjust' and 'air conditioner' in "adjust air conditioner" is inappropriate, and it can also be considered that the intention corresponding to "adjust air conditioner" includes increasing and decreasing the cooling temperature / heating temperature of the air conditioner, switching the operating mode of the air conditioner, etc., so the intention of "adjust air conditioner" is unclear.

[0056] Therefore, when processing these voice commands, it is difficult to determine the specific vehicle control instructions used to implement or complete these voice commands due to one or more problems such as unclear intentions, missing components (or ambiguity), incorrect combination of sentence components, etc., which leads to the failure of the user's voice interaction.

[0057] To improve this situation, some solutions are to use timed dynamic advertising examples, that is, to regularly play specific information to teach users how to use specific vehicle functions through voice. However, this solution is difficult to meet the current usage needs of users.

[0058] In other solutions, the user is requested to clarify the voice commands, or in other words, when the user expresses a voice command with ambiguous meanings or incomplete sentence components, the user is asked a question in return. For example, when the user's voice command is "adjust the air conditioner", the vehicle can play a voice message saying "We didn't understand what you meant, please speak more clearly" to request the user to clarify the true intention of the voice command.

[0059] However, when requesting clarification from the user, the interaction with the user is usually based on fixed sentences, such as the above-mentioned "I don't understand what you mean, please speak more clearly". In this case, if the user is using the vehicle and the voice interaction function installed in the vehicle for the first time or has just started to use it, and the user is relatively unfamiliar with the objects and functions in the vehicle that can be controlled by voice, then when the user faces the counter-questions (or clarification requests) raised by the vehicle, the user may not know how to clarify, or in other words, do not know "how to express the voice command so that the vehicle can perform the corresponding function", resulting in the user having to have multiple rounds of dialogue and clarification with the vehicle before the user can control the vehicle through the correct (or the vehicle can understand) voice command.

[0060] In addition, it is also understood that factors related to language style, such as tone and wording, used when requesting clarification from the user can directly affect the user. For example, when requesting clarification from the user, the vehicle may play a sentence with a concise (or blunt) language style, such as "I don't understand, please be more clear".

[0061] For example, when requesting clarification from the user, the vehicle may play a sentence in a detailed (or cumbersome) language style, such as "I'm sorry, I don't understand what you mean, please be more specific. For example, if you need to adjust the air conditioning temperature, you can say: adjust the air conditioning temperature to 26 degrees."

[0062] It is understandable that different users may have different feelings when faced with sentences of the same language style. For example, for the concise style sentence mentioned above, "I don't understand, please be more clear", some users may think that the sentence is concise and clear, while other users may think that the sentence is stiff and too simple.

[0063] For another example, with respect to the detailed sentence “I’m sorry, I don’t understand what you mean. Please be more specific. For example, if you need to adjust the air conditioner temperature, you can say: adjust the air conditioner temperature to 26 degrees,” some users may think that the sentence is detailed and clear, while other users may think that the sentence is verbose and complicated.

[0064] Based on the above problems that may be encountered, please refer to Figure 1 , the present application embodiment provides a voice interaction method, including:

[0065] 01: Receive the current voice request forwarded by the vehicle;

[0066] 02: When a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding the user to adjust the current voice request is determined according to the large language model, the current voice request and the acquired preset target language style information, the target guidance information matches the target language style information, the large language model is pre-trained, and can generate guidance information prediction results matching each language style information according to the voice request sample and multiple language style information;

[0067] 03: Provide target guidance information to guide users to complete voice interaction.

[0068] The embodiment of the present application provides a voice interaction device. The voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes a receiving module, a determining module and an interaction module. Among them, the receiving module is used to receive the current voice request forwarded by the vehicle. The determination module is used to determine the target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the acquired pre-set target language style information when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request. The target guidance information matches the target language style information, and the large language model is pre-trained, and can generate guidance information prediction results matching each language style information according to the voice request sample and multiple language style information. The interaction module is used to feedback the target guidance information to guide the user to complete the voice interaction.

[0069] The embodiment of the present application provides a server, and the server includes a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to receive the current voice request forwarded by the vehicle, and to determine the target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the obtained pre-set target language style information when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request. The target guidance information matches the target language style information, the large language model is pre-trained, and can generate guidance information prediction results matching each language style information according to the voice request sample and multiple language style information, and is used to feedback the target guidance information to guide the user to complete the voice interaction.

[0070] Specifically, in the implementation of the present application, when the user wants to control the vehicle to perform a specific operation through voice control, and thus triggers the corresponding current voice request at the current moment, the vehicle can receive and forward the current voice request to the server. Correspondingly, the server can receive the current voice request forwarded by the vehicle, and determine the vehicle control instruction corresponding to the current voice request according to the algorithm or model implemented by the preset code.

[0071] However, when the current voice request has semantic defects, such as unclear intention, missing components (or ambiguity), incorrect combination of sentence components, etc., the server fails to determine the vehicle control instructions corresponding to the current voice request, and then the current voice request and the target language style information pre-set by the user can be input into the pre-trained large language model, so that the large language model can output target guidance information that can be used to guide the user to adjust the current voice request and matches the target language style information.

[0072] Furthermore, when the server obtains the target guidance information through the large language model, it can send the target guidance information to the vehicle, so as to feed back the target guidance information to the user through the vehicle, and guide the user to adjust the triggered voice request (i.e., the current voice request), such as adjusting it so that the vehicle can determine the valid voice request corresponding to the vehicle control command, and enable the vehicle to complete the interaction with the user through the valid voice request.

[0073] For example, the user wants to adjust the cooling temperature of the vehicle's air conditioner to 24 degrees through voice at the current moment, but the current voice request triggered by the user is "adjust the air conditioner" which has a semantic defect. As a result, when the server receives the current voice request forwarded by the vehicle, it is unable to determine the vehicle control instruction corresponding to the current voice request. Therefore, the server can input the current voice request and the language style information that is acquired in advance and can be set by the user into the large language model.

[0074] Furthermore, the pre-trained large language model can generate guidance information based on the current voice request (i.e., "adjust the air conditioner"), which may include the first language style information style 1 The first boot information response that matches 1 , and second language style information style 2 Matching second guide information response 2 and third language style information style 3 Matching third-party guidance information response 3 .

[0075] Among them, response 1You can include: "I don't understand what you mean. You can say something more standard, such as: adjust the air conditioner to 20 degrees and the air volume to level 3." 2 This may include: "I'm sorry, I didn't understand what you meant. I suggest you say something like: adjust the air conditioner to 20 degrees and the air volume to level 3." 3 It may include: "I'm sorry, I don't understand what you mean. If you want to adjust the temperature, you can say: adjust the air conditioner to 20 degrees. To adjust the air volume, you can say: adjust the air volume to level 3."

[0076] Furthermore, since the input information received by the large language model includes the current voice request (i.e., "adjust the air conditioner") and the target language style information, we have: when the target language style information is style 1 When , the target guidance information output by the large language model is response 1 ; In the target language style information is style 2 When , the target guidance information output by the large language model is response 2 ; In the target language style information is style 3 When , the target guidance information output by the large language model is response 3 .

[0077] It is understandable that in the embodiments of the present application, a voice request can be understood as a sentence (query) with semantic ambiguity, confusion of sentence components, etc., and thus has semantic defects, such as "adjust the air conditioner", "turn on 'the temperature adjuster'", etc.

[0078] It can also be understood that in the implementation mode of the present application, the large language model can output a variety of guidance information matching a variety of language style information when receiving the input of the voice request based on the pre-completed training. For example, in the above example, the large language model can output a variety of guidance information matching the first language style information according to the current voice request. 1 The first boot information response that matches 1 , and second language style information style 2 Matching second guide information response 2 and third language style information style 3 Matching third-party guidance information response 3 .

[0079] It should be noted that the language style information in the implementation manner of the present application may include "detailed", "concise", "cold", "gentle", etc. Also, the setting of the target language style information may be set according to the actual situation. For example, in one example, when the user performs one or more combinations of corresponding operations such as clicking and typing based on the vehicle or user terminal to select one from a plurality of pre-set language style information as the target language style information, the vehicle or user terminal may send corresponding data to the server so that the server determines the target language style information set by the user.

[0080] For example, a user performs one or more combinations of corresponding operations such as clicking and typing through a vehicle or a user terminal to confirm "detailed" in language style information such as "detailed", "concise", "cold", and "gentle" as the target language style information. The vehicle or user terminal can send corresponding data to the server to enable the server to determine that the target language style information set by the user is "detailed".

[0081] It is understandable that, since the target language style information is settable, the user can set the target language style information according to his or her own needs, so that the large language model in the server outputs target guidance information that matches the target language style information, and thus the target guidance information can meet the user's personal usage preferences.

[0082] In summary, in the implementation mode of the present application, the server can determine the target guidance information for guiding the user to adjust the voice request when the corresponding vehicle control instruction cannot be determined through the current voice request, and feedback the target guidance information to the user, so that the user can adjust the current voice request based on the guidance of the target guidance information to complete the voice interaction when the user is unclear about how to express the voice request for controlling the vehicle, thereby ensuring the user's experience of using the vehicle and the vehicle voice interaction function. The implementation mode of the present application can determine the target guidance information that matches the target language style information based on the current voice request, the large language model and the target voice style information, and then when the target language style information is set based on the user's personal language preference, the target guidance information fed back by the server can meet the user's personal language preference, thereby further improving the user's experience of using the vehicle and the vehicle voice interaction function.

[0083] Moreover, the implementation method of the present application can determine the target guidance information according to the current voice request, so the target guidance information can be related to the current voice request, and then the target guidance information can match the user's current usage needs to a certain extent, and the reliability of the target guidance information can be guaranteed. In addition, since the target guidance information is determined based on a large language model, the credibility of the target guidance information can be guaranteed.

[0084] See also Figure 2 In certain embodiments of the present application, the training step of the large language model includes:

[0085] 04: Determine multiple candidate guidance information according to the voice request sample and a predetermined reference model;

[0086] 05: Determine the ranking results of the plurality of candidate guide information corresponding to each language style information;

[0087] 06: According to the ranking results corresponding to each language style information, the reference model is trained to obtain a large language model.

[0088] The voice interaction device of the embodiment of the present application also includes a candidate guidance information determination module, a sorting result determination module and a reference model training module. Among them, the candidate guidance information determination module is used to determine multiple candidate guidance information according to the voice request sample and the predetermined reference model. The sorting result determination module is used to determine the sorting result of multiple candidate guidance information corresponding to each language style information. The reference model training module is used to train the reference model according to the sorting result corresponding to each language style information to obtain a large language model.

[0089] The processor of the embodiment of the present application is also used to determine multiple candidate guide information based on the voice request sample and the predetermined reference model, and to determine the sorting results of the multiple candidate guide information corresponding to each language style information, and to train the reference model according to the sorting results corresponding to each language style information to obtain a large language model.

[0090] Specifically, in order to enable the large language model to generate a variety of guidance information that matches a variety of language style information when receiving a voice request, and to enable the large language model to output target guidance information that matches the target language style information when receiving a voice request and the input of the target language style information, the server of the implementation mode of the present application can output a variety of candidate guidance information through voice request samples based on a reference model (or a large language model that has not yet been trained).

[0091] Next, the server can determine the ranking results of the multiple candidate guide information relative to each language style information to characterize the matching degree of the multiple candidate guide information relative to the different language style information. Finally, the reference model is trained according to the ranking results, so that when the reference model receives the voice request and the input of the target language style information, it can determine "the candidate guide information that best matches the target language style among the multiple candidate guide information that can be generated by itself", that is, the target guide information.

[0092] To more clearly illustrate the implementation of this application, please refer to Figure 3 ,4 and 5, Figure 3 , 4 5 and 5 are flowcharts of the voice interaction method in certain embodiments of the present application. Figure 3 As shown, after receiving the voice request sample "adjust the air conditioner", the reference model can predict the guidance information of "adjust the air conditioner" multiple times to output 3 candidate guidance information, respectively, res 1 、res 2 and res 3 .

[0093] Among them, res 1 You can include: "I don't understand what you mean. You can say something more standard, such as: adjust the air conditioner to 20 degrees and the air volume to level 3." 2 It may include: "I'm sorry, I didn't understand what you meant. I suggest you say something like: adjust the air conditioner to 20 degrees and the air volume to level 3." 3 It may include: "I'm sorry, I don't understand what you mean. If you want to adjust the temperature, you can say: adjust the air conditioner to 20 degrees. To adjust the air volume, you can say: adjust the air volume to level 3."

[0094] Further, after getting res 1 、res 2 and res 3 In the case of 1 、res 2 and res 3 Relative to the matching degree of each language style information, 1 、res 2 and res 3 To sort the annotations, or in other words, the server can receive the annotation staff's response to the res 1 、res 2 and res 3 The annotation sorting instructions are used to determine the res under each language style information. 1 、res 2 and res 3 The sorting result is as follows: Figure 4 and Figure 5 As shown, the sorting result corresponding to "detailed" is formed (res 3 First, res 2 Second, res 1 Last), the sorting results corresponding to "simple" (res 3 Finally, res 2 Second, res 1 First) and other language style information corresponding to the sorting results.

[0095] Furthermore, the server can train the reference model based on the ranking result, so that after the training is completed, the reference model can learn "the matching situation between each candidate guidance information and each language style among the multiple candidate guidance information that it can generate", such as being able to determine the res 1 、res 2 and res 3 In the example, the guidance information that matches the language style information "concise" is res 1 , the guidance information that matches the language style information "detailed" is res 3 .

[0096] Furthermore, when the voice request is “adjust the air conditioner” and the target language style information is “detailed”, the large language model can 3 Correspondingly, when the voice request is "adjust the air conditioner" and the target language style information is "concise", the large language model can 1 Determine the target guidance information.

[0097] It can be understood that in the implementation manner of the present application, the reference model is a pre-trained language model that can output guidance information according to a voice request.

[0098] Furthermore, since the server in the implementation mode of the present application can perform training on a pre-trained reference model, the training process of the reference model in the implementation mode of the present application can be understood as a fine-tuning process of the reference model.

[0099] It can also be understood that in the implementation manner of the present application, the number of voice request samples is multiple.

[0100] Furthermore, for each voice request sample, the server may repeatedly input the voice request sample into the reference model multiple times, so that the reference model generates multiple candidate guidance information for each voice request sample.

[0101] In addition, it can also be understood that in the embodiments of the present application, the sorting method of multiple candidate guide information is content that can be set according to actual conditions, and the sorting based on manual annotation is only one of the feasible methods. For example, in some embodiments of the present application, the server can call a pre-trained neural network model that can determine the degree of matching between the guide information and the preset language style information, thereby determining the degree of matching between each candidate guide information and each language style information, and then forming a sorting result of multiple candidate guide information corresponding to each language style information.

[0102] Thus, in an embodiment of the present application, the server may determine multiple candidate guidance information corresponding to a voice request sample based on a reference model, and after determining that there are multiple candidate guidance information corresponding to the sorting results of each language style information, perform training on the reference model based on the sorting results of each language style information, so that the reference model can learn the association between each language style information and each candidate guidance information corresponding to the language request sample, thereby ensuring to a certain extent the reliability of the large language model in determining the target guidance information that matches the target language style information based on the target language style information and the voice request.

[0103] In some embodiments of the present application, step 06 includes:

[0104] According to the language style information, the sorting result and the pre-configured first prompt information template, the reference model is trained to obtain a large language model.

[0105] The reference model training module of the implementation mode of the present application trains the reference model according to the language style information, the sorting result and the pre-configured first prompt information template to obtain a large language model.

[0106] The processor of the implementation mode of the present application is also used to train the reference model according to the language style information, the sorting result and the pre-configured first prompt information template to obtain a large language model.

[0107] Specifically, in order to ensure that the reference model can learn the correlation or mapping relationship between the guidance information and the language style information in the early training, the server in the implementation mode of the present application performs and completes the training of the reference model based on the language style information and the pre-configured first prompt (prompt or instruction) information template, so that the reference model can infer the guidance information through the reasoning method indicated or implied by the first prompt information target.

[0108] To more clearly illustrate the implementation method of this application, please refer to Figure 4 and Figure 5 , and see Figure 6 and Figure 7 , Figure 6 and Figure 7 All of them are schematic diagrams of application scenarios in certain embodiments of the present application. Figure 4 , 5 , 6 and 7, after determining that Figure 4 After obtaining the sorting result corresponding to the "detailed" language style, the server can use the sorting result, "detailed" and the first prompt information template to train the reference model.

[0109] Furthermore, in one example, the first prompt information template may include: "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information according to the instructions provided by the user, and extract the key information in the instructions for API search related information. The returned result includes the API name: api, and the key information: arguments. The second step: if the API is unclear, recommend relevant annotation instructions to the user. If it is not unclear, directly return the API result. The return style is: [XX]."

[0110] It can be understood that in the above-mentioned first prompt information template example, '[XX]' can be understood as the language style information to be filled in. For example, when obtaining Figure 4 In the case of the sorting result corresponding to the "detailed" language style information shown, the server can fill "detailed" into '[XX]' in the above example, so as to obtain the following Figure 6 The content shown is: "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information according to the instructions provided by the user, and extract the key information in the instructions for API search related information. The returned results include API name: api, key information: arguments. Step 2: If the API is unclear, recommend relevant annotation instructions to the user. If it is not unclear, directly return the API result. The return style is: detailed."

[0111] Similarly, when we get Figure 5 In the case of the sorting result corresponding to the "concise" language style information shown in FIG. 1 , the server may fill "concise" into '[XX]' in the above example, thereby obtaining the following Figure 7 The content shown is: "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information according to the instructions provided by the user, and extract the key information in the instructions for API search related information. The returned results include API name: api, key information: arguments. Step 2: If the API is unclear, recommend relevant annotation instructions to the user. If it is not unclear, directly return the API result. The return style is: concise."

[0112] Therefore, if Figure 6 and Figure 7 As shown, when the server language style information, the sorting results of multiple candidate guidance information corresponding to the language style information, and the model training information composed of the first prompt information template are used to train the reference model, the reference model can predict the guidance information that matches the language style information based on the reasoning method indicated or implied by the first prompt information template.

[0113] Furthermore, the server may update the reference model accordingly based on the guidance information prediction results of the reference model and the sorting results of multiple candidate guidance information corresponding to the language style information, so that the reference model can learn the association, or mapping relationship, between each guidance information it can generate and each language style information received after multiple rounds of updates.

[0114] Thus, in the implementation mode of the present application, the server can train the reference model according to each type of language style information and the sorting result corresponding to each type of language style information, as well as the pre-configured first prompt information template, so that the training effect of the reference model is guaranteed to a certain extent.

[0115] See also Figure 8 In certain embodiments of the present application, step 06 includes:

[0116] 060: Determine first guidance information and second guidance information corresponding to each type of language style information according to the sorting result corresponding to each type of language style information;

[0117] 061: According to the first guidance information and the second guidance information, the reference model is trained to obtain a large language model.

[0118] The reference model training module of the implementation mode of the present application is also used to determine the first guidance information and the second guidance information corresponding to each language style information according to the sorting result corresponding to each language style information, and to train the reference model according to the first guidance information and the second guidance information to obtain a large language model.

[0119] The processor of the embodiment of the present application is also used to determine the first guidance information and the second guidance information corresponding to each language style information according to the sorting result corresponding to each language style information, and to train the reference model according to the first guidance information and the second guidance information to obtain a large language model.

[0120] Specifically, in order to ensure that the large language model can output a guidance information that matches the target language style information when receiving a voice request and the target language style information, the server of the implementation mode of the present application can also determine the first guidance information and the second guidance information corresponding to each language style information among the multiple candidate guidance information based on the sorting results of the multiple candidate guidance information relative to each language style information.

[0121] In some embodiments of the present application, "the first guidance information corresponding to the language style information" can be understood as: among the multiple candidate guidance information predicted by the reference model based on the voice request sample, the guidance information with the highest matching degree with the language style information and thus the highest sequence number. Figure 4 , 5 , 6 and 7, the first guide information corresponding to "detailed" is res 1 The first guide information corresponding to "simple" is res 3 .

[0122] And, in certain embodiments of the present application, "the first guide information corresponding to the language style information" can be understood as: among the multiple candidate guide information predicted by the reference model based on the speech request sample, the guide information whose matching degree with the language style information is lower than that of the first guide information and whose sequence number is lower than that of the first guide information. Figure 4 , 5 , 6 and 7, the second guide information corresponding to "detailed" is res 2 and res 3 The second guide information corresponding to "simple" is res 1 and res 2 .

[0123] Furthermore, the server may train a reference model based on the first guide information and the second guide information corresponding to the voice request sample and the language style information, so that the probability of the reference model predicting the first guide information is higher than the probability of the reference model predicting the second guide information.

[0124] Thus, in an embodiment of the present application, the server can determine the first guidance information and the second guidance information corresponding to each language style information from a variety of candidate guidance information determined by the reference model according to the voice request sample based on the sorting results corresponding to each language style information, so that the reference model can learn the difference between the first guidance information and the second guidance information corresponding to each language style information to a certain extent, thereby reliably completing the prediction of the target guidance information after the training is completed.

[0125] See also Fig. 9 In some embodiments of the present application, the reference model includes a target model and a copy model of the target model, and step 061 includes:

[0126] 0610: Determine, according to the target model, the replica model, the language style information and the voice request sample, a first probability that the target model predicts the first guide information and the second guide information corresponding to the language style information, and a second probability that the replica model predicts the first guide information and the second guide information corresponding to the language style information;

[0127] 0611: According to the similarity between the first probability and the second probability, train the target model to obtain a large language model.

[0128] The reference model training module of the implementation mode of the present application is also used to determine the first probability that the target model predicts the first guidance information and the second guidance information corresponding to the language style information, and the second probability that the copy model predicts the first guidance information and the second guidance information corresponding to the language style information based on the target model, the copy model, the language style information and the voice request sample, and to train the target model to obtain a large language model based on the similarity between the first probability and the second probability.

[0129] The processor of the embodiment of the present application is also used to determine the first probability that the target model predicts the first guidance information and the second guidance information corresponding to the language style information, and the second probability that the copy model predicts the first guidance information and the second guidance information corresponding to the language style information based on the target model, the copy model, the language style information and the voice request sample, and to train the target model to obtain a large language model based on the similarity between the first probability and the second probability.

[0130] Specifically, in order to enhance the model performance of the reference model as the number of training rounds increases during the training process, the server of the implementation mode of the present application may be provided with a target model with updateable model parameters and a replica model with fixed model parameters.

[0131] To more clearly illustrate the implementation of this application, please refer to Fig.10 , Fig.10 This is a schematic diagram of a large language model training process in some embodiments of the present application. That is, for the language model before the training process starts (ie, the target model), the server can copy the target model to obtain a copy model.

[0132] Subsequently, for each type of language style information, the server may input the voice request sample and the language style information into the target model and the copy model, thereby determining the probability that the target model predicts the first guidance information and the second guidance information corresponding to the language style information, and determining the probability that the copy model predicts the first guidance information and the second guidance information corresponding to the language style information.

[0133] To clearly illustrate the implementation of the present application, let the probability that the target model predicts the first guide information and the second guide information corresponding to the language style information be P 1 , the probability that the copy model predicts the first guide information and the second guide information corresponding to the language style information is P 2 , then: the server according to P 1 and P 2 Similar, train the target model to obtain the large language model required for implementation.

[0134] It should be noted that the copy model in the implementation manner of the present application can be understood as a language model obtained by copying the target model / reference model during the training phase (or before the execution of step 06).

[0135] Optionally, in certain embodiments of the present application, the target model and the replica model may be understood as two models obtained by loading the server with the same model parameters.

[0136] Optionally, in certain embodiments of the present application, the similarity level Similar may be calculated based on a KL divergence (Kullback-Leibler Divergence) calculation formula.

[0137] Optionally, in one example, the first guidance information is the guidance information with the highest degree of matching with the language style information among multiple candidate guidance information. The second guidance information is the guidance information with a lower degree of matching with the language style information than the first guidance information among multiple candidate guidance information. Then, when the training process is performed based on the above-mentioned similarity Similar, the training objectives may include: the similarity Similar is maintained within a preset range, and the probability that the reference model predicts the first guidance information corresponding to the language style information is higher than the probability that the reference model predicts the second guidance information corresponding to the language style information.

[0138] Optionally, in another example, the first guidance information is the guidance information that has the highest degree of match with the language style information among multiple candidate guidance information and is therefore ranked first. The second guidance information is the guidance information that has a lower degree of match with the language style information than the first guidance information among multiple candidate guidance information and is therefore ranked after the first guidance information. Furthermore, when the training process is performed based on the above-mentioned similarity Similar, the training objectives may include: the similarity Similar is maintained within a preset range, and the probability that the target model predicts the first guidance information corresponding to the language style information is higher than the probability that the target model predicts the second guidance information corresponding to the language style information, and the probability that the copy model predicts the first guidance information corresponding to the language style information is higher than the probability that the copy model predicts the second guidance information corresponding to the language style information.

[0139] In this way, in an embodiment of the present application, the target model can be trained based on the first probability of the target model predicting the first guidance information and the second guidance information, and the second probability of the copy model predicting the first guidance information and the second guidance information, so that the training of the target model can be carried out with reference to the copy model, thereby reducing the possibility of model performance degradation due to training to a certain extent, and the training effect of the target model can be guaranteed.

[0140] In some embodiments of the present application, the first probability includes a first sub-probability corresponding to the first guide information and a second sub-probability corresponding to the second guide information, the second probability includes a third sub-probability corresponding to the first guide information and a fourth sub-probability corresponding to the second guide information, and step 0611 includes:

[0141] According to the similarity between the first sub-probability and the third sub-probability, and the similarity between the second sub-probability and the fourth sub-probability, the target model is trained to obtain a large language model.

[0142] The reference model training module of the implementation mode of the present application is also used to train the target model to obtain a large language model according to the similarity between the first sub-probability and the third sub-probability, and the similarity between the second sub-probability and the fourth sub-probability.

[0143] The processor of the embodiment of the present application is also used to train the target model to obtain a large language model based on the similarity between the first sub-probability and the third sub-probability, and the similarity between the second sub-probability and the fourth sub-probability.

[0144] To clearly illustrate the implementation of the present application, let the probability that the target model predicts the first candidate guidance information corresponding to the language style information be P 1-1 , the probability that the target model predicts the second candidate guidance information corresponding to the language style information is P 1-2 , the probability that the copy model predicts the first candidate guidance information corresponding to the language style information is P 2-1 , the probability that the copy model predicts the second candidate guidance information corresponding to the language style information is P 2-2 , then: According to P 1-1 Relative to P 1-2 The similarity of Simi 1 , and P 2-1 Relative to P 2-2 The similarity of Simi 2 , train the target model to obtain the large language model required by the implementation method of this application.

[0145] Optionally, in the implementation manner of the present application, the server is based on the above P 1-1 , P 1-2 , P 2-1 , P 2-2 , Simi1 and Simi2 perform the target model training process, and the training objectives may include: 1-1 Can be higher than P 1-2 , and P 1-1 With P 1-2 The difference between Simi1 and Simi2 increases until convergence, and the similarity between Simi1 and Simi2 is maintained within a preset range.

[0146] Optionally, in some other implementations of the present application, the server is based on the above P 1-1 , P 1-2 , P 2-1 , P 2-2 , Simi1 and Simi2 perform the training process of the target model. The training objectives may include: for the target model, P 1-1 Higher than P 1-2 , and P 1-1 With P 1-2 The difference between the two increases until convergence. For the replica model, P 2-2 Higher than P 2-1 , and P 2-2 With P 2-1 The difference between the two increases until it converges.

[0147] Optionally, in certain embodiments of the present application, the server may calculate the similarity between the above Simi1 and the above Simi2, and train the target model based on the similarity. The similarity may be calculated based on the KL divergence (Kullback-Leibler Divergence) calculation formula.

[0148] Thus, in the implementation mode of the present application, the server can perform training of the target model based on the similarity between the first sub-probability and the third sub-probability, as well as the similarity between the second sub-probability and the fourth sub-probability, with reference to the prediction results of the copy model. This can reduce the possibility of performance degradation of the target model due to training to a certain extent, and the training effect of the target model can be guaranteed.

[0149] Optional, see Fig.11 , Fig.11 Schematic diagram of the large language model training process in certain embodiments of the present application. That is, for each voice request sample received in advance (taking "adjust the air conditioner" as an example), the server can input the voice request sample into the reference model so that the reference model can infer the guidance information according to the voice request sample, thereby forming multiple candidate guidance information, such as Figure 3 res 1 、res 2 and res 3 .

[0150] Subsequently, the server may determine the ranking results of the plurality of candidate guide information relative to each language style information according to the received ranking operation instruction, such as Figure 4 The sorting results corresponding to "Details" in 3 -res 2 -res 1 ”, or Figure 5 The sorting result corresponding to "simple" in "res1 -res 2 -res 2 ”.

[0151] Then, according to the ranking result of each type of language style information, the server can determine the first candidate guide information and the second candidate guide information corresponding to each type of language style information among the multiple candidate guide information generated by the reference model according to the speech request sample. Figure 4 The "res 3 -res 2 -res 1 "For example, "res 3 -res 2 -res 1 " can reflect the relevance of each candidate guide information to the language style of "detailed", so 3 More related to "detailed" than res 1 The degree of relevance to "details" and 1 More related to "detailed" than res 2 The degree of relevance to "details", and thus res 3 Can be regarded as the first guide information, res 1 and res 2 Can be regarded as the second guidance information.

[0152] Then, the server can construct a guidance information combination corresponding to each language style information according to the first candidate guidance information and the second candidate guidance information corresponding to each language style information. Specifically, let the first guidance information be Chosen and the second guidance information be Rejected. Figure 4 For example, Chosen is res 1 ,Rejected is res 2 or res 3 Therefore, the guidance information combination (Chosen, Rejected) corresponding to the language style information of "detailed" may include (res 1 ,res 2 ) and (res 1 ,res 3 ). Similarly, Figure 5 For example, the guidance information combination (Chosen, Rejected) corresponding to the language style information of "concise" may include (res3,res 2 ) and (res 3 ,res 1 ).

[0153] Afterwards, for each type of language style information, the server may input the language style information and the speech request sample into the reference model and its copy, or in other words, input the language style information and the speech request sample into the target model and the copy model, so as to determine the first probability of Chosen and Rejected in the guidance information combination corresponding to the language style information predicted by the target model, and determine the second probability of Chosen and Rejected in the guidance information combination corresponding to the language style information predicted by the copy model, and train the target model according to the similarity between the first probability and the second probability. For details, please refer to Fig.10 .

[0154] Finally, when the pre-defined training termination conditions are met (such as the number of training rounds reaches the preset number, the loss function value converges, etc.), it can be confirmed that the target model training is completed, thereby obtaining a large language model.

[0155] Optionally, in order to ensure that the target model and the replica model can reliably predict reasonable or appropriate results based on the language style information and the voice request sample, the server can input the language style information, the voice request sample and the pre-configured first prompt information template into the target model and the replica model, thereby ensuring that the target model and the replica model can reliably complete the reasoning and ensure the training effect of the target model. The specific content of the first prompt information template can be set according to the actual situation. For example, you can refer to Figure 6 and Figure 7 .

[0156] In certain embodiments of the present application, step 02 includes:

[0157] When a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information is determined according to the large language model, the current voice request, the target language style information and a pre-configured second prompt information template.

[0158] The determination module of the implementation mode of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the target language style information and the pre-configured second prompt information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request.

[0159] The processor of the embodiment of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the target language style information and the pre-configured second prompt information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request.

[0160] Specifically, in order to ensure the credibility of the prediction results of the large language model and to ensure that the large language model can reliably output target guidance information based on the current voice request and target language style information, the server of the implementation mode of the present application can also input the current voice request, target language style information and a pre-configured second prompt information template into the large language model, so that the large language model can understand and infer the current voice request and target language style information according to the reasoning method indicated or implied by the second prompt information template, and thus output the target guidance information.

[0161] It can be understood that the prompt information template can be understood as a natural language text segment, which may include "natural language text that describes the downstream tasks that the model needs to handle" and may also include "explanatory text that describes how the model should reason about the input information."

[0162] It is also understandable that the second prompt information template in the implementation mode of the present application is content that can be set according to actual conditions. Exemplarily, in one example, the second prompt information template may include: "If you are an intelligent voice assistant, recommend relevant annotation instructions to the user according to the instructions provided by the user, and the return style is: [X]."

[0163] Among them, the "instructions provided by the user" in the example of the above prompt information template can be understood as the current voice request in the implementation mode of this application. And, the "[X]" in the example of the above prompt information template can be understood as a fill-in column for the target language style information. For example, when the target language style information is "detailed", the above second prompt information template can be changed to: "If you are an intelligent voice assistant, recommend relevant annotation instructions to the user based on the instructions provided by the user, and the return style is: detailed."

[0164] Optionally, in certain embodiments of the present application, please refer to Figure 6 and Figure 7 That is, the server completes the training of the large language model based on the language style information, the voice request sample and the first prompt information template. Furthermore, the first prompt information target and the second prompt information template in the implementation mode of the present application may be the same, so as to ensure that the ability of the large language model to "output guidance information according to the prompt information template, voice request and language style information" learned through training can be fully utilized.

[0165] Thus, in the implementation mode of the present application, the large language model can perform natural language understanding and processing on the current voice request and the target language style information based on the instructions of the prompt information template, thereby generating target guidance information corresponding to the current voice request and matching the target language style information, thereby ensuring the prediction accuracy of the large language model to a certain extent.

[0166] In certain embodiments of the present application, the second prompt information template includes a first sub-information template and a second sub-information template. The above step of determining the target guidance information according to the large language model, the current voice request, the target language style information and the pre-configured second prompt information template when the vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request includes:

[0167] When the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-information template, the target guidance information is determined based on the large language model, the current voice request, the prompt information template, the target language style information and the second sub-information template.

[0168] The determination module of the implementation mode of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the prompt information template, the target language style information and the second sub-information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-information template.

[0169] The processor of the embodiment of the present application is also used to determine the target guidance information based on the large language model, the current voice request, the prompt information template, the target language style information and the second sub-information template when the vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-information template.

[0170] Specifically, in order to ensure the credibility of the inference results of the large language model in the implementation mode of the present application, and to avoid the situation where a "functional module for determining the vehicle control instruction corresponding to the voice request" needs to be separately set up in the server, the prompt information template in the implementation mode of the present application can be divided into two parts, one part can be understood as the first sub-prompt information template for prompting the large language model to identify the "vehicle control instruction corresponding to the current voice request". The other part can be understood as the second sub-prompt information template for prompting the large language model to generate "target guidance information corresponding to the current voice request and matching the target language style information".

[0171] In one example, a prompt information template including a first sub-prompt information template and a second sub-prompt information template includes "If you are an intelligent voice assistant, the first step is to select an API that can be executed or provide information based on the instructions provided by the user, and extract the key information in the instructions for API search for relevant information. The returned results include, API name: api, key information: arguments. Step 2: If the API is unclear, recommend relevant annotation instructions to the user. If it is not unclear, directly return the API result. The return style is: [XX]".

[0172] Further, it can be understood that the "first step, according to the instructions provided by the user, select the API that can be executed or provide information, and extract the key information in the instructions for API search related information. The returned results include, API name: api, key information: arguments" in the above example can be the first sub-prompt information template.

[0173] Also, in the above example, "Step 2: If the API is unclear, recommend relevant annotation instructions to the user. If it is not unclear, directly return the API result. The return style is: [XX]" can be the second sub-prompt information template.

[0174] For example, see Fig.12 , Fig.12 This is a flow chart of the voice interaction method in some embodiments of the present application. That is, when the current voice request forwarded by the vehicle received by the server is "adjust the air conditioner", and the preset target language style information is "detailed", the server inputs "adjust the air conditioner", the second prompt information template and "detailed" into the large language model.

[0175] When the large language model confirms that the API corresponding to "adjust the air conditioner" is unclear based on the first sub-prompt information template, the large language model can infer the recommended voice request (i.e., "related annotation instructions") and target guidance information generation corresponding to "adjust the air conditioner" based on the second sub-prompt information template and the target language style information.

[0176] Furthermore, when inferring recommended voice requests such as "adjust the air conditioner to 20 degrees", "adjust the air volume to the third gear" and "open the air conditioner page", the large language model can complete and improve these recommended voice requests to form target guidance information that matches the "detailed" language style, such as "I can't understand what you mean. You can say standard points, such as: adjust the air conditioner to 20 degrees, adjust the air volume to the third gear, open the air conditioner page".

[0177] Furthermore, after the target guidance information is sent to the vehicle, the vehicle can play the target guidance information to prompt the user to complete the adjustment of the current voice request "adjust the air conditioner" based on the target guidance information.

[0178] Optionally, it is understood that in order to ensure the reliability of the large language model, the server of the embodiment of the present application can perform model training based on the prompt information template, samples and sample labels during the early training of the large language model, and in the post-landing reasoning process of the large language model, the large language model performs corresponding reasoning work through the prompt information template and voice request. It is understandable that the prompt information template used in the early training process and the prompt information template used in the post-landing reasoning process can be the same.

[0179] Thus, in the implementation mode of the present application, the large language model can confirm the correspondence between the current voice request and the vehicle control instruction based on the first sub-prompt information template and the second sub-prompt information template in the prompt information template, and when it is confirmed that the current voice request fails to correspond to the vehicle control instruction, confirm the target guidance information corresponding to the current voice request, thereby ensuring the accuracy of the target guidance information inferred by the large language model.

[0180] Optional, see Fig.13 , Fig.13 The flowchart of the voice interaction method in some embodiments of the present application is shown in FIG. That is, in order to ensure that the large language model can identify whether the current voice request corresponds to the vehicle control instruction, the embodiment of the present application is provided with corresponding downstream tasks and prompts, so that the large language model can distinguish between "voice requests corresponding to vehicle control instructions" and "voice requests not corresponding to vehicle control instructions" and perform natural language processing respectively.

[0181] Specifically, Fig.13 As shown, the server can train a basic model with certain natural language processing capabilities based on the second voice request sample, the API label corresponding to the second voice request sample, and the prompt corresponding to the second voice request sample, so that the basic model can distinguish between "voice requests with API label unclear" and "voice requests of other APIs other than label unclear".

[0182] It can be understood that the prompt corresponding to the second voice request sample is "If you are an intelligent voice assistant, you can select the API that can execute this information according to the instructions provided by the user, and extract the key information in this instruction for API search related information. The returned results include: API name: api, key information: arguments. Command: Help me open the window. Output: {API: WindowsOpen, arguments: {drivce: car window}}", and this prompt can be set according to actual conditions. And, the "Command: Help me open the window Output: {API: WindowsOpen, arguments: {drivce: car window}}" in the prompt can be understood as an example.

[0183] Furthermore, the server may train the basic model based on the third voice request sample, the vehicle control instruction label corresponding to the third voice request sample, and the prompt corresponding to the second voice request sample so that the basic model can infer the vehicle control instructions corresponding to the "voice requests of different APIs".

[0184] It can be understood that the prompt corresponding to the third voice request sample is "If you are an intelligent voice assistant, you can generate standard user instructions based on the function points, API and key information arguments provided by the user. Input: {API: WindowsOpen, Arguments: {device: car window}}. Output: Standard instruction: Open the car window, open the car window", and the prompt can be set according to the actual situation. And, the "Input: {API: WindowsOpen, Arguments: {device: car window}}. Output: Standard instruction: Open the car window, open the car window" in the prompt can be understood as an example.

[0185] In addition, the server can train the basic model based on the "fourth voice request sample that does not correspond to the vehicle control command", the recommended voice request label corresponding to the fourth voice request sample, and the prompt corresponding to the second voice request sample so that when the basic model faces a voice request with an API of unclear, it can infer the recommended voice request corresponding to the voice request (or in other words, a semantically similar voice request) (that is, Fig.13 'Standard user instructions' in .

[0186] It can be understood that the prompt corresponding to the fourth voice request sample is "If you are an intelligent voice assistant, you can provide users with standard instructions based on the incomplete or unclear instructions provided by the user, and provide users with choices. Input: car window. Output: standard instruction: open car window, close car window", and the prompt can be set according to actual conditions. And, the "Input: car window. Output: standard instruction: open car window, close car window" in the prompt can be understood as an example.

[0187] It can also be understood that after completing the training of the basic model based on the above-mentioned second voice request sample, third voice request sample and fourth voice request sample, and the labels and prompts corresponding to these three voice request samples, the first reference model in the implementation manner of the present application can be obtained.

[0188] Optionally, in some embodiments of the present application, the basic model can be understood as a conventional natural language processing task applicable to general fields, but a model that is relatively unfamiliar to voice request processing tasks in the vehicle field. Therefore, the server of the embodiment of the present application can also perform Fig.13 Before the training shown, the basic model is subjected to corresponding pre-training and / or knowledge injection so that the basic model can understand the semantics of unique characters, words and sentences in the vehicle field, thereby improving the adaptability of the basic model to voice request processing tasks in the vehicle field.

[0189] The present application also provides a computer-readable storage medium storing a computer program, which implements the above-mentioned voice interaction method when the computer program is executed by one or more processors.

[0190] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0191] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0192] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A voice interaction method, characterized in that: include: Receive the current voice request forwarded by the vehicle; In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, target guidance information for guiding the user to adjust the current voice request is determined according to the large language model, the current voice request and the acquired preset target language style information, the target guidance information matches the target language style information, the large language model is pre-trained, and can generate guidance information prediction results matching each of the language style information according to the voice request sample and the multiple language style information; The target guidance information is fed back to guide the user to complete the voice interaction.

2. The method according to claim 1, characterized in that The training steps of the large language model include: Determining a plurality of candidate guidance information according to the voice request sample and a predetermined reference model; Determine a ranking result of the plurality of candidate guide information corresponding to each type of the language style information; According to the ranking results corresponding to each type of language style information, the reference model is trained to obtain the large language model.

3. The method according to claim 2, characterized in that The step of training the reference model according to the ranking result corresponding to each type of the language style information to obtain the large language model includes: The reference model is trained according to the language style information, the sorting result and a pre-configured first prompt information template to obtain the large language model.

4. The method according to claim 2, characterized in that: The step of training the reference model according to the ranking result corresponding to each type of the language style information to obtain the large language model includes: Determining first guidance information and second guidance information corresponding to each type of language style information according to the sorting result corresponding to each type of language style information; The reference model is trained according to the first guide information and the second guide information to obtain the large language model.

5. The method according to claim 4, characterized in that The reference model includes a target model and a replica model of the target model, and the training of the reference model according to the first guidance information and the second guidance information to obtain the large language model includes: Determine, according to the target model, the replica model, the language style information and the voice request sample, a first probability that the target model predicts the first guidance information and the second guidance information corresponding to the language style information, and a second probability that the replica model predicts the first guidance information and the second guidance information corresponding to the language style information; According to the similarity between the first probability and the second probability, the target model is trained to obtain the large language model.

6. The method according to claim 5, characterized in that The first probability includes a first sub-probability corresponding to the first guide information and a second sub-probability corresponding to the second guide information, the second probability includes a third sub-probability corresponding to the first guide information and a fourth sub-probability corresponding to the second guide information, and training the target model to obtain the large language model according to the similarity between the first probability and the second probability includes: According to the similarity between the first sub-probability and the third sub-probability, and the similarity between the second sub-probability and the fourth sub-probability, the target model is trained to obtain the large language model.

7. The method according to claim 1, characterized in that In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining target guidance information for guiding the user to adjust the current voice request according to the large language model, the current voice request and the acquired preset target language style information, includes: In the case that a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, the target guidance information is determined according to the large language model, the current voice request, the target language style information and a pre-configured second prompt information template.

8. The method according to claim 7, characterized in that The second prompt information template includes a first sub-information template and a second sub-information template, and when a vehicle control instruction corresponding to the current voice request cannot be determined according to the current voice request, determining the target guidance information according to the large language model, the current voice request, the target language style information and a pre-configured second prompt information template includes: When a vehicle control instruction corresponding to the current voice request cannot be determined based on the current voice request, the large language model and the first sub-information template, the target guidance information is determined based on the large language model, the current voice request, the prompt information template, the target language style information and the second sub-information template.

9. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images