Large language model training method, vehicle voice interaction method and server

Abstract

The invention discloses a big language model training method, a vehicle voice interaction method, a server and a computer readable storage medium. The method comprises the following steps: constructing a first cue word, wherein the first cue word is configured to guide a large language model to perform multi-intention recognition processing; then, according to the first prompt word and a preset training data set, determining a first optimized training data set; and finally, training the large language model according to the first optimization training data set. Therefore, by constructing the first cue word and the first optimization training data set, the large language model can accurately perform multi-intention recognition and slot filling, so that the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a large language model training method, a vehicle voice interaction method, a server and a computer-readable storage medium. Background Art

[0002] In the related art, the in-vehicle voice assistant is used to interact with the user to facilitate the user's operation. However, for voice requests including multiple intents, the reasoning ability of the in-vehicle voice assistant cannot recognize all intents, resulting in incomplete understanding of the intent and poor user experience. Summary of the invention

[0003] The present application provides a large language model training method, a vehicle voice interaction method, a server and a computer-readable storage medium.

[0004] The present application embodiment provides a method for training a large language model, the method comprising:

[0005] Constructing a first prompt word, wherein the first prompt word is configured to guide the large language model to perform multi-intent recognition processing;

[0006] Determining a first optimized training data set according to the first prompt word and a preset training data set;

[0007] The large language model is trained according to the first optimized training data set.

[0008] In this way, the server constructs a first prompt word, and the first prompt word is configured to guide the large language model to perform multi-intent recognition processing. Then, the server determines a first optimized training data set based on the first prompt word and a preset training data set. Finally, the server trains the large language model based on the first optimized training data set. In this way, by constructing the first prompt word and the first optimized training data set, the large language model can accurately perform multi-intent recognition and slot filling, thereby improving the user experience.

[0009] In some embodiments, the preset training data set includes a plurality of preset training data, and determining the first optimized training data set according to the first prompt word and the preset training data set includes:

[0010] Determining first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data;

[0011] The first optimized training data set is determined according to the first optimized training data.

[0012] In this way, the server determines the first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data. Then, the server determines the first optimized training data set according to the first optimized training data. In this way, the missing data information can be supplemented through the historical conversation data, and the accuracy of multi-intent recognition and slot filling can be improved, thereby improving the user experience.

[0013] In certain embodiments, the method further comprises:

[0014] Constructing a second prompt word according to the first prompt word, the preset intent list and the preset entity set;

[0015] Determining a second optimized training data set according to the second prompt word and the preset training data set;

[0016] The large language model is trained according to the second optimized training data set.

[0017] In this way, the server constructs a second prompt word according to the first prompt word, the preset intent list and the preset entity set. Then, the server determines a second optimized training data set according to the second prompt word and the preset training data set. Finally, the server trains the large language model according to the second optimized training data set. In this way, by limiting the reasoning scope of the large language model through the preset intent list and the preset entity set, the probability of the large language model outputting the wrong entity type can be reduced, and the accuracy of the multi-intent recognition and slot filling of the large language model can be improved, thereby improving the user experience.

[0018] In some embodiments, the preset training data set includes a plurality of preset training data, and determining the second optimized training data set according to the second prompt word and the preset training data set includes:

[0019] Determining second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data;

[0020] The second optimized training data set is determined according to the second optimized training data.

[0021] In this way, the server determines the second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data. Then, the server determines the second optimized training data set according to the second optimized training data. In this way, by using the historical conversation data, the large language model can understand the user's intentions and preferences, improve the accuracy of multi-intent recognition and slot filling, and thus improve the user experience.

[0022] In certain embodiments, the method further comprises:

[0023] Determine an intent recognition training data set and an entity recognition training data set according to the second prompt word and the preset training data set;

[0024] The second preset large language model is trained according to the second optimized training data set, the intent recognition training data set and the entity recognition training data set.

[0025] In this way, the server determines the intent recognition training data set and the entity recognition training data set based on the second prompt word and the preset training data set. Then, the server trains the second preset large language model based on the second optimized training data set, the intent recognition training data set and the entity recognition training data set. In this way, the large language model is trained with the intent recognition training data set and the second optimized training data set to improve the multi-intent recognition capability of the large language model, and the large language model is trained with the entity recognition training data set and the second optimized training data set to improve the slot filling capability of the large language model, so that the large language model can accurately understand the user's intentions and needs, and provide more accurate services to improve the user experience.

[0026] In some embodiments, determining the intention recognition training data set and the entity recognition training data set according to the second prompt word and the preset training data set includes:

[0027] Determine an intent recognition prompt word and an entity recognition prompt word according to the second prompt word;

[0028] Determining an intention recognition training data set according to the intention recognition prompt word and the preset training data set;

[0029] An entity recognition training data set is determined according to the entity recognition prompt word and the preset training data set.

[0030] In this way, the server determines the intent recognition prompt word and the entity recognition prompt word based on the second prompt word. Next, the server determines the intent recognition training data set based on the intent recognition prompt word and the preset training data set. Finally, the server determines the entity recognition training data set based on the entity recognition prompt word and the preset training data set. In this way, by using the intent recognition prompt word, the large language model can focus on learning the knowledge of intent recognition, thereby improving the accuracy of intent recognition. By using the entity recognition prompt word, the large language model can focus on learning the distribution of entity types, thereby improving the accuracy of slot filling.

[0031] In certain embodiments, the method further comprises:

[0032] Based on a preset ratio, configuring the amount of first optimized training data in the first optimized training data set and the amount of second optimized training data in the second optimized training data set, and determining a third optimized training data set;

[0033] The large language model is trained according to the third optimized training data set.

[0034] In this way, based on the preset ratio, the server configures the number of first optimized training data in the first optimized training data set and the number of second optimized training data in the second optimized training data set, and determines the third optimized training data set. Then, the server trains the large language model according to the third optimized training data set. In this way, by mixing different types of training data, the generalization ability of the large language model can be improved, so that it can better adapt to various different input data, and the robustness of the large language model can be improved. The large language model can accurately understand the user's intentions and needs, and provide more accurate services, thereby improving the user experience.

[0035] The embodiment of the present application provides a vehicle voice interaction method based on a preset large language model, wherein the preset large language model is trained based on the above-mentioned training method, and the method includes:

[0036] Get voice request;

[0037] Determining a vehicle control instruction based on the preset large language model and according to the voice request;

[0038] The vehicle control command is sent to the vehicle to complete the voice interaction.

[0039] In this way, the server obtains the voice request. Then, based on the preset large language model, the server determines the vehicle control command according to the voice request. Finally, the server sends the vehicle control command to the vehicle to complete the voice interaction. In this way, based on the preset large language model, the user can control the vehicle to perform various operations through voice commands, improving driving convenience.

[0040] An embodiment of the present application provides a server, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above method is implemented.

[0041] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above method are implemented.

[0042] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 It is one of the flowcharts of the training method of the large language model in certain embodiments of the present application;

[0045] Figure 2 This is the second flow chart of the training method of a large language model in certain embodiments of the present application;

[0046] Figure 3 This is the third flow chart of the training method of a large language model in certain implementation modes of the present application;

[0047] Figure 4 This is a fourth flowchart of a method for training a large language model in certain embodiments of the present application;

[0048] Figure 5 This is a fifth flowchart of a method for training a large language model in certain embodiments of the present application;

[0049] Figure 6 This is the sixth flow chart of the training method of a large language model in certain embodiments of the present application;

[0050] Figure 7 This is the seventh flow chart of the method for training a large language model in certain embodiments of the present application;

[0051] Figure 8 It is a flowchart of the voice interaction method of certain embodiments of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0053] In smart car systems, in-car voice assistants, as a convenient way of human-computer interaction, have greatly improved driving experience and safety. However, traditional in-car voice assistants can only recognize one intent and corresponding slot in a user's request sentence. When processing voice requests with multiple intents, they often have insufficient reasoning capabilities and are unable to recognize multiple intents and slots, resulting in incomplete intent recognition and inability to provide accurate and applicable feedback, which affects the user experience.

[0054] For example, when the user voice request is "Turn on the air conditioner and turn on the reading light", after the model receives the user voice request "Turn on the air conditioner and turn on the reading light", it processes the user voice request as follows:

[0055] "instruction": "Identify the user's intent in the following sentence, and the slot information corresponding to the intent.",

[0056] "input": turn on the air conditioner and the reading light",

[0057] "output": {

[0058] "intent": "Open device function",

[0059] "slot":{"device name":"air conditioner"}

[0060] }.

[0061] The in-car voice assistant may only recognize ""intent":"turn on device function","slot":{"device name":"air conditioner"}", but fail to recognize ""intent":"turn on device function","slot":{"device name":"reading light"}". The incomplete intent recognition cannot fully understand all the semantics of the user's voice request, resulting in the user's needs not being fully processed and a poor user experience.

[0062] Based on the above questions, please refer to Figure 1 The present application embodiment provides a method for training a large language model, the method comprising:

[0063] 011: Construct the first prompt word;

[0064] 012: Determine a first optimized training data set according to the first prompt word and the preset training data set;

[0065] 013: Train the large language model according to the first optimized training data set.

[0066] The embodiment of the present application also provides a server, including a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to construct a first prompt word. And according to the first prompt word and a preset training data set, a first optimized training data set is determined. And according to the first optimized training data set, a large language model is trained.

[0067] The embodiment of the present application also provides a model training device. The training method of the large language model of the embodiment of the present application can be implemented by the model training device of the embodiment of the present application. Specifically, the model training device includes a construction module, a determination module and a training module. The construction module is used to construct a first prompt word. The determination module is used to determine a first optimized training data set based on the first prompt word and a preset training data set. The training module is used to train the large language model based on the first optimized training data set.

[0068] Specifically, in the related art, for voice requests that include multiple intents, the voice request can be split and processed using rules or models to obtain multiple clauses, and then single intent recognition and slot recognition are performed on each clause. For example, for the user voice request "turn on the air conditioner, and turn on the reading light", the voice request will be split first to obtain the clauses "turn on the air conditioner" and "and turn on the reading light", and then single intent recognition and slot recognition are performed on the clauses "turn on the air conditioner" and "and turn on the reading light" respectively, and "intent: "turn on device function", slot: {"device name": "air conditioner"}" and "intent: "turn on device function", slot: {"device name": "reading light"}" are obtained. However, when processing user voice requests that include multiple intents in one sentence, the above method may not be able to perform multi-intent and slot recognition, which ultimately results in incomplete understanding of user intent. For example, the user voice request is "turn on the air conditioner and reading light". The user voice request can no longer be split, but it still includes two intents. Single intent recognition and slot recognition may only recognize "intent": "turn on device function", "slot": {"device name": "air conditioner"}", but cannot recognize "intent": "turn on device function", "slot": {"device name": "reading light"}". In addition, the above method needs to be performed serially, that is, first split the clauses and then perform intent recognition and slot filling, which takes a long time and affects the user experience.

[0069] The large language model obtained by fine-tuning the training method of the large language model provided in the embodiment of the present application can recognize the voice request containing multiple intentions and fully understand the user's semantics when processing such a user voice request containing multiple intentions. In addition, it avoids the serial processing of splitting clauses and then performing recognition, which is more efficient and time-saving, thereby quickly understanding the user's intentions and needs, providing more accurate services, and improving the user experience.

[0070] The first prompt word refers to the prompt word constructed in the vehicle intelligent system to guide the large language model to perform multi-intent recognition processing. The first prompt word clearly defines the tasks that the large language model needs to complete during the fine-tuning training process and specifies the output format. The first prompt word can guide the identification of the number of intents of the user's voice request, the identification of all intents of the user's voice request, the identification of the entity and type of the user's voice request, and specifies the output format. In some embodiments, the first prompt word may be as follows:

[0071] First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified. At the same time, all entities and entity pair types under each intent need to be identified and assembled into JSON format for output. The output format is: [{“intent”: Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1':'entity 1','entity type 2':'entity 2']}}.

[0072] The preset training data set refers to the sample data set used to fine-tune the large language model. These data sets include label information associated with vehicle functions, such as intent labels, entity labels, etc., which are used to guide the large language model to learn to recognize and fill in intents and entities, and improve the accuracy of intent recognition and entity filling.

[0073] The first optimized training data set refers to a training data set generated according to the first prompt word and the preset training data set, which can be used to train the multi-intent recognition capability of the large language model. It should be noted that the data format of the first optimized training data in the first optimized training data set can be Alpaca format, including instruction field, input field, history field and output field. Among them, the instruction field refers to the task description required of the large language model: to generate relevant questions and answers based on the input text data extraction. The input field refers to the user input text data, that is, the user voice request. The history field refers to the historical conversation data of the most recent N rounds. For the convenience of explanation, the implementation method of this application uses the historical conversation data of the previous round as the history field. The output field refers to the result output by the large language model according to the requirements of the instruction field, that is, the result that the large language model is expected to output, including all intents contained in the user voice request, as well as the entities under each intent and the type information of the entity.

[0074] In some embodiments, the first optimized training data in the first optimized training data set may be as follows:

[0075] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified. At the same time, all entities and entity pair types under each intent need to be identified and assembled into JSON format for output. The output format is: [{"intent": " Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0076] "input": "Turn on the air conditioner and play song A",

[0077] "history": [],

[0078] "output": [

[0079] {"intent":"Open device function","slot":["device name":"air conditioner"]},

[0080] {"intent":"Play music","slot":["song name":"Song A"]}

[0081] ].

[0082] The server constructs a first prompt word, which can guide the large language model to perform multi-intent recognition processing. By splitting the user voice request including multiple intents, integrating the single sentence intent recognition task and the slot filling task into the first prompt word, the large language model can be guided to recognize all intents and the slots corresponding to the intents contained in the user voice request at one time.

[0083] Next, the server determines a first optimized training data set according to the first prompt word and the preset training data set.

[0084] Finally, the server trains the large language model according to the first optimized training data set.

[0085] In summary, in the training method and server of the large language model provided in the embodiment of the present application, the server constructs a first prompt word, and the first prompt word is configured to guide the large language model to perform multi-intent recognition processing. Then, the server determines the first optimized training data set based on the first prompt word and the preset training data set. Finally, the server trains the large language model based on the first optimized training data set. In this way, by constructing the first prompt word and the first optimized training data set, the large language model can accurately perform multi-intent recognition and slot filling, thereby improving the user experience.

[0086] See also Figure 2 In some embodiments, the preset training data set includes a plurality of preset training data, and step 012 (determining a first optimized training data set according to the first prompt word and the preset training data set) includes:

[0087] 0121: determining first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data;

[0088] 0122: Determine a first optimized training data set according to the first optimized training data.

[0089] In some embodiments, the determination module is further configured to determine first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data, and determine a first optimized training data set according to the first optimized training data.

[0090] In some embodiments, the processor is further configured to determine first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data, and determine a first optimized training data set according to the first optimized training data.

[0091] Specifically, historical conversation data refers to the most recent one or several conversation records between the user and the system, which can provide contextual information for the large language model, help the large language model understand the user's current intentions and slots, identify the inheritance relationship between intentions and slots, understand the conversation scenario, and refer to personalized information.

[0092] The server determines the first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data. It should be noted that if there is no historical conversation data, the history field can be empty, that is, no historical conversation data is input. For example, the first optimized training data may be as follows:

[0093] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified. At the same time, all entities and entity pair types under each intent need to be identified and assembled into JSON format for output. The output format is: [{"intent": " Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0094] "input": "open",

[0095] "history": [air conditioning and lighting],

[0096] "output": [

[0097] {"intent":"Open device function","slot":["device name":"air conditioner"]},

[0098] {"intent":"Turn on device function","slot":["device name":"light"]}

[0099] ].

[0100] The first optimization training data may also be as follows:

[0101] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified. At the same time, all entities and entity pair types under each intent need to be identified and assembled into JSON format for output. The output format is: [{"intent": " Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0102] "input": "Turn on the air conditioner and play song A",

[0103] "history": [],

[0104] "output": [

[0105] {"intent":"Open device function","slot":["device name":"air conditioner"]},

[0106] {"intent":"Play music","slot":["song name":"Song A"]}

[0107] ].

[0108] Next, the server determines a first optimized training data set according to the first optimized training data.

[0109] In this way, historical conversation data can be used to supplement missing data information, improve the accuracy of multi-intent recognition and slot filling, and thus improve the user experience.

[0110] See also Figure 3 In some embodiments, the method further comprises:

[0111] 014: construct a second prompt word according to the first prompt word, the preset intent list and the preset entity set;

[0112] 015: Determine a second optimized training data set according to the second prompt word and the preset training data set;

[0113] 016: Train the large language model based on the second optimized training data set.

[0114] In some embodiments, the construction module is further used to construct a second prompt word according to the first prompt word, the preset intent list and the preset entity set. The determination module is further used to determine a second optimized training data set according to the second prompt word and the preset training data set. The training module is further used to train the large language model according to the second optimized training data set.

[0115] In some embodiments, the processor is further configured to construct a second prompt word according to the first prompt word, the preset intent list, and the preset entity set, and to determine a second optimized training data set according to the second prompt word and the preset training data set, and to train the large language model according to the second optimized training data set.

[0116] Specifically, the preset intent list refers to a set of intents related to the business field, including all possible intents related to the in-vehicle business field, such as music playback, audio book playback, device control, etc., which can provide a reference for the large language model to identify intents and help the large language model identify the most appropriate intent from the user's voice request. In addition, it can limit the output range of the large language model to prevent the large language model from outputting intents that are not related to the business field. For example, the preset intent list: [play music, play audio books, turn on device functions, turn off device functions, others]. It should be noted that the preset intent list can be adjusted according to business needs, such as adding new intents or deleting intents that are no longer needed.

[0117] The preset entity set includes all possible entity types related to each intent. For example, the entity types related to the "play music" intent are "song name" and "singer name". The preset entity set can provide a reference for the large language model to identify entities, helping the large language model to identify entities related to the intent from the user's voice request. In addition, the preset entity set can also limit the output range of the large language model to prevent the large language model from outputting entities that are irrelevant to the intent. For example, the entity type set corresponding to the "play radio" intent is: [singer name, song name, label, language, album name, song version]; the entity type set corresponding to the "play audiobook" intent is: [audiobook name, type, host name, label, author, chapter number, section number]; the entity type set corresponding to the "play radio" intent is: [radio station name, label, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent. It should be noted that the preset entity set can be adjusted according to business needs, such as adding new entities or deleting entities that are no longer needed.

[0118] Based on the above preset intent list and preset entity set, the large language model can ensure that the output results are within a fixed range and no erroneous fantasy results appear.

[0119] The server constructs a second prompt word according to the first prompt word, the preset intent list and the preset entity set. In some embodiments, the second prompt word may be as follows:

[0120] "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified, and the most appropriate intent needs to be selected from the specified intent set. At the same time, all entities and entity types in the sentence are identified. The entity type is selected from the entity type set corresponding to the intent. Intent list: [play music, play audiobooks, turn on device functions, turn off device functions, other]; the entity type set corresponding to the "play radio" intent is: [singer name, song name, label, language, album name, song version]; the entity type set corresponding to the "play audiobook" intent is: [audiobook name, type, host name, label, author, chapter number, section number]; the entity type set corresponding to the "play radio" intent is: [radio station name, label, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents of "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0121] The output format is: [{“intent”: Figure 1, "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1':'entity 1','entity type 2':'entity 2']}}'.

[0122] Next, the server determines a second optimized training data set according to the second prompt word and the preset training data set.

[0123] Finally, the server trains the large language model based on the second optimized training data set.

[0124] In this way, by limiting the reasoning scope of the large language model through preset intent lists and preset entity sets, the probability of the large language model outputting incorrect entity types can be reduced, and the accuracy of multi-intent recognition and slot filling of the large language model can be improved, thereby improving the user experience.

[0125] See also Figure 4 In some embodiments, the preset training data set includes a plurality of preset training data, and step 015 (determining a second optimized training data set according to the second prompt word and the preset training data set) includes:

[0126] 0151: Determine second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data;

[0127] 0152: Determine a second optimized training data set according to the second optimized training data.

[0128] In some embodiments, the determination module is further configured to determine second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data, and determine a second optimized training data set according to the second optimized training data.

[0129] In some embodiments, the processor is further configured to determine second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data, and determine a second optimized training data set according to the second optimized training data.

[0130] Specifically, historical conversation data refers to the most recent one or several conversation records between the user and the system, which can provide contextual information for the large language model, help the large language model understand the user's current intentions and slots, identify the inheritance relationship between intentions and slots, understand the conversation scenario, and refer to personalized information.

[0131] The server determines the second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data. It should be noted that if there is no historical conversation data, the history field can be empty, that is, no historical conversation data is input. For example, the second optimized training data may be as follows:

[0132] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified, and the intent needs to select the most appropriate one from the specified intent set. At the same time, identify all entities and entity types in the sentence. The entity type is selected from the entity type set corresponding to the intent. Intent list: [play music, play audiobooks, turn on device functions, turn off device functions, other]; the entity type set corresponding to the "play radio" intent is: [singer name, song name, label, language, album name, song version]; the entity type set corresponding to the "play audiobook" intent is: [audiobook name, type, host name, label, author, chapter number, section number]; the entity type set corresponding to the "play radio" intent is: [radio station name, label, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents of "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0133] The output format is: [{“intent”: Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0134] "input": "open",

[0135] "history": [air conditioning and lighting],

[0136] "output": [

[0137] {"intent":"Open device function","slot":["device name":"air conditioner"]},

[0138] {"intent":"Turn on device function","slot":["device name":"light"]}

[0139] ].

[0140] The second optimization training data may also be as follows:

[0141] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified, and the intent needs to select the most appropriate one from the specified intent set. At the same time, identify all entities and entity types in the sentence. The entity type is selected from the entity type set corresponding to the intent. Intent list: [play music, play audiobooks, turn on device functions, turn off device functions, other]; the entity type set corresponding to the "play radio" intent is: [singer name, song name, label, language, album name, song version]; the entity type set corresponding to the "play audiobook" intent is: [audiobook name, type, host name, label, author, chapter number, section number]; the entity type set corresponding to the "play radio" intent is: [radio station name, label, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents of "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0142] The output format is: [{“intent”: Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0143] "input": "Turn on the air conditioner and play song A",

[0144] "history": [],

[0145] "output": [

[0146] {"intent":"Open device function","slot":["device name":"air conditioner"]},

[0147] {"intent":"Play music","slot":["song name":"Song A"]}

[0148] ].

[0149] The second optimization training data may also be as follows:

[0150] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, all intents need to be identified, and the intent needs to select the most appropriate one from the specified intent set. At the same time, identify all entities and entity types in the sentence. The entity type is selected from the entity type set corresponding to the intent. Intent list: [play music, play audiobooks, turn on device functions, turn off device functions, other]; the entity type set corresponding to the "play radio" intent is: [singer name, song name, label, language, album name, song version]; the entity type set corresponding to the "play audiobook" intent is: [audiobook name, type, host name, label, author, chapter number, section number]; the entity type set corresponding to the "play radio" intent is: [radio station name, label, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents of "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0151] The output format is: [{“intent”: Figure 1 , "slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent Figure 2 ','slot':['entity type 1': 'entity 1','entity type 2': 'entity 2']}}',

[0152] "input": "Play Pingshu "Pingshu B"",

[0153] "history": [],

[0154] "output": [

[0155] {"intent":"Play audiobook","slot":["Audiobook title":"Pingshu B","category":"Pingshu"]}

[0156] ].

[0157] Next, the server determines a second optimized training data set according to the second optimized training data.

[0158] In this way, by leveraging historical conversation data, the large language model can understand the user's intentions and preferences, improve the accuracy of multi-intent recognition and slot filling, and thus enhance the user experience.

[0159] See also Figure 5 In some embodiments, the method further comprises:

[0160] 017: Determine an intent recognition training data set and an entity recognition training data set according to the second prompt word and the preset training data set;

[0161] 018: Perform training processing on a second preset large language model according to the second optimized training data set, the intent recognition training data set and the entity recognition training data set.

[0162] In some embodiments, the determination module is further used to determine the intent recognition training data set and the entity recognition training data set based on the second prompt word and the preset training data set. The training module is further used to train the second preset large language model based on the second optimized training data set, the intent recognition training data set and the entity recognition training data set.

[0163] In some embodiments, the processor is further configured to determine an intention recognition training data set and an entity recognition training data set based on the second prompt word and the preset training data set, and to perform training processing on the second preset large language model based on the second optimized training data set, the intention recognition training data set, and the entity recognition training data set.

[0164] Specifically, the intent recognition training dataset refers to an important resource used to assist large language models in multi-intent recognition, including user voice requests and expected intent recognition results, which can help large language models learn to understand and recognize multi-intent sentences, thereby improving the efficiency of human-computer interaction and user experience.

[0165] The entity recognition training dataset refers to an important resource for training entity recognition models, including user voice requests and expected slot filling results. It can help large language models learn to recognize entities in user voice requests and support various natural language processing tasks.

[0166] The server determines the intent recognition training data set and the entity recognition training data set based on the second prompt word and the preset training data set. The intent recognition training data set and the entity recognition training data set can be used to assist the intent recognition training and slot filling training of the large language model, thereby improving the accuracy of intent recognition and slot filling of the large language model. It should be noted that one second optimized training data corresponds to one intent recognition training data and one entity recognition training data. By training the large language model with the intent recognition training data and the entity recognition training data, the recognition accuracy of the large language model for the second optimized training data can be improved.

[0167] Next, the server performs training processing on the second preset large language model according to the second optimized training data set, the intent recognition training data set and the entity recognition training data set.

[0168] In this way, the large language model is trained with the intent recognition training data set and the second optimized training data set to improve the multi-intent recognition capability of the large language model, and the large language model is trained with the entity recognition training data set and the second optimized training data set to improve the slot filling capability of the large language model, so that the large language model can accurately understand the user's intentions and needs, provide more accurate services, and improve user experience.

[0169] See also Figure 6 In some embodiments, step 017 (determining the intention recognition training data set and the entity recognition training data set according to the second prompt word and the preset training data set) includes:

[0170] 0171: Determine the intention recognition prompt word and the entity recognition prompt word according to the second prompt word;

[0171] 0172: Determine an intent recognition training data set according to an intent recognition prompt word and a preset training data set;

[0172] 0173: Determine an entity recognition training data set based on entity recognition prompt words and a preset training data set.

[0173] In some embodiments, the determination module is further configured to determine the intention recognition prompt word and the entity recognition prompt word according to the second prompt word, determine the intention recognition training data set according to the intention recognition prompt word and the preset training data set, and determine the entity recognition training data set according to the entity recognition prompt word and the preset training data set.

[0174] In some embodiments, the processor is further configured to determine the intent recognition prompt word and the entity recognition prompt word according to the second prompt word, determine the intent recognition training data set according to the intent recognition prompt word and the preset training data set, and determine the entity recognition training data set according to the entity recognition prompt word and the preset training data set.

[0175] Specifically, the intent recognition prompt word refers to the text information obtained based on the second prompt word, which is used to guide the large language model to perform the intent recognition task. It is necessary to clearly tell the large language model that the task goal is to recognize the intent of the user's voice request, and emphasize the need to recognize multiple intents that may exist in the user's voice request, and provide a list of possible intents.

[0176] The entity recognition prompt word refers to the text information obtained based on the second prompt word, which is used to guide the large language model to perform entity recognition tasks. It is necessary to clearly tell the large language model that the task goal is to recognize the entities in the user's voice request, and emphasize the need to recognize all entities in the user's voice request, and provide a possible entity set.

[0177] The server determines the intent recognition prompt word and the entity recognition prompt word based on the second prompt word, that is, splits the second prompt word into the intent recognition prompt word and the entity recognition prompt word, which are used to guide the model to perform intent recognition and entity recognition respectively. In some embodiments, the determined intent recognition prompt word may be as follows:

[0178] "First, identify whether the following sentence contains multiple intents. If it does, the original sentence needs to be converted into multiple clauses. For each clause, the most appropriate intent needs to be selected from the specified intent list.

[0179] Intent list: [play music, play audiobook, turn on device function, turn off device function, other];

[0180] The output format is: [{"clause 1": clause 1, "intention": intention Figure 1}, {"clause 2": clause 2, "intention": intention Figure 2}]".

[0181] In some embodiments, the determined intent recognition prompt words may be as follows:

[0182] “Identify all entities and entity types for a given sentence, where the entity type is selected from a set of entity types corresponding to the intent.

[0183] The entity type set corresponding to the "play music" intent is: [singer name, song name, label, language, album name, song version];

[0184] The entity type set corresponding to the "play audiobook" intent is: [audiobook title, type, host name, tag, author, chapter number, section number];

[0185] The entity type set corresponding to the "play radio" intent is: [radio name, label, radio frequency, radio region, type];

[0186] The entity type lists corresponding to the two intents "Turn on device function" and "Turn off device function" are: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0187] The output format is: [{"entity type 1":"entity name 1"},{"entity type 2":"entity 2"}]".

[0188] Next, the server determines the intention recognition training data set based on the intention recognition prompt word and the preset training data set. In some embodiments, the determined intention recognition training data may be as follows:

[0189] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, the original sentence needs to be converted into multiple clauses. Each clause to intent needs to select the most appropriate intent from the specified intent list.

[0190] Intent list: [play music, play audiobook, turn on device function, turn off device function, other];

[0191] The output format is: [{"clause 1": clause 1, "intention": intention Figure 1}, {"clause 2": clause 2, "intention": intention Figure 2}],

[0192] "input": "Turn on the air conditioner and reading light",

[0193] "history": [],

[0194] "output": [

[0195] {"sentence1":"Turn on the air conditioner","intent":"Turn on device function"},

[0196] {"sentence2":"Turn on the reading light","intent":"Turn on device function"},

[0197] ].

[0198] Finally, the server determines the entity recognition training data set based on the entity recognition prompt word and the preset training data set. In some embodiments, the determined entity recognition training data may be as follows:

[0199] "instruction": "Identify all entities and entity types for a given sentence. The entity type is selected from the set of entity types corresponding to the intent.

[0200] The entity type set corresponding to the "play music" intent is: [singer name, song name, label, language, album name, song version];

[0201] The entity type set corresponding to the "play audiobook" intent is: [audiobook title, type, host name, tag, author, chapter number, section number];

[0202] The entity type set corresponding to the "play radio" intent is: [radio name, label, radio frequency, radio region, type];

[0203] The entity type lists corresponding to the two intents "Turn on device function" and "Turn off device function" are: [device name, function name, mode name, value, level]; entity recognition is not required for the "other" intent.

[0204] The output format is: [{"entity type 1":"entity name 1"}, {"entity type 2":"entity 2"}]",

[0205] "input": "Turn on the air conditioner and reading light",

[0206] "history": [],

[0207] "output": [

[0208] {"Device Name": "Air Conditioner"},

[0209] {"Device Name":"Reading Light"}

[0210] ].

[0211] In this way, by using intent recognition prompts, the large language model can focus on learning the knowledge of intent recognition, thereby improving the accuracy of intent recognition. By using entity recognition prompts, the large language model can focus on learning the distribution of entity types, thereby improving the accuracy of slot filling.

[0212] See also Figure 7 In some embodiments, the method further comprises:

[0213] 019: Based on a preset ratio, configure the number of first optimized training data in the first optimized training data set and the number of second optimized training data in the second optimized training data set, and determine a third optimized training data set;

[0214] 020: Train the large language model according to the third optimized training data set.

[0215] In some embodiments, the model training device further includes a configuration module, which is used to configure the number of first optimized training data in the first optimized training data set and the number of second optimized training data in the second optimized training data set based on a preset ratio, and determine the third optimized training data set. The training module is also used to train the large language model according to the third optimized training data set.

[0216] In some embodiments, the processor is further configured to configure the amount of first optimized training data in the first optimized training data set and the amount of second optimized training data in the second optimized training data set based on a preset ratio, determine a third optimized training data set, and train the large language model according to the third optimized training data set.

[0217] Specifically, the preset ratio refers to a preset ratio of the first optimized training data to the second optimized training data, such as 1:1 or 2:1, etc., which is determined according to actual needs.

[0218] Based on the preset ratio, the server configures the number of first optimized training data in the first optimized training data set and the number of second optimized training data in the second optimized training data set to determine the third optimized training data set. That is, according to the preset ratio, corresponding numbers of first optimized training data and second optimized training data are respectively extracted from the first optimized training data set and the second optimized training data set, and mixed to form the third optimized training data set.

[0219] Next, the server trains the large language model according to the third optimized training data set.

[0220] In this way, by mixing different types of training data, the generalization ability of the large language model can be improved, enabling it to better adapt to various different input data and improve the robustness of the large language model. The large language model can accurately understand the user's intentions and needs and provide more accurate services, thereby improving the user experience.

[0221] See also Figure 8 The embodiment of the present application provides a vehicle voice interaction method based on a preset large language model, the preset large language model is trained based on the above-mentioned training method, and the method includes:

[0222] 021: Get voice request;

[0223] 022: Determine vehicle control instructions based on the preset large language model and voice request;

[0224] 023: Send vehicle control commands to the vehicle to complete voice interaction.

[0225] The embodiment of the present application also provides a server, including a memory and a processor. The voice interaction method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain a voice request. And based on a preset large language model, according to the voice request, a vehicle control instruction is determined. And the vehicle control instruction is sent to the vehicle to complete the voice interaction.

[0226] The embodiment of the present application also provides a voice interaction device. The vehicle voice interaction method of the embodiment of the present application can be implemented by the voice interaction device of the embodiment of the present application. Specifically, the voice interaction device includes an acquisition module, a determination module and a sending module. The acquisition module is used to acquire a voice request. The determination module is used to determine a vehicle control instruction according to the voice request based on a preset large language model. The sending module is used to send a vehicle control instruction to the vehicle to complete the voice interaction.

[0227] Specifically, the server obtains a voice request.

[0228] Next, based on the preset large language model, the server determines the vehicle control command according to the voice request. That is, the server uses the trained preset large language model to analyze the user's voice request, identify the user's intention and needs, and generate corresponding vehicle control commands.

[0229] Finally, the server sends the vehicle control command to the vehicle to complete the voice interaction. That is, the generated vehicle control command is sent to the vehicle to control the vehicle to perform corresponding operations, such as turning on the air conditioner, playing music, etc.

[0230] In this way, based on the preset large language model, users can control the vehicle to perform various operations through voice commands, improving driving convenience.

[0231] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0232] It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, executable file or some intermediate form. Computer readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media.

[0233] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0234] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code that includes one or more executable requests for implementing specific logical functions or steps of a process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0235] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for training a large language model, characterized in that: The method comprises: Constructing a first prompt word, wherein the first prompt word is configured to guide the large language model to perform multi-intent recognition processing; Determining a first optimized training data set according to the first prompt word and a preset training data set; The large language model is trained according to the first optimized training data set.

2. The method according to claim 1, characterized in that The preset training data set includes a plurality of preset training data, and determining the first optimized training data set according to the first prompt word and the preset training data set includes: Determining first optimized training data according to the first prompt word, the preset training data, and the historical conversation data associated with the preset training data; The first optimized training data set is determined according to the first optimized training data.

3. The method according to claim 1, characterized in that The method further comprises: Constructing a second prompt word according to the first prompt word, the preset intent list and the preset entity set; Determining a second optimized training data set according to the second prompt word and the preset training data set; The large language model is trained according to the second optimized training data set.

4. The method according to claim 3, characterized in that The preset training data set includes a plurality of preset training data, and determining the second optimized training data set according to the second prompt word and the preset training data set includes: Determining second optimized training data according to the second prompt word, the preset training data, and the historical conversation data associated with the preset training data; The second optimized training data set is determined according to the second optimized training data.

5. The method according to claim 3, characterized in that: The method further comprises: Determine an intent recognition training data set and an entity recognition training data set according to the second prompt word and the preset training data set; The second preset large language model is trained according to the second optimized training data set, the intent recognition training data set and the entity recognition training data set.

6. The method according to claim 5, characterized in that The step of determining the intention recognition training data set and the entity recognition training data set according to the second prompt word and the preset training data set includes: Determine an intent recognition prompt word and an entity recognition prompt word according to the second prompt word; Determining an intention recognition training data set according to the intention recognition prompt word and the preset training data set; An entity recognition training data set is determined according to the entity recognition prompt word and the preset training data set.

7. The method according to claim 3, characterized in that The method further comprises: Based on a preset ratio, configuring the amount of first optimized training data in the first optimized training data set and the amount of second optimized training data in the second optimized training data set, and determining a third optimized training data set; The large language model is trained according to the third optimized training data set.

8. A vehicle voice interaction method based on a preset large language model, characterized in that: The preset large language model is obtained by training based on the training method according to claims 1-7, and the method comprises: Get voice request; Determining a vehicle control instruction based on the preset large language model and according to the voice request; The vehicle control command is sent to the vehicle to complete the voice interaction.

9. A server, characterized in that: The server includes a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images