Training method of large language model, vehicle voice interaction method and server

Abstract

The application discloses a large language model training method, a vehicle voice interaction method, a server and a computer readable storage medium. The method comprises the following steps: constructing a first prompt word, the first prompt word is configured to guide the large language model to perform multi-intent recognition processing. Then, according to the first prompt word and a preset training data set, a first optimized training data set is determined. Finally, the large language model is trained according to the first optimized training data set. In this way, by constructing the first prompt word and the first optimized training data set, the large language model can accurately perform multi-intent recognition and slot filling, thereby improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a training method for a large language model, a vehicle voice interaction method, a server, and a computer-readable storage medium. Background Technology

[0002] In related technologies, in-vehicle voice assistants facilitate human-computer interaction with users. However, for voice requests involving multiple intents, the reasoning ability of in-vehicle voice assistants may be unable to recognize all intents, resulting in incomplete intent understanding and a poor user experience. Summary of the Invention

[0003] This application provides a method for training a large language model, a vehicle voice interaction method, a server, and a computer-readable storage medium.

[0004] This application provides a method for training a large language model, the method comprising:

[0005] A first prompt word is constructed, which is configured to guide the large language model to perform multi-intent recognition processing;

[0006] Based on the first prompt word and the preset training dataset, determine the first optimized training dataset;

[0007] The large language model is trained based on the first optimized training dataset.

[0008] In this way, the server constructs a first prompt word, which is configured to guide the large language model in multi-intent recognition processing. Next, based on the first prompt word and a pre-set training dataset, the server determines a first optimized training dataset. Finally, the server trains the large language model using the first optimized training dataset. Thus, by constructing the first prompt word and the first optimized training dataset, the large language model can accurately perform multi-intent recognition and slot filling, thereby improving the user experience.

[0009] In some implementations, the preset training dataset includes multiple preset training datasets, and determining the first optimized training dataset based on the first prompt word and the preset training dataset includes:

[0010] Based on the first prompt word, the preset training data, and the historical dialogue data associated with the preset training data, the first optimized training data is determined;

[0011] Based on the first optimized training data, the first optimized training dataset is determined.

[0012] Thus, the server determines the first optimized training data based on the first prompt word, preset training data, and historical dialogue data associated with the preset training data. Next, the server determines the first optimized training dataset based on the first optimized training data. In this way, by using historical dialogue data, missing data information can be supplemented, improving the accuracy of multi-intent recognition and slot filling, thereby enhancing the user experience.

[0013] In some embodiments, the method further includes:

[0014] Based on the first prompt word, the preset intent list, and the preset entity set, construct the second prompt word;

[0015] Based on the second prompt word and the preset training dataset, determine the second optimized training dataset;

[0016] The large language model is trained based on the second optimized training dataset.

[0017] Thus, the server constructs a second prompt word based on the first prompt word, a preset intent list, and a preset entity set. Next, the server determines a second optimized training dataset based on the second prompt word and a preset training dataset. Finally, the server trains the large language model using the second optimized training dataset. This method, by limiting the inference scope of the large language model through the preset intent list and entity set, reduces the probability of the large language model outputting incorrect entity types, improves the accuracy of multi-intent recognition and slot filling, and ultimately enhances the user experience.

[0018] In some implementations, the preset training dataset includes multiple preset training datasets, and determining the second optimized training dataset based on the second prompt word and the preset training dataset includes:

[0019] The second optimized training data is determined based on the second prompt word, the preset training data, and the historical dialogue data associated with the preset training data;

[0020] The second optimized training dataset is determined based on the second optimized training data.

[0021] Thus, the server determines the second optimized training data based on the second prompt word, the preset training data, and historical dialogue data associated with the preset training data. Next, the server determines the second optimized training dataset based on the second optimized training data. In this way, by utilizing historical dialogue data, the large language model can understand the user's intent and preferences, improving the accuracy of multi-intent recognition and slot filling, thereby enhancing the user experience.

[0022] In some embodiments, the method further includes:

[0023] Based on the second prompt word and the preset training dataset, determine the intent recognition training dataset and the entity recognition training dataset;

[0024] The second preset large language model is trained based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

[0025] Thus, the server determines the intent recognition training dataset and the entity recognition training dataset based on the second prompt word and the preset training dataset. Next, the server trains the second preset large language model using the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset. In this way, training the large language model using the intent recognition training dataset and the second optimized training dataset improves its multi-intent recognition capability, while training it using the entity recognition training dataset and the second optimized training dataset improves its slot-filling capability. This allows the large language model to accurately understand the user's intent and needs, providing more precise services and improving the user experience.

[0026] In some implementations, determining the intent recognition training dataset and the entity recognition training dataset based on the second prompt word and the preset training dataset includes:

[0027] Based on the second prompt word, determine the intent recognition prompt word and the entity recognition prompt word;

[0028] Based on the intent recognition prompt words and the preset training dataset, determine the intent recognition training dataset;

[0029] The entity recognition training dataset is determined based on the entity recognition prompt words and the preset training dataset.

[0030] Thus, the server determines the intent recognition prompt and entity recognition prompt based on the second prompt. Next, the server determines the intent recognition training dataset based on the intent recognition prompt and the pre-set training dataset. Finally, the server determines the entity recognition training dataset based on the entity recognition prompt and the pre-set training dataset. In this way, by using the intent recognition prompt, the large language model can focus on learning knowledge about intent recognition, thereby improving the accuracy of intent recognition. By using the entity recognition prompt, the large language model can focus on learning the distribution of entity types, thereby improving the accuracy of slot filling.

[0031] In some embodiments, the method further includes:

[0032] Based on a preset ratio, the number of first optimized training data in the first optimized training dataset and the number of second optimized training data in the second optimized training dataset are configured to determine the third optimized training dataset;

[0033] The large language model is trained based on the third optimized training dataset.

[0034] Thus, based on a preset ratio, the server configures the number of first-optimized training data points in the first optimized training dataset and the number of second-optimized training data points in the second optimized training dataset to determine the third optimized training dataset. Next, the server trains the large language model using the third optimized training dataset. In this way, by mixing different types of training data, the generalization ability of the large language model can be improved, enabling it to better adapt to various input data, enhancing its robustness. The large language model can accurately understand user intent and needs, providing more precise services, thereby improving the user experience.

[0035] This application provides a vehicle voice interaction method based on a preset large language model, wherein the preset large language model is trained using the training method described above, and the method includes:

[0036] Get voice request;

[0037] Based on the preset large language model, the vehicle control command is determined according to the voice request;

[0038] The vehicle control command is sent to the vehicle to complete the voice interaction.

[0039] Thus, the server receives the voice request. Next, based on a pre-defined large language model, the server determines the vehicle control commands according to the voice request. Finally, the server sends the vehicle control commands to the vehicle to complete the voice interaction. In this way, based on the pre-defined large language model, users can control the vehicle to perform various operations via voice commands, improving driving convenience.

[0040] This application provides a server that includes a processor and a memory. The memory stores a computer program that, when executed by the processor, implements the method described above.

[0041] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described above.

[0042] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0043] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0044] Figure 1 This is one of the flowcharts illustrating the training method of a large language model in certain embodiments of this application;

[0045] Figure 2 This is a second flowchart illustrating the training method of a large language model according to certain embodiments of this application;

[0046] Figure 3 This is the third flowchart illustrating the training method of a large language model in some embodiments of this application;

[0047] Figure 4 This is the fourth flowchart illustrating the training method of a large language model in some embodiments of this application;

[0048] Figure 5 This is the fifth flowchart illustrating the training method of a large language model in some embodiments of this application;

[0049] Figure 6 This is the sixth flowchart illustrating the training method of a large language model in some embodiments of this application;

[0050] Figure 7 This is the seventh flowchart illustrating the training method of a large language model in some embodiments of this application;

[0051] Figure 8 This is a flowchart illustrating a voice interaction method according to certain embodiments of this application. Detailed Implementation

[0052] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0053] In intelligent vehicle systems, in-vehicle voice assistants serve as a convenient human-computer interaction method, greatly enhancing the driving experience and safety. However, traditional in-vehicle voice assistants can only recognize one intent and corresponding slot in a user's request. When processing voice requests with multiple intents, they often suffer from insufficient reasoning ability, failing to recognize multiple intents and slots. This results in incomplete intent recognition and an inability to provide accurate and appropriate feedback, thus impacting the user experience.

[0054] For example, when a user's voice request is "Turn on the air conditioner and turn on the reading light," the model processes the user's voice request as follows:

[0055] "instruction": "Identify the user's intent in the following sentences, and the corresponding slot information for that intent."

[0056] "input": Turn on the air conditioner and turn on the reading light.

[0057] "output": {

[0058] "intent": "Enable device functionality"

[0059] "slot": {"device name": "air conditioner"}

[0060] }

[0061] In-car voice assistants may only recognize "intent": "turn on device function" and "slot": {"device name": "air conditioning"}, but cannot recognize "intent": "turn on device function" and "slot": {"device name": "reading light"}. Incomplete intent recognition means they cannot fully understand all the semantics of the user's voice request, resulting in the user's needs not being fully processed and a poor user experience.

[0062] Based on the above issues, please refer to Figure 1 This application provides a method for training a large language model, the method comprising:

[0063] 011: Construct the first prompt word;

[0064] 012: Determine the first optimized training dataset based on the first prompt word and the preset training dataset;

[0065] 013: Train the large language model based on the first optimized training dataset.

[0066] This application also provides a server, including a memory and a processor. The voice interaction method of this application can be implemented by the server of this application. Specifically, the memory stores a computer program, and the processor is used to construct a first prompt word, determine a first optimized training dataset based on the first prompt word and a preset training dataset, and train a large language model based on the first optimized training dataset.

[0067] This application also provides a model training apparatus. The training method for the large language model of this application can be implemented by the model training apparatus of this application. Specifically, the model training apparatus includes a construction module, a determination module, and a training module. The construction module is used to construct a first prompt word. The determination module is used to determine a first optimized training dataset based on the first prompt word and a preset training dataset. The training module is used to train the large language model based on the first optimized training dataset.

[0068] Specifically, in related technologies, for voice requests containing multiple intents, rules or models can be used to break down the voice request into multiple clauses, and then single intent recognition and slot recognition are performed on each clause. For example, for a user voice request "turn on the air conditioner and turn on the reading light," the voice request is first broken down into the clauses "turn on the air conditioner" and "turn on the reading light," and then single intent recognition and slot recognition are performed on the clauses "turn on the air conditioner" and "turn on the reading light" respectively, resulting in "intent": "turn on device function," "slot": {"device name": "air conditioner"} and "intent": "turn on device function," "slot": {"device name": "reading light"}. However, when processing user voice requests that contain multiple intents in a single sentence, the above method may fail to recognize multiple intents and slots, ultimately resulting in incomplete understanding of the user's intent. For example, a user's voice request "turn on the air conditioner and reading light" cannot be broken down into its own words, but it still contains two intents. Performing single intent recognition and slot recognition might only identify "intent": "turn on device function", "slot": {"device name": "air conditioner"}, but not "intent": "turn on device function", "slot": {"device name": "reading light"}. Furthermore, the above method needs to be performed sequentially—first splitting the clauses, then performing intent recognition and slot filling—which takes a long time and negatively impacts the user experience.

[0069] The large language model trained using the training method provided in this application can recognize voice requests containing multiple intentions and fully understand user semantics when processing such user voice requests. Furthermore, it avoids the serial processing of splitting clauses before recognition, resulting in higher efficiency and shorter processing time. This allows for a faster understanding of user intentions and needs, providing more accurate services and improving user experience.

[0070] The first cue word refers to the cue word constructed in a vehicle intelligent system to guide a large language model in multi-intent recognition processing. The first cue word clearly defines the tasks that the large language model needs to complete during fine-tuning training and specifies the output format. The first cue word can guide the recognition of the number of intents in a user's voice request, the recognition of all intents in the user's voice request, the recognition of the entity and type of the user's voice request, and specifies the output format. In some implementations, the first cue word may be as follows:

[0071] First, identify whether the following sentence contains multiple intents. If it does, identify all intents, and for each intent, identify all entities and entity pair types, then assemble them into a JSON format for output. The output format is: [{"intent": "intent";} ... Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 "", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}.

[0072] Preset training datasets refer to sample datasets used for fine-tuning training of large language models. These datasets include label information associated with vehicle functions, such as intent labels and entity labels, which are used to guide the large language model to learn to recognize and fill in intents and entities, thereby improving the accuracy of intent recognition and entity filling.

[0073] The first optimized training dataset refers to a training dataset generated based on the first prompt word and a preset training dataset, which can be used to train the multi-intent recognition capability of the large language model. It should be noted that the data format of the first optimized training data in the first optimized training dataset can be Alpaca format, including an instruction field, an input field, a history field, and an output field. The instruction field refers to the task description required of the large language model: extracting relevant questions and answers based on the input text data. The input field refers to the user's input text data, i.e., the user's voice request. The history field refers to the historical dialogue data from the most recent N rounds; for ease of explanation, this application uses the historical dialogue data from the previous round as the history field. The output field refers to the result output by the large language model according to the requirements of the instruction field, i.e., the result expected from the large language model, including all intents contained in the user's voice request, as well as entity and entity type information under each intent.

[0074] In some implementations, the first optimized training data in the first optimized training dataset may be as follows:

[0075] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, identify all intents, and for each intent, identify all entities and entity pair types, and assemble them into a JSON format for output. The output format is: [{"intent": "intent Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0076] "input": "Turn on the air conditioner and play song A",

[0077] "history": [],

[0078] "output": [

[0079] {"intent": "Enable device function", "slot": ["Device name": "Air conditioner"]},

[0080] {"intent": "Play music", "slot": ["Song name": "Song A"]}

[0081] ].

[0082] The server constructs the first prompt word, which guides the large language model to perform multi-intent recognition processing. By splitting user voice requests containing multiple intents, the single-sentence intent recognition task and slot filling task are integrated into the first prompt word, enabling the large language model to identify all intents and their corresponding slots contained in the user voice request in one go.

[0083] Next, the server determines the first optimized training dataset based on the first prompt word and the preset training dataset.

[0084] Finally, the server trains the large language model based on the first optimized training dataset.

[0085] In summary, in the large language model training method and server provided in this application, the server constructs a first prompt word, which is configured to guide the large language model in multi-intent recognition processing. Next, the server determines a first optimized training dataset based on the first prompt word and a preset training dataset. Finally, the server trains the large language model based on the first optimized training dataset. Thus, by constructing the first prompt word and the first optimized training dataset, the large language model can accurately perform multi-intent recognition and slot filling, thereby improving the user experience.

[0086] Please see Figure 2 In some implementations, the preset training dataset includes multiple preset training datasets. Step 012 (determining the first optimized training dataset based on the first prompt word and the preset training dataset) includes:

[0087] 0121: Determine the first optimized training data based on the first prompt word, the preset training data, and the historical dialogue data associated with the preset training data;

[0088] 0122: Determine the first optimized training dataset based on the first optimized training data.

[0089] In some implementations, the determining module is further configured to determine first optimized training data based on a first prompt word, preset training data, and historical dialogue data associated with the preset training data, and to determine a first optimized training dataset based on the first optimized training data.

[0090] In some implementations, the processor is further configured to determine first optimized training data based on a first prompt word, preset training data, and historical dialogue data associated with the preset training data, and to determine a first optimized training dataset based on the first optimized training data.

[0091] Specifically, historical dialogue data refers to the records of the user's most recent dialogue with the system. It can provide contextual information for the large language model, help the large language model understand the user's current intent and slot, identify the inheritance relationship between intent and slot, understand the dialogue scenario, and refer to personalized information.

[0092] The server determines the first optimized training data based on the first prompt word, preset training data, and historical dialogue data associated with the preset training data. It should be noted that if no historical dialogue data exists, the `history` field can be empty, meaning no historical dialogue data should be entered. For example, the first optimized training data might look like this:

[0093] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, identify all intents, and for each intent, identify all entities and entity pair types, and assemble them into a JSON format for output. The output format is: [{"intent": "intent Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0094] "input": "Open",

[0095] "history": [air conditioning and lighting],

[0096] "output": [

[0097] {"intent": "Enable device function", "slot": ["Device name": "Air conditioner"]},

[0098] {"intent": "Enable device function", "slot": ["Device name": "Lighting"]}

[0099] ].

[0100] The first optimized training data may also be as follows:

[0101] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, identify all intents, and for each intent, identify all entities and entity pair types, and assemble them into a JSON format for output. The output format is: [{"intent": "intent Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0102] "input": "Turn on the air conditioner and play song A",

[0103] "history": [],

[0104] "output": [

[0105] {"intent": "Enable device function", "slot": ["Device name": "Air conditioner"]},

[0106] {"intent": "Play music", "slot": ["Song name": "Song A"]}

[0107] ].

[0108] Next, the server determines the first optimized training dataset based on the first optimized training data.

[0109] In this way, by using historical dialogue data, missing data information can be supplemented, improving the accuracy of multi-intent recognition and slot filling, thereby enhancing the user experience.

[0110] Please see Figure 3 In some implementations, the method further includes:

[0111] 014: Construct a second prompt word based on the first prompt word, the preset intent list, and the preset entity set;

[0112] 015: Determine the second optimized training dataset based on the second prompt word and the preset training dataset;

[0113] 016: Train the large language model based on the second optimized training dataset.

[0114] In some implementations, the construction module is further configured to construct a second prompt word based on the first prompt word, a preset intent list, and a preset entity set. The determination module is further configured to determine a second optimized training dataset based on the second prompt word and a preset training dataset. The training module is further configured to train the large language model based on the second optimized training dataset.

[0115] In some implementations, the processor is further configured to construct a second prompt word based on a first prompt word, a preset intent list, and a preset entity set; determine a second optimized training dataset based on the second prompt word and a preset training dataset; and train a large language model based on the second optimized training dataset.

[0116] Specifically, the preset intent list refers to a set of intents related to the business domain, including all possible intents related to the in-vehicle business domain, such as music playback, audiobook playback, and device control. It provides a reference for the large language model to recognize intents, helping it identify the most appropriate intent from user voice requests. Furthermore, it can limit the output range of the large language model, preventing it from outputting intents unrelated to the business domain. For example, a preset intent list might include: [Play music, Play audiobook, Turn on device function, Turn off device function, Other]. It should be noted that the preset intent list can be adjusted according to business needs, such as adding new intents or deleting unnecessary ones.

[0117] The preset entity set includes all possible entity types associated with each intent. For example, the entity types associated with the intent to "play music" are "song name" and "artist name," etc. The preset entity set provides a reference for the large language model to recognize entities, helping it identify intent-related entities from user voice requests. Furthermore, the preset entity set can limit the output range of the large language model, preventing it from outputting entities unrelated to the intent. For example, the entity type set corresponding to the intent to "play radio" is: [artist name, song name, tag, language, album name, song version]; the entity type set corresponding to the intent to "play audiobook" is: [audiobook name, type, narrator name, tag, author, chapter number, section number]; the entity type set corresponding to the intent to "play radio" is: [radio station name, tag, radio station frequency, radio station region, type]; the entity type list corresponding to the two intents to "turn on device function" and "turn off device function" is: [device name, function name, mode name, value, level]; entity recognition may not be performed for "other" intents. It should be noted that the preset entity set can be adjusted according to business needs, such as adding new entities or deleting entities that are no longer needed.

[0118] Based on the aforementioned list of predefined intentions and set of predefined entities, the large language model can ensure that the output results are within a fixed range and that no erroneous or imagined results occur.

[0119] The server constructs a second prompt word based on the first prompt word, a preset intent list, and a preset entity set. In some implementations, the second prompt word may be as follows:

[0120] First, identify whether the following sentence contains multiple intents. If multiple intents are present, all intents need to be identified, and the most suitable intent must be selected from a specified intent set. Simultaneously, identify all entities and their types within the sentence. Entity types are selected from the entity type set corresponding to the intent. Intent list: [Play music, Play audiobook, Turn on device function, Turn off device function, Other]; The entity type set corresponding to the "Play radio" intent is: [Artist name, Song name, Tag, Language, Album name, Song version]; The entity type set corresponding to the "Play audiobook" intent is: [Audiobook name, Type, Narrator name, Tag, Author, Chapter number, Section number]; The entity type set corresponding to the "Play radio" intent is: [Radio station name, Tag, Radio frequency, Radio region, Type]; The entity type lists corresponding to the "Turn on device function" and "Turn off device function" intents are: [Device name, Function name, Mode name, Value, Level]; Entity identification is not required for the "Other" intent.

[0121] The output format is: [{"intent": "intent"; ... Figure 1{"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 "", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}"

[0122] Next, the server determines the second optimized training dataset based on the second prompt word and the preset training dataset.

[0123] Finally, the server trains the large language model based on the second optimized training dataset.

[0124] In this way, by limiting the reasoning scope of the large language model through a preset intent list and a preset entity set, the probability of the large language model outputting incorrect entity types can be reduced, the accuracy of multi-intent recognition and slot filling of the large language model can be improved, and thus the user experience can be enhanced.

[0125] Please see Figure 4 In some implementations, the preset training dataset includes multiple preset training datasets. Step 015 (determining the second optimized training dataset based on the second prompt word and the preset training dataset) includes:

[0126] 0151: Determine the second optimized training data based on the second prompt word, the preset training data, and the historical dialogue data associated with the preset training data;

[0127] 0152: Determine the second optimized training dataset based on the second optimized training data.

[0128] In some implementations, the determining module is further configured to determine second optimized training data based on the second prompt word, preset training data, and historical dialogue data associated with the preset training data, and to determine a second optimized training dataset based on the second optimized training data.

[0129] In some implementations, the processor is further configured to determine second optimized training data based on a second prompt word, preset training data, and historical dialogue data associated with the preset training data, and to determine a second optimized training dataset based on the second optimized training data.

[0130] Specifically, historical dialogue data refers to the records of the user's most recent dialogue with the system. It can provide contextual information for the large language model, help the large language model understand the user's current intent and slot, identify the inheritance relationship between intent and slot, understand the dialogue scenario, and refer to personalized information.

[0131] The server determines the second optimized training data based on the second prompt word, preset training data, and historical dialogue data associated with the preset training data. It should be noted that if no historical dialogue data exists, the `history` field can be empty, meaning no historical dialogue data should be entered. For example, the second optimized training data might look like this:

[0132] "instruction": "First, identify whether the following sentence contains multiple intents. If multiple intents are present, all intents need to be identified, and the most suitable intent must be selected from the specified intent set. Simultaneously, identify all entities and their types within the sentence. The entity types are selected from the entity type set corresponding to the intent. Intent list: [Play music, Play audiobook, Turn on device function, Turn off device function, Other]; The entity type set corresponding to the intent "Play radio" is: [Artist name, Song name, Tag, Language, Album name, Song version]; The entity type set corresponding to the intent "Play audiobook" is: [Audiobook name, Type, Narrator name, Tag, Author, Chapter number, Section number]; The entity type set corresponding to the intent "Play radio" is: [Radio station name, Tag, Radio frequency, Radio region, Type]; The entity type lists corresponding to the intents "Turn on device function" and "Turn off device function" are: [Device name, Function name, Mode name, Value, Level]; Entity recognition is not required for the "Other" intent."

[0133] The output format is: [{"intent": "intent"; ... Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0134] "input": "Open",

[0135] "history": [air conditioning and lighting],

[0136] "output": [

[0137] {"intent": "Enable device function", "slot": ["Device name": "Air conditioner"]},

[0138] {"intent": "Enable device function", "slot": ["Device name": "Lighting"]}

[0139] ].

[0140] The second optimized training data may also be as follows:

[0141] "instruction": "First, identify whether the following sentence contains multiple intents. If multiple intents are present, all intents need to be identified, and the most suitable intent must be selected from the specified intent set. Simultaneously, identify all entities and their types within the sentence. The entity types are selected from the entity type set corresponding to the intent. Intent list: [Play music, Play audiobook, Turn on device function, Turn off device function, Other]; The entity type set corresponding to the intent "Play radio" is: [Artist name, Song name, Tag, Language, Album name, Song version]; The entity type set corresponding to the intent "Play audiobook" is: [Audiobook name, Type, Narrator name, Tag, Author, Chapter number, Section number]; The entity type set corresponding to the intent "Play radio" is: [Radio station name, Tag, Radio frequency, Radio region, Type]; The entity type lists corresponding to the intents "Turn on device function" and "Turn off device function" are: [Device name, Function name, Mode name, Value, Level]; Entity recognition is not required for the "Other" intent."

[0142] The output format is: [{"intent": "intent"; ... Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0143] "input": "Turn on the air conditioner and play song A",

[0144] "history": [],

[0145] "output": [

[0146] {"intent": "Enable device function", "slot": ["Device name": "Air conditioner"]},

[0147] {"intent": "Play music", "slot": ["Song name": "Song A"]}

[0148] ].

[0149] The second optimized training data may also be as follows:

[0150] "instruction": "First, identify whether the following sentence contains multiple intents. If multiple intents are present, all intents need to be identified, and the most suitable intent must be selected from the specified intent set. Simultaneously, identify all entities and their types within the sentence. The entity types are selected from the entity type set corresponding to the intent. Intent list: [Play music, Play audiobook, Turn on device function, Turn off device function, Other]; The entity type set corresponding to the intent "Play radio" is: [Artist name, Song name, Tag, Language, Album name, Song version]; The entity type set corresponding to the intent "Play audiobook" is: [Audiobook name, Type, Narrator name, Tag, Author, Chapter number, Section number]; The entity type set corresponding to the intent "Play radio" is: [Radio station name, Tag, Radio frequency, Radio region, Type]; The entity type lists corresponding to the intents "Turn on device function" and "Turn off device function" are: [Device name, Function name, Mode name, Value, Level]; Entity recognition is not required for the "Other" intent."

[0151] The output format is: [{"intent": "intent"; ... Figure 1 {"slot": ["entity type 1": "entity 1", "entity type 2": "entity 2"]}, {"intent": "intent";} Figure 2 ", "slot": ["Entity type 1": "Entity 1", "Entity type 2": "Entity 2"]}}",

[0152] "input": "Play the storytelling performance 'Storytelling B'",

[0153] "history": [],

[0154] "output": [

[0155] {"intent": "Play audiobook", "slot": ["audiobook title": "Storytelling B", "category": "Storytelling"]}

[0156] ].

[0157] Next, the server determines the second optimized training dataset based on the second optimized training data.

[0158] In this way, by utilizing historical dialogue data, large language models can understand users' intentions and preferences, improve the accuracy of multi-intention recognition and slot filling, and thus enhance the user experience.

[0159] Please see Figure 5 In some implementations, the method further includes:

[0160] 017: Based on the second prompt word and the preset training dataset, determine the intent recognition training dataset and the entity recognition training dataset;

[0161] 018: The second preset large language model is trained based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

[0162] In some implementations, the determining module is further configured to determine an intent recognition training dataset and an entity recognition training dataset based on the second prompt word and the preset training dataset. The training module is further configured to train the second preset large language model based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

[0163] In some embodiments, the processor is further configured to determine an intent recognition training dataset and an entity recognition training dataset based on the second prompt word and a preset training dataset, and to train a second preset large language model based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

[0164] Specifically, the intent recognition training dataset refers to an important resource used to assist large language models in multi-intent recognition. It includes the intent recognition results of user voice requests and expectations, which can help large language models learn to understand and recognize multi-intent sentences, thereby improving the efficiency of human-computer interaction and user experience.

[0165] Entity recognition training datasets refer to important resources for training entity recognition models. They include user voice requests and expected slot filling results, which can help large language models learn to recognize entities in user voice requests and support various natural language processing tasks.

[0166] Based on the second prompt word and a preset training dataset, the server determines the intent recognition training dataset and the entity recognition training dataset. These datasets can be used to assist in the intent recognition and slot filling training of the large language model, improving the accuracy of intent recognition and slot filling. It should be noted that one set of second-optimized training data corresponds to one set of intent recognition training data and one set of entity recognition training data. Training the large language model using these datasets improves the model's accuracy in recognizing the second-optimized training data.

[0167] Next, the server trains the second preset large language model based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

[0168] In this way, by training the large language model with the intent recognition training dataset and the second optimized training dataset, the multi-intent recognition capability of the large language model is improved. By training the large language model with the entity recognition training dataset and the second optimized training dataset, the slot filling capability of the large language model is improved. This enables the large language model to accurately understand the user's intent and needs, and provide more accurate services, thereby improving the user experience.

[0169] Please see Figure 6 In some implementations, step 017 (determining the intent recognition training dataset and the entity recognition training dataset based on the second prompt word and the preset training dataset) includes:

[0170] 0171: Based on the second prompt word, determine the intent recognition prompt word and the entity recognition prompt word;

[0171] 0172: Determine the intent recognition training dataset based on the intent recognition prompt words and the preset training dataset;

[0172] 0173: Determine the entity recognition training dataset based on the entity recognition prompts and the preset training dataset.

[0173] In some embodiments, the determining module is further configured to determine an intent recognition prompt and an entity recognition prompt based on the second prompt. It also determines an intent recognition training dataset based on the intent recognition prompt and a preset training dataset. Finally, it determines an entity recognition training dataset based on the entity recognition prompt and the preset training dataset.

[0174] In some embodiments, the processor is further configured to determine an intent recognition prompt and an entity recognition prompt based on the second prompt; and to determine an intent recognition training dataset based on the intent recognition prompt and a preset training dataset; and to determine an entity recognition training dataset based on the entity recognition prompt and the preset training dataset.

[0175] Specifically, the intent recognition cue words refer to the text information obtained based on the second cue words, which is used to guide the large language model in the intent recognition task. It is necessary to explicitly tell the large language model that the task goal is to recognize the intent of the user's voice request, and to emphasize that it is necessary to recognize multiple intents that may exist in the user's voice request, and also to provide a list of possible intents.

[0176] Entity recognition prompts refer to the text information obtained based on the second prompt, which is used to guide the large language model in performing entity recognition tasks. It is necessary to explicitly tell the large language model that the task objective is to recognize entities in the user's voice request, and to emphasize that all entities in the user's voice request need to be recognized. It is also necessary to provide a possible set of entities.

[0177] Based on the second prompt word, the server determines the intent recognition prompt word and the entity recognition prompt word, that is, it splits the second prompt word into an intent recognition prompt word and an entity recognition prompt word, which are used to guide the model to perform intent recognition and entity recognition, respectively. In some implementations, the determined intent recognition prompt word may be as follows:

[0178] "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, the original sentence needs to be converted into multiple clauses. For each clause, the most appropriate intent needs to be selected from the specified intent list."

[0179] Intent list: [Play music, play audiobook, turn on device function, turn off device function, others];

[0180] The output format is: [{"clause 1":clause 1, "intention": intention Figure 1}, {"Clause 2": Clause 2, "Intention": Intention Figure 2}]”.

[0181] In some implementations, the determined intent recognition prompt may be as follows:

[0182] "Identify all entities and their types for a given sentence, with the entity type selected from the set of entity types corresponding to the intent."

[0183] The entity type set corresponding to the intent to "play music" is: [singer name, song name, tag, language, album name, song version];

[0184] The entity type set corresponding to the intent to "play an audiobook" is: [audiobook name, type, narrator name, tag, author, chapter number, section number];

[0185] The set of entity types corresponding to the intent to “play a broadcast” is: [radio station name, tag, radio station frequency, radio station region, type];

[0186] The entity type list corresponding to the two intents "Turn on device function" and "Turn off device function" is: [Device name, function name, mode name, value, level]; entity recognition is not required for the "Other" intent.

[0187] The output format is: [{"Entity Type 1": "Entity Name 1"}, {"Entity Type 2": "Entity 2"}]".

[0188] Next, the server determines the intent recognition training dataset based on the intent recognition prompts and a pre-set training dataset. In some implementations, the determined intent recognition training data may look like this:

[0189] "instruction": "First, identify whether the following sentence contains multiple intents. If it contains multiple intents, the original sentence needs to be converted into multiple clauses. For each clause, the most appropriate intent needs to be selected from the specified intent list."

[0190] Intent list: [Play music, play audiobook, turn on device function, turn off device function, others];

[0191] The output format is: [{"clause 1":clause 1, "intention": intention Figure 1}, {"Clause 2": Clause 2, "Intention": Intention Figure 2}],

[0192] "input": "Turn on the air conditioner and reading light",

[0193] "history": [],

[0194] "output": [

[0195] {"sentence1": "Turn on the air conditioner", "intent": "Activate device function"}

[0196] {"sentence2": "Turn on reading light", "intent": "Enable device function"}

[0197] ].

[0198] Finally, the server determines the entity recognition training dataset based on the entity recognition prompts and the preset training dataset. In some implementations, the determined entity recognition training data may look like the following:

[0199] "instruction": "Identifies all entities and their types for a given sentence, with the entity type selected from the set of entity types corresponding to the intent."

[0200] The entity type set corresponding to the intent to "play music" is: [singer name, song name, tag, language, album name, song version];

[0201] The entity type set corresponding to the intent to "play an audiobook" is: [audiobook name, type, narrator name, tag, author, chapter number, section number];

[0202] The set of entity types corresponding to the intent to “play a broadcast” is: [radio station name, tag, radio station frequency, radio station region, type];

[0203] The entity type list corresponding to the two intents "Turn on device function" and "Turn off device function" is: [Device name, function name, mode name, value, level]; entity recognition is not required for the "Other" intent.

[0204] The output format is: [{"Entity Type 1": "Entity Name 1"}, {"Entity Type 2": "Entity 2"}]",

[0205] "input": "Turn on the air conditioner and reading light",

[0206] "history": [],

[0207] "output": [

[0208] {"Device Name": "Air Conditioner"}

[0209] {"Device Name": "Reading Light"}

[0210] ].

[0211] Thus, by using intent recognition cues, large language models can focus on learning intent recognition knowledge, thereby improving intent recognition accuracy. By using entity recognition cues, large language models can focus on learning the distribution of entity types, thereby improving slot filling accuracy.

[0212] Please see Figure 7 In some implementations, the method further includes:

[0213] 019: Based on a preset ratio, configure the number of first optimized training data in the first optimized training dataset and the number of second optimized training data in the second optimized training dataset, and determine the third optimized training dataset;

[0214] 020: Train the large language model based on the third optimized training dataset.

[0215] In some embodiments, the model training apparatus further includes a configuration module, which is used to configure the number of first optimized training data in the first optimized training dataset and the number of second optimized training data in the second optimized training dataset based on a preset ratio, and to determine a third optimized training dataset. The training module is also used to train the large language model based on the third optimized training dataset.

[0216] In some implementations, the processor is further configured to configure the number of first optimized training data in the first optimized training dataset and the number of second optimized training data in the second optimized training dataset based on a preset ratio, and determine a third optimized training dataset. Then, the processor trains the large language model based on the third optimized training dataset.

[0217] Specifically, the preset ratio refers to the pre-set ratio of the first optimized training data and the second optimized training data, such as 1:1 or 2:1, which is determined according to actual needs.

[0218] Based on a preset ratio, the server configures the number of first optimized training data points in the first optimized training dataset and the number of second optimized training data points in the second optimized training dataset to determine the third optimized training dataset. That is, according to the preset ratio, a corresponding number of first optimized training data points and second optimized training data points are extracted from the first optimized training dataset and the second optimized training dataset, respectively, and mixed to form the third optimized training dataset.

[0219] Next, the server trains the large language model using the third optimized training dataset.

[0220] In this way, by mixing different types of training data, the generalization ability of large language models can be improved, enabling them to better adapt to various different input data, thus improving the robustness of large language models. Large language models can accurately understand users' intentions and needs and provide more precise services, thereby enhancing the user experience.

[0221] Please see Figure 8 This application provides a vehicle voice interaction method based on a preset large language model. The preset large language model is trained using the training method described above. The method includes:

[0222] 021: Obtain voice request;

[0223] 022: Based on a pre-set large language model, determine vehicle control commands according to voice requests;

[0224] 023: Issue vehicle control commands to the vehicle to complete the voice interaction.

[0225] This application also provides a server, including a memory and a processor. The voice interaction method of this application can be implemented by the server of this application. Specifically, the memory stores a computer program, and the processor is used to acquire voice requests, determine vehicle control commands based on a preset large language model and the voice requests, and issue vehicle control commands to the vehicle to complete the voice interaction.

[0226] This application also provides a voice interaction device. The vehicle voice interaction method of this application can be implemented by the voice interaction device of this application. Specifically, the voice interaction device includes an acquisition module, a determination module, and a delivery module. The acquisition module is used to acquire voice requests. The determination module is used to determine vehicle control commands based on a preset large language model and the voice request. The delivery module is used to deliver vehicle control commands to the vehicle to complete the voice interaction.

[0227] Specifically, the server receives voice requests.

[0228] Next, based on a pre-set large language model, the server determines the vehicle control commands according to the voice request. That is, using the trained pre-set large language model, the server analyzes the user's voice request, identifies the user's intent and needs, and generates corresponding vehicle control commands.

[0229] Finally, the server sends vehicle control commands to the vehicle to complete the voice interaction. That is, the generated vehicle control commands are sent to the vehicle to control it to perform corresponding operations, such as turning on the air conditioning or playing music.

[0230] Thus, based on a pre-set large language model, users can control the vehicle to perform various operations through voice commands, improving driving convenience.

[0231] This application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps of the method described above.

[0232] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0233] In this specification, the terms "specifically," "furthermore," "particularly," "understandably," etc., refer to specific features, structures, materials, or characteristics described in connection with embodiments or examples that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0234] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of executable request code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0235] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A training method for a large language model, characterized in that, The method includes: A first prompt word is constructed, which is configured to guide the large language model to recognize all intents in the user's voice request and the entities and entity types corresponding to each intent, and output the recognition results in a predetermined format; Based on the first prompt word and the preset training dataset, determine the first optimized training dataset; The large language model is trained based on the first optimized training dataset.

2. The method according to claim 1, characterized in that, The preset training dataset includes multiple preset training data sets. The step of determining the first optimized training dataset based on the first prompt word and the preset training dataset includes: Based on the first prompt word, the preset training data, and the historical dialogue data associated with the preset training data, the first optimized training data is determined; Based on the first optimized training data, the first optimized training dataset is determined.

3. The method according to claim 1, characterized in that, The method further includes: Based on the first prompt word, the preset intent list, and the preset entity set, construct the second prompt word; Based on the second prompt word and the preset training dataset, determine the second optimized training dataset; The large language model is trained based on the second optimized training dataset.

4. The method according to claim 3, characterized in that, The preset training dataset includes multiple preset training data sets. The step of determining the second optimized training dataset based on the second prompt word and the preset training dataset includes: The second optimized training data is determined based on the second prompt word, the preset training data, and the historical dialogue data associated with the preset training data; The second optimized training dataset is determined based on the second optimized training data.

5. The method according to claim 3, characterized in that, The method further includes: Based on the second prompt word and the preset training dataset, determine the intent recognition training dataset and the entity recognition training dataset; The second preset large language model is trained based on the second optimized training dataset, the intent recognition training dataset, and the entity recognition training dataset.

6. The method according to claim 5, characterized in that, The step of determining the intent recognition training dataset and the entity recognition training dataset based on the second prompt word and the preset training dataset includes: Based on the second prompt word, determine the intent recognition prompt word and the entity recognition prompt word; Based on the intent recognition prompt words and the preset training dataset, determine the intent recognition training dataset; The entity recognition training dataset is determined based on the entity recognition prompt words and the preset training dataset.

7. The method according to claim 3, characterized in that, The method further includes: Based on a preset ratio, the number of first optimized training data in the first optimized training dataset and the number of second optimized training data in the second optimized training dataset are configured to determine the third optimized training dataset; The large language model is trained based on the third optimized training dataset.

8. A vehicle voice interaction method based on a pre-set large language model, characterized in that, The preset large language model is trained based on the training method described in any one of claims 1-7, the method comprising: Get voice request; Based on the preset large language model, the vehicle control command is determined according to the voice request; The vehicle control command is sent to the vehicle to complete the voice interaction.

9. A server, characterized in that, The server includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-8.

Images