Language model optimization method and device, program product, electronic equipment and medium
By analyzing the corruption ability of the language model and updating the training data set, the problem of insufficient training data quality is solved, and the optimization effect of the language model and task-specific performance are improved.
Patent Information
- Application Number
- CN202411053576.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-29
AI Technical Summary
In the optimization training process of pre-trained language models, it is crucial to select the appropriate training data, but the existing technology is difficult to effectively improve the performance of the model for specific tasks, especially when the quality of the training data is insufficient, resulting in some of the model's capabilities not being possible. Achieve the desired optimization effect.
By obtaining the initial language model, using the test data set to analyze its corruption ability, obtaining the training data set corresponding to the corruption ability, performing optimization processing and updating the original training data set, and then optimizing the pre-trained language model to improve the quality of the training data and Model performance.
It improves the quality of training data, improves the optimization effect of the language model and its performance on specific tasks, and ensures that the model can handle specific tasks more effectively.
Smart Images

Figure CN119047577A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a language model optimization method, device, program product, electronic device and medium. Background Art
[0002] When optimizing a pre-trained language model, such as performing SFT (Standard Fine-Tuning), it is crucial to select appropriate training data. The quality of the training data directly determines the final performance and application scope of the model. When improving the performance of the model for specific tasks such as conversation tasks, how to obtain appropriate training data to improve the optimization effect of the model is a very important issue. Summary of the invention
[0003] In view of this, an object of the present invention is to provide a language model optimization method, device, program product, electronic device and medium.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0005] In a first aspect, the present invention provides a language model optimization method, the language model optimization method comprising:
[0006] Obtaining an initial language model; the initial language model is obtained by optimizing and training a pre-trained language model using an original training data set;
[0007] Performing a capability analysis on the initial language model using a test data set to determine a damage capability of the initial language model;
[0008] From all the training data groups included in the original training data set, a training data group corresponding to the damage capacity is obtained to obtain a training data group to be processed;
[0009] Optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set;
[0010] The pre-trained language model is optimized and trained using the updated training data set to obtain a target language model.
[0011] In an optional implementation, the step of performing capability analysis on the initial language model using a test data set to determine the damage capability of the initial language model includes:
[0012] Inputting the test data set into the initial language model to generate a first data set;
[0013] Inputting the test data set into a preset reference language model to generate a second data set;
[0014] The first data set is compared with the second data set to obtain the ability of the initial language model to be weaker than the reference language model, thereby obtaining the damage ability of the initial language model.
[0015] In an optional embodiment, the to-be-processed training data group includes a plurality of to-be-processed training samples and their original labels;
[0016] The step of optimizing the training data group to be processed to obtain an optimized training data group comprises:
[0017] Using the initial language model, generating first output data corresponding to each of the training samples to be processed;
[0018] Using a preset reference language model, generating second output data corresponding to each of the training samples to be processed;
[0019] For each of the training samples to be processed, a new label of the training sample to be processed is determined based on the original label of the training sample to be processed and the corresponding first output data and second output data, and each training sample to be processed and its new label are obtained to obtain an optimized training data group.
[0020] In an optional implementation, the step of determining a new label of the training sample to be processed based on the original label of the training sample to be processed and the first output data and the second output data corresponding thereto comprises:
[0021] Calculating the similarity between the first output data and the second output data corresponding to the training sample to be processed;
[0022] When the similarity between the two is greater than a preset first similarity threshold, determining a new label for the training sample to be processed based on the original label and the first output data;
[0023] When the similarity between the two is less than or equal to the first similarity threshold, a new label of the training sample to be processed is determined based on the original label and the second output data.
[0024] In an optional implementation, the step of determining a new label of the training sample to be processed based on the original label and the first output data includes:
[0025] Calculating the similarity between the original label and the first output data;
[0026] If the similarity between the two is greater than a preset second similarity threshold, the original label is used as a new label for the training sample to be processed; wherein the second similarity threshold is greater than the first similarity threshold;
[0027] If the similarity between the two is less than or equal to the second similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
[0028] In an optional implementation, the step of determining a new label of the training sample to be processed based on the original label and the second output data includes:
[0029] Calculating the similarity between the original label and the second output data;
[0030] If the similarity between the two is greater than the first similarity threshold, the second output data is used as a new label for the training sample to be processed;
[0031] If the similarity between the two is less than or equal to the first similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
[0032] In a second aspect, the present invention provides a language model optimization device, the language model optimization device comprising:
[0033] An acquisition module is used to acquire an initial language model; the initial language model is obtained by optimizing and training a pre-trained language model using an original training data set;
[0034] An analysis module, configured to perform capability analysis on the initial language model using a test data set to determine the damage capability of the initial language model;
[0035] An updating module, configured to obtain a training data group corresponding to the damage capacity from all training data groups included in the original training data set, to obtain a training data group to be processed;
[0036] Optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set;
[0037] The training module is used to optimize the pre-trained language model using the updated training data set to obtain a target language model.
[0038] In a third aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the language model optimization method described in any one of the aforementioned embodiments.
[0039] In a fourth aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the language model optimization method described in any one of the aforementioned embodiments is implemented.
[0040] In a fifth aspect, the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the language model optimization method described in any one of the aforementioned embodiments is implemented.
[0041] The language model optimization method, device, program product, electronic device and medium provided by the present invention include: firstly obtaining an initial language model, which is obtained by optimizing and training a pre-trained language model using an original training data set; and performing capability analysis on the initial language model using a test data set to determine the damage capability of the initial language model; then obtaining a training data set corresponding to the damage capability from all training data sets contained in the original training data set to obtain a training data set to be processed; then optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set; and finally optimizing and training the pre-trained language model using the updated training data set to obtain a target language model. Thus, the quality of the training data is improved, and the optimization effect of the language model and its performance on specific tasks are improved.
[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0044] Figure 1 A block diagram of an electronic device provided by an embodiment of the present invention is shown;
[0045] Figure 2 One of the flow charts of the language model optimization method provided by an embodiment of the present invention is shown;
[0046] Figure 3 The second flowchart of the language model optimization method provided by the embodiment of the present invention is shown;
[0047] Figure 4 A functional module diagram of a language model optimization device provided by an embodiment of the present invention is shown.
[0048] Icon: 100 - electronic device; 110 - processor; 120 - memory; 130 - communication module; 300 - language model optimization device; 310 - acquisition module; 330 - analysis module; 350 - update module; 370 - training module. DETAILED DESCRIPTION
[0049] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.
[0051] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0052] Please refer to Figure 1 , is a block diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes a processor 110, a memory 120, and a communication module 130, and each component is directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.
[0053] The processor 110 is used to read / write data or programs stored in the memory 120 and execute corresponding functions. It can be a general-purpose processor, including a CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be a DSP digital signal processor, an ASIC application-specific integrated circuit, an FPGA off-the-shelf programmable gate array or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.
[0054] The memory 120 is used to store programs or data. The memory 120 can be a RAM (Random Access Memory), a ROM (Read Only Memory), a PROM (Programmable Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electric Erasable Programmable Read-Only Memory), etc.
[0055] The communication module 130 is used to perform signaling or data communications with other devices.
[0056] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.
[0057] It is understandable that the electronic device can be a server, which is used to execute the language model optimization method of an embodiment of the present invention to obtain an optimized language model, and send the optimized language model to the intelligent central control platform of the car to realize voice interaction with the user.
[0058] The above-mentioned electronic device will be used as an execution subject to execute each step in each method provided in the embodiment of the present invention and the corresponding technical effects.
[0059] See also Figure 2 , is a flow chart of a language model optimization method provided in an embodiment of the present invention.
[0060] Step S202, obtaining an initial language model; the initial language model is obtained by optimizing and training the pre-trained language model using the original training data set.
[0061] In this embodiment, the pre-trained language model can be understood as a language model that can handle specific tasks. The pre-trained language model can adopt the Qwen-14B-Chat model, or other language models, which is not limited in the embodiment of the present invention. The original training data set can be understood as a pre-set training data set for improving the performance of the pre-trained language model for a specific task. In addition, the original training data set includes multiple training data groups, each training data group corresponds to a specific capability, and the specific capability can be understood as an capability for processing a specific task.
[0062] For ease of understanding, the embodiment of the present invention takes the specific task of interactive dialogue with the user in a car driving scene as an example for explanation. For example, the original training data set includes three training data groups, namely training data group 1, training data group 2 and training data group 3. And training data group 1 corresponds to voice control capability, training data group 2 corresponds to voice communication capability, and training data group 3 corresponds to voice broadcast capability.
[0063] Among them, voice control capability can be understood as the ability to control the working status of the car's windows or audio equipment and respond according to the voice commands input by the user. Voice communication capability can be understood as the ability to make and receive calls or send text messages and respond according to the voice commands input by the user. Voice broadcast capability can be understood as the ability to broadcast the corresponding content according to the voice commands input by the user. Then, based on the training data group corresponding to each specific capability in the original training data set, the pre-trained language model is optimized and trained to obtain the initial language model.
[0064] It should be understood that the number of training data groups in the original training data set and the specific capabilities corresponding to each of them can be set according to actual conditions, and the implementation of the present invention is not limited to this.
[0065] Step S204: perform capability analysis on the initial language model using the test data set to determine the damage capability of the initial language model.
[0066] It is understandable that since the training data set is very important for the optimization of the language model, and the original training data set is pre-set, using the original training data set to optimize the pre-trained language model may result in the specific capabilities of the initial language model not achieving the desired optimization effect.
[0067] Therefore, in an embodiment of the present invention, after the initial language model is obtained, a capability analysis is performed on it using a test data set, that is, to analyze whether each specific capability of the initial language model achieves the expected optimization effect, and the specific capabilities that do not achieve the expected optimization effect are regarded as damaged capabilities of the initial language model.
[0068] For ease of understanding, we continue to use the above example for illustration. For example, we use the test data set to analyze the capabilities of the initial language model, that is, to analyze whether its voice control capability, voice communication capability, and voice broadcast capability have achieved the expected optimization effect. Assuming that the voice broadcast capability has not achieved the expected optimization effect, the voice broadcast capability is regarded as the damaged capability of the initial language model.
[0069] Step S206, obtaining a training data group corresponding to the damage capacity from all training data groups included in the original training data set, to obtain a training data group to be processed.
[0070] Step S208, optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set.
[0071] Step S210, optimizing and training the pre-trained language model using the updated training data set to obtain a target language model.
[0072] In this embodiment, if a certain specific ability of the initial language model does not achieve the expected optimization effect, it may be because the quality of the training data group corresponding to the specific ability is poor. In this case, the training data group corresponding to the specific ability, i.e., the damaged ability, can be obtained from the original training data set to obtain the training data group to be processed.
[0073] Then, the training data set to be processed is optimized to obtain an optimized training data set, and the original training data set is updated based on the optimized training data set to obtain an updated training data set. Then, the pre-trained language model is optimized again using the updated training data set, and the optimized language model obtained is the target language model.
[0074] For ease of understanding, the above example is continued for explanation. Since the speech broadcasting capability of the initial language model does not achieve the desired optimization effect, the speech broadcasting capability is regarded as a damaged capability, and the corresponding training data group, i.e., training data group 3, is obtained from the three training data groups included in the original training data set, thereby obtaining the training data group to be processed.
[0075] Then, the training data set 3 is optimized to improve its quality, and the optimized training data set, i.e., the training data set 3', is obtained. The training data set 3 in the original training data set is replaced with the training data set 3' to update the original training data set, and the updated training data set is obtained. Then, the pre-trained language model is optimized and trained using the updated training data set, and the target language model is obtained.
[0076] It can be understood that the embodiment of the present invention analyzes the capability of the initial language model, determines the damage capability and obtains a training data set corresponding to the damage capability, so as to obtain training data with poor data quality and optimize it to update the original training data set, and optimizes the pre-trained language model with the updated training data set to obtain the target language model. This improves the quality of the training data, the optimization effect of the language model and its performance on specific tasks.
[0077] It can be seen that based on the above steps, firstly, the initial language model is obtained, which is obtained by optimizing and training the pre-trained language model using the original training data set; and the initial language model is analyzed for its ability using the test data set to determine the damage ability of the initial language model; then, in all the training data sets contained in the original training data set, the training data set corresponding to the damage ability is obtained to obtain the training data set to be processed; then, the training data set to be processed is optimized to obtain the optimized training data set, and the original training data set is updated based on the optimized training data set to obtain the updated training data set; finally, the pre-trained language model is optimized and trained using the updated training data set to obtain the target language model. This improves the quality of the training data, the optimization effect of the language model, and its performance on specific tasks.
[0078] Optionally, for step S204, the present invention provides a possible implementation method, see Figure 3 .
[0079] Step S204 - 1 , inputting the test data set into the initial language model to generate a first data set.
[0080] Step S204 - 3 , inputting the test data set into a preset reference language model to generate a second data set.
[0081] Step S204-5, comparing the first data set with the second data set, obtaining the ability of the initial language model to be weaker than the reference language model, and obtaining the damage ability of the initial language model.
[0082] In this embodiment, the preset reference language model can be understood as a language model that can handle specific tasks and whose various specific capabilities all achieve the desired optimization effect. The reference language model can adopt the Wenxinyiyan model or other language models. It should be noted that the number of parameters of the reference language model is greater than the number of parameters of the initial language model, that is, the reference language model is a larger language model than the initial language model.
[0083] The test data set includes a test data group corresponding to each specific capability. The test data set can be input into the initial language model to generate a first data set, which includes a first data group corresponding to each specific capability. And the test data set is input into the reference language model to generate a second data set, which includes a second data group corresponding to each specific capability. Then for each specific capability, a preset quality assessment method such as the GSB (Good-Same-Bad) method is used to compare the quality of the first data group and the second data group corresponding to the specific capability to compare the specific capability of the initial language model with the specific capability of the reference language model.
[0084] If the quality of the first data group corresponding to the specific capability is better than the quality of the second data group corresponding to it, or the quality of the first data group corresponding to the specific capability is the same as that of the second data group, then the specific capability of the initial language model is determined to be stronger than or equal to that of the reference language model, that is, the specific capability of the initial language model is considered to have achieved the desired optimization effect, and the specific capability will not be considered as a damaged capability. If the quality of the first data group corresponding to the specific capability is worse than the quality of the second data group corresponding to it, then the specific capability of the initial language model is determined to be weaker than that of the reference language model, that is, the specific capability of the initial language model is considered to have not achieved the desired optimization effect, and the specific capability will be considered as a damaged capability.
[0085] For ease of understanding, the above example is continued for explanation. The test data set includes test data group 1, test data group 2 and test data group 3. Test data group 1 corresponds to voice control capability, test data group 2 corresponds to voice communication capability, and test data group 3 corresponds to voice broadcast capability.
[0086] The test data sets, i.e., test data set 1, test data set 2, and test data set 3, are input into the initial language model to generate the corresponding first data set, which includes data set 1-1, data set 1-2, and data set 1-3. And the test data sets, i.e., test data set 1, test data set 2, and test data set 3, are input into the reference language model to generate the corresponding second data set, which includes data set 2-1, data set 2-2, and data set 2-3. Among them, data set 1-1 and data set 2-1 correspond to voice control capabilities, data set 1-2 and data set 2-2 correspond to voice communication capabilities, and data set 1-3 and data set 2-3 correspond to voice broadcast capabilities.
[0087] For the voice control ability, the GSB method is used to compare the voice control ability of the initial language model with the voice control ability of the reference language model by comparing the quality of data group 1-1 and data group 2-1. Assuming that the quality of data group 1-1 is better than the quality of data group 2-1, that is, the comparison result is Good, it is determined that the voice control ability of the initial language model is stronger than the voice control ability of the reference language model, and the voice control ability will not be regarded as a damaged ability.
[0088] For voice communication capability, the GSB method is used to compare the voice communication capability of the initial language model with the voice communication capability of the reference language model by comparing the quality of data group 1-2 and data group 2-2. Assuming that the quality of data group 1-2 and data group 2-2 is the same, that is, the comparison result is Same, then the voice communication capability of the initial language model is determined to be equal to the voice communication capability of the reference language model, and the voice communication capability will not be regarded as a damaged capability.
[0089] For the speech broadcast capability, the GSB method is used to compare the speech broadcast capability of the initial language model with the speech broadcast capability of the reference language model by comparing the quality of data group 1-2 and data group 2-2. Assuming that the quality of data group 1-3 is worse than that of data group 2-3, that is, the comparison result is Bad, it is determined that the speech communication capability of the initial language model is weaker than that of the reference language model, and the speech broadcast capability will be regarded as damaged capability.
[0090] Optionally, for the process of optimizing the training data set to be processed to obtain an optimized training data set in step S208, an embodiment of the present invention provides a possible implementation method.
[0091] Step S208-1: Generate first output data corresponding to each training sample to be processed by using the initial language model.
[0092] Step S208-3: Generate second output data corresponding to each training sample to be processed by using a preset reference language model.
[0093] Step S208-5, for each training sample to be processed, based on the original label of the training sample to be processed and its corresponding first output data and second output data, determine the new label of the training sample to be processed, obtain each training sample to be processed and its new label, and obtain an optimized training data group.
[0094] In this embodiment, the training data group to be processed includes multiple training samples to be processed and their original labels. It is understandable that if a certain specific capability of the initial language model does not achieve the expected optimization effect, it may be because the label of the training sample corresponding to the specific capability is inaccurate. Therefore, the embodiment of the present invention optimizes the training sample by inputting each training sample corresponding to the damaged capability into the initial language model and the reference language model to generate corresponding output data respectively and compare them with the original labels of the training samples, and selects the most accurate one as the new label of the training sample.
[0095] For ease of understanding, the above example is continued for explanation. Training data group 3 includes multiple training samples and their original labels. Each training sample in training data group 3 is used as each training sample to be processed, and is input into the initial language model and the reference language model respectively, and the initial language model is used to generate the first output data corresponding to each training sample to be processed, and the reference language model is used to generate the second output data corresponding to each training sample to be processed. It can be understood that in order to ensure the optimization effect of the language model, the output data corresponding to each training sample to be processed generated by the reference language model can also be cleaned, and the cleaned data can be used as the second output data.
[0096] By performing similar processing on each training sample to be processed, the original label, the corresponding first output data and the second output data of each training sample to be processed can be obtained. Then, for each training sample to be processed, based on the original label of the training sample to be processed and the corresponding first output data and second output data, the new label of the training sample to be processed is determined, and each training sample to be processed and its new label are obtained, and the optimized training data group, i.e., training data group 3', is obtained.
[0097] For ease of understanding, the embodiment of the present invention is explained by taking a training sample in the training data group 3, namely training sample A, as an example. Each training sample consists of an instruction (Instruction) and input data (Input). For example, the instruction of training sample A is "As an intelligent voice assistant, you need to extract information related to 'Little P Broadcast' from the given sentence. 'Little P Broadcast' refers to the vehicle's built-in voice assistant Little P, which can provide the driver with real-time road information, navigation instructions, vehicle status and other reminder services through voice"; the input data of training sample A is "Remind me to get up at ten o'clock"; the original label of training sample A is "Master, you should get up."
[0098] Then, the initial language model is used to generate the first output data corresponding to training sample A, i.e., "It is ten o'clock now, please get up". And the reference language model is used to generate the second output data corresponding to training sample A, i.e., "It is ten o'clock, time to get up". Then, based on the original label of training sample A, i.e., "Master, you should get up", the corresponding first output data, i.e., "It is ten o'clock now, please get up", and the second output data, i.e., "It is ten o'clock, time to get up", the new label of training sample A is determined to optimize the label of training sample A, so as to obtain training sample A and its new label.
[0099] Optionally, for the process of determining a new label of the training sample to be processed based on the original label of the training sample to be processed and the corresponding first output data and second output data in step S208-5, an embodiment of the present invention provides a possible implementation method.
[0100] Step S208-5-1, calculating the similarity between the first output data and the second output data corresponding to the training sample to be processed.
[0101] Step S208-5-3a: when the similarity between the two is greater than a preset first similarity threshold, a new label of the training sample to be processed is determined based on the original label and the first output data.
[0102] Step S208-5-3b: when the similarity between the two is less than or equal to the first similarity threshold, determine a new label for the training sample to be processed based on the original label and the second output data.
[0103] In this embodiment, based on the original labels of the training samples to be processed and the first output data and the second output data corresponding thereto, the similarity between the first output data and the second output data is calculated. For example, the similarity between the first output data and the second output data can be obtained by calculating the Levenshtein distance between them and normalizing the Levenshtein distance.
[0104] If the similarity between the two is greater than a preset first similarity threshold, such as 0.9, it is considered that the output results of the initial language model and the reference language model for the training sample to be processed are consistent, which means that the original label of the training sample to be processed has little influence on the loss ability not achieving the expected optimization effect. Then, based on the original label and the first output data, it is further judged whether the original label is accurate.
[0105] If the similarity between the two is less than or equal to the first similarity threshold value, such as 0.9, it is considered that the output results of the initial language model and the reference language model for the training sample to be processed are inconsistent, which means that the original label of the training sample to be processed has a greater impact on the damage ability not achieving the expected optimization effect. Then, based on the original label and the second output data, it is determined whether the second output data is accurate, that is, whether to use the second output data to replace the original label. It should be understood that the first similarity threshold value can be set according to actual conditions, and the embodiment of the present invention is not limited to this.
[0106] For ease of understanding, we continue to use the training sample A in the above example for explanation. Calculate the similarity between the first output data corresponding to the training sample A, i.e., "It is ten o'clock now, please get up", and the second output data, i.e., "It is ten o'clock, time to get up". Assuming that the similarity between the two is less than the first similarity threshold of 0.9, it is considered that the output results of the initial language model and the reference language model for the training sample A are inconsistent, which means that the original label of the training sample A, i.e., "Master, you should get up", has a greater impact on the voice broadcasting ability not achieving the expected optimization effect. Then, based on the original label, i.e., "Master, you should get up" and the second output data, i.e., "It is ten o'clock, time to get up", it is determined whether the second output data, i.e., "It is ten o'clock, time to get up", is used to replace the original label, i.e., "Master, you should get up".
[0107] Optionally, for step S208-5-3a, an embodiment of the present invention provides a possible implementation method.
[0108] Step S208-5-3a-1, calculating the similarity between the original label and the first output data.
[0109] Step S208-5-3a-3: If the similarity between the two is greater than a preset second similarity threshold, the original label is used as a new label for the training sample to be processed; wherein the second similarity threshold is greater than the first similarity threshold.
[0110] Step S208-5-3a-5, if the similarity between the two is less than or equal to the second similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
[0111] In this embodiment, when the similarity between the first output data and the second output data is greater than the first similarity threshold, the similarity between the original label and the first output data is calculated. For example, the similarity between the original label and the first output data can be obtained by calculating the Levenshtein distance between the original label and the first output data and normalizing the Levenshtein distance.
[0112] If the similarity between the two is greater than a preset second similarity threshold value, such as 0.95, the original label of the training sample to be processed is considered accurate, and the original label is used as the new label of the training sample to be processed, that is, the original label of the training sample to be processed is retained. The second similarity threshold value is greater than the first similarity threshold value, and the second similarity threshold value can be set according to actual conditions, which is not limited in the embodiment of the present invention.
[0113] If the similarity between the two is less than or equal to a second similarity threshold value such as 0.95, it is considered that the original label of the training sample to be processed is accurate or inaccurate, and then a new label of the training sample to be processed is selected from the original label, the first output data, and the second output data according to a preset selection mechanism. For example, the preset selection mechanism includes four annotation evaluation methods, and according to each annotation evaluation method, one is selected from the original label, the first output data, and the second output data, and the number of selections of the original label, the first output data, and the second output data is counted, and the one with the largest number of selections is used as the new label of the training sample to be processed.
[0114] Optionally, for step S208-5-3b, an embodiment of the present invention provides a possible implementation method.
[0115] Step S208-5-3b-1, calculate the similarity between the original label and the second output data.
[0116] Step S208-5-3b-3: if the similarity between the two is greater than the first similarity threshold, the second output data is used as a new label for the training sample to be processed.
[0117] Step S208-5-3b-5, if the similarity between the two is less than or equal to the first similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
[0118] In this embodiment, when the similarity between the first output data and the second output data is less than or equal to the first similarity threshold, the similarity between the original label and the second output data is calculated. For example, the similarity between the original label and the second output data can be obtained by calculating the Levenshtein distance between the original label and the second output data and normalizing the Levenshtein distance.
[0119] If the similarity between the two is greater than a first similarity threshold such as 0.9, the second output data corresponding to the training sample to be processed is considered accurate, and the second output data is used as a new label for the training sample to be processed, that is, the second output data is used to replace the original label.
[0120] If the similarity between the two is less than or equal to the first similarity threshold value, such as 0.9, the second output data corresponding to the training sample to be processed is inaccurate, and then a new label for the training sample to be processed is selected from the original label, the first output data, and the second output data according to a preset selection mechanism. For example, the preset selection mechanism includes four annotation evaluation methods, and according to each annotation evaluation method, one is selected from the original label, the first output data, and the second output data, and the number of selections of the original label, the first output data, and the second output data is counted, and the one with the largest number of selections is used as the new label for the training sample to be processed.
[0121] For ease of understanding, we continue to use the above training sample A as an example. Since the similarity between the first output data corresponding to training sample A, i.e., "It is ten o'clock now, please get up", and the second output data, i.e., "It is ten o'clock, time to get up", is less than the first similarity threshold of 0.9. Then calculate the similarity between the original label, i.e., "Master, you should get up" and the second output data, i.e., "It is ten o'clock, time to get up". Assuming that the similarity between the two is greater than the first similarity threshold of 0.9, it is considered that the second output data, i.e., "It is ten o'clock, time to get up", is accurate, and then the second output data, i.e., "It is ten o'clock, time to get up", is used as the new label of training sample A. That is, the second output data, i.e., "It is ten o'clock, time to get up", replaces the original label, i.e., "Master, you should get up".
[0122] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a language model optimization device is given below. Figure 4 , is a functional module diagram of a language model optimization device 300 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the language model optimization device 300 provided in this embodiment are the same as those in the above embodiment. For the sake of brief description, for matters not mentioned in this embodiment, reference may be made to the corresponding contents in the above embodiment. The language model optimization device 300 includes:
[0123] The acquisition module 310 is used to acquire an initial language model; the initial language model is obtained by optimizing and training the pre-trained language model using the original training data set;
[0124] An analysis module 330 is used to perform capability analysis on the initial language model using the test data set to determine the damage capability of the initial language model;
[0125] An updating module 350 is used to obtain a training data group corresponding to the damage capacity from all training data groups included in the original training data set, so as to obtain a training data group to be processed;
[0126] Optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set;
[0127] The training module 370 is used to optimize the pre-trained language model using the updated training data set to obtain a target language model.
[0128] Optionally, the analysis module 330 is further used to: input the test data set into the initial language model to generate a first data set; input the test data set into a preset reference language model to generate a second data set; compare the first data set with the second data set to obtain the ability of the initial language model to be weaker than the reference language model, and obtain the damage ability of the initial language model.
[0129] Optionally, the update module 350 is also used to: use the initial language model to generate first output data corresponding to each training sample to be processed; use the preset reference language model to generate second output data corresponding to each training sample to be processed; for each training sample to be processed, based on the original label of the training sample to be processed and its corresponding first output data and second output data, determine a new label for the training sample to be processed, obtain each training sample to be processed and its new label, and obtain an optimized training data group.
[0130] Optionally, the update module 350 is also used to: calculate the similarity between the first output data and the second output data corresponding to the training sample to be processed; when the similarity between the two is greater than a preset first similarity threshold, determine the new label of the training sample to be processed based on the original label and the first output data; when the similarity between the two is less than or equal to the first similarity threshold, determine the new label of the training sample to be processed based on the original label and the second output data.
[0131] Optionally, the update module 350 is also used to: calculate the similarity between the original label and the first output data; if the similarity between the two is greater than a preset second similarity threshold, use the original label as the new label for the training sample to be processed; wherein the second similarity threshold is greater than the first similarity threshold; if the similarity between the two is less than or equal to the second similarity threshold, select the new label for the training sample to be processed from the original label, the first output data and the second output data according to a preset selection mechanism.
[0132] Optionally, the update module 350 is also used to: calculate the similarity between the original label and the second output data; if the similarity between the two is greater than a first similarity threshold, use the second output data as a new label for the training sample to be processed; if the similarity between the two is less than or equal to the first similarity threshold, select the new label for the training sample to be processed from the original label, the first output data and the second output data according to a preset selection mechanism.
[0133] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the language model optimization method disclosed in the embodiment of the present invention is implemented.
[0134] An embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the language model optimization method disclosed in the embodiment of the present invention is implemented.
[0135] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the language model optimization method disclosed in the embodiment of the present invention is implemented.
[0136] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0137] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
[0138] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A language model optimization method, characterized in that: The language model optimization method comprises: Obtaining an initial language model; the initial language model is obtained by optimizing and training a pre-trained language model using an original training data set; Performing a capability analysis on the initial language model using a test data set to determine a damage capability of the initial language model; From all the training data groups included in the original training data set, a training data group corresponding to the damage capacity is obtained to obtain a training data group to be processed; Optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set; The pre-trained language model is optimized and trained using the updated training data set to obtain a target language model.
2. The language model optimization method according to claim 1, characterized in that: The step of performing capability analysis on the initial language model using the test data set to determine the damage capability of the initial language model comprises: Inputting the test data set into the initial language model to generate a first data set; Inputting the test data set into a preset reference language model to generate a second data set; The first data set is compared with the second data set to obtain the ability of the initial language model to be weaker than the reference language model, thereby obtaining the damage ability of the initial language model.
3. The language model optimization method according to claim 1, characterized in that: The training data group to be processed includes a plurality of training samples to be processed and their original labels; The step of optimizing the training data group to be processed to obtain an optimized training data group comprises: Using the initial language model, generating first output data corresponding to each of the training samples to be processed; Using a preset reference language model, generating second output data corresponding to each of the training samples to be processed; For each of the training samples to be processed, a new label of the training sample to be processed is determined based on the original label of the training sample to be processed and the corresponding first output data and second output data, and each training sample to be processed and its new label are obtained to obtain an optimized training data group.
4. The language model optimization method according to claim 3, characterized in that: The step of determining a new label of the training sample to be processed based on the original label of the training sample to be processed and the first output data and the second output data corresponding thereto comprises: Calculating the similarity between the first output data and the second output data corresponding to the training sample to be processed; When the similarity between the two is greater than a preset first similarity threshold, determining a new label for the training sample to be processed based on the original label and the first output data; When the similarity between the two is less than or equal to the first similarity threshold, a new label of the training sample to be processed is determined based on the original label and the second output data.
5. The language model optimization method according to claim 4, characterized in that: The step of determining a new label of the training sample to be processed based on the original label and the first output data comprises: Calculating the similarity between the original label and the first output data; If the similarity between the two is greater than a preset second similarity threshold, the original label is used as a new label for the training sample to be processed; wherein the second similarity threshold is greater than the first similarity threshold; If the similarity between the two is less than or equal to the second similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
6. The language model optimization method according to claim 4, characterized in that: The step of determining a new label of the training sample to be processed based on the original label and the second output data comprises: Calculating the similarity between the original label and the second output data; If the similarity between the two is greater than the first similarity threshold, the second output data is used as a new label for the training sample to be processed; If the similarity between the two is less than or equal to the first similarity threshold, a new label for the training sample to be processed is selected from the original label, the first output data and the second output data according to a preset selection mechanism.
7. A language model optimization device, characterized in that: The language model optimization device comprises: An acquisition module is used to acquire an initial language model; the initial language model is obtained by optimizing and training a pre-trained language model using an original training data set; An analysis module, configured to perform capability analysis on the initial language model using a test data set to determine the damage capability of the initial language model; An updating module, configured to obtain a training data group corresponding to the damage capacity from all training data groups included in the original training data set, to obtain a training data group to be processed; Optimizing the training data set to be processed to obtain an optimized training data set, and updating the original training data set based on the optimized training data set to obtain an updated training data set; The training module is used to optimize the pre-trained language model using the updated training data set to obtain a target language model.
8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the language model optimization method according to any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the language model optimization method according to any one of claims 1 to 6 is implemented.
10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the language model optimization method according to any one of claims 1 to 6.
