Continuous model training method, device and equipment based on joint measurement index selection

Abstract

The invention provides a model continuous training method, device and equipment based on joint measurement index selection, and the method comprises the steps: determining a first cross entropy loss value of a candidate data set for a first small language model, and determining a second cross entropy loss value of a training data set for a second small language model, calculating a first difference value between the first cross entropy loss value and the second cross entropy loss value, determining a first target data set from the candidate data set based on the first difference value and a first optimal transmission distance between the candidate data set and the training data set, and continuously training a first small language model based on the first target data set to obtain a second small language model; according to the method, the data quality of the first target data set for training the first small language model can be improved to a certain extent, so that the prediction accuracy of the target small language model can be improved to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a method, device and equipment for continuous model training based on joint metric selection. Background Art

[0002] In past research, there has been an increasing demand for improving the reasoning capabilities of small language models. Although several existing large-scale language models have demonstrated powerful reasoning capabilities in complex tasks, their huge computing resource requirements limit their widespread use in practical applications. How to effectively utilize the reasoning capabilities of language models in an environment with limited resources has become an emerging research topic.

[0003] In related technologies, the reasoning ability of a large model is "distilled" into a smaller model mainly through knowledge distillation. This method usually relies on the reasoning path or data generated by the large teacher model, and uses this data to fine-tune the small model. It is also possible to use a large model to generate synthetic data and use this data to fine-tune the small model. This type of method is common in task scenarios that require complex reasoning, such as arithmetic reasoning, logical reasoning, etc.

[0004] However, in the above method, the quality of the training data set used in the training stage of the small model is low, resulting in low prediction accuracy of the trained small language model. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a method, device, electronic device and readable storage medium for continuous model training based on joint metric selection, so as to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application provides a model continuous training method based on joint metric selection, the method comprising: Determine a first cross entropy loss value of the candidate data set for the first small language model, and determine a second cross entropy loss value of the training data set for the second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different; Calculating a first difference between the first cross entropy loss value and the second cross entropy loss value; Determine a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set; Based on the first target data set, the first small language model is continuously trained to obtain a target small language model.

[0007] Optionally, determining a first cross entropy loss value of the candidate dataset for the first small language model includes: Inputting first sample text data in the candidate data set into a first small language model to obtain a first prediction vector output by the first small language model; Based on the first prediction vector and the first annotation vector of the first sample text data, a first cross entropy loss value of the candidate data set for the first small language model is determined.

[0008] Optionally, determining a second cross entropy loss value of the training dataset for a second small language model includes: Inputting second sample text data in the training data set into a second small language model to obtain a second prediction vector output by the second small language model; Based on the second prediction vector and the second annotation vector of the second sample text data, a second cross entropy loss value of the training data set for the second small language model is determined.

[0009] Optionally, determining a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set includes: A first target data set is determined from the candidate data sets, wherein the first difference is greater than or equal to a first threshold, and a first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold.

[0010] Optionally, determining a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set includes: Determine a first gradient of a first optimal transmission distance between the candidate data set and the training data set for each of the candidate data sets; Calculating a first quality evaluation index of the candidate data set based on the first gradient, a regularization coefficient of the first gradient, and the first difference; Determine a first target data set from the candidate data sets, in which the first quality evaluation index is greater than or equal to a third threshold.

[0011] Optionally, the continuously training the first small language model based on the first target data set to obtain a target small language model includes: Vectorizing the first sample text data contained in the first target data set to obtain a first sample text vector; Inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; Calculating a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data; Based on the third cross entropy loss value, the model parameters of the first small language model are adjusted to obtain a target small language model.

[0012] Optionally, adjusting the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model includes: Based on the third cross entropy loss value, adjusting the model parameters of the first small language model to obtain the first small language model after parameter adjustment; Calculating a second difference between the third cross entropy loss value and the second cross entropy loss value; Determining a second quality evaluation index of the first target data set based on the second difference, a regularization coefficient of the second difference, and a second optimal transmission distance between the first target data set and the training data set; Determine, from the first target data set, a second target data set in which the second quality evaluation index is greater than or equal to a third threshold; Vectorizing the second sample text data contained in the second target data set to obtain a second sample text vector; Inputting the second sample text vector into the first small language model after parameter adjustment to obtain a second sample prediction vector output by the first small language model after parameter adjustment; Calculating a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data; Calculate the second sample prediction vector and the second sample text based on the fourth cross entropy loss value, adjust the model parameters of the first small language model after the parameter adjustment, and obtain a fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model.

[0013] Optionally, adjusting the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model includes: Based on the third cross entropy loss value, adjusting the model parameters of the first small language model to obtain the first small language model after parameter adjustment; Re-performing the step of inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; Counting the number of first consecutive occurrences of the situation where the third cross entropy loss value is less than or equal to the fourth threshold and the number of training cycles of the first small language model; When the first number is greater than or equal to a fifth threshold, or the number of training cycles is greater than or equal to a sixth threshold, stop adjusting the model parameters of the first small language model to obtain a target small language model.

[0014] In a second aspect, an embodiment of the present application provides a model continuous training device based on joint metric selection, the device comprising: A first determination module is used to determine a first cross entropy loss value of a candidate data set for a first small language model, and to determine a second cross entropy loss value of a training data set for a second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different; A calculation module, used to calculate a first difference between the first cross entropy loss value and the second cross entropy loss value; a second determination module, configured to determine a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set; A continuous training module is used to continuously train the first small language model based on the first target data set to obtain a target small language model.

[0015] Optionally, the first determining module includes: A first input-output submodule, configured to input first sample text data in the candidate data set into a first small language model to obtain a first prediction vector output by the first small language model; The first determination submodule is used to determine a first cross entropy loss value of the candidate data set for the first small language model based on the first prediction vector and the first annotation vector of the first sample text data.

[0016] Optionally, the first determining module includes: A second input-output submodule, configured to input second sample text data in the training data set into a second small language model to obtain a second prediction vector output by the second small language model; The second determination submodule is used to determine a second cross entropy loss value of the training data set for the second small language model based on the second prediction vector and the second annotation vector of the second sample text data.

[0017] Optionally, the second determining module includes: The third determination submodule is used to determine, from the candidate data sets, a first target data set in which the first difference is greater than or equal to a first threshold and a first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold.

[0018] Optionally, the second determining module includes: A fourth determination submodule, configured to determine a first gradient of a first optimal transmission distance between the candidate data set and the training data set for each of the candidate data sets; a first calculation submodule, configured to calculate a first quality evaluation index of the candidate data set based on the first gradient, a regularization coefficient of the first gradient, and the first difference; The fifth determination submodule is configured to determine, from the candidate data sets, a first target data set whose first quality evaluation index is greater than or equal to a third threshold.

[0019] Optionally, the continuous training module includes: A vectorization submodule, used for vectorizing the first sample text data contained in the first target data set to obtain a first sample text vector; A third input-output submodule, configured to input the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; A second calculation submodule, used to calculate a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data; An adjustment submodule is used to adjust the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model.

[0020] Optionally, the adjustment submodule includes: A first adjustment unit, configured to adjust a model parameter of the first small language model based on the third cross entropy loss value to obtain a first small language model after parameter adjustment; A first calculation unit, configured to calculate a second difference between the third cross entropy loss value and the second cross entropy loss value; A first determining unit, configured to determine a second quality evaluation index of the first target data set based on the second difference, a regularization coefficient of the second difference, and a second optimal transmission distance between the first target data set and the training data set; A second determining unit, configured to determine, from the first target data set, a second target data set in which the second quality evaluation index is greater than or equal to a third threshold; A vectorization unit, used for vectorizing the second sample text data contained in the second target data set to obtain a second sample text vector; An input-output unit, used for inputting the second sample text vector into the first small language model after parameter adjustment to obtain a second sample prediction vector output by the first small language model after parameter adjustment; A second calculation unit, used to calculate a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data; The second adjustment unit is used to calculate the second sample prediction vector and the second sample text number based on the fourth cross entropy loss value, adjust the model parameters of the first small language model after the parameter adjustment, and obtain the fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model.

[0021] Optionally, the adjustment submodule includes: A third adjustment unit is used to adjust the model parameters of the first small language model based on the third cross entropy loss value to obtain the first small language model after parameter adjustment; an execution unit, configured to re-execute the step of inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; A statistical unit, configured to count the number of first consecutive occurrences of the situation where the third cross entropy loss value is less than or equal to a fourth threshold and the number of training cycles of the first small language model; The stopping module is used to stop adjusting the model parameters of the first small language model to obtain a target small language model when the first number is greater than or equal to a fifth threshold or the number of training cycles is greater than or equal to a sixth threshold.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement a model continuous training method based on joint metric selection as described in any one of the above.

[0023] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, a model continuous training method based on joint metric selection as described in any one of the above items is implemented.

[0024] The specific beneficial effects are: In the embodiment of the present application, a first cross entropy loss value of a candidate data set for a first small language model is determined, and a second cross entropy loss value of a training data set for a second small language model is determined; wherein the second small language model is a model obtained after the model parameters of the first small language model are adjusted; the sample text data contained in the candidate data set and the training data set are different, a first difference between the first cross entropy loss value and the second cross entropy loss value is calculated, and a first target data set is determined from the candidate data set based on the first difference and a first optimal transmission distance between the candidate data set and the training data set, and the first small language model is continuously trained based on the first target data set to obtain a target small language model, and a first difference between the first cross entropy loss value of the candidate data set for the first small language model and the second cross entropy loss value of the training data set for the second small language model can be calculated, and the first target data set can be screened from the candidate set according to the first difference and the first optimal transmission distance between the candidate data set and the training data set, so that the first small language model can be continuously trained according to the first target data set to obtain the target small language model, and the data quality of the first target data set used for training the first small language model can be improved to a certain extent, thereby improving the prediction accuracy of the target small language model to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 It is a flowchart of a method for continuous model training based on joint metric selection provided in an embodiment of the present application; Figure 2 It is a flowchart of another method for continuous model training based on joint metric selection provided in an embodiment of the present application; Figure 3 It is a flowchart of a specific implementation method of a model continuous training method based on joint metric indicator selection provided in an embodiment of the present application; Figure 4 It is a logic block diagram of a model continuous training device based on joint metric selection provided by an embodiment of the present application; Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0028] Reference Figure 1 , Figure 1 A flowchart of a method for continuous model training based on joint metric selection provided in an embodiment of the present application, the method comprising: Step 101, determining a first cross entropy loss value of a candidate data set for a first small language model, and determining a second cross entropy loss value of a training data set for a second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different.

[0029] In an embodiment of the present application, a small language model (SLM) is a technology in the field of artificial intelligence, which refers to a model with a smaller scale and a smaller number of parameters compared to a large language model. These models can usually achieve better performance at a lower computing cost when dealing with specific tasks or fields. The second small language model may be a model obtained after the model parameters of the first small language model are adjusted. The candidate data set may include multiple data sets, and the training data set may be a data set that has been determined to meet the data quality requirements, and the sample text data contained in the candidate data set and the training data set are different, that is, the training data set can be understood as a reference data set for the candidate data set. The cross entropy loss value in the embodiment of the present application can be defined as the form shown in Formula 1 below, as follows: (Formula 1) In the above formula 1, represents the first cross entropy loss value or the second cross entropy loss value, represents the logarithmic sign, Represents the input data of the small language model as a dataset D Sample text data in x When , the output is label y The probability of x The corresponding output data With data labels The similarity between them is substituted. D Can be a candidate dataset or a training dataset.

[0030] In an embodiment of the present application, the sample text data in the candidate data set can be input into the first small language model to obtain the predicted data output by the first small language model, and then the first cross entropy loss value can be calculated based on the predicted data and the data labels in the candidate data set, that is, the first cross entropy loss value of the candidate data set for the first small language model. Similarly, the sample text data in the training data set can be input into the second small language model to obtain the predicted data output by the second small language model, and then the second cross entropy loss value can be calculated based on the predicted data and the data labels in the predicted data set, that is, the second cross entropy loss value of the training data set for the second small language model.

[0031] Step 102: Calculate a first difference between the first cross entropy loss value and the second cross entropy loss value.

[0032] In an embodiment of the present application, a first difference between the first cross entropy loss value and the second cross entropy loss value may be calculated as shown in Formula 2 below: (Formula 2) In the above formula 2, represents the first difference, represents the first cross entropy loss value, Represents the second cross entropy loss value, and the symbol “||” represents the absolute value.

[0033] Step 103: determining a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set.

[0034] In an embodiment of the present application, the first target data set can be screened from the candidate data set based on the first difference and the first optimal transmission distance between the candidate data set and the training data set. Among them, the optimal transmission (OptimalTransport) distance is a measurement method for measuring the difference between two probability distributions. The core idea is to "convert" or "transport" one probability distribution to another probability distribution in some way so that the total cost of this conversion is minimized. First, both the candidate data set and the training data set can be vectorized to obtain the data distribution of the candidate data set and the data distribution of the training data set. After that, the first optimal transmission distance between the candidate data set and the training data set can be calculated based on the data distribution of the selected data set and the data distribution of the training data set. The detailed process of calculating the first optimal transmission distance can be implemented by some computer programs. For example, the POT toolbox in Python can be used to calculate the first optimal transmission distance.

[0035] Step 104: Based on the first target data set, continuously train the first small language model to obtain a target small language model.

[0036] In an embodiment of the present application, the first small language model can be continuously trained according to the first target data set to obtain a target small language model. Among them, continuous training can mean that the termination condition of the training process of the first small language model is that the training cycle of the first small language model reaches a preset number of times. That is, after the training cycle of the first small language model reaches a preset number of times, the training process of the first small language model can be stopped, so that the first small language model finally obtained can be determined as the target small language model. The training process of the first small language model is the same as the training process of the common deep learning language model, which will not be repeated here.

[0037] In an embodiment of the present application, a first cross entropy loss value of a candidate data set for a first small language model is determined, and a second cross entropy loss value of a training data set for a second small language model is determined; wherein the second small language model is a model obtained after the model parameters of the first small language model are adjusted; the sample text data contained in the candidate data set and the training data set are different, a first difference between the first cross entropy loss value and the second cross entropy loss value is calculated, and based on the first difference and the first optimal transmission distance between the candidate data set and the training data set, a first target data set is determined from the candidate data set, and based on the first target data set, the first small language model is continuously trained to obtain a target small language model, and the first difference between the first cross entropy loss value of the candidate data set for the first small language model and the second cross entropy loss value of the training data set for the second small language model can be calculated, and the first target data set can be screened from the candidate set according to the first difference and the first optimal transmission distance between the candidate data set and the training data set, so that the first small language model can be continuously trained according to the first target data set to obtain the target small language model, which can improve the data quality of the first target data set used to train the first small language model to a certain extent, thereby improving the prediction accuracy of the target small language model to a certain extent.

[0038] Reference Figure 2 , Figure 2 A flowchart of another method for continuous model training based on joint metric selection provided in an embodiment of the present application, the method may include: Step 201, determining a first cross entropy loss value of a candidate data set for a first small language model, and determining a second cross entropy loss value of a training data set for a second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different.

[0039] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 101, which will not be repeated here.

[0040] Optionally, step 201 may include the following sub-steps: Sub-step 2011: inputting the first sample text data in the candidate data set into the first small language model to obtain a first prediction vector output by the first small language model.

[0041] In an embodiment of the present application, the first sample text data in the candidate data set can be input into the first small language model, so as to obtain the first prediction vector output by the first small language model. The first sample text data can be vectorized through an embedding module (embedding) to obtain the first sample text vector, and then the first sample text vector is input into the first small language model as the input data of the first small language model, so as to obtain the first prediction vector output by the first small language model. Generally, the text information corresponding to the first prediction vector and the first sample text vector can constitute a conversation, which can include questions and answers.

[0042] Sub-step 2012: determining a first cross entropy loss value of the candidate data set for the first small language model based on the first prediction vector and the first annotation vector of the first sample text data.

[0043] In an embodiment of the present application, a first cross entropy loss value of the candidate data set for the first small language model can be calculated based on the first prediction vector and the first annotation vector of the first sample text data. The first annotation vector can be obtained by vectorizing the label of the first sample text data. The calculation method of the first cross entropy loss value can refer to Formula 1, where, The first sample text data can be used x The similarity between the corresponding first prediction vector and the first annotation vector is replaced.

[0044] In an embodiment of the present application, by inputting the first sample text data in the candidate data set into the first small language model, a first prediction vector output by the first small language model is obtained, and based on the first prediction vector and the first annotation vector of the first sample text data, a first cross-entropy loss value of the candidate data set for the first small language model is determined. The first small language model can be used, combined with calculation to obtain the first cross-entropy loss value corresponding to the candidate data set, thereby improving the accuracy of the first cross-entropy loss value to a certain extent.

[0045] Sub-step 2013: input the second sample text data in the training data set into the second small language model to obtain a second prediction vector output by the second small language model.

[0046] In an embodiment of the present application, the second sample text data of the training data set can be input into the second small language model, so as to obtain a second prediction vector output by the second small language model. The second sample text data can be first vectorized through the embedding module to obtain a second sample text vector, and then the second sample text vector is input into the second small language model as input data of the second small language model, so as to obtain a second prediction vector output by the second small language model.

[0047] Sub-step 2014: determining a second cross entropy loss value of the training data set for the second small language model based on the second prediction vector and the second annotation vector of the second sample text data.

[0048] In an embodiment of the present application, a second cross entropy loss value of the training data set for the second small language model can be calculated based on the second prediction vector and the second annotation vector of the second sample text data. The second annotation vector can be obtained by vectorizing the label of the second sample text data. The calculation method of the second cross entropy loss value can refer to Formula 1, where, The second sample text data can be used x The similarity between the corresponding second prediction vector and the second annotation vector is used instead.

[0049] In an embodiment of the present application, by inputting the second sample text data in the training data set into the second small language model, a second prediction vector output by the second small language model is obtained, and based on the second prediction vector and the second annotation vector of the second sample text data, a second cross-entropy loss value of the training data set for the second small language model is determined. The second small language model can be used, combined with calculation to obtain the second cross-entropy loss value corresponding to the training data set, thereby improving the accuracy of the second cross-entropy loss value to a certain extent.

[0050] Step 202: Calculate a first difference between the first cross entropy loss value and the second cross entropy loss value.

[0051] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 102, which will not be repeated here.

[0052] Step 203: determining a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set.

[0053] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 103, which will not be repeated here.

[0054] Optionally, step 203 may include the following sub-steps: Sub-step 2031: determining, from the candidate data sets, a first target data set whose first difference is greater than or equal to a first threshold and whose first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold.

[0055] In an embodiment of the present application, a candidate data set whose first difference is greater than or equal to a first threshold and whose first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold may be used as the first target data set. There may be multiple candidate data sets.

[0056] In an embodiment of the present application, by determining from the candidate data sets a first target data set whose first difference is greater than or equal to the first threshold, and whose first optimal transmission distance between the candidate data set and the training data set is greater than or equal to the second threshold, the relationship between the first threshold and the first difference, as well as the relationship between the first optimal transmission distance between the candidate data set and the training data set and the second threshold can be used to screen out the first target data set from the candidate data sets, thereby improving the data quality of the first target data set to a certain extent.

[0057] Sub-step 2032: determining a first gradient of a first optimal transmission distance between the candidate data set and the training data set for each of the candidate data sets.

[0058] In an embodiment of the present application, the first gradient of the first optimal transmission distance between the candidate data set and the training data set for each candidate set may be calculated, and the calculation method thereof is shown in the following formula 3: (Formula 3) In the above formula 3, Indicates the first optimal transmission distance The first gradient for each candidate data set, represents candidate data set i, and N is the number of candidate data sets.

[0059] Sub-step 2033: calculating a first quality evaluation index of the candidate data set based on the first gradient, the regularization coefficient of the first gradient and the first difference.

[0060] In an embodiment of the present application, the first gradient may have a corresponding regularization coefficient, and the first quality evaluation index of the candidate data set may be calculated by the first gradient, the regularization coefficient of the first gradient, and the first difference, and the calculation method thereof is shown in Formula 4 below: (Formula 4) In the above formula 4, represents the first quality evaluation indicator of the candidate data set, represents the first difference, represents the first gradient, Represents the regularization coefficient of the first gradient, and its value range is usually (0, 1).

[0061] Sub-step 2034: determining, from the candidate data sets, a first target data set whose first quality evaluation index is greater than or equal to a third threshold.

[0062] In an embodiment of the present application, the first quality evaluation index can be used as a joint measurement index for the quality of the data set composed of the cross entropy loss and the optimal transmission distance. Therefore, the candidate data set whose first quality evaluation index is greater than or equal to the third threshold can be used as the first target data set.

[0063] In an embodiment of the present application, by determining the first gradient of the first optimal transmission distance between the candidate data set and the training data set for each candidate data set, a first quality evaluation index of the candidate data set is calculated based on the first gradient, the regularization coefficient of the first gradient and the first difference, and a first target data set whose first quality evaluation index is greater than or equal to a third threshold is determined from the candidate data set. The first quality evaluation index of the candidate data set can be calculated according to the first gradient and the first difference in combination with the regularization coefficient, and the first target data set can be screened according to the first quality evaluation index, which improves the data quality of the first target data set to a certain extent. When the first target data set is used for training the first small language model, overfitting of the first small language model can be avoided, thereby improving the prediction accuracy of the target small language model finally obtained.

[0064] Step 204: vectorize the first sample text data contained in the first target data set to obtain a first sample text vector.

[0065] In an embodiment of the present application, the first sample text data contained in the first target data set may be vectorized through an embedding module, thereby obtaining a first sample text vector.

[0066] Step 205: input the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model.

[0067] In an embodiment of the present application, the first sample text vector may be input into the first small language model, so as to obtain a first sample prediction vector output by the first small language model.

[0068] Step 206: Calculate a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data.

[0069] In an embodiment of the present application, a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data can be calculated. The sample annotation vector can be obtained by vectorizing the annotation label corresponding to the first sample text data. The calculation method of the third cross entropy loss value can refer to Formula 1, where The first sample text data can be used x The similarity between the corresponding first sample prediction vector and the sample annotation vector is replaced.

[0070] Step 207: Based on the third cross entropy loss value, adjust the model parameters of the first small language model to obtain a target small language model.

[0071] Optionally, step 207 may include the following sub-steps: Sub-step 20701: Based on the third cross entropy loss value, adjust the model parameters of the first small language model to obtain the first small language model after parameter adjustment.

[0072] In an embodiment of the present application, the model parameters of the first small language model can be adjusted according to the third cross entropy loss value, so as to obtain the first small language model after parameter adjustment. The adjustment direction of the model parameters can be a direction that reduces the third cross entropy loss value.

[0073] Sub-step 20702, calculating a second difference between the third cross entropy loss value and the second cross entropy loss value.

[0074] In an embodiment of the present application, a second difference between the third cross entropy loss value and the second cross entropy loss value may be calculated. The calculation method thereof may refer to Formula 2, which will not be described in detail here.

[0075] Sub-step 20703, determining a second quality evaluation index of the first target data set based on the second difference, the regularization coefficient of the second difference, and the second optimal transmission distance between the first target data set and the training data set.

[0076] In an embodiment of the present application, a second quality evaluation index of the first target data set can be calculated based on the second difference, the regularization coefficient of the second difference, and the second optimal transmission distance between the first target data set and the training data set. The calculation method of the second quality evaluation index is similar to that of the first quality evaluation index, and can be referred to Formula 4, which is not repeated here.

[0077] Sub-step 20704: determining, from the first target data set, a second target data set whose second quality evaluation index is greater than or equal to a third threshold.

[0078] In an embodiment of the present application, a second target data set whose second quality evaluation index is greater than or equal to a third threshold value can be obtained by screening from the first target data set. That is, the second target data set is obtained by screening from the first target data set with the second quality evaluation index being greater than or equal to the third threshold value as a screening condition.

[0079] Sub-step 20705: vectorize the second sample text data contained in the second target data set to obtain a second sample text vector.

[0080] In an embodiment of the present application, the second sample text data contained in the second target data set may be vectorized through an embedding module, thereby obtaining a second sample text vector.

[0081] Sub-step 20706: input the second sample text vector into the first small language model after parameter adjustment to obtain a second sample prediction vector output by the first small language model after parameter adjustment.

[0082] In an embodiment of the present application, the second sample text vector may be input into the first small language model after parameter adjustment, so as to obtain a second sample prediction vector output by the first small language model after parameter adjustment.

[0083] Sub-step 20707, calculating a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data.

[0084] In an embodiment of the present application, a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data can be calculated. The specific calculation process can refer to Formula 1, where It can be replaced by the similarity between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data x.

[0085] Sub-step 20708, calculating the second sample prediction vector and the second sample text number based on the fourth cross entropy loss value, adjusting the model parameters of the first small language model after the parameter adjustment, and obtaining the fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model.

[0086] In an embodiment of the present application, the model parameters of the first small language model after parameter adjustment can be adjusted again according to the fourth cross entropy loss value, so as to obtain a target small language model. The direction of adjusting the above model parameters can be the direction in which the fourth cross entropy loss value is reduced. In addition, the above steps 20701 to 20708 can be repeated multiple times to perform multiple model training on the first small language model, and the training of the first small language model can be stopped when the training process of the first small language model meets the convergence condition, so that the latest first small language model can be used as the target small language model.

[0087] In an embodiment of the present application, by adjusting the model parameters of the first small language model based on the third cross entropy loss value, obtaining the first small language model after parameter adjustment, calculating the second difference between the third cross entropy loss value and the second cross entropy loss value, determining the second quality evaluation index of the first target data set based on the second difference, the regularization coefficient of the second difference, and the second optimal transmission distance between the first target data set and the training data set, determining a second target data set whose second quality evaluation index is greater than or equal to the third threshold from the first target data set, vectorizing the second sample text data contained in the second target data set to obtain a second sample text vector, and inputting the second sample text vector into the first small language model after parameter adjustment. In the model, a second sample prediction vector output by the first small language model after parameter adjustment is obtained, a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data is calculated, and the second sample prediction vector and the second sample text data are calculated based on the fourth cross entropy loss value. The model parameters of the first small language model after parameter adjustment are adjusted to obtain the fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model. During the training process of the first small language model, the data set used for training can be continuously screened, the data quality of the screened data set can be improved, the convergence speed of the first small language model can be improved, and the training time of the first small language model can be shortened.

[0088] Sub-step 20709: adjusting the model parameters of the first small language model based on the third cross entropy loss value to obtain the first small language model after parameter adjustment.

[0089] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 20701 and will not be repeated here.

[0090] Sub-step 20710: re-execute the step of inputting the first sample text vector into the first small language model to obtain the first sample prediction vector output by the first small language model.

[0091] In an embodiment of the present application, after obtaining the first small language model after parameter adjustment, the step of inputting the first sample text vector into the first small language model to obtain the first sample prediction vector output by the first small language model can be re-executed to repeatedly train the first small language model.

[0092] Sub-step 20711, counting the number of first consecutive occurrences of the situation where the third cross entropy loss value is less than or equal to the fourth threshold and the number of training cycles of the first small language model.

[0093] In an embodiment of the present application, the first number of consecutive occurrences of the third cross entropy loss value being less than or equal to the fourth threshold value can be counted, and the number of training cycles of the first small language model can be counted at the same time. Among them, the initial value of the first number is 0. When the third cross entropy loss value is less than or equal to the fourth threshold value for the first time, the first number can be increased by one. If the third cross entropy loss value is less than or equal to the fourth threshold value continues to appear, the first number is continuously accumulated. If the third cross entropy loss value is greater than the fourth threshold value once, the first number is set to zero, and the detection of the third cross entropy loss value being less than or equal to the fourth threshold value is restarted. The fourth threshold value can be set manually.

[0094] Sub-step 20712: When the first number is greater than or equal to a fifth threshold, or the number of training cycles is greater than or equal to a sixth threshold, stop adjusting the model parameters of the first small language model to obtain a target small language model.

[0095] In an embodiment of the present application, if the first number is greater than or equal to the fifth threshold, or the number of training cycles of the first small language model is greater than or equal to the sixth threshold, it can be considered that the training of the first small language model meets the convergence condition. At this time, the adjustment of the model parameters of the first small language model can be stopped, and the first small language model obtained after the last model parameter adjustment is used as the target small language model.

[0096] In an embodiment of the present application, by adjusting the model parameters of the first small language model based on the third cross entropy loss value, obtaining the first small language model after parameter adjustment, re-executing the step of inputting the first sample text vector into the first small language model to obtain the first sample prediction vector output by the first small language model, counting the first number of consecutive occurrences of the third cross entropy loss value being less than or equal to the fourth threshold and the number of training cycles of the first small language model, when the first number is greater than or equal to the fifth threshold, or the number of training cycles is greater than or equal to the sixth threshold, stopping adjusting the model parameters of the first small language model to obtain a target small language model, the first small language model can be continuously trained to obtain the target small language model, overfitting of the first small language model can be avoided, and the prediction accuracy of the target small language model can be improved to a certain extent.

[0097] In an embodiment of the present application, the first sample text data contained in the first target data set is vectorized to obtain a first sample text vector, the first sample text vector is input into the first small language model, and the first sample prediction vector output by the first small language model is obtained, and a third cross-entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data is calculated. Based on the third cross-entropy loss value, the model parameters of the first small language model are adjusted to obtain a target small language model. The first target data set can be used to train the first small language model to obtain the target small language model, which can avoid overfitting of the first small language model during the training process and improve the prediction accuracy of the target small language model to a certain extent.

[0098] Reference Figure 3 , Figure 3 It is a flowchart of a specific implementation method of a model continuous training method based on joint metric selection provided by an embodiment of the present application. In the figure, the implementation process of the method can be divided into two stages, namely the data screening stage and the continuous training stage. In the data screening stage, the training data set can be used as a reference data set, and the joint quality index composed of the difference in cross entropy loss values ​​and the gradient of OT distance can be used to screen the candidate data set to obtain a first target data set. In the continuous training stage, the first target data set can be used to continuously train the small language model, and after the training is completed, the target small language model can be obtained. In addition, the training data of a specific task can be used to fine-tune the data of the target small language model so that the target small language model can better meet the needs of the specific task.

[0099] Reference Figure 4 , Figure 4A logical block diagram of a model continuous training device based on joint metric selection provided in an embodiment of the present application, the device 400 may include: A first determination module 401 is used to determine a first cross entropy loss value of a candidate data set for a first small language model, and to determine a second cross entropy loss value of a training data set for a second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different; A calculation module 402 is used to calculate a first difference between the first cross entropy loss value and the second cross entropy loss value; A second determination module 403 is configured to determine a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set; The continuous training module 404 is used to continuously train the first small language model based on the first target data set to obtain a target small language model.

[0100] Optionally, the first determining module 401 includes: A first input-output submodule, configured to input first sample text data in the candidate data set into a first small language model to obtain a first prediction vector output by the first small language model; The first determination submodule is used to determine a first cross entropy loss value of the candidate data set for the first small language model based on the first prediction vector and the first annotation vector of the first sample text data.

[0101] Optionally, the first determining module 401 includes: A second input-output submodule, configured to input second sample text data in the training data set into a second small language model to obtain a second prediction vector output by the second small language model; The second determination submodule is used to determine a second cross entropy loss value of the training data set for the second small language model based on the second prediction vector and the second annotation vector of the second sample text data.

[0102] Optionally, the second determining module 403 includes: The third determination submodule is used to determine, from the candidate data sets, a first target data set in which the first difference is greater than or equal to a first threshold and a first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold.

[0103] Optionally, the second determining module 403 includes: A fourth determination submodule, configured to determine a first gradient of a first optimal transmission distance between the candidate data set and the training data set for each of the candidate data sets; a first calculation submodule, configured to calculate a first quality evaluation index of the candidate data set based on the first gradient, a regularization coefficient of the first gradient, and the first difference; The fifth determination submodule is configured to determine, from the candidate data sets, a first target data set whose first quality evaluation index is greater than or equal to a third threshold.

[0104] Optionally, the continuous training module 404 includes: A vectorization submodule, used for vectorizing the first sample text data contained in the first target data set to obtain a first sample text vector; A third input-output submodule, configured to input the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; A second calculation submodule, used to calculate a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data; An adjustment submodule is used to adjust the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model.

[0105] Optionally, the adjustment submodule includes: A first adjustment unit, configured to adjust a model parameter of the first small language model based on the third cross entropy loss value to obtain a first small language model after parameter adjustment; A first calculation unit, configured to calculate a second difference between the third cross entropy loss value and the second cross entropy loss value; A first determining unit, configured to determine a second quality evaluation index of the first target data set based on the second difference, a regularization coefficient of the second difference, and a second optimal transmission distance between the first target data set and the training data set; A second determining unit, configured to determine, from the first target data set, a second target data set in which the second quality evaluation index is greater than or equal to a third threshold; A vectorization unit, used for vectorizing the second sample text data contained in the second target data set to obtain a second sample text vector; An input-output unit, used for inputting the second sample text vector into the first small language model after parameter adjustment to obtain a second sample prediction vector output by the first small language model after parameter adjustment; A second calculation unit, used to calculate a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data; The second adjustment unit is used to calculate the second sample prediction vector and the second sample text number based on the fourth cross entropy loss value, adjust the model parameters of the first small language model after the parameter adjustment, and obtain the fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model.

[0106] Optionally, the adjustment submodule includes: A third adjustment unit is used to adjust the model parameters of the first small language model based on the third cross entropy loss value to obtain the first small language model after parameter adjustment; an execution unit, configured to re-execute the step of inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; A statistical unit, configured to count the number of first consecutive occurrences of the situation where the third cross entropy loss value is less than or equal to a fourth threshold and the number of training cycles of the first small language model; The stopping module is used to stop adjusting the model parameters of the first small language model to obtain a target small language model when the first number is greater than or equal to a fifth threshold or the number of training cycles is greater than or equal to a sixth threshold.

[0107] The model continuous training device based on the joint metric selection in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a GPUBOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0108] The model continuous training device based on joint metric selection in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, a Linux, a Windows operating system, etc., and can also be other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0109] The model continuous training device based on joint metric selection provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0110] The present application embodiment provides an electronic device, see Figure 5 The electronic device 50 includes: a processor 501, a memory 502, and a computer program 5021 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the program, the model continuous training method based on joint metric selection of the aforementioned embodiment is implemented.

[0111] An embodiment of the present application also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps in the model continuous training method based on joint metric indicator selection as disclosed in the embodiment of the present application.

[0112] An embodiment of the present application also provides a computer program product, which, when executed on an electronic device, enables a processor to implement the steps in the model continuous training method based on joint metric indicator selection as disclosed in the embodiment of the present application.

[0113] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0114] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0117] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0118] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0119] The above is a detailed introduction to a model continuous training method based on joint metric indicator selection provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model continuous training method based on joint metric selection, characterized in that: The method comprises: Determine a first cross entropy loss value of the candidate data set for the first small language model, and determine a second cross entropy loss value of the training data set for the second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different; Calculating a first difference between the first cross entropy loss value and the second cross entropy loss value; Determine a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set; Based on the first target data set, the first small language model is continuously trained to obtain a target small language model.

2. The method according to claim 1, characterized in that The determining a first cross entropy loss value of the candidate data set for the first small language model includes: Inputting first sample text data in the candidate data set into a first small language model to obtain a first prediction vector output by the first small language model; Based on the first prediction vector and the first annotation vector of the first sample text data, a first cross entropy loss value of the candidate data set for the first small language model is determined.

3. The method according to claim 1, characterized in that The determining a second cross entropy loss value of the training data set for the second small language model includes: Inputting second sample text data in the training data set into a second small language model to obtain a second prediction vector output by the second small language model; Based on the second prediction vector and the second annotation vector of the second sample text data, a second cross entropy loss value of the training data set for the second small language model is determined.

4. The method according to claim 1, characterized in that: The determining a first target data set from the candidate data set based on the first difference and a first optimal transmission distance between the candidate data set and the training data set includes: A first target data set is determined from the candidate data sets, wherein the first difference is greater than or equal to a first threshold, and a first optimal transmission distance between the candidate data set and the training data set is greater than or equal to a second threshold.

5. The method according to claim 1, characterized in that The determining a first target data set from the candidate data set based on the first difference and a first optimal transmission distance between the candidate data set and the training data set includes: Determine a first gradient of a first optimal transmission distance between the candidate data set and the training data set for each of the candidate data sets; Calculating a first quality evaluation index of the candidate data set based on the first gradient, a regularization coefficient of the first gradient, and the first difference; Determine a first target data set from the candidate data sets, in which the first quality evaluation index is greater than or equal to a third threshold.

6. The method according to claim 1, characterized in that The step of continuously training the first small language model based on the first target data set to obtain a target small language model includes: Vectorizing the first sample text data contained in the first target data set to obtain a first sample text vector; Inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; Calculating a third cross entropy loss value between the first sample prediction vector and the sample annotation vector corresponding to the first sample text data; Based on the third cross entropy loss value, the model parameters of the first small language model are adjusted to obtain a target small language model.

7. The method according to claim 6, characterized in that The adjusting the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model includes: Based on the third cross entropy loss value, adjusting the model parameters of the first small language model to obtain the first small language model after parameter adjustment; Calculating a second difference between the third cross entropy loss value and the second cross entropy loss value; Determining a second quality evaluation index of the first target data set based on the second difference, a regularization coefficient of the second difference, and a second optimal transmission distance between the first target data set and the training data set; Determine, from the first target data set, a second target data set in which the second quality evaluation index is greater than or equal to a third threshold; Vectorizing the second sample text data contained in the second target data set to obtain a second sample text vector; Inputting the second sample text vector into the first small language model after parameter adjustment to obtain a second sample prediction vector output by the first small language model after parameter adjustment; Calculating a fourth cross entropy loss value between the second sample prediction vector and the sample annotation vector corresponding to the second sample text data; Calculate the second sample prediction vector and the second sample text based on the fourth cross entropy loss value, adjust the model parameters of the first small language model after the parameter adjustment, and obtain a fourth cross entropy loss value between the sample annotation vectors corresponding to the target small language model.

8. The method according to claim 6, characterized in that The adjusting the model parameters of the first small language model based on the third cross entropy loss value to obtain a target small language model includes: Based on the third cross entropy loss value, adjusting the model parameters of the first small language model to obtain the first small language model after parameter adjustment; Re-performing the step of inputting the first sample text vector into the first small language model to obtain a first sample prediction vector output by the first small language model; Counting the number of first consecutive occurrences of the situation where the third cross entropy loss value is less than or equal to the fourth threshold and the number of training cycles of the first small language model; When the first number is greater than or equal to a fifth threshold, or the number of training cycles is greater than or equal to a sixth threshold, stop adjusting the model parameters of the first small language model to obtain a target small language model.

9. A model continuous training device based on joint metric selection, characterized in that: The device comprises: A first determination module is used to determine a first cross entropy loss value of a candidate data set for a first small language model, and to determine a second cross entropy loss value of a training data set for a second small language model; wherein the second small language model is a model obtained after adjusting model parameters of the first small language model; and the sample text data contained in the candidate data set and the training data set are different; A calculation module, used to calculate a first difference between the first cross entropy loss value and the second cross entropy loss value; a second determination module, configured to determine a first target data set from the candidate data sets based on the first difference and a first optimal transmission distance between the candidate data set and the training data set; A continuous training module is used to continuously train the first small language model based on the first target data set to obtain a target small language model.

10. An electronic device, characterized in that: It includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the model continuous training method based on joint metric selection as described in any one of claims 1 to 8.

Images