Training method and device of large vertical domain model and storage medium

Abstract

The invention discloses a training method and device for a large vertical domain model and a storage medium, and relates to the technical field of model training, and the method comprises the steps: carrying out the first-stage training of a preset reference model according to an initial training set, and obtaining evaluation indexes before and after the first-stage training; the data in the initial training set are classified according to the evaluation indexes, the data in the initial training set are divided into efficient data, invalid data and difficult case data, and the difficult case data comprise dirty data and actual difficult cases; adjusting the proportion of various types of data in the initial training set, carrying out second-stage training on the reference model, and determining a target training set according to a training result after a preset training round is reached; and performing third-stage training on the reference model according to the target training set, and obtaining a target vertical domain model after a preset stop condition is reached. According to the invention, data can be screened according to a large model and iterative training is carried out.

Description

Technical Field

[0001] The present application relates to the field of model training technology, and in particular to a training method, device and storage medium for a large vertical model. Background Art

[0002] When training the vertical domain model, a pre-trained model in a general field (such as the open source model qwen series, GLM series, etc.) is used as the benchmark model, and the basic model is trained based on the training data set that has undergone basic cleaning.

[0003] Vertical domain models need to focus on knowledge and skills in a specific field to provide high-quality output and better specific task results. However, although the data sets obtained from open source data set websites and general text data have undergone basic cleaning, the data quality is still uneven. In order to ensure the quality and professionalism of the training set, experts in related fields are usually required to manually evaluate and revise the data set, which has low efficiency in data screening and generates high labor costs.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a training method, device and storage medium for a large vertical model, aiming to solve the technical problem of how to screen training data and improve the quality of training data during the training of the large model.

[0006] To achieve the above objectives, the present application proposes a training method for a vertical large model, the method comprising:

[0007] Performing a first-stage training on a preset benchmark model according to an initial training set, and obtaining evaluation indicators before and after the first-stage training;

[0008] Classifying the data in the initial training set according to the evaluation index, dividing the data in the initial training set into efficient data, invalid data, and difficult data, wherein the difficult data includes dirty data and actual difficult examples;

[0009] Adjusting the proportion of each type of data in the initial training set, performing a second phase of training on the benchmark model, and determining a target training set according to the training results after reaching a preset number of training rounds;

[0010] The benchmark model is trained in the third stage according to the target training set, and a target vertical domain model is obtained after a preset stop condition is reached.

[0011] In one embodiment, the step of performing a first-stage training on a preset benchmark model according to an initial training set and obtaining evaluation indicators before and after the first-stage training includes:

[0012] Inputting the initial training set into the benchmark model for evaluation, and obtaining a first evaluation index for each data in the initial training set;

[0013] Fine-tune the benchmark model according to the initial training set to obtain a first control model, and evaluate the first control model to obtain a second evaluation index for each data in the initial training set;

[0014] The benchmark model is fine-tuned and trained according to the first training set to obtain a second control model, and the second control model is evaluated to obtain a third evaluation index for each data in the first training set.

[0015] In one embodiment, after the step of fine-tuning the benchmark model according to the initial training set to obtain a first control model, evaluating the first control model, and obtaining a second evaluation index for each data in the initial training set, the following steps are included:

[0016] In the initial training set, the proportion of the invalid data is reduced and the proportion of the difficult data is increased to obtain the first training set.

[0017] In one embodiment, the step of classifying the data in the initial training set according to the evaluation index and dividing the data in the initial training set into efficient data, invalid data, and difficult data, wherein the difficult data includes dirty data and actual difficult data, comprises:

[0018] Determining the efficient data, the invalid data, and the difficult data according to the first evaluation index and the second evaluation index;

[0019] The dirty data and the actual difficult examples are distinguished from the difficult example data according to the second evaluation index and the third evaluation index.

[0020] In one embodiment, the step of determining the efficient data, the invalid data, and the difficult data according to the first evaluation index and the second evaluation index comprises:

[0021] Obtaining a first average loss value of the benchmark model for the initial training set, and a second average loss value of the first control model for the initial training set;

[0022] Perform binary clustering on the first average loss value and the second average loss value. If both the first average loss value and the second average loss value belong to the low loss class, the corresponding data is the invalid data.

[0023] Obtaining the comprehensive information entropy of the first control model for the initial training set, and the average loss difference value between the first average loss value and the second average loss value, and performing binary classification clustering on the comprehensive information entropy and the average loss difference value;

[0024] If the average loss difference value belongs to the high loss difference value class, and the comprehensive information entropy belongs to the low comprehensive information entropy class, then the corresponding data is the efficient data;

[0025] If there is data in the initial training set that is neither invalid data nor efficient data, then the data is difficult data.

[0026] In one embodiment, the step of distinguishing the dirty data and the actual hard examples from the hard example data according to the second evaluation index and the third evaluation index comprises:

[0027] Obtaining a predicted probability distribution difference of the second control model for each data in the first training set, and performing binary classification clustering on the predicted probability distribution difference of the second control model;

[0028] Obtaining the comprehensive information entropy of the second control model for the second training set, and performing binary classification clustering on the comprehensive information entropy of the second control model;

[0029] If the predicted probability distribution difference of the second control model belongs to a low distribution difference, the difficult example data is dirty data;

[0030] Alternatively, if the predicted probability distribution difference of the second control model belongs to a high distribution difference, and the comprehensive information entropy of the second control model belongs to a high comprehensive information entropy, then the difficult example data is dirty data;

[0031] The hard example data that is not the dirty data is confirmed as the actual hard example.

[0032] In one embodiment, the step of adjusting the proportion of each type of data in the initial training set, performing second-stage training on the benchmark model, and determining the target training set according to the training results after reaching a preset number of training rounds includes:

[0033] In each round of iteration, the initial training set is perturbed a preset number of times according to the perturbation rules of reducing the invalid data, maintaining the invalid data, increasing the actual difficult examples, and discarding the dirty data, to obtain multiple second training sets;

[0034] Performing a second phase of training on the benchmark model according to the second training set;

[0035] After reaching the preset training rounds, the second training set corresponding to the last round is used as the target training set, and the intersection of the second training sets in each round of iteration is used as the third training set.

[0036] In one embodiment, the step of performing the third stage training on the benchmark model according to the target training set and obtaining the target vertical domain model after reaching a preset stop condition comprises:

[0037] Performing a third phase of training on the benchmark model according to the target training set to obtain a first benchmark model;

[0038] The benchmark model is subjected to annealing training according to the third training set, and a target vertical domain model is obtained after a preset stop condition is reached.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the training method of the vertical domain large model as described above are implemented.

[0040] The present application provides a method for training a large vertical domain model. By fine-tuning the baseline model and classifying the data in the training set according to the evaluation indicators before and after training, the training set can be adjusted according to the type of data to avoid over-training of the model on the efficient data that has been mastered, so as to focus more on the processing of difficult data. The baseline model is trained in the third stage according to the target training set, and the target vertical domain model is obtained after the preset stop condition is reached; by using the screened and optimized data set for training, the model can be more focused on learning the core features of vertical domain knowledge. It helps to improve the model's ability to understand and process vertical domain data, thereby improving the accuracy and reliability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 A flowchart diagram of the first embodiment of the training method of the vertical domain large model of the present application is provided;

[0044] Figure 2 A detailed flow chart of the first embodiment of the training method of the vertical domain large model of the present application is provided;

[0045] Figure 3 A flow chart of the second embodiment of the training method of the vertical domain large model of the present application;

[0046] Figure 4 Another flowchart diagram provided for the second embodiment of the training method of the vertical domain large model of the present application;

[0047] Figure 5 A flowchart diagram of the third embodiment of the training method of the vertical domain large model of the present application;

[0048] Figure 6 A flowchart diagram of the fourth embodiment of the training method of the vertical domain large model of the present application is provided;

[0049] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the training method of the vertical domain large model in the embodiment of the present application. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution of the embodiment of the present application is: perform the first stage training on the preset benchmark model according to the initial training set, and obtain the evaluation indicators before and after the first stage training; classify the data in the initial training set according to the evaluation indicators, and divide the data in the initial training set into efficient data, invalid data and difficult data, and the difficult data includes dirty data and actual difficult examples; adjust the proportion of each type of data in the initial training set, perform the second stage training on the benchmark model, and after reaching the preset training rounds, determine the target training set according to the training results; perform the third stage training on the benchmark model according to the target training set, and obtain the target vertical domain model after reaching the preset stop condition.

[0053] When training a vertical domain model, a pre-trained model in a general field (such as the open source model qwen series, GLM series, etc.) is used as a benchmark model, and the basic model is trained based on a training data set that has undergone basic cleaning. The vertical domain model needs to focus on the knowledge and skills in a specific field to provide high-quality output and better specific task results. However, although the data sets obtained from open source data set websites and general text data have undergone basic cleaning, the data quality is still uneven. In order to ensure the quality and professionalism of the training set, experts in related fields are usually required to manually evaluate and revise the data set. The efficiency of data screening is low and high labor costs are incurred.

[0054] The present application provides a method for training a large vertical domain model. By fine-tuning the baseline model and classifying the data in the training set according to the evaluation indicators before and after training, the training set can be adjusted according to the type of data to avoid over-training of the model on the efficient data that has been mastered, so as to focus more on the processing of difficult data. The baseline model is trained in the third stage according to the target training set, and the target vertical domain model is obtained after the preset stop condition is reached; by using the screened and optimized data set for training, the model can be more focused on learning the core features of vertical domain knowledge. It helps to improve the model's ability to understand and process vertical domain data, thereby improving the accuracy and reliability of the model.

[0055] It should be noted that the execution subject of this embodiment can be a computing service device with network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a device, etc. that can realize the above functions. The following takes the training device of the vertical domain large model as an example to illustrate this embodiment and the following embodiments.

[0056] Based on this, the present application embodiment provides a training method for a vertical domain large model, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the training method for a vertical large model of the present application.

[0057] In this embodiment, the training method of the vertical domain large model includes steps S100 to S400:

[0058] Step S100, performing first-stage training on a preset benchmark model according to an initial training set, and obtaining evaluation indicators before and after the first-stage training.

[0059] It should be noted that the benchmark model refers to a pre-trained model in a general field, such as the open source models qwen series, GLM series, etc.

[0060] In this embodiment, first, a benchmark model and an initial training set are obtained, such as BERT, GPT or other pre-trained models in the field of NLP. The initial training set can be obtained from various sources (such as databases, APIs, files, etc.), and is processed by basic cleaning of missing values, deletion or merging of duplicate records, and deletion of invalid characters, garbled characters, data that does not conform to business logic, and other invalid data. Before the first stage of training, the initial training set is used to evaluate the initial performance of the benchmark model. Secondly, the benchmark model is trained in the first stage using the initial training set. After the training is completed, the initial training set is used again to evaluate the performance of the benchmark model after the first stage of training. Evaluation indicators may include: probability prediction value, comprehensive information entropy, average loss value (average loss value), embedding layer output of each keyword, last layer output, etc.

[0061] Please refer to Figure 2 In a feasible implementation manner, step S100 may include steps S110 to S140:

[0062] Step S110, inputting the initial training set into the benchmark model for evaluation, and obtaining a first evaluation index for each data in the initial training set.

[0063] Step S120, fine-tuning the benchmark model according to the initial training set to obtain a first control model, and evaluating the first control model to obtain a second evaluation index for each data in the initial training set.

[0064] In this embodiment, the benchmark model is used to perform a one-time reasoning on the training data in the initial training set, and the evaluation index of the benchmark model for each data is recorded to obtain the first evaluation index. After the benchmark model evaluates the data in the initial training set, the benchmark model is fine-tuned using the full amount of data in the initial training set. The number of training rounds of the fine-tuning training is less than the normal training process. After the fine-tuning training is completed, the first control model is obtained, and the process of obtaining the evaluation index is repeated based on the first control model to obtain the second evaluation index.

[0065] Step S130, in the initial training set, reducing the proportion of the invalid data and increasing the proportion of the difficult data to obtain the first training set.

[0066] In this implementation, after the first stage of training, the data in the initial training set is divided into efficient data, invalid data, and difficult data according to the evaluation indicators obtained before and after training, and the proportion of invalid data is reduced, the proportion of difficult data is increased, and the efficient data is kept unchanged. After the above data ratio adjustment, a new training set is obtained, which is called the first training set.

[0067] Step S140, fine-tune the benchmark model according to the first training set to obtain a second control model, and evaluate the second control model to obtain a third evaluation index for each data in the first training set.

[0068] In this embodiment, the first training set is used to further fine-tune the benchmark model, and in the fine-tune training, the number of training rounds is also less than the normal training process. After the fine-tune training, another new model is obtained, which is called the second control model. The second control model is evaluated using the first training set, and the third evaluation index for each data is calculated. The third evaluation index will be used to compare the performance difference between the second control model and the first control model on the first training set.

[0069] In this embodiment, during the first stage of training to obtain the first control model and the second control model, the rounds of training are less than the rounds of normal training. The purpose of the first stage of training is to further classify the data in the initial training set. By reducing the number of training rounds, the model can be prevented from being over-trained on the efficient data that has been mastered, thereby focusing more on the processing of difficult data. By reducing the number of training rounds, the training time and the consumption of computer resources can also be shortened. In addition, long-term training may cause large changes in model parameters, thereby affecting the stability and consistency of the model. Shortening the training rounds helps to maintain the stability of the model before and after fine-tuning, so that the model can maintain consistent performance when processing new data.

[0070] Step S200, classifying the data in the initial training set according to the evaluation index, dividing the data in the initial training set into efficient data, invalid data and difficult data, wherein the difficult data includes dirty data and actual difficult data.

[0071] It should be noted that efficient data refers to data that significantly improves the performance of the model during the training process, that is, the model performs well on these data. Invalid data refers to data that does not contribute to the improvement of model performance and may even cause the model performance to deteriorate. Difficult data refers to data that is difficult for the model to classify, including dirty data (such as format errors, outliers, etc.) and actual difficult examples (that is, samples that are difficult for the model to correctly classify).

[0072] Please refer to Figure 2 In a feasible implementation manner, step S200 may include steps S210 to S230:

[0073] Step S210: determining the efficient data, the invalid data, and the difficult data according to the first evaluation index and the second evaluation index.

[0074] Step S220: distinguish the dirty data and the actual difficult examples from the difficult example data according to the second evaluation index and the third evaluation index.

[0075] In this embodiment, by comparing the first evaluation index and the second evaluation index, it is possible to distinguish which data has significantly improved the model performance after fine-tuning (efficient data), which data has no change or even decreases the model performance after fine-tuning (invalid data), and which data is still difficult for the model to handle after fine-tuning (difficult data). By comparing the second evaluation index and the third evaluation index, dirty data and actual difficult examples are further distinguished from the difficult data. Dirty data may be data that performs inconsistently on different training sets, which may be caused by labeling errors, data corruption, etc. Actual difficult examples are data that are difficult to handle in different training sets. These data may have complex features or patterns, making it difficult for the model to accurately predict.

[0076] Step S300, adjust the proportion of each type of data in the initial training set, reduce the proportion of invalid data, keep the proportion of efficient data unchanged, increase the proportion of difficult data, perform second-stage training on the benchmark model, and determine the target training set based on the training results after reaching a preset number of training rounds.

[0077] In this embodiment, the adjusted training set is used to train the benchmark model until a preset training round is reached. The training round can be determined according to the actual computing power.

[0078] Step S400, performing a third phase of training on the benchmark model according to the target training set, and obtaining a target vertical domain model after reaching a preset stop condition.

[0079] In this embodiment, the benchmark model is trained using the target training set until the stopping condition is reached. During the training process, the learning rate is gradually reduced to promote the convergence of the model. When the training is completed, the obtained model is the target model for a specific vertical domain (vertical field). Optionally, the preset stopping condition can be that the learning rate reaches a preset minimum learning rate, or that the training loss and verification loss of the model change less than a preset change threshold.

[0080] In this embodiment, using the screened and optimized target data set for training can make the model more focused on learning the core features of vertical domain knowledge, which helps to improve the model's ability to understand and process vertical domain data, thereby improving the accuracy and reliability of the model.

[0081] In this embodiment, by continuously training the benchmark model, the benchmark model can autonomously screen out data related to vertical domain knowledge and of higher quality based on the evaluation indicators before and after training, which helps to reduce noise and invalid information in the data set, thereby improving the overall data quality. Through iterative training, the benchmark model can continuously learn and adapt to the characteristics of vertical domain data, so as to more accurately screen and classify data. This allows the data set to be continuously optimized with the iteration of the model, further improving data quality. In addition, the use of a method in which the model autonomously screens data can significantly reduce the workload of domain experts in quality assessment and revision of the data set. By automating the process of autonomously screening data and iterative training by the model, labor costs and time costs can be further reduced, making the process of training the vertical domain model more efficient.

[0082] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S210 may further include steps S211 to S215:

[0083] Step S211, obtaining a first average loss value of the benchmark model for the initial training set, and a second average loss value of the first control model for the initial training set.

[0084] Step S212: performing binary clustering on the first average loss value and the second average loss value. If both the first average loss value and the second average loss value belong to the low loss class, the corresponding data is the invalid data.

[0085] It should be noted that the loss value (loss function value) is an indicator to judge the degree of model fit. The smaller the value, the closer the model's predicted results are to the actual results, and the better the model performance.

[0086] In this embodiment, a suitable loss function is selected according to the problem type (such as regression, classification). Exemplarily, for a large language model, a cross entropy loss is selected. For each sample in the initial training set, a loss function is used to calculate the difference or error between its predicted value and the actual value, that is, the loss value of the sample. The loss values ​​of all tokens of the sample in the last iteration of the training process are added together and then divided by the total number of tokens to obtain the average loss value of the sample.

[0087] In this embodiment, after obtaining the average loss value, the first average loss value of the benchmark model is compared with the second average loss value of the comparison model to perform binary clustering, and the average loss value is divided into two categories: high and low. If a piece of data is classified as low loss in both the benchmark model and the first comparison model, the data is classified as invalid data, that is, it is highly likely to belong to the knowledge that the benchmark model has already mastered, and the sampling ratio should be reduced.

[0088] In this implementation, when the first average loss value and the second average loss value are clustered into two categories, for example, using the kmeans clustering method, the method iteratively obtains an adaptive threshold value to divide the loss values ​​into a "low loss class" and a "high loss class".

[0089] Step S213: Obtain the comprehensive information entropy of the first control model for the initial training set, and the average loss difference value between the first average loss value and the second average loss value, and perform binary classification clustering on the comprehensive information entropy and the average loss difference value.

[0090] Step S214: If the average loss difference value belongs to the high loss difference value class and the comprehensive information entropy belongs to the low comprehensive information entropy class, the corresponding data is the efficient data.

[0091] Step S215: If there is data in the initial training set that is neither invalid data nor efficient data, then the data is difficult data.

[0092] It should be noted that information entropy describes the uncertainty of possible events in the information source. The higher the information entropy, the greater the uncertainty of the information, and the more information is required. In the field of natural language processing (NLP) and machine learning, tokens usually refer to units such as words, subwords, or characters obtained after preprocessing text data. These tokens are used as the basic units of model input and output.

[0093] In this implementation, when calculating the comprehensive information entropy, first, for each piece of data, for the probability prediction value, the model probability prediction value is taken for the token of each sampled data in the initial training set, and then the information entropy is calculated based on the probability prediction value. The calculation formula of the information entropy is: Among them, pi is the predicted probability of the i-th possible value of the token, and n is the vocabulary size (or the number of possible values). After calculating the information entropy of a single token, take the top 20% with the largest information entropy and average it as the comprehensive information entropy. Subtract the first average loss value from the first average loss value to get the average loss difference value. Binary clustering is performed on the average loss difference value and the comprehensive information entropy of the first control model. If a piece of data belongs to a high loss difference value and the comprehensive information entropy of the first control model belongs to a low comprehensive information entropy, and the data is not invalid data, this data is classified as efficient data, that is, knowledge that the model can quickly master through training, and its sampling ratio remains unchanged. If a piece of data is neither efficient data nor invalid data, it is classified as difficult data, that is, data that is difficult for the model to learn. It may be dirty data that has been contaminated, or it may be vertical domain professional knowledge that is difficult for the model (called actual difficult examples). The sampling ratio of this part of data should be increased.

[0094] In this embodiment, the first control model is used to predict each data in the initial training set. A two-dimensional probability matrix is ​​output, in which each element P(t i ∣x j ) means that given an input x j In the case of i The probability of occurrence. The dimension of this matrix is ​​usually "vocabulary size" multiplied by "sequence length". Extract the probability prediction value of each token at each position. For each token, get a probability distribution (that is, the probability value of the token at each position in the sequence). Calculate the cosine similarity of the probability prediction value of each token by the baseline model and the first control model, get the sequence length similarity values ​​of this data, and arrange them from low to high. Select the top 20% of the data, calculate the average similarity value, and use it as the probability distribution difference value.

[0095] In this embodiment, by setting clear classification criteria, data with different characteristics can be clearly distinguished. After invalid data is identified, these data are ignored to avoid wasting computing resources during model training. For efficient data, the characteristics of these data can be analyzed more deeply to optimize the performance of the model on these data. By focusing on efficient data and difficult data, the strengths and weaknesses of the model can be more accurately located, and the architecture, parameters or training strategies of the model can be adjusted to improve the overall performance of the model. By optimizing the data set, the stability and accuracy of model training can be improved.

[0096] In one possible implementation, please refer to Figure 4 Step S220 may further include steps A221 to A225:

[0097] Step A221, obtaining the predicted probability distribution difference of the second control model for each data in the first training set, and performing binary classification clustering on the predicted probability distribution difference of the second control model.

[0098] In this embodiment, the predicted probability distribution difference of the second control model for each data point is calculated, and the predicted probability distribution difference is clustered by two categories, and the predicted probability distribution difference is divided into two categories: low distribution difference value and high distribution difference value. Low difference means that the predictions of the two models for the same data point are relatively consistent, while high difference indicates that there is a large prediction difference. The method for calculating the predicted probability distribution difference is the same as in the above method, and will not be repeated here.

[0099] Step A222, obtaining the comprehensive information entropy of the second control model for the second training set, and performing binary classification clustering on the comprehensive information entropy of the second control model.

[0100] In this embodiment, the comprehensive information entropy of the second control model for each data point is calculated, and the comprehensive information entropy is clustered into two categories: low comprehensive information entropy and high comprehensive information entropy. High comprehensive information entropy indicates that the uncertainty of the data is high, that is, it is difficult for the model to extract useful information from it. The method for calculating the comprehensive information entropy of the second control model is the same as the method for calculating the comprehensive information entropy of the first control model in the above embodiment, and will not be repeated here.

[0101] Step A223: If the predicted probability distribution difference of the second control model belongs to a low distribution difference, the difficult example data is dirty data.

[0102] Step A224, or, if the predicted probability distribution difference of the second control model belongs to a high distribution difference, and the comprehensive information entropy of the second control model belongs to a high comprehensive information entropy, then the difficult example data is dirty data.

[0103] Step A225: confirm the hard example data that is not dirty data as the actual hard example.

[0104] In this embodiment, dirty data is identified based on the above two clustering results. If the predicted probability distribution difference of a difficult data belongs to a low distribution difference value, or the predicted probability distribution difference of a difficult data belongs to a high distribution difference value and the comprehensive information entropy of the second control model belongs to a high comprehensive information entropy, then this difficult data is identified as dirty data, that is, data that confuses the model after model learning and should be discarded. Difficult data other than dirty data is identified as actual difficult data.

[0105] In this embodiment, the difficult data in the model are screened and classified to distinguish dirty data from actual difficult examples. By excluding dirty data, the impact of data noise on model training can be reduced, so that the model can more accurately learn the true distribution and characteristics of the data. By tuning the model for actual difficult examples, the performance of the model in complex or marginal situations can be improved, making the model more reliable. In addition, by identifying and cleaning dirty data in the data set, the overall data quality can be improved, computing resources can be saved, training time can be shortened, and resource utilization efficiency can be improved.

[0106] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 5 , step S300 may further include steps S310 to S330:

[0107] Step S310, in each round of iteration, according to the perturbation rules of reducing the invalid data, retaining the invalid data, increasing the actual difficult examples and discarding the dirty data, the initial training set is perturbed a preset number of times to obtain multiple second training sets.

[0108] Step S320: performing a second phase of training on the benchmark model according to the second training set.

[0109] Step S330, after reaching the preset training rounds, the second training set corresponding to the last round is used as the target training set, and the intersection of the second training sets in the iterations is used as the third training set.

[0110] In this embodiment, the second stage training is performed on the benchmark model. During the second stage training, M rounds of iterations are performed. In each round of iteration, perturbations are performed based on preset perturbation rules to generate N data mixing ratios as the second training set for fine-tuning training. The benchmark model is fine-tuned with fewer training rounds than the normal training process to obtain N models in each round of iteration, and the evaluation index acquisition process is repeated for the N models.

[0111] In this embodiment, when determining the second training set corresponding to the current round, that is, the latest data mixing ratio, in each round of iterative training, a subset of the initial training set (that is, the second training set) is used for fine-tuning training, and the general keywords and vertical domain keywords embedding vectors output by N models in each iteration are recorded, wherein the general keywords refer to keywords that may be used in a wide range of knowledge fields or disciplines. These keywords usually cover a wide range, can span different disciplines and majors, and are universal and universal. Vertical domain keywords, that is, vertical field keywords, refer to keywords used in specific industries or professional fields. These keywords are highly professional and targeted, and are usually only applicable to a specific field or industry. Sort the general keyword embedding layer difference values ​​from low to high, and assign a ranking value to each model; sort the vertical domain keyword embedding layer difference values ​​from high to low, and assign a ranking value to each model. Add the two ranking values, take the data mixing ratio of the model with the lowest value as the current data mixing ratio, and iterate in sequence.

[0112] Optionally, the preset perturbation rules are to reduce invalid data, keep invalid data, increase actual difficult examples, and discard dirty data, and the perturbation is performed at any time within ±30% of the original ratio. In the first round of iteration, according to the perturbation rules, the initial training set is perturbed for the first time to obtain the first second training set, and the first second training set is used to perform the second stage training on the benchmark model to obtain N models. Record the general knowledge keywords and vertical domain keywords embedding vectors output by the N models. Sort the general knowledge keywords embedding layer difference values ​​from low to high, and assign a ranking value to each model; sort the vertical domain keywords embedding layer difference values ​​from high to low, and assign a ranking value to each model. Add the two ranking values ​​and take the data mixing ratio of the model with the lowest value as the second second training set. In the second round of iteration, the second second training set is also perturbed according to the rules of reducing invalid data, keeping invalid data, increasing actual difficult examples, and discarding dirty data. At the end of each round of iteration, the second training set of the current round is recorded. After reaching the preset training round, the second training set corresponding to the last round is selected as the target training set. The intersection of all second training sets generated in each round of iteration is taken as the third training set, and the data in the third training set is considered to be high-quality data.

[0113] In this embodiment, in M ​​rounds of iterative training, for each round of N models, the data in the corresponding training set are segmented, and word frequency statistics are performed, and the words are sorted in order from high to low frequency. Meaningless stop words (such as: de, le, we, etc.) are screened out through large model prompt word engineering, and high-frequency words are classified as general knowledge keywords or vertical domain keywords, and manual screening is performed. Finally, 100 general knowledge and 100 vertical domain keywords are screened out, and the number of keywords can be adjusted according to the size of the data set and the specific situation. Record the output of the embedding layer of each keyword model (a one-dimensional vector, the length is the dimension of the embedding vector) and the output of the last layer of the model (the hidden layer before LM_head) (a one-dimensional vector, the length is the dimension of the last layer output vector).

[0114] In this implementation, "reducing invalid data" and "discarding dirty data" in the perturbation rules help clean up the data set and improve data quality, while "keeping invalid data" and "increasing actual difficult examples" help the model better learn the complexity and boundary conditions of the data. Through iterative training, the model can gradually optimize its performance until the preset training rounds are reached.

[0115] Based on the first embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 6 , step S400 may further include steps S410 to S420:

[0116] Step S410, performing third-stage training on the benchmark model according to the target training set to obtain a first benchmark model.

[0117] Step S420, performing annealing training on the benchmark model according to the third training set, and obtaining a target vertical domain model after reaching a preset stop condition.

[0118] It should be noted that annealing training is an optimization algorithm that simulates the physical annealing process. It gradually reduces the learning rate during the training process so that the model gradually converges to the global optimal solution or an approximate global optimal solution.

[0119] In this embodiment, the third stage of model training is performed based on the second training set corresponding to the last round of the second stage training as the target training set. After the model training is completed, the intersection of all the second training sets generated in each round of iteration is used as the third training set, and the annealing training of small-scale data in the last stage is performed based on the third training set. In the early stage of training, the learning rate is large, and the model can quickly adjust the parameters to approach the optimal solution; as the training progresses, the learning rate gradually decreases, and the pace of model parameter adjustment also gradually slows down to avoid skipping the global optimal solution or falling into the local optimal solution.

[0120] In this embodiment, the first benchmark model obtained by the third stage training is used as the starting point of the annealing training, and the parameters such as the initial learning rate, learning rate threshold and learning rate decay rate in the annealing training are set. The initial learning rate is a relatively high initial learning rate to ensure that the model can quickly adjust the parameters in the early stage of training. The learning rate decay rate is the ratio or fixed step size of the learning rate reduction in each training cycle (or iteration). The minimum learning rate is set to a very small learning rate threshold. The stopping condition can be that the learning rate reaches a preset minimum learning rate, and the preset minimum learning rate is 0. When the learning rate drops to this threshold, the annealing training is considered to be over. In each training cycle, the model is trained using the third training set, and the learning rate is updated at the end of each cycle according to the learning rate decay rate, and the learning rate is reduced by a fixed step size. When the learning rate drops below the minimum learning rate, the annealing training is stopped. After the annealing training is completed, a fully optimized and generalized target vertical domain model is obtained.

[0121] Optionally, the preset stop condition can also be that the training loss and validation loss of the model no longer decrease significantly. Before the annealing training begins, two lists are initialized to record the training loss and validation loss of each training cycle. In each training cycle, the model is trained using the third training set, and the training loss of the current cycle is calculated. The calculated training loss is added to the training loss record list. The model is evaluated using the validation set, and the validation loss of the current cycle is calculated. The calculated validation loss is added to the validation loss record list. After each training cycle, check that the training loss and validation loss no longer decrease significantly (for example, the change is less than a threshold, such as 0.01% or 0.1%) within multiple consecutive cycles (such as 5 or 10 cycles, the specific number is determined based on experience and data scale).

[0122] Optionally, the preset stop condition may also be to meet a learning rate condition or a loss change condition. The learning rate condition is that the learning rate has dropped to a set minimum learning rate. The loss change condition is that the training loss and the validation loss no longer decrease significantly over multiple consecutive cycles. If any of the above stop conditions is met, the annealing training is stopped, and the weights and parameters of the current model are saved as the final model. If the stop condition is not met, the next training cycle is continued.

[0123] In this embodiment, the third training set is the intersection of all second training sets generated in each round of iteration, and contains higher quality small-scale data. Through the third stage of training, the model can further adapt to the data distribution and characteristics of the target training set, thereby improving the performance in the target vertical domain. Annealing training gradually reduces the learning rate so that the model gradually converges to the global optimal solution or an approximate global optimal solution during the training process, further improving the accuracy and stability of the model.

[0124] The present application provides a training device for a vertical domain large model, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method for the vertical domain large model in the above-mentioned embodiment one.

[0125] Reference below Figure 7 , which shows a schematic diagram of the structure of a training device for a vertical domain large model suitable for implementing an embodiment of the present application. The training device for the vertical domain large model in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (portable android devices), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The training device for the vertical large-scale model shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0126] like Figure 7As shown, the training device of the vertical domain large model may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the training device of the vertical domain large model are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the training device of the vertical domain large model to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a training device of the vertical domain large model with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or provided alternatively.

[0127] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0128] The training device for the vertical domain large model provided in the present application adopts the training method for the vertical domain large model in the above-mentioned embodiment, which can solve the technical problem of how to screen the training data in the process of the large model and improve the quality of the training data. Compared with the prior art, the beneficial effects of the training device for the vertical domain large model provided in the present application are the same as the beneficial effects of the training method for the vertical domain large model provided in the above-mentioned embodiment, and the other technical features in the training device for the vertical domain large model are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0130] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0131] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the training method of the vertical domain large model in the above-mentioned embodiment.

[0132] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0133] The above-mentioned computer-readable storage medium may be included in the training device of the vertical domain large model; or it may exist independently without being assembled into the training device of the vertical domain large model.

[0134] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the training device of the vertical domain large model, the training device of the vertical domain large model enables: to perform the first stage training on the preset benchmark model according to the initial training set, and obtain the evaluation indicators before and after the first stage training; to classify the data in the initial training set according to the evaluation indicators, and divide the data in the initial training set into efficient data, invalid data and difficult example data, and the difficult example data includes dirty data and actual difficult examples; to adjust the proportion of each type of data in the initial training set, to perform the second stage training on the benchmark model, and after reaching the preset training rounds, to determine the target training set according to the training results; to perform the third stage training on the benchmark model according to the target training set, and after reaching the preset stop condition, to obtain the target vertical domain model.

[0135] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0137] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0138] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned vertical domain large model training method, and can solve the technical problem of how to screen the training data in the process of the large model and improve the quality of the training data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the vertical domain large model training method provided in the above-mentioned embodiments, which will not be repeated here.

[0139] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A training method for a large vertical model, characterized in that: The method includes: Performing a first-stage training on a preset benchmark model according to an initial training set, and obtaining evaluation indicators before and after the first-stage training; Classifying the data in the initial training set according to the evaluation index, dividing the data in the initial training set into efficient data, invalid data, and difficult data, wherein the difficult data includes dirty data and actual difficult examples; Adjusting the proportion of each type of data in the initial training set, performing a second phase of training on the benchmark model, and determining a target training set according to the training results after reaching a preset number of training rounds; The benchmark model is trained in the third stage according to the target training set, and a target vertical domain model is obtained after a preset stop condition is reached.

2. The method according to claim 1, characterized in that The step of performing the first phase training on the preset benchmark model according to the initial training set and obtaining the evaluation index before and after the first phase training includes: Inputting the initial training set into the benchmark model for evaluation, and obtaining a first evaluation index for each data in the initial training set; Fine-tune the benchmark model according to the initial training set to obtain a first control model, and evaluate the first control model to obtain a second evaluation index for each data in the initial training set; The benchmark model is fine-tuned and trained according to the first training set to obtain a second control model, and the second control model is evaluated to obtain a third evaluation index for each data in the first training set.

3. The method according to claim 2, characterized in that After the step of fine-tuning the benchmark model according to the initial training set to obtain a first control model, evaluating the first control model, and obtaining a second evaluation index for each data in the initial training set, the method includes: In the initial training set, the proportion of the invalid data is reduced and the proportion of the difficult data is increased to obtain the first training set.

4. The method according to claim 2, characterized in that The step of classifying the data in the initial training set according to the evaluation index to divide the data in the initial training set into efficient data, invalid data and difficult data comprises: Determining the efficient data, the invalid data, and the difficult data according to the first evaluation index and the second evaluation index; The dirty data and the actual difficult examples are distinguished from the difficult example data according to the second evaluation index and the third evaluation index.

5. The method according to claim 4, characterized in that The step of determining the efficient data, the invalid data, and the difficult data according to the first evaluation index and the second evaluation index comprises: Obtaining a first average loss value of the benchmark model for the initial training set, and a second average loss value of the first control model for the initial training set; Perform binary clustering on the first average loss value and the second average loss value, and if both the first average loss value and the second average loss value belong to the low loss class, the corresponding data is the invalid data; Obtaining the comprehensive information entropy of the first control model for the initial training set, and the average loss difference value between the first average loss value and the second average loss value, and performing binary classification clustering on the comprehensive information entropy and the average loss difference value; If the average loss difference value belongs to the high loss difference value class, and the comprehensive information entropy belongs to the low comprehensive information entropy class, then the corresponding data is the efficient data; If there is data in the initial training set that is neither invalid data nor efficient data, then the data is difficult data.

6. The method according to claim 4, characterized in that The step of distinguishing the dirty data and the actual difficult examples from the difficult example data according to the second evaluation index and the third evaluation index comprises: Obtaining a predicted probability distribution difference of the second control model for each data in the first training set, and performing binary classification clustering on the predicted probability distribution difference of the second control model; Obtaining the comprehensive information entropy of the second control model for the second training set, and performing binary classification clustering on the comprehensive information entropy of the second control model; If the predicted probability distribution difference of the second control model belongs to a low distribution difference, the difficult example data is dirty data; Alternatively, if the predicted probability distribution difference of the second control model belongs to a high distribution difference, and the comprehensive information entropy of the second control model belongs to a high comprehensive information entropy, then the difficult example data is dirty data; The hard example data that is not the dirty data is confirmed as the actual hard example.

7. The method according to claim 1, characterized in that The step of adjusting the proportion of each type of data in the initial training set, performing second-stage training on the benchmark model, and determining the target training set according to the training results after reaching a preset training round includes: In each round of iteration, the initial training set is perturbed a preset number of times according to the perturbation rules of reducing the invalid data, maintaining the invalid data, increasing the actual difficult examples, and discarding the dirty data, to obtain multiple second training sets; Performing a second phase of training on the benchmark model according to the second training set; After reaching the preset training rounds, the second training set corresponding to the last round is used as the target training set, and the intersection of the second training sets in each iteration is used as the third training set.

8. The method according to claim 7, characterized in that The step of performing the third phase training on the benchmark model according to the target training set and obtaining the target vertical domain model after reaching a preset stop condition comprises: Performing a third phase of training on the benchmark model according to the target training set to obtain a first benchmark model; The benchmark model is subjected to annealing training according to the third training set, and a target vertical domain model is obtained after a preset stop condition is reached.

9. A training device for a vertical large model, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for training a vertical domain large model as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the vertical domain large model as described in any one of claims 1 to 8 are implemented.

Images