Sample selection method and device for large model training
By designing phased sample selection criteria in large model training, using the uncertainty of the trained model and the model to be trained, the sample selection efficiency and accuracy problems in large model training are solved, and the performance and generalization of the model are improved.
Patent Information
- Application Number
- CN202510463425.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-11
- Publication Date
- 2025-08-01
AI Technical Summary
In the prior art, it is difficult to efficiently select high-quality pre-training samples, continue to pre-train and fine-tune samples in large-scale model training, which has problems such as large time overhead and poor generalization.
By designing sample selection criteria for different training stages, using the degree of uncertainty of the trained large models and the large models to be trained, the comprehensive sorting of candidate texts is determined, and high-quality pre-training samples are selected, the continued pre-training samples and fine-tuning samples are selected.
It improves the efficiency and accuracy of sample selection, ensures that the large model selects full, diverse and helpful samples at each training stage, and improves the generalization of the model and the prediction accuracy of downstream tasks.
Smart Images
Figure CN120409619A_ABST
Abstract
Description
Technical Field
[0001] One or more embodiments of this specification relate to the field of computer technology, and in particular, to a method and device for sample selection for large model training. Background Art
[0002] With the rapid development of large language models (referred to as large models), samples (including pre-training samples, continued pre-training samples, fine-tuning samples, etc.) are the cornerstone of training large models. High-quality samples can significantly improve the performance of large models, while low-quality samples may lead to poor performance of large models. Therefore, it is crucial to collect and screen high-quality samples. Summary of the Invention
[0003] One or more embodiments of this specification describe a method and device for sample selection for large model training, which can effectively select high-quality samples for large models.
[0004] In a first aspect, a method for sample selection for large model training is provided, including:
[0005] For any first candidate text in the candidate text set, input it into the first large model to be trained, and obtain a first metric value indicating the uncertainty degree of the first large model with respect to the first candidate text;
[0006] Determine a first ranking of the candidate texts according to the first metric values of the candidate texts;
[0007] Obtain a second ranking of the candidate texts determined according to the second metric values of the candidate texts; the second metric value of any candidate text indicates the uncertainty degree of the second trained large model with respect to the candidate text; the number of parameters of the first large model is much smaller than that of the second large model;
[0008] Determine a comprehensive ranking of the candidate texts according to the first ranking and the second ranking; where the priority of the second ranking is higher than that of the first ranking;
[0009] Based on the comprehensive ranking, select a predetermined number of candidate texts from the candidate text set, and determine a pre-training sample set for the first large model based on them.
[0010] In a second aspect, a device for sample selection for large model training is provided, including:
[0011] An input unit, configured to input any first candidate text in the candidate text set into the first large model to be trained, and obtain a first metric value indicating the uncertainty degree of the first large model with respect to the first candidate text;
[0012] A determining unit, configured to determine a first ranking of each candidate text according to a first index value of each candidate text;
[0013] An obtaining unit, configured to obtain a second ranking of each candidate text determined according to a second index value of each candidate text; the second index value of any candidate text indicates the degree of uncertainty of the second trained large model with respect to this candidate text; the number of parameters of the first large model is much smaller than that of the second large model;
[0014] The determining unit is further configured to determine a comprehensive ranking of each candidate text according to the first ranking and the second ranking; wherein the priority of the second ranking is higher than that of the first ranking;
[0015] A selecting unit, configured to select a predetermined number of candidate texts from the candidate text set based on the comprehensive ranking, and determine a pre-training sample set of the first large model based on the selected candidate texts.
[0016] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed on a computer, the computer is made to execute the method of the first aspect.
[0017] In a fourth aspect, a computing device is provided, including a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the method of the first aspect is implemented.
[0018] The sample selection method for large model training provided by one or more embodiments of this specification determines two rankings of each candidate text respectively according to the degrees of uncertainty of each candidate text with respect to a trained large model and a large model to be trained, and then takes the ranking of each candidate text corresponding to the trained large model as the main factor and the ranking corresponding to the large model to be trained as the auxiliary factor to determine the comprehensive ranking of each candidate text, and selects a predetermined number of candidate texts based on this comprehensive ranking to form a pre-training sample set of the large model to be trained. Among them, the above-mentioned trained large model has a relatively large number of parameters and can be regarded as a teacher model, while the large model to be trained has a relatively small number of parameters and can be regarded as a student model. It can be seen that this solution uses the knowledge learned by the teacher model to select a pre-training sample set for the student model. Thus, the selection efficiency and accuracy of the samples can be improved. In addition, this solution selects pre-training samples based on the degrees of uncertainty of the teacher model and the student model with respect to the samples, and can select high-quality samples for the student model. Brief Description of the Drawings
[0019] To more clearly illustrate the technical solutions of the embodiments of this specification, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.
[0020] Figure 1 Schematic diagram showing the sample selection criteria for each training stage in an example of this specification;
[0021] Figure 2 Schematic diagram showing the method for selecting pre-training samples for a large model in an example of this specification;
[0022] Figure 3 Flowchart showing the method for selecting pre-training samples for a large model according to an embodiment of this specification;
[0023] Figure 4 Schematic diagram showing the result of the Coreset clustering algorithm in an example of this specification;
[0024] Figure 5 Schematic diagram showing the method for selecting pre-training samples for a large model in an example of this specification;
[0025] Figure 6 Flowchart showing the method for selecting fine-tuning samples for a large model according to an embodiment of this specification;
[0026] Figure 7 Schematic diagram showing the method for selecting fine-tuning samples for a large model according to an embodiment of this specification;
[0027] Figure 8 Schematic diagram showing the sample selection device for training a large model according to an embodiment of this specification. Detailed implementation manners
[0028] The following describes the solutions provided in this specification in conjunction with the accompanying drawings.
[0029] As mentioned above, it is necessary to select high-quality pre-training samples, continued pre-training samples, fine-tuning samples, etc. for the large model. Among them, pre-training samples refer to the samples used in the pre-training stage, continued pre-training samples refer to the samples used in the continued pre-training stage, and fine-tuning samples refer to the samples used in the fine-tuning stage.
[0030] Among them, regarding the selection of pre-training samples, there are mainly the following several solutions:
[0031] First, by fine-tuning a small data influence model to select pre-training samples for the large model, which mainly includes the following steps:
[0032] 1. Sample division: That is, dividing the sample set to be selected into multiple batches;
[0033] 2. Sample impact scoring: Using a data impact model to score each sample;
[0034] 3. Selection and training: Selecting the optimal k% of the samples according to the scores for the next round of pre-training, and using these samples to update the parameters of the large model.
[0035] The pre-training samples selected by the above method can ensure that they are high-quality samples helpful for the pre-training of the large model. However, due to the extremely large size of the sample set to be selected and dynamic adjustment during the pre-training stage, it will incur a large time overhead. On the other hand, selecting the optimal k% of the samples according to the scores, where the setting of k will affect the performance of the large model. For example, if k is set too small, the selected sample size will be small, which will affect the generalization ability of the large model, while if k is set too large, low-quality samples will be selected.
[0036] Second, according to the impact of each pre-training sample on the degree of model gradient descent calculated during the pre-training process, dynamically select pre-training samples. However, this solution has the following two disadvantages: 1) The time overhead for dynamically selecting pre-training samples during the pre-training process is large. 2) It is not suitable for being extended to scenarios with larger data volumes.
[0037] Third, by clustering the sample set to be selected and training a small language model on the data set guiding data selection, evaluating the quality of the sample subsets corresponding to each cluster to select pre-training samples. However, this solution has the following two disadvantages: 1) The sample subsets with low evaluation quality will be deleted, which does not meet the comprehensive coverage characteristic in the data selection criteria. 2) The data set guiding data selection has a small scale and does not have generalization ability.
[0038] Considering that the sample set to be selected has different characteristics at different training stages of the large model. For example, during the pre-training stage, the data volume of the sample set to be selected is usually extremely large (the data volume reaches hundreds of millions or even billions). This solution first designs corresponding sample selection criteria for each training stage, and then further proposes corresponding sample selection schemes based on these sample selection criteria.
[0039] Figure 1 Shows a schematic diagram of the sample selection criteria for each training stage in an example of this specification. Figure 1Among them, in the pre-training stage, the sample selection criteria can include but are not limited to high quality, that is, the samples are accurate; coverage, that is, the samples should contain data from as many fields as possible. For example, some are natural language data, some are mathematical data, and some are other data. When the fields covered by the samples are comprehensive enough, the prediction performance of the large model for individual fields can be improved; diversity, that is, the samples should involve as many tasks as possible. For example, some samples involve task A, and some samples involve task B. Diverse samples help improve the generalization of the large model; helpful, that is, the samples are helpful for improving the performance of downstream tasks, etc.
[0040] That is to say, in the pre-training stage, it is usually desirable to collect a sufficient number of high-quality samples that should cover various knowledge as much as possible, and this knowledge is diverse, that is, it should include various topics and fields and be helpful for downstream tasks. Taking a code large model as an example, it is necessary to collect a sufficient number of code samples that can be correctly executed. These code samples should cover programming languages such as "C++", "Python", "Java", etc. as much as possible, and the code samples of each programming language should cover various code functions as much as possible, such as "dynamic programming", "greedy algorithm", "depth-first search", etc. After pre-training the code large model based on such code samples, the prediction accuracy of the code large model in downstream tasks such as code generation and code completion can be improved.
[0041] The sample selection scheme corresponding to this sample selection criterion can include constructing prompts at least based on the above sample selection criteria, and then using these prompts to instruct the trained large model to evaluate the quality of the samples in the set of samples to be selected (corresponding to the pre-training stage), so as to select the pre-training sample set for the large model to be trained.
[0042] In addition, in the continued pre-training stage, the sample selection criteria can include but are not limited to coverage, diversity, and helpfulness, etc. The continued pre-training stage refers to the process of continuing to pre-train the pre-trained large model. This process usually occurs when a large model has been trained based on a high-quality, comprehensive, diverse, and helpful sample set, but it is hoped that the model can learn the knowledge that was not learned in the pre-training stage. In other words, in the continued pre-training stage, usually more attention is paid to whether the samples cover enough knowledge and the diversity of the samples, and what knowledge the large model has not learned yet.
[0043] The sample selection scheme corresponding to the sample selection criteria may include using a large pre-trained model with a large number of parameters as the teacher model and the large model to be trained as the student model, and then respectively determining the degree of uncertainty of the teacher model and the student model with respect to the samples in the sample set to be selected (corresponding to the continued pre-training stage), and selecting the continued pre-training sample set for the student model based on this degree of uncertainty.
[0044] Finally, in the fine-tuning stage, the sample selection criteria may include, but are not limited to, being helpful, underfitting (i.e., the sample is unfamiliar or uncertain to the model), etc. Fine-tuning refers to, based on the pre-trained large model (abbreviated as the pre-trained model), for specific downstream tasks (in NLP tasks, such as text classification, question answering, and named entity recognition, etc.), using a relatively small-scale but labeled sample set to further train some or all of the parameters of the pre-trained model. In the fine-tuning stage, it is usually necessary to consider more whether the selected samples are helpful for specific tasks and whether the model will learn the samples during fine-tuning.
[0045] The sample selection scheme corresponding to the above sample selection criteria may include first obtaining the test set of the large model to be trained, and then for the training samples in the sample set to be selected (corresponding to the fine-tuning stage), combining them with the test samples, and selecting the fine-tuning sample set for the large model to be trained based on the probabilities predicted by the trained large model for the combined result and the individual test samples respectively.
[0046] In summary, this solution proposes corresponding sample selection schemes for the pre-training stage, continued pre-training stage, and fine-tuning stage of the large model respectively, and the samples selected based on this sample selection scheme meet the corresponding sample selection criteria.
[0047] The sample selection schemes for the above-mentioned training stages will be described in detail below.
[0048] Figure 2 A schematic diagram showing a method for selecting pre-training samples for a large model in an example of this specification. Figure 2 Among them, for any corpus text x in the corpus i , a prompt word P can be constructed based on this corpus text x i and several judgment criteria. This prompt word P askLLM indicates judging whether the corpus text x askLLM is a high-quality text based on several judgment criteria. Then the prompt word P i can be input into the target large model to obtain an indication that the corpus text x askLLM is iWhether it is the output result of high-quality text. After obtaining the output results corresponding to each corpus text in the corpus, a pre-training text set for the large model to be trained can be formed based on the target corpus texts in the corpus that indicate high-quality text corresponding to the output results.
[0049] In one example, the output of the target large model can be expressed by the following formula:
[0050] Y = LLM(P askLLM ), P askLLM = Cat(x i , s1, s2, …), Y ∈ {yes, no} (Formula 1)
[0051] Wherein, x i represents the corpus text, s i represents the judgment criterion, and Y represents the output of the target large model. Specifically, when Y = yes, it means that the corpus text x i is high-quality text, and when Y = no, it means that the corpus text x i is low-quality text.
[0052] In addition, the above-mentioned target large model can be a high-performance pre-trained large model, such as GPT-4o, Gemini, GPT-5, etc.
[0053] It should be understood that although the amount of data of the corpus texts in the corpus is relatively large, due to the powerful language understanding ability of the large model, the pre-training text set can be quickly selected for the large model.
[0054] In addition, the above-mentioned several judgment criteria can include, but are not limited to, one or more of the following:
[0055] Clarity, which indicates whether the corpus text is specific, well-defined, and without any ambiguity. Domainality, which indicates whether the corpus text tests the knowledge and understanding ability of artificial intelligence in a specific field or a series of fields. Complexity, which indicates whether the corpus text contains multiple components, variables, or has different levels of depth and nuances. Problem-solving ability, which indicates whether the corpus text actively solves problems, analyzes and clearly defines problems, and systematically designs and implements solutions. Creativity, which indicates whether the corpus text proposes creative methods or solutions. Technical accuracy, which indicates whether the corpus text is a highly technically accurate, correct, and precise answer. Application relevance, which indicates whether the corpus text is relevant to actual applications.
[0056] In summary, based on the above-mentioned several judgment criteria, this solution can select a high-quality pre-training text set for the large model.
[0057] Figure 3The flowchart shows a method for selecting pre-training samples for a large model according to an embodiment of this specification. This method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities. As Figure 3 shown, this method may include the following steps:
[0058] Step S302: For any candidate text in the candidate text set, input it into the large model S to be trained, and obtain an index value m1 indicating the degree of uncertainty of the large model S with respect to this candidate text.
[0059] Here, the large model S can also be referred to as the student model.
[0060] In one embodiment, the above-mentioned candidate text set contains all texts in the original text set.
[0061] In another embodiment, the above-mentioned candidate text set is a text subset corresponding to any one of the N clusters obtained by clustering the original text set using a clustering algorithm.
[0062] Here, the clustering algorithm may include, but is not limited to, the K-Means algorithm, the DBSCAN density clustering algorithm, the spectral clustering algorithm, or the Coreset clustering algorithm, etc.
[0063] Figure 4 The schematic diagram shows the result of the Coreset clustering algorithm in an example of this specification. Figure 4 In it, a circle represents a cluster, and a point represents a text. Taking any cluster as an example, the texts located within the corresponding circle form the text subset corresponding to this cluster. Among them, the distance between each text in the text subset corresponding to any cluster and the cluster center (i.e., the center point of the circle) is less than the preset threshold δ s (that is, the radius of the circle). Among them, the data volume of the text subset corresponding to any cluster is much smaller than that of the original text set, so it can also be called the core set.
[0064] In a more specific embodiment, the above-mentioned original text set may be the above-mentioned pre-training text set (i.e., Figure 2 the pre-training text set selected in it), and the large model S to be trained may be a large model pre-trained based on the above-mentioned pre-training text set, that is, this large model S is a pre-trained model. At this time Figure 3 the shown method can be regarded as a method for selecting continued pre-training samples for the pre-trained model.
[0065] It should be noted that in the case of performing the above clustering operation, it is possible to select the pre-training samples or continue the pre-training samples of the large model S based on the core set corresponding to each cluster. Since the data volume of the core set is much smaller than that of the original sample set, the drawback of global search in a large-scale original sample set can be overcome, and the sample selection time is greatly reduced.
[0066] In step S302, after inputting any candidate text into the large model S to be trained, the large model S can predict various probabilities for each character included in the candidate text. Then, the logarithms of the probabilities can be averaged and then negated to obtain the target operation value. Then, the target operation value is directly used as the index value m1 corresponding to the candidate text.
[0067] In one example, the target operation value can be determined according to the following formula:
[0068]
[0069] where H is the target operation value, n is the number of characters included in the candidate text, and x i is any character in the candidate text, and P(x i ) is the probability predicted by the large model S for the character x i in the candidate text.
[0070] Of course, in practice, the above target operation value can also be subjected to a power operation. For example, a power operation is performed with 2 as the base and the target operation value as the power to obtain the index value m1. The index value m1 at this time can also be called the perplexity (PPL).
[0071] It should be noted that the smaller the above index value m1, the more familiar the large model S is with the candidate text. Or rather, the larger the index value m1, the more uncertain the large model S is about the candidate text, that is, it has not learned the candidate text.
[0072] It should be understood that to further improve the prediction performance of the large model, it is necessary to select samples that the large model is not familiar with or is uncertain about (that is, samples that have not been learned) to train it. That is to say, it is necessary to select texts with a larger index value m1 to train the large model S.
[0073] In short, through step S302, the index value m1 of each candidate text in the candidate text set can be obtained.
[0074] Step S304: Determine the sorting Sort1 of each candidate text according to the index value m1 of each candidate text.
[0075] In one example, each candidate text can be sorted in descending order according to the metric value m1 of each candidate text, so as to obtain the sorting Sort1 of each candidate text. That is, the candidate texts that the large model S is not familiar with or is uncertain about are arranged in the front.
[0076] Of course, in practice, the sorting Sort1 can also be sorted in ascending order according to the metric value m1 of each candidate text, and this specification does not limit this.
[0077] Step S306, obtain the sorting Sort2 of each candidate text determined according to the metric value m2 of each candidate text.
[0078] The metric value m2 of each candidate text here can be obtained by inputting each candidate text into the trained large model T. This large model T can also be called the teacher model, and its number of parameters is much larger than that of the above-mentioned large model S.
[0079] The metric value m2 of any of the above candidate texts indicates the degree of uncertainty of the large model T about this candidate text, and it is obtained through the following steps: input this candidate text into the trained large model T to obtain the probabilities predicted by the large model T for each character included in this candidate text. Then, the logarithms of the probabilities can be averaged and then negated to obtain the target operation value. Finally, directly use this target operation value as the metric value m2, or perform a power operation on this target operation value to obtain the metric value m2. The power operation method here can refer to the above text and will not be elaborated here.
[0080] Similarly, the smaller the above-mentioned metric value m2, the more familiar the large model T is with this candidate text. Or rather, the larger the metric value m2, the more uncertain the large model T is about this candidate text.
[0081] It should be noted that since the large model T is a trained model, those samples that are relatively familiar or certain to the large model T may be high-quality and helpful training samples for the large model S. That is, select the texts with smaller metric values m2 to train the large model S.
[0082] In one example, each candidate text can be sorted in ascending order according to the metric value m2 of each candidate text, so as to obtain the sorting Sort2 of each candidate text. That is, the candidate texts that the large model T is familiar with or is certain about are arranged in the front.
[0083] Of course, in practice, the sorting Sort2 can also be sorted in descending order according to the metric value m2 of each candidate text, and this specification does not limit this.
[0084] Step S308, determine the comprehensive sorting of each candidate text according to the sorting Sort1 and the sorting Sort2, where the priority of the sorting Sort2 is higher than that of the sorting Sort1.
[0085] Specifically, for each candidate text under Sort2, if the metric values m2 of any two or more candidate texts are the same, then the final sorting position is determined according to the positions of these any two or more candidate texts under Sort1, and then the comprehensive sorting determined according to Sort1 and Sort2 is obtained.
[0086] In another embodiment, the comprehensive sorting of each candidate text can also be determined by taking the intersection of Sort1 and Sort2.
[0087] Additionally, the difference between the respective metric value m1 and metric value m2 of each candidate text can be calculated separately, and each candidate text is sorted from largest to smallest according to this difference to obtain Sort3.
[0088] Among them, the smaller the difference here, the more familiar the large model S is with this candidate text. Or rather, the larger the difference, the more uncertain the large model S is about this candidate text, that is, it has not learned this candidate text. Therefore, texts with larger differences need to be selected to train the large model S.
[0089] It should be understood that sorting each candidate text from largest to smallest according to this difference actually ranks the candidate texts that the large model S is not familiar with or is uncertain about in the front.
[0090] When Sort3 is also obtained, the comprehensive sorting of each candidate text can be determined according to Sort1, Sort2, and Sort3. Among them, the priority of Sort3 is higher than that of Sort1 and lower than that of Sort2.
[0091] Specifically, the initial comprehensive sorting can be determined first according to Sort2 and Sort3 (the method is similar to determining the comprehensive sorting according to Sort2 and Sort1). If the metric values m2 and m1 of any two or more candidate texts are the same, then the final sorting position is determined according to the positions of these any two or more candidate texts under Sort1, and thus the final comprehensive sorting can be obtained.
[0092] Taking the above metric values m1 and m2 both being PPL as an example, the determination method of the comprehensive sorting can be expressed as the following formula:
[0093] ArgSort(minimize(PPL(X i ; T)), maximize(PPL(X i ; S) -
[0094] PPL(X i; T)), maximize(PPL(X i ; S)); Ascending = True) (Formula 3)
[0095] Wherein, PPL(X i ; T) is the perplexity of the teacher model for the candidate text X i , PPL(X i ; S) is the perplexity of the student model for the candidate text X i , and PPL(X i ; S) - PPL(X i ; T) is the difference between the above two perplexities.
[0096] The meaning of the above Formula 2 is that for the large model S, candidate texts with a small PPL(X i ; T), a large PPL(X i ; S), and a large difference are needed to form the pre-training sample set of the large model S.
[0097] Step S310, based on the comprehensive ranking, select a predetermined number of candidate texts from the candidate text set, and determine the pre-training sample set of the large model S based on them.
[0098] Taking the example where in the comprehensive ranking, the index value m1 of each candidate text is arranged from large to small, the index value m2 is arranged from small to large, and the difference is arranged from large to small, a predetermined number of candidate texts with a forward comprehensive ranking can be selected from each candidate text to determine the pre-training sample set of the large model S.
[0099] Specifically, when the candidate text set contains all texts in the original text set, directly form the pre-training sample set of the large model S based on a predetermined number of candidate texts with a forward comprehensive ranking. When the candidate text set is a text subset corresponding to any class cluster, it is also necessary to select each candidate text from the text subsets corresponding to other class clusters. Finally, form the pre-training sample set of the large model S based on each candidate text selected from the N text subsets corresponding to the N class clusters.
[0100] In this way, a high-quality and unlearned pre-training sample set is selected for the large model S.
[0101] It should be understood that when the large model S is a pre-trained model, this pre-training sample set is used to continue pre-train the large model S, so this pre-training sample set can also be called a continued pre-training sample set.
[0102] Figure 5 The schematic diagram of the method for selecting pre-training samples for the large model in an example of this specification is shown. Figure 5In the method, for the original text set, the Coreset clustering algorithm can be used to cluster it, so as to obtain N core sets corresponding to N clusters. After that, for each core set, each text in it can be input into the teacher model and the student model to obtain the perplexity PPL1 of the teacher model for each text in the core set and the perplexity PPL2 of the student model for each text in the core set. Then, each text in the core set can be sorted based on the perplexity PPL1, and each text in the core set can be sorted based on the perplexity PPL2. In addition, each text in the core set can be sorted according to the difference between the perplexity PPL1 and the perplexity PPL2. Finally, the three sorts can be integrated to obtain the comprehensive sort of each text in the core set, and based on this comprehensive sort, the target text is selected from the core set. After the target texts corresponding to each core set of each cluster are selected, a pre-training sample set of the student model is formed based on the selected target texts.
[0103] The above is a detailed description of the sample selection schemes corresponding to the pre-training stage and the continued pre-training stage respectively. The following describes the sample selection scheme corresponding to the fine-tuning stage.
[0104] Figure 6 The flowchart of the method for selecting fine-tuning samples for a large model according to an embodiment of the present specification is shown. This method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. As Figure 6 shown, this method may include the following steps:
[0105] Step S602, obtain the target training set and the test set Testset of the pre-trained large model LLM1.
[0106] Among them, the large model LLM1 here may refer to the large model S pre-trained based on the pre-training sample set selected by the method steps shown Figure 3 Of course, it may also refer to any pre-trained model.
[0107] The training texts in the above target training set and the test texts in the test set both have sample labels, and the sample labels are related to the downstream tasks performed by the large model LLM1. For example, they can be classification labels (corresponding to text classification tasks), or question answers (corresponding to question answering tasks), etc.
[0108] In one embodiment, the above target training set can be obtained by screening the initial training set Trainset. The screening process is as follows: for any test text Y in the test set Testset, calculate the test text Y jThe similarity with each initial text in the initial training set Trainset, and based on this similarity, determine from the initial training set Trainset the k initial texts that are closest to the test text Y j The k initial texts corresponding to each test text in the test set Testset are classified into the target training set. In the following description of this specification, the initial texts included in the target training set are referred to as training texts.
[0109] Taking any initial text x in the initial training set Trainset i as an example, the semantic representation model can be used to process the initial text x i and the test text Y j respectively, to obtain the representation vector Embed(x i ) and the representation vector Embed(Y j ). After that, based on the representation vector Embed(x i ) and the representation vector Embed(Y j ), calculate the similarity between the initial text x i and the test text Y j .
[0110] Among them, the above semantic representation model can include but is not limited to pre-trained models such as BERT, GPT series, etc. In addition, the above similarity can be, for example, cosine similarity, Euclidean distance, Manhattan distance, and Pearson correlation coefficient, etc.
[0111] Generally, the data volume of the initial training set Trainset is relatively large. Therefore, in this solution, the initial training set Trainset is first screened, that is, the initial texts that are far from the test text are screened out, which can reduce the scale of the training set and then improve the selection efficiency of the fine-tuning samples.
[0112] Of course, in practice, when the scale of the initial training set Trainset is relatively small, the initial training set Trainset can also be directly used as the above target training set, and this specification does not limit this.
[0113] Step S604, for any test text Y in the test set j , input the combined results of the test text Y j and each training text in the target training set into the trained large model LLM2 respectively, and obtain the respective first comprehensive probabilities corresponding to each combined result.
[0114] Among them, the large model LLM2 and the large model LLM1 here are used to perform the same tasks. For example, they are both used to perform text classification, question answering, or named entity recognition, etc.
[0115] Taking any training text xi For example, after inputting the combined result of the test text Y j and the training text x i into the large model LLM2, the probabilities predicted by the large model LLM2 for each character included in the combined result can be obtained. Then, the average of each probability can be calculated, and the average result can be used as the first comprehensive probability P(Y j |x i ), which means the probability predicted by the large model LLM2 for the test text Y i under the condition of using the training text x j .
[0116] Step S606: Obtain the second comprehensive probability obtained by inputting any test text Y j into the large model LLM2.
[0117] Specifically, after inputting the test text Y j into the large model LLM2, the probabilities predicted by the large model LLM2 for each character included in the test text Y j can be obtained. Then, the average of each probability can be calculated, and the average result can be used as the second comprehensive probability P(Y j ), which means the probability predicted by the large model LLM2 for the test text Y j when the test text Y j is input into the large model LLM2 alone.
[0118] Step S608: Select the target text corresponding to the test text Y j from the target training set by comparing the first comprehensive probabilities corresponding to the above combined results with the above second comprehensive probability.
[0119] For example, for the first comprehensive probability P(Y j |x i ) corresponding to the combined result of the test text Y j and the training text x i ), if P(Y j |x i ) > P(Y j ), it proves that using the training text x i as a condition improves the probability predicted by the large model LLM2 for the test text Y j . Therefore, the training text x i can be considered as a helpful sample and thus can be included in the fine-tuning sample set of the large model LLM1.
[0120] In one embodiment, training texts in each combination result corresponding to each first comprehensive probability greater than the second comprehensive probability can be used as the target text corresponding to the test text Y j respectively.
[0121] In another embodiment, only the training text in the combination result corresponding to the largest first comprehensive probability among the first comprehensive probabilities greater than the second comprehensive probability is used as the target text corresponding to the test text Y j respectively.
[0122] In a more specific embodiment, the difference between the logarithm value of each first comprehensive probability and the logarithm value of the second comprehensive probability is calculated respectively to obtain each difference corresponding to each combination result; the largest difference among the differences is determined, and when the largest difference is greater than a preset threshold, the training text in the combination result corresponding to the largest difference is determined as the target text corresponding to the test text Y j respectively.
[0123] Specifically, the target text corresponding to the test text Y can be determined according to the following formula: j respectively.
[0124]
[0125] It should be understood that the main idea of this formula is to transform the comparison of the magnitudes between two probabilities into the subtraction of two log-likelihoods.
[0126] The reason for adding the relu function to the above formula 4 is that in this solution, it is not necessary to screen the training texts where P(Y j |x i ) < P(Y j ).
[0127] Step S610: Based on the target texts selected for each test text in the test set Testset, a fine-tuning sample set of the large model LLM1 is formed.
[0128] Generally speaking, this solution forms a fine-tuning sample set of the large model LLM1 by selecting helpful samples from the target training set.
[0129] The following is an explanation of the theoretical support for the above formula 4:
[0130] It should be understood that the essence of formula 4 is to measure the mutual information (MI) between two variables, and the mutual information measures the degree of mutual dependence between two variables. Let the random variables (x, y) be a pair of random variables in the space. If their joint distribution is p(x, y), and the marginal distributions are p(x) and p(y) respectively, then the mutual information between them can be defined as:
[0131] I(x,y) = D KL (p(x,y) || p(x) ⊙ p(y)) (Formula 5)
[0132] where D KL is the KL divergence. If the joint distribution p(x,y) equals the product of the marginal distributions p(x) and p(y), then I(x,y) = 0. That is, when x and y are independent, observing y is of no help in predicting x, and their mutual information is 0 at this time.
[0133] The mutual information of discrete random variables x and y can be calculated as follows:
[0134]
[0135] Intuitively, if the entropy H(Y) is regarded as a measure of the uncertainty of a random variable, then H(Y|X) is the uncertainty of "the event Y will occur after the event X is known". Thus, the right side of the final equation can be read as "subtracting the uncertainty of the event Y from the uncertainty of the event Y that occurs based on the event c".
[0136] Therefore, it is necessary to minimize Formula 5, that is, to maximize Formula 4.
[0137] Figure 7 Schematic diagram showing a method for selecting fine-tuning samples for a large model according to an embodiment of this specification. Figure 7 In this method, based on each test text in the test set Ttestset of the large model LLM1 to be trained, k training texts similar to each test text can be screened out from the initial training set Trainset of the large model LLM1, and then the target training set can be obtained. After that, for each test text in the test set Testset, the test text can be combined with each training text in the target training set and then input into the trained large model LLM2, so that the first comprehensive probabilities corresponding to each combination result can be obtained. Finally, compare each first comprehensive probability with the second comprehensive probability determined by the large model LLM2 for this test text, select the largest first comprehensive probability from the first comprehensive probabilities greater than the second comprehensive probability, and classify the training text in the corresponding combination result into the fine-tuning sample set of the large model LLM2. It should be understood that after performing the above operation of selecting training texts for each test text in the test set Testset, the final fine-tuning sample set is obtained.
[0138] In summary, this solution proposes an efficient sample selection framework that selects high-quality samples from the original sample set for helping improve the performance of the large model in the pre-training, continued pre-training, and fine-tuning stages from six aspects: high quality, full coverage, diversity, helpfulness, non-mastery, and efficiency. In addition, this solution designs a mutual information theory based on information theory to efficiently calculate the correlation between the training text to be selected and the test text to quantitatively measure the quality of the training text. Finally, this solution unifies the multi-dimensional criteria for multi-stage sample selection, overcoming the problems of single-stage single-dimensional and multi-stage single-dimensional sample selection.
[0139] Correspondingly to the above sample selection method for large model training, an embodiment of this specification also provides a sample selection device for large model training, as Figure 8 shown, the device may include:
[0140] An input unit 802, configured to input any first candidate text in the candidate text set into the first large model to be trained, and obtain a first metric value indicating the degree of uncertainty of the first large model with respect to the first candidate text.
[0141] A determination unit 804, configured to determine a first ranking of each candidate text according to the first metric values of each candidate text.
[0142] An acquisition unit 806, configured to acquire a second ranking of each candidate text determined according to the second metric value of each candidate text, where the second metric value of any candidate text indicates the degree of uncertainty of the second trained large model with respect to the candidate text, and the number of parameters of the first large model is much smaller than that of the second large model.
[0143] The determination unit 804 is further configured to determine a comprehensive ranking of each candidate text according to the first ranking and the second ranking, where the priority of the second ranking is higher than that of the first ranking.
[0144] A selection unit 808, configured to select a predetermined number of candidate texts from the candidate text set based on the comprehensive ranking, and determine a pre-training sample set of the first large model based on the selected candidate texts.
[0145] In one embodiment, the first ranking is sorted from largest to smallest according to the first metric values of each candidate text, and the second ranking is sorted from smallest to largest according to the second metric values of each candidate text;
[0146] The selection unit 808 is specifically configured to:
[0147] Select a predetermined number of candidate texts with a higher comprehensive ranking from each candidate text.
[0148] In one embodiment, the device further includes:
[0149] A calculation unit 810 is configured to calculate the difference between the first index value and the second index value of each candidate text respectively, and sort the candidate texts in descending order according to the difference to obtain a third sorting result.
[0150] The determination unit 804 is specifically configured to:
[0151] Determine the comprehensive sorting result of each candidate text according to the first sorting result, the second sorting result and the third sorting result, where the priority of the third sorting result is higher than that of the first sorting result and lower than that of the second sorting result.
[0152] In one embodiment, the input unit 802 includes:
[0153] An acquisition sub-module 8022 is configured to acquire the probabilities predicted by the first large model for each character included in the first candidate text.
[0154] A calculation sub-module 8024 is configured to average the logarithms of the probabilities and then take the negative value to obtain a first operation value.
[0155] A determination sub-module is configured to determine the first index value according to the first operation value.
[0156] In one embodiment, the above candidate text set is a text subset corresponding to any one of the N clusters obtained by clustering the original text set using a clustering algorithm.
[0157] The selection unit 808 is specifically configured to:
[0158] Form a pre-training sample set of the first large model based on the candidate texts selected from the N text subsets corresponding to the N clusters.
[0159] In one embodiment, the text subset corresponding to any one of the clusters is formed based on the original texts in the original text set whose distances from the cluster center of the cluster are less than a preset threshold.
[0160] In one embodiment, the apparatus further includes:
[0161] A construction unit 812 is configured to construct a prompt word for any first corpus text in the corpus, where the prompt word indicates whether the first corpus text is a high-quality text based on several judgment criteria.
[0162] The input unit 802 is further configured to input the prompt word into the target large model to obtain an output result indicating whether the first corpus text is a high-quality text.
[0163] A formation unit 814 is configured to form the above original text set based on the target corpus texts in each corpus text whose corresponding output results indicate high-quality texts.
[0164] In one embodiment, the above-mentioned several judgment criteria include one or more of the following:
[0165] Clarity, domain, complexity, problem-solving ability, creativity, technical accuracy, application relevance.
[0166] In one embodiment, the above-mentioned first large model is a pre-trained model, and it is trained based on the original text set. The above-mentioned pre-training sample set is used to continue pre-training the first large model.
[0167] In one embodiment, the obtaining unit 806 is further configured to obtain the target training set and the test set of the first large model after pre-training;
[0168] The input unit 802 is further configured to, for any first test text in the test set, input the combination results of the first test text and each training text in the target training set into the trained third large model, and obtain the first comprehensive probabilities corresponding to each combination result. The third large model and the first large model are used to perform the same task;
[0169] The obtaining unit 806 is further configured to obtain the second comprehensive probability obtained by inputting the first test text into the third large model;
[0170] The selection unit 808 is further configured to select the target text corresponding to the first test text from the target training set by comparing each first comprehensive probability with the second comprehensive probability;
[0171] The determining unit 804 is further configured to form a fine-tuning sample set of the first large model based on the target texts selected for each test text in the test set.
[0172] The obtaining unit 806 is specifically configured to:
[0173] Obtain the initial training set of the first large model;
[0174] For any test text in the test set, calculate the similarity between the test text and each initial text in the initial training set, and determine the k initial texts closest to the test text from the initial training set according to the similarity;
[0175] Use the k initial texts corresponding to each test text in the test set as each training text to form the target training set.
[0176] The input unit 802 is specifically configured to:
[0177] Obtain the probabilities predicted by the third large model for each character included in the first combination result of the first test text and any first training text;
[0178] Average the respective probabilities, and use the averaged result as the first comprehensive probability corresponding to the first combined result.
[0179] In one embodiment, the selection unit 808 is specifically configured to:
[0180] Respectively subtract the logarithm of the second comprehensive probability from the logarithm of each first comprehensive probability to obtain respective differences corresponding to each combined result;
[0181] Determine the maximum difference among the respective differences. When the maximum difference is greater than a preset threshold, determine the training text in the combined result corresponding to the maximum difference as the target text corresponding to the first test text.
[0182] The functions of the functional units of the device in the above embodiments of this specification can be implemented by the respective steps of the method embodiments above. Therefore, the specific working process of the device provided in an embodiment of this specification will not be repeated here.
[0183] The sample selection device for large model training provided in an embodiment of this specification can effectively select high-quality samples for the large model.
[0184] According to an embodiment of another aspect, there is also provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed in a computer, the computer is made to execute the method described in combination with Figure 3 or Figure 6 as described.
[0185] According to an embodiment of still another aspect, there is also provided a computing device, including a memory and a processor. An executable code is stored in the memory. When the processor executes the executable code, the method described in combination with Figure 3 or Figure 6 as described is implemented.
[0186] The various embodiments in this specification are all described in a progressive manner. The same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the medium or device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0187] The above has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0188] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of this specification. It should be understood that the above are only specific embodiments of this specification and are not used to limit the protection scope of this specification. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of this specification shall be included within the protection scope of this specification.
Claims
1. A method for sample selection in large model training, comprising: For any first candidate text in the candidate text set, input it into the first large model to be trained, and obtain a first index value indicating the uncertainty degree of the first large model for the first candidate text; Determine the first ranking of the candidate texts according to the first index values of the candidate texts; Obtain the second ranking of the candidate texts determined according to the second index values of the candidate texts; the second index value of any candidate text indicates the uncertainty degree of the trained second large model for the candidate text; the number of parameters of the first large model is much smaller than that of the second large model; Determine the comprehensive ranking of the candidate texts according to the first ranking and the second ranking; Wherein the priority of the second ranking is higher than that of the first ranking; Based on the comprehensive ranking, select a predetermined number of candidate texts from the candidate text set, and determine the pre-training sample set of the first large model based on them.
2. The method according to claim 1, wherein, The first ranking is sorted from large to small according to the first index values of the candidate texts, and the second ranking is sorted from small to large according to the second index values of the candidate texts; The step of selecting a predetermined number of candidate texts from the candidate text set includes: Select a predetermined number of candidate texts with a higher comprehensive ranking from the candidate texts.
3. The method according to claim 2, further comprising: Calculate the difference between the first index value and the second index value of each candidate text respectively, and sort the candidate texts from large to small according to the difference to obtain a third ranking; The step of determining the comprehensive ranking of the candidate texts includes: Determine the comprehensive ranking of the candidate texts according to the first ranking, the second ranking and the third ranking; The priority of the third ranking is higher than that of the first ranking and lower than that of the second ranking.
4. The method according to claim 1, wherein The step of obtaining a first index value indicating the uncertainty degree of the first large model for the first candidate text includes: Obtain the probabilities predicted by the first large model for each character included in the first candidate text; Take the average of the logarithms of the probabilities and then take the negative value to obtain a first operation value; Determine the first index value according to the first operation value.
5. The method according to claim 1, wherein The candidate text set is a text subset corresponding to any one of the N clusters obtained by clustering the original text set using a clustering algorithm; The step of determining the pre-training sample set of the first large model includes: Based on the candidate texts selected from the N text subsets corresponding to the N clusters, form the pre-training sample set of the first large model.
6. The method according to claim 5, wherein, The text subset corresponding to any one of the clusters is formed based on the original texts in the original text set whose distances from the cluster center of the cluster are less than a preset threshold.
7. The method according to claim 5, wherein The original text set is obtained through the following steps: For any first corpus text in the corpus, construct a prompt word based on the first corpus text, and the prompt word indicates whether the first corpus text is a high-quality text based on several judgment criteria; Input the prompt into the target large model to obtain an output result indicating whether the first corpus text is a high-quality text; Based on each corpus text, form the original text set from each target corpus text whose corresponding output result indicates a high-quality text.
8. The method according to claim 7, wherein The several judgment criteria include one or more of the following: Clarity, domain specificity, complexity, problem-solving ability, creativity, technical accuracy, application relevance.
9. The method according to claim 7, wherein The first large model is a pre-trained model and is trained based on the original text set; the pre-training sample set is used to continue pre-training the first large model.
10. The method according to claim 1, further comprising: Obtain a target training set and a test set of the first large model after pre-training; For any first test text in the test set, input the first test text and each combination result of each training text in the target training set into the trained third large model to obtain each first comprehensive probability corresponding to each combination result; the third large model and the first large model are used to perform the same task; Obtain a second comprehensive probability obtained by inputting the first test text into the third large model; By comparing each first comprehensive probability with the second comprehensive probability, select a target text corresponding to the first test text from the target training set; Based on the target texts selected for each test text in the test set, form a fine-tuning sample set of the first large model.
11. The method according to claim 10, wherein, The obtaining the target training set of the first large model after pre-training includes: Obtain the initial training set of the first large model; For any test text in the test set, calculate the similarity between the test text and each initial text in the initial training set, and based on the similarity, determine the k initial texts closest to the test text from the initial training set; Use the k initial texts corresponding to each test text in the test set as the respective training texts to form the target training set.
12. The method according to claim 10, wherein, The obtaining each first comprehensive probability corresponding to each combination result includes: Obtain the probabilities predicted by the third large model for each character included in the first combination result of the first test text and any first training text; Average the probabilities, and use the averaged result as the first comprehensive probability corresponding to the first combination result.
13. The method according to claim 10, wherein The selecting the target text corresponding to the first test text from the target training set includes: Respectively calculate the difference between the logarithm of each first comprehensive probability and the logarithm of the second comprehensive probability to obtain each difference corresponding to each combination result; Determine the maximum difference among the differences. When the maximum difference is greater than a preset threshold, determine the training text in the combination result corresponding to the maximum difference as the target text corresponding to the first test text.
14. A sample selection device for large model training, comprising: An input unit, configured to, for any first candidate text in the candidate text set, input it into the first large model to be trained to obtain a first index value indicating the degree of uncertainty of the first large model with respect to the first candidate text; A determining unit, configured to determine a first ranking of each candidate text according to a first metric value of each candidate text; An obtaining unit, configured to obtain a second ranking of each candidate text determined according to a second metric value of each candidate text; the second metric value of any candidate text indicates the degree of uncertainty of the trained second large model with respect to the candidate text; the number of parameters of the first large model is much smaller than that of the second large model; The determining unit is further configured to determine a comprehensive ranking of each candidate text according to the first ranking and the second ranking; wherein the priority of the second ranking is higher than that of the first ranking; A selecting unit, configured to select a predetermined number of candidate texts from the candidate text set based on the comprehensive ranking, and determine a pre-training sample set of the first large model based thereon.
15. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed on a computer, the computer is caused to execute the method according to any one of claims 1-13.
16. A computing device, comprising a memory and a processor, wherein, An executable code is stored in the memory, and when the processor executes the executable code, the method according to any one of claims 1-13 is implemented.