Screening method and equipment of instruction data, medium and product

Abstract

The invention relates to the technical field of information, and discloses an instruction data screening method and device, a medium and a product. The method comprises the following steps: determining a first confusion degree and a second confusion degree according to a pre-trained lightweight general language model and a target sample set; the first confusion degree is used for representing the prediction confusion degree under the instruction condition; the second confusion degree is used for representing the prediction confusion degree under the instruction-free condition; determining an information gain score according to the first confusion degree and the second confusion degree; the information gain score is used for quantifying the contribution degree of the instruction content to reduction of the response content generation difficulty; determining a training data set according to the information gain score and the target sample set; the training data set is used for improving the generalization ability and instruction following precision of the large language model in the instruction fine tuning stage. The technical problems of high deployment threshold, high cost, supervision dependence, low training efficiency and poor model generalization ability can be at least solved.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a method, device, medium, and product for screening instruction data. Background Art

[0002] Currently, command fine-tuning has become the mainstream paradigm for large-scale language models to improve their ability to respond to human commands. Command fine-tuning is used after model pre-training to adjust the model behavior with a small amount of high-quality data to make its output more in line with human needs.

[0003] As the scale of the model expands, the amount of training data involved in instruction fine-tuning explodes. This data contains a large amount of redundant, ambiguous, or even misleading information. The current mainstream training data screening solutions in the industry include strong model evaluation, training-specific scorers, or reward models.

[0004] However, the inventors found that there are at least the following technical problems in the relevant technology: the expansion of the model scale leads to an increase in computing demand, and the demand for powerful hardware increases accordingly, resulting in low data screening efficiency and poor generalization ability of data screening methods. Summary of the Invention

[0005] One purpose of the present application is to provide a method for screening instruction data, at least to solve the current problems of low data screening efficiency and poor generalization ability of data screening methods.

[0006] To achieve the above objectives, some embodiments of the present application provide the following aspects: In a first aspect, some embodiments of the present application provide a method for screening instruction data, the method comprising: determining a first perplexity and a second perplexity based on a pre-trained lightweight general language model and a target sample set; the first perplexity is used to characterize the predicted perplexity under the condition of instructions; the second perplexity is used to characterize the predicted perplexity under the condition of no instructions; determining an information gain score based on the first perplexity and the second perplexity; the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating response content; determining a training data set based on the information gain score and the target sample set; the training data set is used to improve the generalization ability and instruction following accuracy of the large language model in the instruction fine-tuning stage.

[0007] In a second aspect, some embodiments of the present application further provide an electronic device comprising: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, cause the processor to perform the steps of the method described above.

[0008] In a third aspect, some embodiments of the present application further provide a computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method described above.

[0009] In a fourth aspect, some embodiments of the present application further provide a computer program product, comprising a computer program / instruction, which implements the steps of the above-described method when executed by a processor.

[0010] Compared with the related art, the solution provided in the embodiment of the present application is based on a pre-trained lightweight universal language model for modeling, which can avoid the use of large-scale models to lower the deployment threshold. The first perplexity and the second perplexity are determined based on the lightweight universal language model and the pre-processed target sample set. In this way, the predicted perplexity of the model under instruction conditions and without instruction conditions is determined to measure the overall quality of the model generation; then, the information gain score is calculated by comparing the perplexity under instruction conditions and without instruction conditions. This calculation step reflects the contribution of the instructions in the target sample set to the generation of response content; finally, the training data set is determined based on the information gain score and the target sample set. The entire calculation process is based on reasoning based on the pre-trained model, without the need for any fine-tuning or manual labeling. In this way, by providing an instruction data screening method based on a lightweight model, not only can the training efficiency be improved, but also the screening of hundreds of thousands of samples can be completed on a single consumer-grade graphics card. The speed is improved compared to traditional methods. Moreover, by using a small number of instruction data samples, the model performance of pre-training on a large amount of non-screened data can be achieved. It has strong generalization ability and can be flexibly adapted to various instruction data formats such as Alpaca, WizardLM, ShareGPT, etc. In addition, it can be widely used in training adjustments in vertical fields such as finance, medical care, and law. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0012] Figure 1 An exemplary flow chart of a method for screening instruction data provided in some embodiments of the present application; Figure 2 A schematic structural diagram of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0013] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] The following terms are used in this article.

[0015] In natural language processing and language models, a token represents the basic unit of text processing by the model, usually a word after the text is segmented.

[0016] PPL, Perplexity, is used to measure the overall generation quality of the model; the lower the PPL value, the higher the model prediction accuracy and the stronger the generation ability.

[0017] GPU, Graphics Processing Unit, graphics processor, is used to process graphics rendering related tasks and has highly parallel computing capabilities.

[0018] First embodiment The first embodiment of the present application relates to a method for screening instruction data. Figure 1 As shown, the method may include the following steps: Step S10, determining a first perplexity and a second perplexity based on the pre-trained lightweight universal language model and the target sample set; the first perplexity is used to represent the prediction perplexity under the condition of instructions; the second perplexity is used to represent the prediction perplexity under the condition of no instructions; Step S20: determining an information gain score based on the first perplexity and the second perplexity; the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating the response content; Step S30: determining a training data set based on the information gain score and the target sample set; the training data set is used to improve the generalization ability and instruction following accuracy of the large language model in the instruction fine-tuning stage.

[0019] First, it's important to note that instruction data is a unified format specifically constructed to tune the behavior of language models. It's used to train the model's ability to understand and respond to human instructions. Instruction data is typically formatted as a triple (Instruction, Input, Response). For example, in the instruction "Translate the following text into English," Instruction represents the instruction, which can be a clear description of the task in natural language; Input represents the input, which can be a sentence or paragraph from the text; and Response represents the output, which presents the ideal answer that meets the instruction's requirements.

[0020] The following describes each of the above steps in detail.

[0021] Regarding step S10, specifically, the lightweight universal language model may use a small or medium-sized model with a parameter magnitude of 0.5B-1.0B, such as Qwen2.5-0.5B.

[0022] Specifically, the target sample set is a collection of processed original instruction datasets, which may include relevant training data when adjusted for vertical fields. A data preprocessor is used to standardize the structure of the training data, remove invalid samples containing illegal or missing fields, and normalize the samples in the original instruction dataset into a triple format to ensure that each sample has a standardized structure, resulting in the target sample set. Illegal fields can be garbled characters or special symbols, and missing fields can be no instruction or an empty response.

[0023] Specifically, a pre-trained lightweight general language model is used to input each sample in the preprocessed dataset under both conditions of instructions and no instructions. The response generation process is simulated by the lightweight model analyzer, and the response sequence is evaluated to obtain the response probabilities under the conditions of instructions and no instructions. The perplexity under the conditions of instructions and no instructions is then calculated based on the response probabilities; the predicted perplexity under the conditions of instructions is the first perplexity; the predicted perplexity under the conditions of no instructions is the second perplexity.

[0024] Regarding step S20, specifically, the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating the response content. By converting the difference between the first perplexity and the second perplexity into a quantifiable instruction contribution indicator, it can be used to screen high-value data.

[0025] For example, in a text translation task, the samples in the target sample set include the following: { "instruction": "Translate the following English into Chinese", "input": "The rapid development of AI technology requires robustethical guidelines.", "output": "The rapid development of artificial intelligence technology requires strong ethical guidelines." } Instructions are used to clarify the task type. Without instructions, the model may continue writing in English or translating into other languages, resulting in an increase in the second perplexity. This, in turn, increases the difference between the second perplexity and the first perplexity, which in turn increases the information gain score. A higher information gain score indicates a greater contribution of the instructions to the generation quality of the pre-trained lightweight general language model. Therefore, the information gain score, as a quantitative indicator, reflects the effectiveness of instructions.

[0026] Regarding step S30, specifically, the information gain score is used as a dimension to evaluate sample quality, and other evaluation dimensions need to be determined based on the target sample set. After determining the indicators of all evaluation dimensions, the indicators are combined to determine the training data set for instruction fine-tuning.

[0027] Understandably, in related technologies, training and inference of large-scale models requires high-end GPU clusters, which are extremely expensive. Furthermore, as the number of parameters increases, the model requires more instruction data to achieve stable performance, and data training efficiency decreases as the number of parameters increases. Large models tend to memorize details of the training data, which also leads to reduced generalization capabilities when faced with new instructions. In contrast, small and medium-sized models, with their fewer parameters and less memory usage, can run on consumer-grade GPUs or mobile phones, significantly lowering the hardware barrier. Each instruction data has a more significant impact on the model, resulting in higher training efficiency. Therefore, using lightweight small and medium-sized general language models, paired with high-quality instruction data for training, is more cost-effective.

[0028] It is not difficult to find that compared with the related art, the solution provided in the embodiment of the present application is based on a pre-trained lightweight universal language model for modeling, which can avoid the use of large-scale models to lower the deployment threshold. The first perplexity and the second perplexity are determined based on the lightweight universal language model and the pre-processed target sample set. In this way, the predicted perplexity of the model under instruction conditions and without instruction conditions is determined to measure the quality of the overall model generation; then, the information gain score is calculated by comparing the perplexity under instruction conditions and without instruction conditions. This calculation step reflects the contribution of the instructions in the target sample set to the generation of response content; finally, the training data set is determined based on the information gain score and the target sample set. The entire calculation process is based on reasoning based on the pre-trained model, without the need for any fine-tuning or manual labeling. In this way, by providing an instruction data screening method based on a lightweight model, not only can the training efficiency be improved, but the screening of hundreds of thousands of samples can be completed on a single consumer-grade graphics card, and the speed is improved compared to traditional methods. Moreover, by using a small number of instruction data samples, the model performance of a large amount of non-screened data pre-training can be achieved. It has strong generalization ability and can be flexibly adapted to various instruction data formats such as Alpaca, WizardLM, ShareGPT, etc. In addition, it can be widely used in training adjustments in vertical fields such as finance, medical care, and law.

[0029] Second embodiment The second embodiment of the present application relates to a method for filtering instruction data. The second embodiment is an improvement on the first embodiment. Specifically, the improvement is that: in this embodiment, a method for calculating a first perplexity and a second perplexity based on a target sample set is provided, as well as a method for calculating an information gain score.

[0030] Optionally, in some embodiments, the target sample set includes the instruction content, input content, and output content of each sample; The first perplexity is specifically used to represent the degree of prediction uncertainty of the response content generated by the lightweight universal language model under the joint constraints of the instruction content and the input content; The second perplexity is used to characterize the degree of prediction uncertainty of the response content generated by the lightweight universal language model under the constraints of the input content alone and when the instruction content is missing.

[0031] Specifically, the target sample set can provide a clear learning signal for the lightweight general language model through a triple in the format of (Instruction, Input, Response). This instruction constraint of precise instructions, necessary inputs, and strict output compliance helps to obtain high-quality samples.

[0032] For example, in the presence of a command, the input format may include the command and input text. The lightweight universal language model generates a response based on the complete command and context, and calculates the output perplexity as the first perplexity. In the absence of a command, the input format only includes the input text, and the lightweight universal language model calculates the perplexity when generating a response based solely on the input text as the second perplexity. It can be seen that if the first perplexity is less than the second perplexity, it means that the command can significantly reduce the generation difficulty of the lightweight universal language model.

[0033] Specifically, the method for calculating the first perplexity and the second perplexity is as follows: Using a pre-trained lightweight general language model, each sample in the dataset is input under both instructions and no instructions. The lightweight model analyzer simulates the response generation process, evaluates the response sequence, and then obtains the response probability. The corresponding perplexity is then calculated based on the response probability.

[0034] The calculation method of the first perplexity is:

[0035] The calculation method of the second perplexity is:

[0036] in, Indicates the first samples; Indicates the The input of samples; Indicates the After samples are input, the model outputs a response sequence; Indicates the The first sample Token, that is, the first token in the response sequence word; Indicates the The first sample All tokens before the token; Indicates the The length of the response sequence of each sample, that is, the number of tokens; Represents the response probability, which is a pre-trained lightweight universal language model based on Or the probability of a historical Token outputting the next Token.

[0037] Specifically, Used to distinguish different samples. For example, there are 100 samples in the target sample set. It can represent the first instruction in the data set and its corresponding answer. Can contain directives and / or context. Response Sequence It can be a text sequence generated by the model, for example, the answer generated by the model after inputting a command. If the original sentence is "Artificial intelligence is very good.", after the word segmentation strategy, 5 tokens are obtained, and the token sequence is ["artificial", "intelligent", "very", "good", "."], , the second token is "intelligent", and when the model generates the sequence, it outputs each token one by one, first outputting "artificial" and then "intelligent". The number of tokens cannot be equated with the number of words or phrases. The number of tokens depends on the specific word segmentation strategy. Different word segmentation strategies will produce different numbers of tokens. In the non-instruction scenario, Indicates that the model only predicts the probability of the next token based on the historical token; in the case of instructions, the model also combines And the probability of predicting the next Token based on the generated Token.

[0038] Optionally, in some embodiments, determining the information gain score according to the first perplexity and the second perplexity, that is, step S20 includes: Calculate the difference between the first perplexity and the second perplexity of each sample in the target sample set to obtain an information gain score; or, Perform negative logarithmic mean operations on the first perplexity and the second perplexity to obtain the corresponding first conditional entropy and second conditional entropy, and calculate the difference between the second conditional entropy and the first conditional entropy of each sample to obtain the information gain score; The first conditional entropy is used to characterize the average prediction uncertainty of the response content generated under the condition of instructions; the second conditional entropy is used to characterize the average prediction uncertainty of the response content generated under the condition of no instructions.

[0039] Specifically, the information gain calculator compares the difference in perplexity between the two conditions of instruction and no instruction to calculate the information gain score. The specific calculation method can be referred to as follows:

[0040] in, Indicates the first samples; Indicates the The input of samples; Indicates the After samples are input, the model outputs a response sequence; represents the perplexity under no-instruction conditions, i.e., the second perplexity, which characterizes the uncertainty of the generated response sequence under no-instruction conditions; Represents the uncertainty of generating a response sequence under instruction conditions, that is, the first perplexity; Represents the information gain score, that is, the information gain brought by the instruction. The larger the value, the more effective the instruction.

[0041] or, Specifically, the information gain calculator compares the entropy under the conditions of instruction and no instruction to calculate the information gain score. The specific calculation method can be referred to as follows:

[0042] in,

[0043]

[0044] in, Indicates the first samples; Indicates the The input of samples; Indicates the After samples are input, the model outputs a response sequence; Indicates the The first sample Token, that is, the first token in the response sequence word; Indicates the The first sample All tokens before the token; Indicates the The length of the response sequence of each sample, that is, the number of tokens; Represents the response probability, which is a pre-trained lightweight universal language model based on Or the probability of the historical token outputting the next token; It represents the entropy when there is an instruction, that is, the first conditional entropy; It represents the entropy when there is no instruction, that is, the second conditional entropy.

[0045] Specifically, entropy is the average uncertainty of each token, while perplexity is its exponential form. The definitions of entropy and perplexity are equivalent. Therefore, the information gain score can be obtained by calculating the difference in perplexity or entropy.

[0046] It is not difficult to find that in the embodiment of the present application, by comparing the difference in perplexity between the two scenarios with and without instructions, the contribution of instructions to the quality of model generation is directly quantified, subjective evaluation bias is avoided, and high information gain samples can be identified and retained, and inefficient data can be filtered out; and because the model trained with high information gain samples can better understand the nature of instructions, it can not only reduce computing power consumption, but also enhance the model's ability to understand the nature of instructions.

[0047] Third embodiment The third embodiment of the present application relates to a method for determining a training data set. The third embodiment is an improvement on the first embodiment, and the specific improvement is that: in this embodiment, a method for determining a factor set and a training data set is provided.

[0048] Optionally, in some embodiments, in step S30, determining the training data set according to the information gain score and the target sample set includes: Step S301: Obtaining a goodness score for each sample based on the target sample set and the information gain score; the goodness score is used to reflect the quality of the sample, and the higher the sample quality, the higher the goodness score; Step S302: Determine the training data set based on the quality score of each sample.

[0049] Specifically, in step S301, the purpose of calculating a goodness score is to assess the quality of each sample in the target sample set. Generating a goodness score allows for quantifying the training value of each sample. A sample goodness scorer supports weighting, normalization, and dynamic threshold adjustment, integrating information gain scores and other quantitative metrics to construct a unified sample value metric. All metrics provide goodness scores within the [0, 1] range. Exemplary other quantitative metrics may include, but are not limited to, response entropy, word length factor, and content coverage.

[0050] For step S302, specifically, based on the goodness score, a refined sample generator is used to screen out samples with high goodness scores, and construct a training data set that is ultimately used for instruction fine-tuning. The refined sample generator refers to a tool or process that screens, processes, and optimizes the original samples based on the goodness scores of the samples during the construction of the instruction fine-tuning training data set, and ultimately produces high-quality training data. Exemplarily, the generated goodness scores can be screened, and when the goodness score is >0.8, it is high goodness, when the goodness score is <0.5, it is low goodness, and the rest are medium goodness; the high-goodness samples are retained, and the medium-goodness samples are manually reviewed, and the medium-goodness samples are retained after correcting the ambiguous instructions or erroneous outputs. The screened samples constitute the training data set, and the training data set accurately adjusts the model behavior through high-quality samples, covers a variety of valid instruction-output pairs, can enhance generalization, reduce invalid calculations of low-value samples, and optimize training efficiency.

[0051] Optionally, in some embodiments, the obtaining of a goodness score for each sample based on the target sample set and the information gain score, that is, step S301 includes: Step S3011, calculating the response entropy, word length factor, and content coverage of each sample in the target sample set; Step S3012: Obtain information gain score, and combine the response entropy, word length factor, and content coverage to form a factor set; the factor set is used to characterize a collection of indicators for evaluating sample quality; Step S3013: Perform sample scoring based on the factor set to obtain a goodness score for each sample.

[0052] For step S3011, specifically, the response entropy To measure the Generate a response sequence under the conditions The uncertainty or diversity reflects the uniformity of the model's predicted probability distribution for each Token. The smaller the value of , the higher the certainty of the model prediction.

[0053] The specific method for calculating response entropy can be referred to as follows:

[0054] in, Indicates the first samples; Indicates the The input of samples; Indicates the After samples are input, the model outputs a response sequence; Indicates the The first sample Token, that is, the first token in the response sequence word; Indicates the The first sample All tokens before the token; Indicates the The length of the response sequence of each sample, that is, the number of tokens; Indicates the The response entropy of samples; The vocabulary of the model (Vocabulary), which contains all possible token collections; Representation vocabulary Any Token in; The presentation model is based on instructions and the generated sequence , predict the next Token to become probability.

[0055] It is worth noting that both perplexity and response entropy are used to evaluate the quality of language model generation, but their calculation methods and application scenarios are significantly different. Specifically, perplexity is mainly used to measure the difficulty of the model in predicting the real sequence, and only calculates the response probability of the real token. The smaller the perplexity value, the more accurate the model, with an ideal perplexity value of 1. Response entropy is primarily used to measure the uncertainty or diversity of model predictions. It requires calculating the probability distribution of all tokens in the vocabulary. Smaller response entropy values ​​reflect higher model certainty and a lack of diversity. Larger response entropy values ​​fail to reflect model certainty, and the specific task must be considered. For example, in factual question answering, response entropy values ​​are very low, while creative text may require higher response entropy values.

[0056] The method for calculating word length factor can be referred to as follows:

[0057] in, Indicates the first samples; Represents a response sequence The length of , that is, the number of tokens; represents the preset ideal response length mean; Indicates length tolerance, Indicates the The word length factor of the samples.

[0058] Specifically, It is used to control the allowable deviation range between the length of the model output response sequence and the preset ideal response length. It can be set manually. The larger the set value, the wider the allowable deviation range. The value range is (0,1], which is used to penalize the generation results that deviate from the preset ideal response length mean. The greater the difference between the actual length and the target, the stronger the penalty. The penalty logic is as follows: When the response sequence Length Deviation from the preset ideal response length mean hour, Decreases and approaches 0, and the score decreases; When the response sequence Length hour, , at this time the ideal state is reached and there is no penalty.

[0059] The method for calculating content coverage can be referred to as follows:

[0060] in, Indicates the first samples; Represents a predefined set of keywords; Representing a collection The total number of keywords contained in Indicates the After samples are input, the model outputs a response sequence; Represents a unique set of Tokens in the response sequence; Indicates the Content coverage of the samples.

[0061] Specifically, The value range is (0,1], which is used to measure the response sequence The ratio of the number of unique keywords to all keywords reflects the relevance and completeness of the content produced. The closer the value is to 1, the more it represents a response sequence. The more domain keywords it contains. It can usually be determined by manual setting or automatic extraction, depending on the task requirements and data characteristics. For example, in the medical field, . It refers to the set obtained by removing duplicates from the token sequence of the generated text. That is, each token is retained once, and repeated tokens are not counted. Deduplication can avoid repeated tokens from interfering with the calculation and prevent the model from artificially increasing the score by repeating keywords. In addition, unique tokens can more truly reflect the diversity of the response content and ensure that each keyword contributes equally.

[0062] Specifically, in step S3012, the information gain score is integrated with the indicators from step S3011 to form a multi-dimensional quality assessment system, namely a factor set. This factor set overcomes the limitations of a single indicator and comprehensively quantifies sample quality across multiple dimensions, allowing for more accurate identification of high-quality samples.

[0063] Specifically, in step S3013, sample scoring is the process of numerically evaluating the quality of generated samples by constructing a unified sample value index based on the comprehensive scores of multiple factors. Scoring relies on a set of factors. By integrating evaluation metrics within the factor set, such as information gain, response entropy, and word length factor, into a unified score, it facilitates sorting, filtering, or optimizing the generated model.

[0064] Optionally, in some embodiments, a training data set is determined based on the goodness score of each sample, and step S302 includes: Obtain the goodness score of each sample, arrange the samples according to the descending rule, output the samples that are greater than the set threshold range, and obtain the training data set.

[0065] Specifically, we first obtain the goodness scores of all samples. Using the Refine Sample Generator, we sort them in descending order of goodness score, prioritizing high-quality samples. We then select samples with a threshold greater than the top k% as the final fine-tuning training set. For example, the value of k% can be 5%-10%, but this also needs to be fine-tuned based on the application scenario. Finally, we output a high-quality training dataset.

[0066] It should be noted that this embodiment may also be an improvement based on the second embodiment.

[0067] It is not difficult to find that in the embodiment of the present application, a method for determining the goodness score is provided. The response entropy, word length factor, and content coverage of each sample in the target sample set are calculated. These three indicators and the information gain score are combined to form a factor set. For the factor set, according to the preset sample scoring rules, the goodness score of each sample is obtained. The goodness scores are then sorted in descending order, and samples greater than the set threshold range are output as the training data set. In this way, the screening of high-quality samples can be completed without supervision or manual methods, while ensuring the effectiveness of the model and greatly improving the training efficiency.

[0068] Fourth embodiment The fourth embodiment of the present application relates to a method for generating fog of war. The fourth embodiment is an improvement based on the first, second, and third embodiments. The specific improvement is that: in this embodiment, a method for performing sample scoring is provided. Optionally, in some embodiments, the sample scoring is performed according to the factor set to obtain a goodness score for each sample, that is, step S3013 includes: Step S30131: Introduce artificial rules or external models to optimize sample scoring. and / or, Step S30132: Obtain the weight of each indicator in the factor set, and obtain the quality score of each sample based on the normalized data set.

[0069] Specifically, for step S30131, a multi-strategy aggregator can be invoked to introduce manual rules or external models (such as the BERT-based quality discrimination model or an information entropy-based diversity scoring model) to enhance screening robustness. For example, each participant (using manual rules or the aforementioned models) can first provide their opinions or scores. A weighted voting strategy is then employed to sum the results based on preset weights to produce a comprehensive quality score. Since the comprehensive score changes from the initial score, the relative quality of samples may change. Therefore, the sample quality ranking can be updated based on this comprehensive score, ultimately improving the accuracy of the quality score.

[0070] Regarding step S30132, specifically, before performing the goodness score for each sample, it is necessary to normalize the various indicators in the factor set. The specific calculation method can be referred to as follows:

[0071] in, Indicates the first samples; Indicates the candidate pool The maximum value of Indicates the candidate pool The minimum value of Indicates the first indicator values; represents the information gain score , word length factor and content coverage The normalized result is Represents response entropy The normalized result.

[0072] It should be noted that due to the differences in units and numerical ranges of different indicators, such as content coverage The value of can be 5, and the information gain score It will dominate the results, and large numerical indicators will suppress small numerical indicators. Therefore, the purpose of normalization is to uniformly scale the values ​​of different indicators to the same domain value range. After normalization, all indicators are in the same interval.

[0073] Specifically, The value range is [0,1]. According to the minimum-maximum linear scaling rule, the original data is linearly mapped to a fixed range in proportion, the minimum value of the data is mapped to 0, the maximum value is mapped to 1, and other values ​​are evenly distributed in proportion in [0,1].

[0074] After normalization, the weight coefficients of each indicator are obtained through the sample goodness scorer, and the sample scoring function is executed to obtain the final sample goodness score. The specific execution of the sample scoring function can be referred to as follows:

[0075] in, Represents the weight coefficient of each indicator, represents the normalized information gain score; represents the normalized response entropy; Represents the normalized word length factor; Indicates the normalized content coverage; Indicates the The sample goodness score of the generated samples.

[0076] Specifically, the weight coefficients can be adjusted by pre-manual configuration or other methods, and other methods can be data-driven weight adjustment. In the general instruction fine-tuning scenario, the optimal combination of weight coefficients is determined by grid search or Bayesian optimization method. The optimal combination refers to a set of weight coefficients found in the general instruction fine-tuning scenario, which can make the sample quality score after multi-strategy fusion most in line with actual needs. For example, the optimal combination of weight coefficients can refer to the following values:

[0077] It is not difficult to find that in the embodiment of the present application, by introducing artificial rules to filter the samples for the second time to see whether the rigid standards meet the requirements, key errors can be avoided from being missed, and external model supplements, such as BERT, can capture more complex features; in addition, by adjusting the weight coefficients through manual configuration or other means, it can be applied to different fields, and the weights can also intuitively reflect the priority, directly reflect the focus of instruction fine-tuning, and meet the needs of later debugging.

[0078] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0079] Fifth embodiment Some embodiments of the present application further provide an electronic device. The electronic device may be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and the like. The electronic device may also be various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0080] The electronic device includes: one or more processors; and a memory storing computer program instructions, wherein the computer program instructions, when executed, enable the processor to perform the steps of the method provided in any one or more of the above embodiments. Figure 2 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes: one or more processors 1101, a memory 1102, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, with each device providing some of the necessary operations. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0081] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, the memory 1102, the input device 1103 and the output device 1104 may be connected via a bus or other means, with the bus connection being used as an example in the figure.

[0082] Input device 1103 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. Examples include a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, and other input devices. Output device 1104 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibration motors). The display device may include, but is not limited to, a liquid crystal display, a light emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0083] To provide user interaction, the electronic device may be a computer. The computer includes a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse) through which the user can provide input to the computer. Other types of devices may also be used to provide user interaction; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback), and input from the user may be received in any form (e.g., voice input or tactile input).

[0084] In the embodiments of the present application, a computer program / instruction is stored on a computer-readable medium. When executed by a processor, the computer program / instruction implements the steps of the method provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments, or it may exist independently and not be incorporated into the device. The computer-readable medium carries one or more computer-readable instructions.

[0085] The memory 1102 can be used as a non-transitory computer-readable storage medium to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor 1101 executes the non-transitory software programs, instructions, and modules stored in the memory 1102 to execute various functional applications and data processing of the server, thereby implementing the program instructions / modules corresponding to the method provided in any one or more of the above embodiments of the present application.

[0086] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely located relative to the processor 1101, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. Computer-readable media may be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0088] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, compact discs, digital versatile discs or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0089] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] In the above embodiments, all or part of the steps or functions of the present invention may be implemented using software, hardware, firmware, or any combination thereof. For example, implementation may be achieved using a dedicated integrated circuit, a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application may be executed by a processor to implement the above steps or functions. Similarly, the software program of the present application (including related data structures) may be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, a floppy disk, or the like. In addition, some steps or functions of the present application may be implemented using hardware, for example, as a circuit that cooperates with a processor to perform the various steps or functions.

[0091] The computer program product provided in the embodiments of the present application includes one or more computer programs / instructions that, when executed by a processor, fully or partially produce the processes or functions described in accordance with the embodiments of the present application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0092] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-specific system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The scope of this application is defined by the appended claims rather than the foregoing description and is therefore intended to encompass within this application all changes that come within the meaning and range of equivalents of the claims. Any reference signs in the claims should not be construed as limiting the claims to which they relate. In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim may also be implemented by one unit or device through software or hardware. Words such as "first" and "second" are only used to distinguish the description and do not indicate any particular order, nor should they be understood as indicating or implying relative importance.

[0094] The above descriptions are merely specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art may easily propose variations or substitutions within the technical scope disclosed in the present application, and such variations or substitutions shall be encompassed within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims, and the above descriptions shall be regarded as exemplary and non-limiting.

Claims

1. A method for screening instruction data, characterized in that: The method comprises: Determine a first perplexity and a second perplexity based on a pre-trained lightweight universal language model and a target sample set; the first perplexity is used to characterize the prediction perplexity under an instruction condition; the second perplexity is used to characterize the prediction perplexity under a no-instruction condition; Determining an information gain score based on the first perplexity and the second perplexity; the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating the response content; A training data set is determined based on the information gain score and the target sample set; the training data set is used to improve the generalization ability and instruction following accuracy of the large language model during the instruction fine-tuning stage.

2. The method according to claim 1, characterized in that The target sample set includes the instruction content, input content and output content of each sample; The first perplexity is specifically used to represent the degree of prediction uncertainty of the response content generated by the lightweight universal language model under the joint constraints of the instruction content and the input content; The second perplexity is used to characterize the degree of prediction uncertainty of the response content generated by the lightweight universal language model under the constraints of the input content alone and when the instruction content is missing.

3. The method according to claim 1, characterized in that Determining an information gain score according to the first perplexity and the second perplexity includes: Calculate the difference between the first perplexity and the second perplexity of each sample in the target sample set to obtain an information gain score; or, Perform negative logarithmic mean operations on the first perplexity and the second perplexity to obtain the corresponding first conditional entropy and second conditional entropy, and calculate the difference between the second conditional entropy and the first conditional entropy of each sample to obtain the information gain score; The first conditional entropy is used to characterize the average prediction uncertainty of the response content generated under the condition of instructions; the second conditional entropy is used to characterize the average prediction uncertainty of the response content generated under the condition of no instructions.

4. The method according to claim 1, wherein The determining of the training data set according to the information gain score and the target sample set includes: Obtaining a goodness score for each sample based on the target sample set and the information gain score; the goodness score is used to reflect the quality of the sample, and the higher the sample quality, the higher the goodness score; The training data set is determined based on the goodness score of each sample.

5. The method according to claim 4, characterized in that Obtaining a goodness score for each sample based on the target sample set and the information gain score includes: Calculate the response entropy, word length factor, and content coverage of each sample in the target sample set; Obtaining an information gain score, and combining the response entropy, word length factor, and content coverage to form a factor set; the factor set is used to characterize a collection of indicators for evaluating sample quality; Sample scoring is performed based on the factor set to obtain a goodness score for each sample.

6. The method according to claim 4, characterized in that Determining the training data set based on the goodness score of each sample includes: Obtain the goodness score of each sample, arrange the samples according to the descending rule, output the samples that are greater than the set threshold range, and obtain the training data set.

7. The method according to claim 5, characterized in that The step of performing sample scoring according to the factor set to obtain a goodness score for each sample includes: Introducing artificial rules or external models to optimize sample scoring, and / or, Obtain the weight of each indicator in the factor set, and obtain the goodness score of each sample based on the normalized data set. The calculation expression of the goodness score is: ； in, Represents the weight coefficient of each indicator, represents the normalized information gain score; represents the normalized response entropy; Represents the normalized word length factor; Indicates the normalized content coverage; Indicates the The sample goodness score of the generated samples.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images