A method, device, medium and product for screening instruction data

Abstract

The application relates to the technical field of information technology and discloses a kind of screening method, equipment, medium and product of instruction data.The method comprises the following steps: determining first perplexity and second perplexity according to a pre-trained lightweight general language model and a target sample set;The first perplexity is used to represent the predicted perplexity under the instruction condition;The second perplexity is used to represent the predicted perplexity under the non-instruction condition;According to the first perplexity and the second perplexity, determine the information gain score;Information gain score is used to quantify the contribution of instruction content to reduce the difficulty of response content generation;According to the information gain score and the target sample set, determine the training data set;The training data set is used to improve the generalization ability and instruction compliance accuracy of large language model in the instruction fine-tuning stage.The technical problems of high deployment threshold, high cost, supervision dependence, low training efficiency and poor model generalization ability can be solved.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method, device, medium and product for filtering instruction data. Background Technology

[0002] Currently, in order to improve the ability of large-scale language models to respond to human commands, command fine-tuning has become the mainstream paradigm. Command fine-tuning is used after model pre-training, using a small amount of high-quality data to adjust the model behavior so that its output is more in line with human needs.

[0003] As the model size increases, the amount of training data involved in instruction fine-tuning explodes. This data contains a large amount of redundant, ambiguous, or even misleading information. Currently, the mainstream training data screening solutions in the industry include strong model evaluation, training a dedicated scorer, or a reward model.

[0004] However, the inventors discovered that the related technologies have at least the following technical problems: the increase in model size leads to increased computational demands, which in turn increases the demand for powerful hardware, resulting in low data filtering efficiency and poor generalization ability of data filtering methods. Summary of the Invention

[0005] One objective of this application is to provide a method for filtering instruction data, which at least addresses the current problems of low data filtering efficiency and poor generalization ability of data filtering methods.

[0006] To achieve the above objectives, some embodiments of this application provide the following aspects:

[0007] In a first aspect, some embodiments of this application provide a method for filtering instruction data, the method comprising: determining a first perplexity and a second perplexity based on a pre-trained lightweight general-purpose language model and a target sample set; the first perplexity being used to characterize the predicted perplexity under the condition of having an instruction; the second perplexity being used to characterize the predicted perplexity under the condition of not having an instruction; determining an information gain score based on the first perplexity and the second perplexity; the information gain score being used to quantify the contribution of instruction content to reducing the difficulty of generating response content; determining a training dataset based on the information gain score and the target sample set; the training dataset being used to improve the generalization ability and instruction compliance accuracy of a large language model during the instruction fine-tuning stage.

[0008] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method described above.

[0009] Thirdly, some embodiments of this application also provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method described above.

[0010] Fourthly, some embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0011] Compared with related technologies, the solution provided in this application uses a pre-trained lightweight general-purpose language model for modeling, which avoids the use of large-scale models and reduces the deployment threshold. Based on the lightweight general-purpose language model and the pre-processed target sample set, a first perplexity and a second perplexity are determined. Thus, the predicted perplexity of the model under conditions with and without instructions is determined to measure the overall quality of the model's generation. Then, an information gain score is calculated by comparing the perplexity under the two conditions with and without instructions. This calculation step reflects the contribution of instructions in the target sample set to the generated response content. Finally, the training dataset is determined based on the information gain score and the target sample set. The entire calculation process is based on inference performed by the pre-trained model, without any fine-tuning or manual annotation. Thus, by providing a lightweight model-based instruction data filtering method, not only can training efficiency be improved, but also the filtering of hundreds of thousands of samples can be completed on a single consumer-grade graphics card, which is faster than traditional methods. Moreover, by using a small number of instruction data samples, it can achieve the performance of a model pre-trained with a large amount of unfiltered data, with strong generalization ability and flexible adaptation to various instruction data formats such as Alpaca, WizardLM, and ShareGPT. In addition, it can be widely used for training and adjustment in vertical fields such as finance, healthcare, and law. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 An exemplary flowchart of an instruction data filtering method provided for some embodiments of this application;

[0014] Figure 2 This is a schematic diagram of the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] The following terms are used in this document.

[0017] In natural language processing and language modeling, a token represents the basic unit when the model processes text; it is usually a word after the text has been segmented.

[0018] PPL, or Perplexity, measures the overall quality of model generation; the lower the PPL value, the higher the model's prediction accuracy and the stronger its generation ability.

[0019] GPU, Graphics Processing Unit, is used to handle graphics rendering-related tasks and has highly parallel computing capabilities.

[0020] First Embodiment

[0021] The first embodiment of this application relates to a method for filtering instruction data. For example... Figure 1 As shown, the method may include the following steps:

[0022] Step S10: Determine a first perplexity and a second perplexity based on the pre-trained lightweight general-purpose language model and the target sample set; the first perplexity is used to characterize the predicted perplexity under the condition of having instructions; the second perplexity is used to characterize the predicted perplexity under the condition of not having instructions.

[0023] Step S20: Determine the information gain score based on the first perplexity and the second perplexity; the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating the response content.

[0024] Step S30: Determine the training dataset based on the information gain score and the target sample set; the training dataset is used to improve the generalization ability and instruction following accuracy of the large language model in the instruction fine-tuning stage.

[0025] First, it's important to clarify that instruction data is a standardized data format specifically designed to tune the behavior of language models, used to train the model's ability to understand and respond to human instructions. The format of instruction data is generally a triple (Instruction, Input, Response). Taking the instruction "Translate the following text into English" as an example, Instruction represents the instruction, which can be a clear description of the task in natural language; Input represents the input, which can be a sentence or paragraph from the text; and Response represents the output, used to present the ideal answer that meets the requirements of the instruction.

[0026] The following sections will provide a detailed explanation of each of the above steps.

[0027] Specifically, for step S10, a lightweight general-purpose language model can use a small to medium-sized model with 0.5B-1.0B parameters, such as Qwen2.5-0.5B.

[0028] Specifically, the target sample set is a collection obtained by processing the original instruction dataset. The original instruction dataset may contain relevant training data for adjustments to the vertical domain. A data preprocessor is used to standardize the structure of the training data, remove invalid samples containing illegal or missing fields, and standardize the samples in the original instruction dataset into a triplet format to ensure that each sample has a standardized structure, thus obtaining the target sample set. Illegal fields can be garbled characters or special symbols, and missing fields can be no instruction or an empty response.

[0029] Specifically, a pre-trained lightweight general-purpose language model is used to input each sample in the preprocessed dataset under both instruction-based and instructionless conditions. The response generation process is simulated through a lightweight model analyzer to evaluate the response sequence and obtain the response probabilities under instruction-based and instructionless conditions. Then, the perplexity under instruction-based and instructionless conditions is calculated based on the response probabilities. The predicted perplexity under instruction-based conditions is the first perplexity, and the predicted perplexity under instructionless conditions is the second perplexity.

[0030] Specifically, in step S20, the information gain score is used to quantify the contribution of instruction content to reducing the difficulty of generating response content. By converting the difference between the first and second perplexity into a quantifiable instruction contribution index, it can be used to screen high-value data.

[0031] For example, in a text translation task, the samples in the target sample set contain the following:

[0032] {

[0033] "instruction": "Translate the following English text into Chinese",

[0034] "input": "The rapid development of AI technology requires robustethical guidelines.",

[0035] Output: "The rapid development of artificial intelligence technology requires strong ethical guidelines."

[0036] }

[0037] Instructions are used to specify the task type. Without instructions, the model might continue writing in English or translate into other languages, leading to an increase in the second perplexity. This, in turn, increases the difference between the second and first perplexities, i.e., an increase in the information gain score. A higher information gain score indicates a greater contribution of the instructions to the quality of the pre-trained lightweight general-purpose language model. Therefore, the information gain score serves as a quantitative indicator reflecting the effectiveness of the instructions.

[0038] Specifically, for step S30, information gain score is one dimension for evaluating sample quality. Other evaluation dimensions need to be determined based on the target sample set. After determining the indicators of all evaluation dimensions, the training dataset is determined by combining the indicators for fine-tuning of instructions.

[0039] It's understandable that in related technologies, training and inference of large-scale models requires high-end GPU clusters, resulting in extremely high costs. Furthermore, as the number of parameters increases, the model needs more instruction data to achieve stable performance, and data training efficiency decreases with increasing parameters. Large models also tend to remember the details of training data, leading to a decline in generalization ability when faced with new instructions. In contrast, small- to medium-scale models, with fewer parameters and lower memory usage, can run on consumer-grade GPUs or mobile devices, significantly lowering the hardware barrier. Each instruction data has a more significant impact on the model, resulting in higher training efficiency. Therefore, using lightweight small- to medium-scale general-purpose language models, coupled with high-quality instruction data for training, offers better cost-effectiveness.

[0040] It is not difficult to see that, compared with related technologies, the solution provided in this application, which is based on a pre-trained lightweight general-purpose language model for modeling, can avoid using large-scale models to reduce the deployment threshold. Based on the lightweight general-purpose language model and the pre-processed target sample set, the first perplexity and the second perplexity are determined. Thus, the predicted perplexity of the model under conditions with and without instructions is determined to measure the overall quality of the model's generation. Then, the information gain score is calculated by comparing the perplexity under the two conditions with and without instructions. This calculation step reflects the contribution of instructions in the target sample set to the generated response content. Finally, the training dataset is determined based on the information gain score and the target sample set. The entire calculation process is based on inference performed by the pre-trained model, without any fine-tuning or manual annotation. Thus, by providing a lightweight model-based instruction data filtering method, we can not only improve training efficiency and complete the filtering of hundreds of thousands of samples on a single consumer-grade graphics card, which is faster than traditional methods, but also achieve the performance of a model pre-trained with a large amount of unfiltered data by using a small number of instruction data samples. It has strong generalization ability and can flexibly adapt to various instruction data formats such as Alpaca, WizardLM, and ShareGPT. In addition, it can be widely used for training and adjustment in vertical fields such as finance, healthcare, and law.

[0041] Second Embodiment

[0042] The second embodiment of this application relates to the filtering of instruction data. The second embodiment is an improvement on the first embodiment, specifically in that: in this embodiment, a method for calculating the first perplexity and the second perplexity based on the target sample set, as well as a method for calculating the information gain score, are provided.

[0043] Optionally, in some embodiments, the target sample set includes the instruction content, input content, and output content of each sample;

[0044] The first perplexity is specifically used to characterize the degree of uncertainty in the prediction of the response content generated by the lightweight general-purpose language model under the joint constraints of instruction content and input content;

[0045] The second perplexity is used to characterize the degree of uncertainty in the prediction of the response content generated by the lightweight general-purpose language model when the input content is constrained alone and the instruction content is missing.

[0046] Specifically, the target sample set can provide explicit learning signals to the lightweight general-purpose language model through triples in the format of (Instruction, Input, Response). This instruction constraint, which ensures precise instructions, necessary inputs, and strict output compliance, helps to obtain high-quality samples.

[0047] For example, with instructions, the input format can include both the instructions and the input text. The lightweight general-purpose language model generates a response based on the complete instructions and context, calculating the output perplexity as the first perplexity. Without instructions, the input format only contains the input text, and the lightweight general-purpose language model generates a response based solely on the perplexity of the input text, using this as the second perplexity. It can be seen that if the first perplexity is less than the second perplexity, it indicates that the instruction significantly reduces the generation difficulty for the lightweight general-purpose language model.

[0048] Specifically, the methods for calculating the first and second perplexity are as follows:

[0049] Using a pre-trained lightweight general-purpose language model, each sample in the dataset is input under both instruction-based and instruction-free conditions. The response generation process is simulated through a lightweight model analyzer to evaluate the response sequence and obtain the response probability. Then, the perplexity is calculated based on the response probability.

[0050] The method for calculating the first degree of perplexity:

[0051]

[0052] The method for calculating the second degree of perplexity:

[0053]

[0054] in, Indicates the first in the data One sample; Indicates the first Input for each sample; Indicates the first The response sequence output by the model after each sample input; Indicates the first The first sample The token, i.e., the first token in the response sequence. One word; Indicates the first The first sample All tokens preceding this token; Indicates the first The length of the response sequence for each sample, i.e., the number of tokens; Representing the response probability, it is based on a pre-trained lightweight general-purpose language model. Or the probability of outputting the next token from the previous token.

[0055] Specifically, Used to distinguish different samples, for example, if the target sample set contains 100 samples. It can represent the first instruction in the dataset and its corresponding response. It can be contained as instructions and / or context. Response sequence It can be a text sequence generated by the model, such as the answer generated by the model after inputting a command. When the original sentence is "Artificial intelligence is very good.", after a word segmentation strategy, it yields 5 tokens, with the token sequence being ["artificial", "intelligent", "very", "good", "."]. The second token is "intelligent," and the model outputs tokens one by one when generating the sequence, first outputting "human," then "intelligent." The number of tokens is not the same as the number of words or phrases; the number of tokens depends on the specific word segmentation strategy, and different strategies yield different numbers of tokens. In instructionless scenarios... This indicates that the model predicts the probability of the next token based solely on historical tokens; in scenarios with instructions, the model also combines... The probability of predicting the next token based on the generated tokens.

[0056] Optionally, in some embodiments, determining the information gain score based on the first perplexity and the second perplexity, i.e., step S20, includes:

[0057] Calculate the difference between the first and second perplexity of each sample in the target sample set to obtain the information gain score;

[0058] or,

[0059] The first and second perplexities are respectively calculated by performing negative logarithmic mean to obtain the corresponding first and second conditional entropies. The difference between the second and first conditional entropies of each sample is calculated to obtain the information gain score.

[0060] The first conditional entropy is used to characterize the average prediction uncertainty of generating response content under instruction conditions; the second conditional entropy is used to characterize the average prediction uncertainty of generating response content under no instruction conditions.

[0061] Specifically, the information gain score is calculated by comparing the difference in perplexity under conditions with and without instructions using an information gain calculator. The calculation method is as follows:

[0062]

[0063] in, Indicates the first in the data One sample; Indicates the first Input for each sample; Indicates the first The response sequence output by the model after each sample input; This represents the perplexity under no-instruction conditions, i.e., the second perplexity, which characterizes the uncertainty of generating response sequences under no-instruction conditions. This represents the uncertainty in generating a response sequence under given instructions, i.e., the first degree of perplexity. This represents the information gain score, i.e., the information gain brought about by the instruction. When the value is greater than 0, the larger the value, the more effective the instruction.

[0064] or,

[0065] Specifically, the information gain score is calculated by comparing the entropy under two conditions: with and without instructions, using an information gain calculator. The calculation method is as follows:

[0066]

[0067] in,

[0068]

[0069]

[0070] in, Indicates the first in the data One sample; Indicates the first Input for each sample; Indicates the first The response sequence output by the model after each sample input; Indicates the first The first sample The token, i.e., the first token in the response sequence. One word; Indicates the first The first sample All tokens preceding this token; Indicates the first The length of the response sequence for each sample, i.e., the number of tokens; Representing the response probability, it is based on a pre-trained lightweight general-purpose language model. Or the probability of outputting the next token from the previous token; This represents the entropy when there is an instruction, i.e., the first conditional entropy. This represents the entropy without instructions, i.e., the second conditional entropy.

[0071] Specifically, entropy is the average uncertainty of each token, while perplexity is its exponential form; the definitions of entropy and perplexity are equivalent. Therefore, information gain scores can be obtained by calculating the difference in perplexity or the difference in entropy.

[0072] It is not difficult to see that in the embodiments of this application, by comparing the difference in perplexity between the two scenarios under the conditions of having instructions and not having instructions, the contribution of instructions to the model generation quality is directly quantified, avoiding subjective evaluation bias, and can identify and retain high information gain samples while filtering inefficient data. Furthermore, since the model trained on high information gain samples can better understand the essence of instructions, it can not only reduce computing power consumption but also enhance the model's ability to understand the essence of instructions.

[0073] Third Embodiment

[0074] The third embodiment of this application relates to a method for determining a training dataset. The third embodiment is an improvement upon the first embodiment, specifically in that it provides a method for determining a factor set and a training dataset.

[0075] Optionally, in some embodiments, step S30, determining the training dataset based on the information gain score and the target sample set, includes:

[0076] Step S301: Based on the target sample set and the information gain score, obtain the goodness score for each sample; the goodness score is used to reflect the quality of the sample, and the higher the sample quality, the higher the goodness score.

[0077] Step S302: Determine the training dataset based on the goodness score of each sample.

[0078] Specifically, in step S301, calculating the goodness score is used to assess the quality of each sample in the target sample set. Generating the goodness score quantifies the training value of each sample. The sample goodness scorer supports weighting, normalization, and dynamic threshold adjustment, integrating information gain scores and other quantitative indicators to construct a unified sample value index. All goodness scores given by these indicators fall within the range [0,1]. For example, other quantitative indicators may include, but are not limited to, response entropy, word length factor, and content coverage.

[0079] Specifically, in step S302, based on the goodness-of-performance scores, a refined sample generator is used to select samples with high goodness-of-performance scores, constructing the final training dataset for instruction fine-tuning. The refined sample generator refers to a tool or process that, during the construction of the instruction fine-tuning training dataset, filters, processes, and optimizes the original samples based on their goodness-of-performance scores, ultimately producing high-quality training data. For example, the generated goodness-of-performance scores can be filtered: scores > 0.8 are considered high goodness-of-performance, scores < 0.5 are considered low goodness-of-performance, and the remainder are considered medium goodness-of-performance. High-goodness samples are retained, while medium-goodness samples undergo manual review, correcting any ambiguous instructions or erroneous outputs before retaining them. The filtered samples constitute the training dataset. This training dataset, through high-quality samples, accurately adjusts the model's behavior, covering diverse and effective instruction-output pairs, enhancing generalization, reducing invalid computations of low-value samples, and optimizing training efficiency.

[0080] Optionally, in some embodiments, the step S301, which involves obtaining the goodness score for each sample based on the target sample set and the information gain score, includes:

[0081] Step S3011: Calculate the response entropy, word length factor, and content coverage of each sample in the target sample set;

[0082] Step S3012: Obtain the information gain score, and combine it with the response entropy, word length factor, and content coverage to form a factor set; the factor set is a collection of indicators used to characterize the quality of the evaluation sample.

[0083] Step S3013: Perform sample scoring based on the factor set to obtain the goodness score for each sample.

[0084] Specifically, regarding step S3011, the response entropy... Used to measure the presence of instructions Generate response sequence under the condition The uncertainty or diversity reflects the uniformity of the probability distribution predicted by the model for each token. The smaller the value, the higher the certainty of the model's prediction.

[0085] For specific methods on calculating response entropy, please refer to the following:

[0086]

[0087] in, Indicates the first in the data One sample; Indicates the first Input for each sample; Indicates the first The response sequence output by the model after each sample input; Indicates the first The first sample The token, i.e., the first token in the response sequence. One word; Indicates the first The first sample All tokens preceding this token; Indicates the first The length of the response sequence for each sample, i.e., the number of tokens; Indicates the first The response entropy of a sample; The vocabulary representing the model contains all possible sets of tokens; Vocabulary Any Token in the list; The representation model is based on instructions. and generated sequences Predicting the next token to become The probability of.

[0088] It's worth noting that both perplexity and response entropy are metrics used to evaluate the quality of language model generation, but their calculation methods and application scenarios differ significantly. Specifically, perplexity primarily measures the difficulty of the model predicting real sequences, calculating only the response probability of the real token. The smaller the perplexity value, the more accurate the model; the ideal perplexity value is 1. Response entropy primarily measures the uncertainty or diversity of a model's predictions. It requires calculating the probability distribution of all tokens in the vocabulary. A smaller response entropy value reflects higher model certainty and a lack of diversity, while a larger response entropy value fails to reflect model certainty. In such cases, the specific task must be considered; for example, the response entropy value is very low in fact-finding questions, while creative text may require a higher response entropy value.

[0089] The method for calculating the word length factor can be found as follows:

[0090]

[0091] in, Indicates the first in the data One sample; Represents the response sequence The length of the token, i.e., the number of tokens; This represents the preset average ideal response length; Indicates length tolerance. Indicates the first Word length factor for each sample.

[0092] Specifically, This is used to control the allowable deviation range between the length of the model's output response sequence and the preset ideal response length. It can be set manually, and the larger the set value, the wider the allowable deviation range. The value range is (0,1], and it is used to penalize generated results that deviate from the preset average ideal response length. The greater the difference between the actual length and the target, the stronger the penalty. The penalty logic is as follows:

[0093] When the response sequence length Deviation from the preset ideal average response length hour, As the value decreases and approaches zero, the score decreases.

[0094] When the response sequence length = hour, =1, at which point the ideal state is reached, and there is no penalty.

[0095] The following are methods for calculating content coverage:

[0096]

[0097] in, Indicates the first in the data One sample; This represents a predefined set of keywords; Represents a set The total number of keywords contained therein; Indicates the first The response sequence output by the model after each sample input; Represents the unique set of tokens in the response sequence; Indicates the first Content coverage of each sample.

[0098] Specifically, The value range is (0,1], used to measure the response sequence. The proportion of unique keywords appearing in a given keyword out of all keywords reflects the relevance and completeness of the produced content. The closer the value is to 1, the better the response sequence. The more domain keywords it contains. This can typically be determined through either manual setting or automated extraction, depending on the task requirements and data characteristics. For example, in the medical field... , . This refers to the set obtained by removing duplicate tokens from the token sequence of the generated text. Each token is kept only once, and duplicate tokens are not counted. Deduplication can avoid interference from duplicate tokens in the calculation, prevent the model from artificially increasing the score by repeating keywords, and the unique token can more realistically reflect the diversity of the response content, ensuring that the contribution of each keyword is equal.

[0099] Specifically, for step S3012, the information gain score is integrated with the indicators from step S3011 to form a multi-dimensional quality assessment system, namely, a factor set. The factor set overcomes the limitations of a single indicator, comprehensively quantifies sample quality from multiple dimensions, and more accurately identifies high-quality samples.

[0100] Specifically, step S3013 involves sample scoring, which is a process of numerically evaluating the quality of generated samples by constructing a unified sample value index through the comprehensive multi-factor scores. The scoring relies on a set of factors; by integrating evaluation indicators from the factor set, such as information gain, response entropy, and word length factors, into a unified score, it facilitates sorting, filtering, or optimization of the generation model.

[0101] Optionally, in some embodiments, the training dataset is determined based on the goodness score of each sample. Step S302 includes:

[0102] Obtain the goodness score for each sample, sort the samples according to the descending order rule, and output the samples that are greater than the set threshold range to obtain the training dataset.

[0103] Specifically, the goodness scores of all samples are first obtained. Then, using a refined sample generator, the samples are sorted in descending order of goodness scores to ensure high-quality samples are prioritized. Samples with a set threshold greater than the top k% are selected as the final fine-tuning training set. For example, k% can range from 5% to 10%, but this needs to be fine-tuned based on the business scenario. Finally, a high-quality training dataset is output.

[0104] It should be noted that this embodiment can also be an improvement based on the second embodiment.

[0105] It is easy to see that the embodiments of this application provide a method for determining the excellence score. This method calculates the response entropy, word length factor, and content coverage of each sample in the target sample set. These three indicators, along with the information gain score, form a factor set. For this factor set, a excellence score is obtained for each sample according to a preset sample scoring rule. The excellence scores are then sorted in descending order, and samples with scores exceeding a set threshold are output as the training dataset. In this way, high-quality sample selection can be completed without supervision or manual intervention, significantly improving training efficiency while ensuring model performance.

[0106] Fourth embodiment

[0107] The fourth embodiment of this application relates to a method for generating fog of war. This fourth embodiment is an improvement upon the first, second, and third embodiments. Specifically, the improvement lies in providing a method for performing sample scoring.

[0108] Optionally, in some embodiments, step S3013, which involves performing sample scoring based on the factor set to obtain a goodness score for each sample, includes:

[0109] Step S30131: Introduce manual rules or external models to optimize sample scores.

[0110] And / or,

[0111] Step S30132: Obtain the weights of each indicator in the factor set, and obtain the goodness score of each sample based on the normalized data set.

[0112] Specifically, for step S30131, a multi-strategy fusion tool can be invoked to introduce manual rules or external models (such as BERT-based quality discrimination models, information entropy-based diversity scoring models, etc.) to improve the robustness of the screening. For example, each participant (manual rules, the aforementioned models) can first provide opinions or scores, and then a weighted voting strategy can be used to sum the results of each party according to preset weights to obtain a comprehensive goodness score. Since the comprehensive score changes compared to the initial score, the relative goodness relationships between samples may change. Therefore, the goodness score ranking of the samples can be updated based on this comprehensive score, ultimately improving the accuracy of the goodness score.

[0113] Regarding step S30132, specifically, before scoring the goodness of each sample, it is necessary to normalize the indicators in the factor set. The specific calculation method can be found below:

[0114]

[0115]

[0116] in, Indicates the first in the data One sample; Indicating in the candidate pool The maximum value; Indicating in the candidate pool The minimum value; Represents the first in the original data Individual indicator values; Information gain score Word length factor and content coverage The result after normalization Represents response entropy The result after normalization.

[0117] It should be noted that, due to differences in the units and numerical ranges of different indicators, there may be situations where larger numerical indicators suppress smaller numerical indicators. Therefore, the purpose of normalization is to uniformly scale the values ​​of different indicators to the same range. After normalization, all indicators are in the same interval.

[0118] Specifically, The value range is [0,1]. According to the minimum-maximum linear scaling rule, the original data is linearly mapped to a fixed range proportionally. The minimum value of the data is mapped to 0, the maximum value is mapped to 1, and other values ​​are evenly distributed in [0,1].

[0119] After normalization, the weight coefficients of each indicator are obtained through the sample goodness-of-fact scorer. The sample scoring function is then executed to obtain the final sample goodness-of-fact score. The specific execution of the sample scoring function can be found below:

[0120]

[0121] in, , , , , This represents the weighting coefficient of each indicator. This represents the normalized information gain score. This represents the normalized response entropy; This represents the normalized word length factor; This indicates the normalized content coverage. Indicates the first The sample goodness score of each generated sample.

[0122] Specifically, the weight coefficients can be adjusted manually in advance or through other methods, such as data-driven weight adjustment. In the scenario of fine-tuning general instructions, the optimal combination of weight coefficients is determined through grid search or Bayesian optimization. The optimal combination refers to a set of weight coefficients found in the scenario of fine-tuning general instructions, which makes the sample goodness score after multi-strategy fusion best match the actual needs. For example, the optimal combination of weight coefficients can refer to the following values:

[0123]

[0124] It is not difficult to see that in the embodiments of this application, the hard standard of whether the samples meet the requirements is filtered by the introduction of manual rules to avoid missing key errors. External models, such as BERT, can capture more complex features. In addition, the weight coefficients can be adjusted by manual configuration or other means to be applicable to different fields. The weights can also intuitively reflect the priority and directly reflect the focus of fine-tuning of instructions, meeting the needs of later debugging.

[0125] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0126] Fifth Embodiment

[0127] Some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0128] The electronic device includes: one or more processors; and a memory storing computer program instructions that, when executed, cause the processor to perform the steps of the methods provided in any one or more of the above embodiments. Figure 2 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0129] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103 and output device 1104 may be connected by a bus or other means, as shown in the figure, which is connected by a bus.

[0130] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0131] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0132] In this embodiment, a computer-readable medium stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in any one or more of the above embodiments. This computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The aforementioned computer-readable medium carries one or more computer-readable instructions.

[0133] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0134] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0135] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0136] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0137] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0139] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0140] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0141] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0142] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for filtering instruction data, characterized in that, The method includes: Based on a pre-trained lightweight general-purpose language model and a target sample set, a first perplexity and a second perplexity are determined. The first perplexity is used to characterize the predicted perplexity under the condition of having instructions, where the input format includes instructions and input text. The second perplexity is used to characterize the predicted perplexity under the condition of not having instructions, where the input format only contains input text. An information gain score is determined based on the first perplexity and the second perplexity; the information gain score is used to quantify the contribution of the instruction content to reducing the difficulty of generating response content. The training dataset is determined based on the information gain score and the target sample set; the training dataset is used to improve the generalization ability and instruction following accuracy of the large language model in the instruction fine-tuning stage. The step of determining the training dataset based on the information gain score and the target sample set includes: Based on the target sample set and the information gain score, the goodness score of each sample is obtained; based on the goodness score of each sample, the training dataset is determined. The process of obtaining the goodness score for each sample based on the target sample set and the information gain score includes: Calculate the response entropy, word length factor, and content coverage for each sample in the target sample set; where response entropy is used to measure the uncertainty or diversity of generating response sequences under instruction conditions, word length factor is used to penalize the generated results that deviate from the preset ideal response length average, and content coverage is the proportion of the number of unique keywords in the response sequence to all keywords. The method for calculating the word length factor is as follows: ； in, Indicates the first in the data One sample; Represents the response sequence The length of the token, i.e., the number of tokens; This represents the preset average ideal response length; This represents the length tolerance, used to control the allowable deviation range between the length of the model's output response sequence and the preset ideal response length; Indicates the first Word length factor for each sample; Obtain the information gain score, and combine it with the response entropy, word length factor, and content coverage to form a factor set; the factor set is used as a set of indicators to characterize the quality of the evaluation sample; The sample scoring is performed based on the set of factors to obtain a goodness score for each sample; The excellence score is used to reflect the quality of the sample; the higher the sample quality, the higher the excellence score.

2. The method according to claim 1, characterized in that, The target sample set includes the instruction content, input content, and output content of each sample; The first perplexity is specifically used to characterize the degree of uncertainty in the prediction of the response content generated by the lightweight general-purpose language model under the joint constraints of instruction content and input content; The second perplexity is used to characterize the degree of uncertainty in the prediction of the response content generated by the lightweight general-purpose language model when the input content is constrained alone and the instruction content is missing.

3. The method according to claim 1, characterized in that, The step of determining the information gain score based on the first perplexity and the second perplexity includes: The information gain score is obtained by calculating the difference between the first and second perplexity of each sample in the target sample set. or, The first and second perplexities are respectively calculated by performing negative logarithmic mean to obtain the corresponding first and second conditional entropies. The difference between the second and first conditional entropies of each sample is calculated to obtain the information gain score. The first conditional entropy is used to characterize the average prediction uncertainty of generating response content under instruction conditions; the second conditional entropy is used to characterize the average prediction uncertainty of generating response content under no instruction conditions.

4. The method according to claim 1, characterized in that, The step of determining the training dataset based on the goodness score of each sample also includes: Obtain the goodness score for each sample, sort the samples according to the descending order rule, and output the samples that are greater than the set threshold range to obtain the training dataset.

5. The method according to claim 1, characterized in that, The step of performing sample scoring based on the factor set to obtain a goodness score for each sample includes: Optimize sample scoring by introducing manual rules or external models. And / or, Obtain the weights of each indicator in the factor set, and based on the normalized dataset, obtain the goodness score for each sample. The formula for calculating the goodness score is as follows: ; in, , , , , This represents the weighting coefficient of each indicator. This represents the normalized information gain score. This represents the normalized response entropy; This represents the normalized word length factor; This indicates the normalized content coverage. Indicates the first The sample goodness score of each generated sample.

6. An electronic device, characterized in that, The electronic device includes: One or more processors; and A memory storing computer program instructions, which, when executed, cause the processor to perform the steps of the method as described in any one of claims 1 to 5.

7. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images