Sample processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a sample processing method and device, equipment, a storage medium and a program product. The method comprises the following steps: determining a group of attention heads from a plurality of attention heads of a basic language model; shielding a group of attention heads in the basic language model to obtain a reference language model; for training samples in a training sample set used for training the target language model, the basic language model and the reference language model are respectively used for processing the training samples to obtain a first processing result from the basic language model and a second processing result from the reference language model; and in response to determining that the difference between the first processing result and the second processing result meets a predetermined condition, classifying the training sample as a key sample for training the target language model. In this way, high-quality and uniformly-distributed key samples can be determined from the training sample set, and the training efficiency of the target language model is improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computer technology, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for processing samples. Background Art

[0002] In recent years, there has been growing interest in collecting inference-intensive pre-training data to improve the complex reasoning capabilities of large language models. Previous approaches typically rely on supervised classifiers to identify such data, which requires manual labeling or leveraging pre-trained language models, often introducing domain-specific biases. Therefore, it is desirable to efficiently and scalably identify diverse, high-quality pre-training data. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for processing a sample is provided. The method includes: determining a set of attention heads from a plurality of attention heads of a base language model; masking the set of attention heads in the base language model to obtain a reference language model; processing a training sample in a training sample set for training a target language model using the base language model and the reference language model, respectively, to obtain a first processing result from the base language model and a second processing result from the reference language model; and, in response to determining that a difference between the first processing result and the second processing result satisfies a predetermined condition, classifying the training sample as a key sample for training the target language model.

[0004] In a second aspect of the present disclosure, a device for processing samples is provided. The device includes: an attention head determination module configured to determine a set of attention heads from a plurality of attention heads of a base language model; a reference language model acquisition module configured to mask a set of attention heads in the base language model to obtain a reference language model; and a key sample classification module configured to process a training sample in a training sample set for training a target language model using the base language model and the reference language model, respectively, to obtain a first processing result from the base language model and a second processing result from the reference language model; and in response to determining that a difference between the first processing result and the second processing result satisfies a predetermined condition, classify the training sample as a key sample for training the target language model.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processing unit, the instructions cause the electronic device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0008] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A schematic block diagram illustrating an architecture for processing samples according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A schematic diagram showing a performance comparison between the solution proposed in the present disclosure and a baseline solution according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A flow chart illustrating a method for processing a sample according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A block diagram illustrating an apparatus for processing a sample according to some embodiments of the present disclosure; and

[0015] Figure 6 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0016] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0018] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0019] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to the electronic device, application, server or storage medium and other software or hardware that performs the operation of the technical solution of the present disclosure based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0023] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0024] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing inputs and providing corresponding outputs. It typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence so that the output of the previous layer is provided as input to the next layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each of which processes the input from the previous layer.

[0025] Generally speaking, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association between input and output (also called input-to-output mapping) from the training data. The parameter values of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. The testing stage can sometimes be integrated into the training stage. In the application or inference stage, the trained model can be used to process the actual model input based on the parameter values obtained through training to determine the corresponding model output.

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In environment 100, an electronic device 110 applies a sample processing model 105 to perform sample processing. The sample processing model 105 is configured to generate key samples 116 based on a training sample set 112, so that the key samples 116 can be used to train a language model 118. In some embodiments, the key samples 116 can include high-quality pre-training data such as academic papers, books, high-quality code and instruction data sets. In some embodiments, the sample processing model 105 can be built based on any machine learning model with an attention head. For example, the sample processing model 105 can be built based on a unidirectional language model, a bidirectional language model, or other models.

[0027] In some embodiments, the determined key samples 116 can be used to train the language model 118. Alternatively and / or additionally, the determined key samples 116 can be used to update the training sample set 112, and the updated training sample set can then be used to train the language model 118. For example, one or more determined key samples 116 can be added to the training sample set 112. For another example, the addition process can be performed once or multiple times for a certain key sample, resulting in multiple repeated training samples being included in the training sample set 112. In this way, the contribution of the key samples to the training process can be enhanced, thereby improving the accuracy of the language model 118.

[0028] In environment 100, electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, personal communication system (PCS) device, personal navigation device, personal digital assistant (PDA), audio / video player, digital camera / camcorder, positioning device, television receiver, radio broadcast receiver, e-book device, gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 can also support any type of interface for the user (such as "wearable" circuit, etc.). The video generation model 105 can be implemented, for example, on various types of computing systems / servers that can provide computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0029] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0030] As mentioned above, current methods typically use supervised classifiers to identify inference-intensive pre-training data. Identifying high-quality pre-training data has always been a key factor in creating language models (LMs). It is generally believed that high-quality pre-training data can include academic papers, books, high-quality code and instruction datasets, etc. Current methods typically rely on manually curated seed data to train classifiers to extract additional high-quality pre-training data from massive web corpora. However, as the scale and diversity of pre-training data requirements for language models continue to grow, these manually curated classifiers face the problems of high manual workload requirements and relatively low diversity of the identified data.

[0031] For example, a related method uses reference information from encyclopedia data (which can be viewed as weakly supervised data) to train a classifier and then identify documents similar to the encyclopedia data. While some other methods aim to adapt to user needs by leveraging user feedback, these methods may introduce potential biases and harm diversity. There are also methods that train classifiers from different fields and combine classifiers from different fields for practical use. However, these methods do not capture the intensive nature of data reasoning, and therefore, they may be labor-intensive and require a lot of effort in data engineering. In addition, there is a risk that the classification results of a small model extracted from the responses of a large model may not improve the performance of the large model.

[0032] In some related work, the quality of pre-training data is evaluated using heuristic filtering rules or the perplexity of existing language models, which do not require training. One filter uses the perplexity difference between two language models trained on the same data to evaluate the quality of text. However, when two language models trained on the same data are not available, the language model needs to be trained at a high computational cost. On the other hand, methods that use high-quality labeled data to train models are needed, which have received more attention due to their higher accuracy and excellent versatility across different data categories. For example, one method uses the responses of the language model to train a classifier, which can be viewed as a form of distillation of a language model that is sparser than knowledge distillation (e.g., up to 70B (billion) dense parameters). In addition, other methods turn to training multi-class classifiers to make fine-grained distinctions between various content types and rely on labeled data generated by proprietary language models.

[0033] Understanding the inner workings of language models is crucial for pushing the boundaries of general artificial intelligence while maintaining its safety, leading to a growing body of work on mechanical interpretability. One related line of work investigates the relationship between some of the heads in a language model and its ability to learn from context. Another related line of work extracts a large number of interpretable features from a single-layer transformer with a sparse autoencoder to infer the behavior of a neural network.

[0034] Influence metrics are used to assess the impact of training data on model predictions. One related work uses influence functions to identify pre-training documents that are important for learning factual knowledge and mathematical reasoning, respectively. Another related work implements efficient data influence estimation to select high-quality data.

[0035] In the process of selecting high-quality pre-training data, in order to solve the above-mentioned problems, in an embodiment of the present disclosure, a method for processing samples is proposed. In summary, a group of attention heads (i.e., key attention heads) in the basic language model that have a greater impact on the performance of the model can be determined, and the above-mentioned group of attention heads can be shielded to generate a reference language model with degraded performance. Two models can be used to process the training samples separately, and then the importance of the training sample can be determined based on the difference between the two processing results. It should be understood that if the difference between the two processing results is large, it indicates that the process of processing the training sample is very dependent on the shielded key attention head. From this, it can be inferred that the training sample is likely to have made a greater contribution to training the key attention head. In other words, the training sample is likely to be a high-quality training sample. In this way, a large number of training samples can be classified into key samples and non-key samples in a simpler and more effective way.

[0036] Specifically, a group of attention heads are determined from multiple attention heads of a base language model; a group of attention heads are shielded in the base language model to obtain a reference language model; and for training samples in a training sample set for training a target language model, the training samples are processed using the base language model and the reference language model respectively to obtain a first processing result from the base language model and a second processing result for the reference language model; and in response to determining that the difference between the first processing result and the second processing result meets a predetermined condition, the training sample is classified as a key sample for training the target language model.

[0037] According to the disclosed solution, without the need for supervisory signals or training, the inherent model behavior, namely the attention head mechanism, can be used to identify key samples from the training samples for training the target language model. In this way, high-quality and evenly distributed key samples are identified from the training sample set, improving the training efficiency of the target language model.

[0038] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings. Figure 2 Schematic diagram of an architecture 200 for processing samples according to some embodiments of the present disclosure is shown. Figure 2 As shown, a set of attention heads 210 can be determined from a plurality of attention heads of the base language model 205. In one example, the set of attention heads 210 is associated with the reasoning and retrieval performance of the base language model 205, and thus the set of attention heads 210 is also referred to as a retrieval head.

[0039] In some embodiments, the basic language model 205 can be any pre-trained model that can be obtained. In one example, the basic language model 205 can be trained on a general task and then fine-tuned according to a specific application scenario (for example, selecting high-quality pre-training data) to adapt to a specific task. In this way, the large amount of computing resources and time required to train the basic language model 205 from scratch is avoided, and the basic language model 205 can achieve better performance after fine-tuning. For example, the basic language model 205 can include a general question-answering model, a question-answering model that focuses on one or more fields, and so on.

[0040] In some embodiments, a performance indicator (also referred to as a retrieval score) of an attention head among the plurality of attention heads may be determined based on a reference sample set, where the performance indicator indicates the performance of the attention head. In one example, the performance indicator may indicate the reasoning and retrieval performance of the attention head. In another example, the performance indicator may indicate the performance of the attention head in correctly answering questions, etc.

[0041] In some embodiments, the reference sample 215 in the reference sample set may include at least a question 216 and an answer 217. In one example, the question 216 and the answer 217 may also be referred to as a context. The context may include multiple pairs of questions 216 and answers 217, and the context may be constructed using a JavaScript Object Notation (JSON) object. In one example, the question 216 in a pair of questions 216 and answers 217 may be a randomly generated string (e.g., consisting of letters and numbers), and the answer 217 may be one or more sentences, or parts of sentences, expressed in natural language and sampled from a corpus of web documents. The basic language model 205 may be used to determine a predicted answer 218 based on the question 216. Then, a performance indicator of the attention head may be determined based on a comparison of the predicted answer 218 and the answer 217. For example, the higher the similarity between the predicted answer 218 and the answer 217, the higher the performance indicator; the lower the similarity between the predicted answer 218 and the answer 217, the lower the performance indicator.

[0042] In some embodiments, reference sample 215 may also include an input prompt word, which may indicate a task to be performed by base language model 205. For example, the input prompt word may instruct base language model 205 to retrieve and output an answer corresponding to the input question from the context. In this way, base language model 205 can better understand the task to be performed and thus provide a more desired answer.

[0043] In some embodiments, the reference sample 215 further includes at least one input-output example 219, and the input-output example 219 may include pairs of questions and answers. Then, the base language model 215 can be utilized to determine a predicted answer 218 based on the question 216 and at least one input-output example 219. In this way, a few-shot learning scenario can be simulated, so that the base language model 205 can better understand the task to be performed under the guidance of the input-output examples.

[0044] In some examples, due to the limited context length of the base language model 215, the total length of the reference sample 215 (e.g., which may include input prompts and the predicted answer 218) can be limited to be close to but not exceed 4096 tokens. Alternatively and / or additionally, based on the performance of the electronic device that invokes the base language model, the total length can be set to other lengths.

[0045] In some embodiments, a target token can be determined from multiple tokens of the predicted answer 218, the target token appears in the answer 217, and the attention score obtained by the target token in the attention head can satisfy a predetermined score condition. The score condition can, for example, include the highest score, the second-highest score, or the top N scores ranked, and so on. In one example, the current token generated by the base language model 205 at the decoding step t can be represented as w, and the attention score of the attention head can be represented as where x represents the vocabulary and |x| represents the size of the vocabulary. If the following condition 1 and condition 2 are satisfied, the attention head will perform a "copy and paste" operation from the answer 217. Condition 1 can be that the generated token w (e.g., the token generated in the predicted answer 218) appears in the answer 217 (represented as k), and condition 1 can be represented as follows:

[0046] w ∈ k (1)

[0047] Condition 2 can be that the generated token w obtains the highest attention score at all positions visible to the current query token of the attention head, and condition 2 can be represented as follows:

[0048] j ∈ i q , where i q = {j|j < t} is the set of positions visible at the decoding step t (2)

[0049] j = argmax(a),x j = w (3)

[0050] After determining the target token, a performance metric of the attention head can be determined based on the intersection of the target token and the answer 217. In one example, the set containing all target tokens (also referred to as the tokens copied and pasted by the attention head) can be represented as gh , the performance index of the attention head can be determined based on the following formula:

[0051]

[0052] In the above formula, h represents the performance metric. In this way, based on the behavior of the attention head, the performance metric of the attention head is modeled as the recall rate at the word level, thereby improving the accuracy of the performance metric.

[0053] After the performance index is determined, a group of attention heads 210 can be determined from the multiple attention heads. In one example, if the performance index is larger, the reasoning and retrieval performance of the corresponding attention head is stronger. The attention heads are sorted in descending order according to the performance index, and the attention heads with the highest ranking (for example, the top 5%, the top 10%, etc.) can be determined as a group of attention heads 210. It should be understood that the attention heads can be located in different layers in the model, and the position of the layer in which the attention heads are located can be ignored; alternatively and / or additionally, the attention heads in one or more specified layers can be given priority to determine a group of attention heads that have the greatest impact on the model performance. In this way, a group of attention heads 210 that play an important role in reasoning and retrieval can be accurately determined.

[0054] After determining a set of attention heads 210, the set of attention heads 210 are masked in the base language model 205 to obtain a reference language model 220. By masking the set of attention heads 210, the retrieval and reasoning capabilities of the reference language model 220 are significantly degraded, or even have almost no ability to handle retrieval and reasoning tasks.

[0055] In some embodiments, the weights of a group of attention heads 210 can be set to equal weights. Figure 2 In the example, the weights of the attention heads 210 can all be set to 0 or other smaller values. In one example, if the length of the output sequence of the reference language model 220 is L, the attention weight of each word can be set to 1 / L.

[0056] After obtaining the reference language model 220, the training samples in the training sample set used to train the target language model are processed using the basic language model 205 and the reference language model 220, respectively, to obtain a first processing result from the basic language model 205 and a second processing result from the reference language model 220. If the difference between the first processing result and the second processing result meets a predetermined condition, the training sample is classified as a key sample (also known as an inference-intensive sample) for training the target language model. The greater the difference, the greater the impact of the masked attention head on the sample, from which it can be inferred that the corresponding training sample plays a more important role in the inference and retrieval process of the training language model. In some examples, key samples may include a large amount of inference data, such as mathematical formulas, codes, etc. In this way, the impact of the training sample 225 on the intrinsic reasoning and retrieval capabilities of the language model can be evaluated.

[0057] In some embodiments, the number of parameters of the target language model is greater than the number of parameters of the base language model 205. In this way, unlike the existing teacher-student model, key samples determined by a small-scale language model can be used to train a large-scale language model, thereby improving the training efficiency of the large-scale language model. This improves the training efficiency without relying on human-labeled data, data generated by the language model, or training any classifier.

[0058] In some embodiments, the training samples in the training sample set may belong to the same domain. Because the distribution of training samples varies significantly, it may not be possible to directly compare the processing results of language models for data from different domains (e.g., general / mathematical / coding). Therefore, the training sample set can be divided by domain, so that the base language model 205 and the reference language model 220 process training samples belonging to the same domain. In this way, the classification of training samples can be refined, thereby further improving the accuracy of the target language model in processing problems in different domains.

[0059] In some embodiments, the base language model 205 may be used to obtain a first loss 226 for the training sample 225. The reference language model 220 may be used to obtain a second loss 228 for the training sample. In one example, the base language model 205 may be used to determine the average cross entropy loss at the word level of the training sample 225 (as an example of the first loss 226, represented as Then, the reference language model 220 may be used to determine the average cross entropy loss at the word level of the training sample 225 (as an example of the second loss 228, expressed as Based on the first loss 226 and the second loss 228, a first processing result and a second processing result can be determined. In one example, the first loss 226 can be determined as the first processing result, and the second loss 228 can be determined as the second processing result. The above method of using loss to determine the processing result is merely exemplary, and any appropriate method can be used to determine the processing result of the language model on the training sample.

[0060] In some embodiments, if the relative ratio change between the first processing result (e.g., the first loss 226) and the second processing result (e.g., the second loss 228) is greater than a predetermined threshold, the training sample 225 can be classified as a key sample. The relative ratio change between the first processing result and the second processing result can be used to quantify the inference intensity of the training sample 225. In one example, the relative ratio change between the first processing result and the second processing result can be expressed as follows:

[0061]

[0062] The above method of determining the difference between the first processing result and the second processing result by using the relative ratio change is merely exemplary. The difference between the first processing result and the second processing result may also be determined using any other appropriate method (e.g., absolute difference, relative difference, logarithmic difference, etc.). For example, the relative rate of change may be determined based on the following formula:

[0063]

[0064] In some examples, the differences between the first processing result and the second processing result may be sorted in descending order, and the training samples with the highest ranking (eg, top 20%, top 25%, etc.) may be classified as key samples.

[0065] In some embodiments, after determining key examples, the training sample set can be updated using the key examples. The target language model is then trained using the updated training sample set. In one example, the training sample set can be updated by making the key examples appear repeatedly (e.g., appearing twice or more) in the training sample set. In this way, more key examples can be used to train the target language model, thereby improving the reasoning and retrieval capabilities of the target language model.

[0066] In order to verify the effectiveness of the sample processing scheme proposed in the present disclosure, the performance of the scheme proposed in the present disclosure can be compared with that of the benchmark scheme. Figure 3 FIG. 3 is a schematic diagram showing a performance comparison 300 of the solution proposed by the present disclosure and a baseline solution according to some embodiments of the present disclosure. Figure 3As shown, the scheme 305 proposed in the present disclosure uses a pre-trained model with a parameter size of 1.3B as the basic language model 205, and determines key samples from the 241B corpus to train a dense language model with a parameter size of 7B. The baseline scheme 310 is the 7B language model trained on the 241B corpus, which has been filtered by a classifier focused on the education field to retain high-quality data. After 100B of words, the average performance of the scheme 305 proposed in the present disclosure in various tasks is consistently better than the baseline scheme 310. The scheme 305 proposed in the present disclosure can produce continuous improvements, proving that it can further improve the overall data quality through better data selection beyond existing heuristics or classifiers.

[0067] Furthermore, the proposed approach was compared with a trained classifier and found that the data selected by the proposed approach was more balanced and more widely distributed across content categories, and it also prioritized longer and more comprehensive samples. Despite being completely unsupervised and untrained, the proposed approach also demonstrated strong consistency with classifier-based models, validating its reliability and generalizability.

[0068] Figure 4 4 is a flow chart showing a method 400 for processing a sample according to some embodiments of the present disclosure. The method 400 may be implemented in Figure 1 The electronic device 110 will refer to Figure 1 The method 400 is described with reference to the environment 100 of FIG.

[0069] In block 410 , the electronic device 110 determines a set of attention heads from among a plurality of attention heads of a base language model.

[0070] In block 420 , the electronic device 110 masks a set of attention heads in the base language model to obtain a reference language model.

[0071] In box 430, for a training sample in the training sample set for training the target language model, the electronic device 110 processes the training sample using the base language model and the reference language model, respectively, to obtain a first processing result from the base language model and a second processing result from the reference language model; and in response to determining that the difference between the first processing result and the second processing result meets a predetermined condition, classifies the training sample as a key sample for training the target language model.

[0072] In some embodiments, determining a set of attention heads includes: determining a performance indicator of an attention head among a plurality of attention heads based on a reference sample set, the performance indicator indicating performance of the attention head; and determining a set of attention heads from the plurality of attention heads based on the performance indicator.

[0073] In some embodiments, the reference samples in the reference sample set include at least questions and answers, and determining the performance index of the attention head includes: using a basic language model to determine a predicted answer based on the question; and determining the performance index of the attention head based on a comparison between the predicted answer and the answer.

[0074] In some embodiments, the reference sample further includes at least one input-output example, and determining the predicted answer includes: utilizing a basic language model to determine the predicted answer based on the question and the at least one input-output example.

[0075] In some embodiments, determining a performance indicator based on a comparison of a predicted answer and the answer includes: determining a target word from a plurality of words of the predicted answer, the target word appearing in the answer and an attention score obtained by the target word in the attention head meeting a predetermined score condition; and determining a performance indicator based on the intersection of the target word and the answer.

[0076] In some embodiments, masking a group of attention heads includes setting weights of the group of attention heads to equal weights.

[0077] In some embodiments, obtaining the first processing result and the second processing result includes: using a basic language model to obtain a first loss for the training sample; using a reference language model to obtain a second loss for the training sample; and determining the first processing result and the second processing result based on the first loss and the second loss.

[0078] In some embodiments, classifying the training sample as a key sample includes: classifying the training sample as a key sample in response to a relative ratio change between the first processing result and the second processing result being greater than a predetermined threshold.

[0079] In some embodiments, each training sample in the training sample set belongs to the same field.

[0080] In some embodiments, the base language model is a pre-trained model, and the number of parameters of the target language model is greater than the number of parameters of the base language model.

[0081] In some embodiments, the method 400 further includes updating the training sample set using the key samples; and training the target language model using the updated training sample set.

[0082] The disclosed embodiments also provide corresponding devices for implementing the above methods or processes. Figure 5 1. An apparatus 500 for processing a sample according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0083] like Figure 5 As shown, the apparatus 500 includes an attention head determination module 510, configured to determine a group of attention heads from a plurality of attention heads of a base language model; a reference language model acquisition module 520, configured to mask a group of attention heads in the base language model to obtain a reference language model; and a key sample classification module 530, configured to process the training samples in a training sample set for training a target language model using the base language model and the reference language model, respectively, to obtain a first processing result from the base language model and a second processing result from the reference language model; and in response to determining that the difference between the first processing result and the second processing result satisfies a predetermined condition, classify the training sample as a key sample for training the target language model.

[0084] In some embodiments, the attention head determination module 510 is further configured to determine a performance indicator of an attention head among the multiple attention heads based on a reference sample set, where the performance indicator indicates the performance of the attention head; and determine a group of attention heads from the multiple attention heads based on the performance indicator.

[0085] In some embodiments, the reference samples in the reference sample set include at least a question and an answer. The attention head determination module 510 is further configured to use the basic language model to determine a predicted answer based on the question; and determine a performance indicator of the attention head based on a comparison between the predicted answer and the answer.

[0086] In some embodiments, the reference sample further includes at least one input-output example. The attention head determination module 510 is further configured to utilize the basic language model to determine a predicted answer based on the question and the at least one input-output example.

[0087] In some embodiments, the attention head determination module 510 is further configured to determine a target word from multiple words of the predicted answer, the target word appears in the answer and the attention score obtained by the target word in the attention head satisfies a predetermined score condition; and determine a performance indicator based on the intersection of the target word and the answer.

[0088] In some embodiments, the reference language model acquisition module 520 is further configured to set the weights of a group of attention heads to be equal.

[0089] In some embodiments, the key sample classification module 530 is further configured to obtain a first loss for the training sample using a basic language model; obtain a second loss for the training sample using a reference language model; and determine a first processing result and a second processing result based on the first loss and the second loss.

[0090] In some embodiments, the key sample classification module 530 is further configured to classify the training sample as a key sample in response to a relative ratio change between the first processing result and the second processing result being greater than a predetermined threshold.

[0091] In some embodiments, each training sample in the training sample set belongs to the same field.

[0092] In some embodiments, the base language model is a pre-trained model, and the number of parameters of the target language model is greater than the number of parameters of the base language model.

[0093] In some embodiments, the apparatus 500 further includes a target language model training module configured to update the training sample set using the key samples; and train the target language model using the updated training sample set.

[0094] The units and / or modules included in the device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 500 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0095] It should be understood that one or more steps in the above method can be performed by a suitable electronic device or combination of electronic devices. Such an electronic device or combination of electronic devices may include, for example, Figure 1 The electronic device 110 in.

[0096] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The illustrated electronic device 600 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 electronic device 110 or Figure 5 device 500.

[0097] like Figure 6As shown, electronic device 600 is in the form of a general electronic device. Components of electronic device 600 may include, but are not limited to, one or more processing units or processors 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processor 610 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 600.

[0098] The electronic device 600 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 600.

[0099] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in FIG, a magnetic disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 620 may include a computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0100] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0101] The input device 650 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 660 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 600, or with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0102] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0103] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0104] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0105] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0106] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some updated implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0107] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing a sample, comprising: Determine a set of attention heads from the multiple attention heads of the base language model; masking the set of attention heads in the base language model to obtain a reference language model; as well as For the training samples in the training sample set used to train the target language model, Processing the training sample using the basic language model and the reference language model respectively to obtain a first processing result from the basic language model and a second processing result from the reference language model; as well as In response to determining that the difference between the first processing result and the second processing result satisfies a predetermined condition, the training sample is classified as a key sample for training the target language model.

2. The method of claim 1 , wherein determining the set of attention heads comprises: determining, based on a reference sample set, a performance indicator of an attention head among the plurality of attention heads, the performance indicator indicating performance of the attention head; as well as Based on the performance indicator, the set of attention heads is determined from the plurality of attention heads.

3. The method according to claim 2, wherein the reference samples in the reference sample set include at least questions and answers, and determining the performance indicator of the attention head comprises: Determining a predicted answer based on the question using the base language model; as well as Based on the comparison between the predicted answer and the answer, the performance indicator of the attention head is determined.

4. The method of claim 3, wherein the reference sample further comprises at least one input-output example, and determining the predicted answer comprises: The predicted answer is determined based on the question and the at least one input and output example using the base language model.

5. The method of claim 3 , wherein determining the performance indicator based on the comparison of the predicted answer and the answer comprises: determining a target word-gram from a plurality of word-grams of the predicted answer, wherein the target word-gram appears in the answer and an attention score obtained by the target word-gram in the attention head satisfies a predetermined score condition; as well as The performance indicator is determined based on an intersection of the target word and the answer.

6. The method of claim 1 , wherein masking the set of attention heads comprises: The weights of the set of attention heads are set to equal weights.

7. The method according to claim 1, wherein obtaining the first processing result and the second processing result comprises: Obtaining a first loss for the training sample using the basic language model; Obtaining a second loss for the training sample using the reference language model; as well as The first processing result and the second processing result are determined based on the first loss and the second loss.

8. The method according to claim 1, wherein classifying the training sample as the key sample comprises: In response to a relative ratio change between the first processing result and the second processing result being greater than a predetermined threshold, the training sample is classified as the key sample.

9. The method according to claim 1, wherein each training sample in the training sample set belongs to the same field. 10 . The method according to claim 1 , wherein the basic language model is a pre-trained model, and the number of parameters of the target language model is greater than the number of parameters of the basic language model.

11. The method according to claim 1 , further comprising: Updating the training sample set using the key samples; as well as The target language model is trained using the updated training sample set.

12. A device for processing a sample, comprising: an attention head determination module, configured to determine a set of attention heads from a plurality of attention heads of a base language model; a reference language model acquisition module, configured to mask the set of attention heads in the base language model to obtain a reference language model; as well as a key sample classification module configured to process, for training samples in a training sample set for training a target language model, the training samples using the basic language model and the reference language model, respectively, to obtain a first processing result from the basic language model and a second processing result from the reference language model; And in response to determining that the difference between the first processing result and the second processing result satisfies a predetermined condition, classifying the training sample as a key sample for training the target language model.

13. An electronic device comprising: at least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, causing the apparatus to perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images