Large model training sample reweighting method

Abstract

The invention discloses a large model training sample reweighting method, which belongs to the technical field of large model training, and comprises the following steps: selecting a training set and a verification set to train a large model, and decomposing a variable quantity matrix of training parameters of the large model into two low-rank matrixes by using a low-rank adaptation method; based on the low-rank matrix, extracting one-dimensional input activation and one-dimensional output gradient of the training sample and the verification sample on a low-rank adaptation layer after forward propagation and back propagation through the large model, and calculating the weight of the training sample according to the one-dimensional input activation and the one-dimensional output gradient; and based on the weights of the training samples, carrying out weighted summation on the loss values of the training samples to obtain the total loss of the training samples, and carrying out back propagation and parameter updating on the large model by utilizing the total loss of the training samples. According to the method, each training sample can be weighted efficiently, and the video memory overhead is reduced.

Description

Technical Field

[0001] The invention relates to a large model training sample re-weighting method, belonging to the technical field of large model training. Background Art

[0002] Supervised fine-tuning is an important and effective strategy in modern deep learning. It uses limited labeled data to fine-tune specific tasks based on pre-trained models, making the pre-trained models better adapted to downstream tasks at a lower cost. However, with the continuous expansion of large model parameters and the increase in training corpus, the cost of model fine-tuning is becoming increasingly high. Furthermore, the massive training samples contain a large amount of biased and noisy data, which may damage model performance if used directly for fine-tuning. Therefore, how to efficiently select high-quality samples that are beneficial for fine-tuning has become an important issue.

[0003] Existing works have mainly explored this issue from two aspects. The first method mainly uses data similarity for sample selection. For example, the third-generation General Pre-trained Transformer-3 (GPT-3) trains a classifier to select samples with similar features to Wikipedia text from a massive corpus. Some works use retrieval-based methods (such as the best match 25 algorithm, BM25) to efficiently select data that is more relevant to the task. Some works use the similarity of N-gram features to select training data that is closer to the target sample. Although these methods do not bring a lot of additional video memory overhead, they all rely on the similarity of text features, so the results of sample selection are often not necessarily accurate. Another type of method evaluates sample importance from the perspective of parameter gradient updates during training (such as Data Shapley and influence functions). The core idea is to compare the changes in model parameter gradient updates after removing a sample from the training set. Although this type of method can accurately evaluate the impact of samples on model training, it involves the calculation of second-order derivatives, which incurs huge additional memory overhead and makes it difficult to apply to large language models (LLMs). Summary of the invention

[0004] The purpose of the present invention is to provide a large model training sample re-weighting method, which can efficiently weight each training sample and reduce video memory overhead.

[0005] In order to achieve the above object, the present invention provides the following technical solutions: The present invention provides a large model training sample re-weighting method, comprising: A training set and a validation set are selected to train the large model, and a low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices; Based on the low-rank matrix, extract the one-dimensional input activation and one-dimensional output gradient of the training sample and the validation sample on the low-rank adaptation layer after forward propagation and backpropagation through the large model, and calculate the weight of the training sample based on the one-dimensional input activation and one-dimensional output gradient; Based on the weights of the training samples, the loss values ​​of the training samples are weighted summed to obtain the total loss of the training samples, and the total loss of the training samples is used to perform backpropagation and parameter update on the large model.

[0006] Combined with the first aspect, further, the training set and validation set are selected according to the downstream tasks of the large model. The training set and validation set are in the form of instruction fine-tuning. The dataset format corresponding to the training set and validation set is ,in, Indicates input samples, Indicates Output response, Represents the total number of samples.

[0007] In combination with the first aspect, further, using a low-rank adaptation method to decompose the change matrix of the training parameters of the large model into two low-rank matrices includes: The training parameters of the large model The variation matrix Decomposed into two low-rank matrices and ,in, , , Respectively represent the dimensions , , The matrix of represents the rank of low-rank adaptation, ; Through two low-rank matrices and The product of The change matrix of the training parameters of the large model Approximate, and after fine-tuning, the two low-rank matrices and The product of With training parameters Merge and get new training parameters.

[0008] Combined with the first aspect, further, extracting the one-dimensional input activation and one-dimensional output gradient on the low-rank adaptation layer after the training samples and the verification samples are forward propagated and back propagated through the large model includes: In each round of training, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after the training samples are forward propagated and back-propagated through the large model are extracted; Each interval After a round of training, the validation samples are extracted and passed through the large model for forward propagation and back propagation, resulting in the one-dimensional input activation and one-dimensional output gradient on the low-rank adaptation layer.

[0009] In combination with the first aspect, further, the interval It is set according to the number of samples corresponding to the selected training set and validation set.

[0010] In combination with the first aspect, further, each interval After a round of training, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after extracting the validation sample and forward propagating and back propagating through the large model include: The one-dimensional input activation and one-dimensional output gradient of each extracted validation sample on the low-rank adaptation layer after forward propagation and back propagation through the large model are temporarily stored in the memory or video memory as intermediate results until the intermediate results are updated during the next extraction.

[0011] In combination with the first aspect, further, according to the one-dimensional input activation and the one-dimensional output gradient, calculating the weight of the training sample includes: Calculate the similarity of the one-dimensional input activations of the training samples and the verification samples in the current training round based on the one-dimensional input activations of the training samples on the low-rank adaptation layer after forward propagation and back propagation through the large model in the current training round and the one-dimensional input activations of the verification samples on the low-rank adaptation layer after forward propagation and back propagation through the large model; Calculate the similarity of the one-dimensional output gradients of the training samples and the verification samples in the current training round according to the one-dimensional output gradients of the training samples on the low-rank adaptation layer after forward propagation and back propagation through the large model in the current training round and the one-dimensional output gradients of the verification samples on the low-rank adaptation layer after forward propagation and back propagation through the large model; Calculate the gradient similarity between the training sample and the verification sample in the current training round according to the one-dimensional input activation similarity and one-dimensional output gradient similarity between the training sample and the verification sample in the current training round; The gradient similarity between the training samples and the validation samples in the current training round is normalized, and the normalized gradient similarity between the training samples and the validation samples in the current training round is used as the weight of the training samples in the current training round.

[0012] Combined with the first aspect, further, the calculation formula of the one-dimensional input activation similarity between the training sample and the verification sample is: ,in, Indicates The trainable modules The one-dimensional input activation on the low-rank adaptation layer after the training samples are forward-propagated and back-propagated through the large model. Indicates The trainable modules The one-dimensional input activation of the low-rank adaptation layer after the validation samples are forward-propagated and back-propagated through the large model. represents transpose; The calculation formula for the one-dimensional output gradient similarity between the training sample and the verification sample is: ,in, Indicates The trainable module The one-dimensional output gradient of the low-rank adaptation layer after the training sample is forward-propagated and back-propagated through the large model, Indicates The trainable modules The one-dimensional output gradient of the validation sample on the low-rank adaptation layer after forward propagation and backpropagation through the large model; The calculation formula for the gradient similarity between the training sample and the verification sample is: .

[0013] In combination with the first aspect, further, if the one-dimensional input activation and one-dimensional output gradient of the verification sample on the low-rank adaptation layer after forward propagation and backward propagation through the large model are not extracted in the current training round, then the one-dimensional input activation and one-dimensional output gradient of the verification sample extracted last time after forward propagation and backward propagation through the large model on the low-rank adaptation layer are used as the one-dimensional input activation and one-dimensional output gradient of the verification sample in the current training round after forward propagation and backward propagation through the large model on the low-rank adaptation layer.

[0014] In combination with the first aspect, further, based on the weights of the training samples, weighted summation is performed on the loss values ​​of the training samples to obtain the total loss of the training samples, and the total loss of the training samples is used to perform back propagation and parameter update on the large model, including: According to the weights of the training samples in the current training round, the loss values ​​of the training samples in the current training round are weighted summed to obtain the total loss of the training samples in the current training round, and the total loss of the training samples in the current round is used to perform backpropagation and parameter update on the large model; Repeat the above steps until the training of the large model reaches the maximum number of iterations; Among them, the calculation formula for the total loss of training samples is: ; in, represents the total loss of training samples, Indicates The weight of the training samples, Indicates The loss value of training samples is Represents the total number of training samples.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The large model training sample reweighting method provided by the present invention is based on the chain rule and the commutative law of multiplication of parameter gradient back propagation. When calculating gradient similarity, the two-dimensional gradient similarity is decomposed into the product of one-dimensional input activation and one-dimensional output gradient, which can greatly reduce the video memory overhead; further combining this method with the low-rank adaptation method and applying the above decomposition on the low-rank adaptation layer can achieve a video memory overhead similar to that of ordinary supervised fine-tuning.

[0016] The large model training sample reweighting method provided by the present invention fully considers the different contributions of each training sample to downstream tasks, and efficiently weights each training sample through adaptive weighting and low-rank approximation methods, and can achieve better performance in downstream task fine-tuning with extremely low additional video memory overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of re-weighting large model training samples provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present application is further described in detail below in conjunction with specific implementation methods.

[0019] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as limitations on the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0020] The present application provides a large model training sample re-weighting method, including: A training set and a validation set are selected to train the large model, and a low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices; Based on the low-rank matrix, extract the one-dimensional input activation and one-dimensional output gradient of the training sample and the validation sample on the low-rank adaptation layer after forward propagation and backpropagation through the large model, and calculate the weight of the training sample based on the one-dimensional input activation and one-dimensional output gradient; Based on the weights of the training samples, the loss values ​​of the training samples are weighted summed to obtain the total loss of the training samples, and the total loss of the training samples is used to perform backpropagation and parameter update on the large model.

[0021] In order to accurately evaluate the quality of training samples while minimizing the additional memory overhead, we can start from the traditional meta-learning paradigm and optimize the sample weighting based on meta-learning for memory efficiency. In the meta-learning paradigm, for a specific downstream task, a training set and a small-scale high-quality validation set are selected, and then the validation set is used to evaluate the quality of the training samples during fine-tuning (specifically, higher weights are given to those training samples that are closer to the validation sample gradient update direction). This type of method essentially evaluates the importance of samples based on the parameter gradient update during training, so it also involves the calculation of second-order derivatives (i.e., the derivative of the gradient), which has huge memory overhead. In order to avoid the calculation of second-order derivatives, the traditional meta-learning paradigm can be modeled as an explicit gradient similarity calculation, that is, directly extracting the parameter gradients corresponding to the training samples and the validation samples for similarity calculation, and giving higher weights to those training samples that are closer to the validation sample gradient direction.

[0022] Modern large models often contain more than 1 billion parameters. In order to reduce the memory overhead during fine-tuning, a study proposed the Low-Rank Adaptation (LoRA) method. The LoRA method can significantly reduce the trainable parameters in large models (to less than 1%), specifically including: decomposing the parameter change matrix during model training into two low-rank matrices, and approximating the original parameter change matrix by the product of these two low-rank matrices; after fine-tuning, multiply the two low-rank matrices and merge them into the original parameters. Combining the meta-learning method with the LoRA method, you only need to calculate the gradient similarity of the model's LoRA parameters.

[0023] However, even after the above approximation and optimization, meta-learning-based training sample weight adjustment still requires a lot of additional video memory overhead compared to ordinary supervised fine-tuning. This is mainly because the dimension of the parameter gradient is the same as the original parameter. When calculating the gradient similarity, the parameter gradient of the verification sample and the two-dimensional gradient similarity are introduced. Therefore, in theory, the video memory overhead will still be doubled compared to ordinary supervised fine-tuning, which greatly limits the practical application of this method, especially in resource-constrained scenarios.

[0024] This embodiment is mainly based on large model fine-tuning and meta-sample weighting. Taking into account that the second-order derivative calculation involved in the traditional meta-learning method will bring great additional memory overhead, the meta-sample weighting is modeled as an explicit gradient similarity calculation, and the chain rule and the commutative law of multiplication of gradient back propagation are used to decompose the two-dimensional gradient similarity calculation of the training sample and the verification sample into the product of the one-dimensional input activation and the one-dimensional output gradient, thereby greatly reducing the memory overhead.

[0025] This embodiment is further combined with the LoRA method. Specifically, LoRA decomposes the parameter change matrix of the model training process into two low-rank matrices, and approximates the original parameter change matrix by the product of the two low-rank matrices. The decomposition of the two-dimensional gradient similarity is performed on the low-rank layer of this series of low-rank matrices, so that the dimensions of the obtained one-dimensional input activation and one-dimensional output gradient are the same as the rank of LoRA, which minimizes the additional video memory overhead and is almost the same as ordinary supervised fine-tuning.

[0026] In this embodiment, the training set and the validation set are selected based on the downstream tasks of the large model, and the training set and the validation set are in the form of instruction fine-tuning.

[0027] Specifically, the dataset formats corresponding to the training set and validation set are ,in, Indicates input samples, Indicates Output response, Represents the total number of samples.

[0028] The size of the training set is generally much larger than the validation set. The training set generally contains more than 10,000 samples, and the validation set can contain about 1,000 samples. In the subsequent process, the validation samples are used to weight the training samples, and the validation samples can also be used in training.

[0029] In order to reduce the memory usage, the low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices.

[0030] Specifically, the low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices including: The training parameters of the large model The variation matrix Decomposed into two low-rank matrices and ,in, , , Respectively represent the dimensions , , The matrix of represents the rank of low-rank adaptation, ; Through two low-rank matrices and The product of The change matrix of the training parameters of the large model Approximate, and after fine-tuning, the two low-rank matrices and The product of With training parameters Merge and get new training parameters. The specific formula is: , .

[0031] In this embodiment, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after the training samples and the verification samples are forward propagated and back propagated through the large model include: In each round of training, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after the training samples are forward propagated and back-propagated through the large model are extracted; Each interval After a round of training, the validation samples are extracted and passed through the large model for forward propagation and back propagation, resulting in the one-dimensional input activation and one-dimensional output gradient on the low-rank adaptation layer.

[0032] Specifically, interval It is set according to the number of samples corresponding to the selected training set and validation set to adapt to the number of training samples and validation samples.

[0033] Each interval The method comprises: temporarily storing the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after the verification sample extracted each time is forward propagated and back-propagated through the large model as intermediate results in the memory or video memory until the intermediate results are updated during the next extraction.

[0034] In this embodiment, according to the one-dimensional input activation and the one-dimensional output gradient, calculating the weight of the training sample specifically includes the following steps: Step 1: Calculate the similarity of the one-dimensional input activation of the training sample and the verification sample in the current training round based on the one-dimensional input activation of the low-rank adaptation layer after the training sample in the current training round is forward propagated and back-propagated through the large model and the one-dimensional input activation of the verification sample in the low-rank adaptation layer after forward propagation and back-propagation through the large model; Specifically, the formula for calculating the one-dimensional input activation similarity between the training sample and the verification sample is: ,in, Indicates The trainable modules The one-dimensional input activation on the low-rank adaptation layer after the training samples are forward-propagated and back-propagated through the large model. Indicates The trainable modules The one-dimensional input activation of the low-rank adaptation layer after the validation samples are forward-propagated and back-propagated through the large model. Indicates transpose.

[0035] It should be noted that if the one-dimensional input activation of the low-rank adaptation layer after the verification sample is forward propagated and backward propagated through the large model in the current training round, the one-dimensional input activation of the low-rank adaptation layer after the verification sample extracted last time is forward propagated and backward propagated through the large model will be used as the one-dimensional input activation of the low-rank adaptation layer after the verification sample in the current training round is forward propagated and backward propagated through the large model.

[0036] Step 2: Calculate the similarity of the one-dimensional output gradients of the training samples and the verification samples in the current training round based on the one-dimensional output gradients of the training samples on the low-rank adaptation layer after forward propagation and back propagation through the large model in the current training round and the one-dimensional output gradients of the verification samples on the low-rank adaptation layer after forward propagation and back propagation through the large model; Specifically, the calculation formula for the one-dimensional output gradient similarity between the training sample and the verification sample is: ,in, Indicates The trainable modules The one-dimensional output gradient of the low-rank adaptation layer after the training sample is forward-propagated and back-propagated through the large model, Indicates The trainable module The one-dimensional output gradient of the low-rank adaptation layer after the validation samples are forward-propagated and back-propagated through the large model.

[0037] It should be noted that if the one-dimensional output gradient of the verification sample on the low-rank adaptation layer after forward propagation and backward propagation through the large model is not extracted in the current training round, the one-dimensional output gradient of the verification sample extracted last time after forward propagation and backward propagation through the large model on the low-rank adaptation layer will be used as the one-dimensional output gradient of the verification sample on the low-rank adaptation layer after forward propagation and backward propagation through the large model in the current training round.

[0038] Step 3: Calculate the gradient similarity between the training sample and the verification sample in the current training round based on the one-dimensional input activation similarity and one-dimensional output gradient similarity between the training sample and the verification sample in the current training round; Specifically, the calculation formula for the gradient similarity between the training sample and the verification sample is: .

[0039] Step 4: Normalize the gradient similarity between the training samples and the validation samples in the current training round, and use the normalized gradient similarity between the training samples and the validation samples in the current training round as the weight of the training samples in the current training round.

[0040] This embodiment decomposes the two-dimensional parameter gradient into the product of one-dimensional input activation and one-dimensional output gradient according to the chain rule of gradient back propagation, and then converts the calculation of the two-dimensional parameter gradient similarity into the product of one-dimensional input activation and one-dimensional output gradient by using the multiplication commutative law, which can greatly reduce the video memory overhead.

[0041] In this embodiment, based on the weights of the training samples, the loss values ​​of the training samples are weighted summed to obtain the total loss of the training samples, and the total loss of the training samples is used to perform back propagation and parameter update on the large model, including: According to the weights of the training samples in the current training round, the loss values ​​of the training samples in the current training round are weighted summed to obtain the total loss of the training samples in the current training round, and the total loss of the training samples in the current round is used to perform backpropagation and parameter update on the large model; Repeat the above steps until the training of the large model reaches the maximum number of iterations.

[0042] Specifically, the total loss of the training sample is calculated as: ; in, represents the total loss of training samples, Indicates The weight of the training samples, Indicates The loss value of training samples is Represents the total number of training samples.

[0043] This embodiment fully considers the different contributions of each training sample to the downstream task, and efficiently weights each sample through adaptive weighting and low-rank approximation methods. The large model training sample reweighting method provided in this embodiment can achieve better performance in downstream task fine-tuning with extremely low additional video memory overhead.

[0044] The large model training sample re-weighting method provided in this embodiment can use computer software technology to implement the corresponding process. Figure 1 As shown, the process of the large model training sample re-weighting method provided in this embodiment is specifically described by taking GSM8K mathematical task fine-tuning as an example.

[0045] Step 1: For GSM8K fine-tuning, select MathInstruct as the training set (containing 262,000 samples), and the GSM8K training set as the high-quality verification set (GSM8K is divided into two parts: training set and test set, with 7473 and 1319 samples respectively. Here, only the training set is selected as the verification sample). The data set adopts the form of instruction fine-tuning. For specific examples, please refer to Figure 1 Verification sample input and training sample input. In the subsequent process, the verification sample is used to weight the training sample, and the verification sample can also be used in training. After fine-tuning, test on the GSM8K test set to evaluate the fine-tuning effect. In order to reduce the memory usage, the training parameters of the LoRA large model are used during training. The variation matrix Decomposed into two low-rank matrices and ,in, , for example, it can be set to 32, and the two low-rank matrices can be multiplied to obtain For example, the model to be fine-tuned is selected as Baichuan-7B, and LoRA is used to decompose the parameter change matrix of each trainable module in the attention module and feedforward module of each layer. Figure 1 As shown, the LoRA decomposition of the descending projection module in one of the feedforward modules is demonstrated.

[0046] Step 2: Each round will training samples are input into the large model ( is the batch size of the training sample, for example, it can be set to 4), and after back propagation, the one-dimensional input activation of each trainable module (i.e., LoRA module) of the model in the LoRA low-rank layer is recorded. (Dimensions are ) and the output gradient (Dimensions are ), the parameters are not updated at this time; every wheel( To verify the update interval of sample activation value and gradient value, it can be set to 10). validation samples are input into the large model ( To verify the batch size of the sample, for example, it can be set to 4), record the one-dimensional input activation of each trainable module of the model after back propagation And the output gradient According to the chain rule of gradient calculation in the back-propagation process, the parameter gradient corresponding to a certain trainable module can be obtained by (or ), and then the gradient similarity can be obtained by matrix multiplication of the parameter gradient value (that is, the gradient similarity of the training and validation samples can be obtained by Calculated, where Indicates that the gradient is expanded into a one-dimensional vector. It should be noted that even through After being expanded into a one-dimensional vector, vector multiplication is performed. The video memory overhead is equivalent to the original two-dimensional matrix multiplication. In order to avoid the video memory overhead caused by the above process, this embodiment uses the multiplication commutative law to convert it into a true one-dimensional vector multiplication. The specific method is to calculate and To get the similarity between the one-dimensional input activation and the one-dimensional output gradient, the two are multiplied to get the final gradient similarity. In particular, if the current round does not calculate the new and , then retrieve the most recently updated and The above operation is performed based on the value. In the specific implementation, after calculating the input activation and output gradient corresponding to the verification sample, these values ​​are temporarily stored in the memory or video memory as intermediate results until the next update. The calculated gradient similarity is normalized and used as the weight of each training sample in the batch, such as Figure 1 The weights of the two training samples shown are 0.2402 and 0.1128 respectively. Figure 1 middle, , represents the one-dimensional input activation and one-dimensional output gradient of the training sample, , Represents the one-dimensional input activation and one-dimensional output gradient of the training samples extracted from the LoRA low-rank layer.

[0047] Step 3: Use the weights obtained in step 2 to perform weighted summation and back propagation on the loss values ​​of each round of training samples and update the parameters. In each round, assume that the weight corresponding to each training sample is obtained after normalizing the gradient similarity: , the formula for weighted summation of training loss using these weight values ​​is: Finally, use Back propagation and parameter update are performed, and the round ends.

[0048] Repeat the above steps until the training of the large model reaches the maximum number of iterations.

[0049] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A large model training sample reweighting method, characterized in that: include: A training set and a validation set are selected to train the large model, and a low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices; Based on the low-rank matrix, extract the one-dimensional input activation and one-dimensional output gradient of the training sample and the validation sample on the low-rank adaptation layer after forward propagation and backpropagation through the large model, and calculate the weight of the training sample based on the one-dimensional input activation and one-dimensional output gradient; Based on the weights of the training samples, the loss values ​​of the training samples are weighted summed to obtain the total loss of the training samples, and the total loss of the training samples is used to perform backpropagation and parameter update on the large model.

2. The large model training sample re-weighting method according to claim 1, characterized in that: The training set and validation set are selected based on the downstream tasks of the large model, and the training set and validation set are in the form of instruction fine-tuning.

3. The large model training sample re-weighting method according to claim 1, characterized in that: The low-rank adaptation method is used to decompose the change matrix of the training parameters of the large model into two low-rank matrices including: The training parameters of the large model The variation matrix Decomposed into two low-rank matrices and ,in, , , Respectively represent the dimensions , , The matrix of represents the rank of low-rank adaptation, ; Through two low-rank matrices and The product of The change matrix of the training parameters of the large model Approximate, and after fine-tuning, the two low-rank matrices and The product of With training parameters Merge and get new training parameters.

4. The large model training sample re-weighting method according to claim 1, characterized in that: The one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after extracting the training samples and validation samples for forward propagation and back propagation through the large model include: In each round of training, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after the training samples are forward propagated and back-propagated through the large model are extracted; Each interval After a round of training, the validation samples are extracted and passed through the large model for forward propagation and back propagation, resulting in the one-dimensional input activation and one-dimensional output gradient on the low-rank adaptation layer.

5. The large model training sample re-weighting method according to claim 4, characterized in that: interval It is set according to the number of samples corresponding to the selected training set and validation set.

6. The large model training sample re-weighting method according to claim 4, characterized in that: Each interval After a round of training, the one-dimensional input activation and one-dimensional output gradient of the low-rank adaptation layer after extracting the validation sample and forward propagating and back propagating through the large model include: The one-dimensional input activation and one-dimensional output gradient of each extracted validation sample on the low-rank adaptation layer after forward propagation and back propagation through the large model are temporarily stored in the memory or video memory as intermediate results until the intermediate results are updated during the next extraction.

7. The large model training sample re-weighting method according to claim 4, characterized in that: Based on the one-dimensional input activation and one-dimensional output gradient, calculating the weight of the training sample includes: Calculate the similarity of the one-dimensional input activations of the training samples and the verification samples in the current training round based on the one-dimensional input activations of the training samples on the low-rank adaptation layer after forward propagation and back propagation through the large model in the current training round and the one-dimensional input activations of the verification samples on the low-rank adaptation layer after forward propagation and back propagation through the large model; Calculate the similarity of the one-dimensional output gradients of the training samples and the verification samples in the current training round according to the one-dimensional output gradients of the training samples on the low-rank adaptation layer after forward propagation and back propagation through the large model in the current training round and the one-dimensional output gradients of the verification samples on the low-rank adaptation layer after forward propagation and back propagation through the large model; Calculate the gradient similarity between the training sample and the verification sample in the current training round according to the one-dimensional input activation similarity and one-dimensional output gradient similarity between the training sample and the verification sample in the current training round; The gradient similarity between the training samples and the validation samples in the current training round is normalized, and the normalized gradient similarity between the training samples and the validation samples in the current training round is used as the weight of the training samples in the current training round.

8. The large model training sample re-weighting method according to claim 7, characterized in that: The formula for calculating the one-dimensional input activation similarity between the training sample and the verification sample is ,in, Indicates The trainable modules The one-dimensional input activation on the low-rank adaptation layer after the training samples are forward-propagated and back-propagated through the large model. Indicates The trainable modules The one-dimensional input activation of the low-rank adaptation layer after the validation samples are forward-propagated and back-propagated through the large model. represents transpose; The calculation formula for the one-dimensional output gradient similarity between the training sample and the verification sample is: ,in, Indicates The trainable modules The one-dimensional output gradient of the low-rank adaptation layer after the training samples are forward propagated and back-propagated through the large model, Indicates The trainable modules The one-dimensional output gradient of the validation sample on the low-rank adaptation layer after forward propagation and back propagation through the large model; The calculation formula for the gradient similarity between the training sample and the verification sample is: .

9. The large model training sample re-weighting method according to claim 7, characterized in that: If the one-dimensional input activation and one-dimensional output gradient of the verification sample on the low-rank adaptation layer after forward propagation and back-propagation through the large model are not extracted in the current training round, the one-dimensional input activation and one-dimensional output gradient of the verification sample extracted last time on the low-rank adaptation layer after forward propagation and back-propagation through the large model are used as the one-dimensional input activation and one-dimensional output gradient of the verification sample on the low-rank adaptation layer after forward propagation and back-propagation through the large model in the current training round.

10. The large model training sample re-weighting method according to claim 1, characterized in that: Based on the weight of the training samples, the loss values ​​of the training samples are weighted summed to obtain the total loss of the training samples, and the total loss of the training samples is used to perform back propagation and parameter update on the large model, including: According to the weights of the training samples in the current training round, the loss values ​​of the training samples in the current training round are weighted summed to obtain the total loss of the training samples in the current training round, and the total loss of the training samples in the current round is used to perform backpropagation and parameter update on the large model; Repeat the above steps until the training of the large model reaches the maximum number of iterations; Among them, the calculation formula for the total loss of training samples is: ； in, represents the total loss of training samples, Indicates The weight of the training samples, Indicates The loss value of training samples is Represents the total number of training samples.

Images