Large model fine tuning method, electronic equipment, readable storage medium and computer program product

Abstract

The invention relates to a large model fine tuning method, electronic equipment, a readable storage medium and a computer program product, and belongs to the field of deep learning. The method comprises the following steps: in the updating process of training, updating selected parameters through masks, and freezing unselected parameters; in the training process, the constraint of the L2 norm is added to the loss function. Compared with full-amount fine tuning, the large-model fine tuning method has the advantages that the training updating cost of the pre-training model is greatly reduced, and the large-model fine tuning method is suitable for multi-training-set and multi-parameter large-scale pre-training model fine tuning. The knowledge of the pre-training model is effectively migrated through data processing and regularization constraint of adding the L2 norm, the generalization ability is improved, the over-fitting condition is effectively solved, and the performance is improved. According to the method, no extra training parameters are added, and no time delay is added in the reasoning process. According to the invention, the calculation efficiency and fine tuning performance of fine tuning of the pre-training model are greatly improved.

Description

Technical Field

[0001] The present invention relates to a large model fine-tuning method, electronic equipment, readable storage medium, and computer program product, and belongs to the technical field of deep learning. Background Art

[0002] Artificial intelligence is developing rapidly, and the application of big models is becoming more and more popular. It has been applied in various industries and brought convenience to people's lives. Based on the multimodal dialogue interaction of big models, intelligent assistants such as ChatGPT, Wenyan Yixin, and Kimi have been developed and implemented. These general big models can be applied in many fields. In specific fields, big models also play an important role. For example, medical big models assist doctors in diagnosing patients; aerospace big models assist in providing a variety of application solutions. The training of these big models is based on pre-trained models that have been trained under large data sets. On the basis of the pre-trained model, according to different downstream tasks, different data sets are selected to fine-tune the pre-trained model, and finally it can be adaptively applied to tasks in different fields. Fine-tuning is to use data sets in specific fields to further train the existing pre-trained big models. Fine-tuning can optimize the performance of the pre-trained model under specific tasks, so that the model can have the ability to complete specific downstream tasks.

[0003] Therefore, the application prospects of large models are very broad. However, due to the large number of parameters in the large model itself, it takes a huge amount of calculation to fine-tune a scientific and accurate large model for downstream tasks based on the pre-trained model. At the same time, there are many problems in training, such as excessive memory usage and high hardware requirements. Simply reducing the capacity of the large model will also lead to a decrease in the training accuracy of the large model itself. These problems have brought great challenges to the training of large models. Balancing the efficiency and performance of fine-tuning large models is an important research direction.

[0004] Existing large model fine-tuning solutions are mainly divided into two categories. One is full fine-tuning, which fine-tunes the parameters of the entire pre-trained model. The other is partial fine-tuning, which focuses on selecting some parameters of the model or fine-tuning additional modules. The former requires a large computational cost and time, but can achieve better performance. The latter has the advantages of low computational cost and short time, but reduces the performance of the model. Although many existing technologies, such as parameter tuning, prompt tuning, adapter tuning, data tuning, etc., are trying to improve performance while maintaining a low cost and have made breakthroughs, there are still shortcomings:

[0005] Performance limitation: Whether fine-tuning some model parameters or adding additional modules for fine-tuning, fine-tuning has fewer parameters than full fine-tuning. Using fine-tuning fewer parameters to reduce training costs will affect its performance on new tasks.

[0006] The potential of pre-trained models is difficult to realize: the existing large model fine-tuning schemes and different parameter fine-tuning selection schemes will affect the knowledge transfer of the original pre-trained model during the training process, thereby limiting the potential of the pre-trained model in downstream tasks. The pre-trained model cannot fully express its knowledge and realize the potential of the pre-trained model.

[0007] Overfitting of data: Although the current fine-tuning is trying to reduce fine-tuning parameters and training costs as much as possible, the existing fine-tuning schemes still have the risk of overfitting when the data distribution of the downstream task dataset and the pre-training dataset is very different. When the distribution of the dataset of the downstream task is very different from the data distribution used for pre-training, it is more likely to cause overfitting.

[0008] Inference delay: Fine-tuning methods such as adapter fine-tuning fine-tune by inserting adapters before and after some modules of the pre-trained model. Although the form of the newly inserted module is simple, the addition of additional modules during the inference process will also cause a certain inference delay.

[0009] In summary, although the existing fine-tuning solutions for large models have made some progress in reducing training parameters, there are still many problems in maintaining model performance, unleashing the potential of pre-trained models, overfitting, and inference delay. Therefore, there is an urgent need for a new solution that can fine-tune large models efficiently and accurately to achieve a trade-off between the efficiency and performance of large models and better meet the development and needs of the field of large models of artificial intelligence. Summary of the invention

[0010] In view of the problems existing in the above-mentioned prior art, the present invention provides a large model fine-tuning method, an electronic device, and a readable storage medium. The fine-tuning method of the present invention is efficient, accurate, and has no delay.

[0011] The technical solution of the present invention is: the present invention provides a large model fine-tuning method, the method comprising:

[0012] During the training update process, the selected parameters are updated through the mask, and the unselected parameters are frozen;

[0013] For the loss function, constraints are added to the traditional cross entropy loss function.

[0014] The mask is calculated as follows:

[0015] (1) Before starting training, the gradient is selected first. In order to prevent the randomly initialized classification head from affecting the gradient of the pre-trained model parameters, the Supervised Contrastive Learning (SCL) loss function is selected to calculate the gradient of the downstream task dataset;

[0016] (2) For each pre-trained model neuron, select the parameters corresponding to the largest first k gradients and update them in the subsequent training process to obtain the mask M. The elements in the mask M are:

[0017]

[0018] Among them, m ij and w ij are the elements of the i-th row and j-th column of the mask M and the pre-trained model parameters W, respectively.

[0019] The core of the present invention is the parameter selection based on gradient. In the process of training and updating, the gradient contains important information. In the updating process, the parameters with larger gradients have larger changes in the updating process, so they play a key role in the training of pre-trained models. Based on this, it can be considered that selecting parameters with larger gradient values ​​in training and updating these gradients can effectively achieve fine-tuning of pre-trained models.

[0020] Further, updating the selected parameters by masking includes:

[0021] In the back propagation update process, only the selected parameters are updated; for the t-th training cycle, the i-th layer of the pre-trained model, the pre-trained model parameters The update formula is:

[0022]

[0023] Among them, η is the learning rate, L is the loss function, ⊙ represents element-by-element multiplication, represents the pre-trained model parameters of the i-th layer in the t-th training cycle, M i Represents the mask corresponding to the i-th layer of the pre-trained model.

[0024] Furthermore, in the present invention, the mask is closely related to parameter selection, which affects the performance of pre-trained model fine-tuning. The determination of the mask is related to the data. Based on this, for downstream tasks, when the data set is large, it is particularly important to select valid data to calculate the mask. Before the mask calculation, the downstream task data is preprocessed:

[0025] For downstream task datasets Randomly divide it into s parts, recorded as After the s data sets are trained for one training cycle by the pre-trained model, the loss function and the corresponding pre-trained model gradient are calculated at this time; the data with the minimum loss function value among the s data sets are selected, and the pre-trained model gradient obtained by the data with the minimum loss function value among the s data sets is used to subsequently calculate the corresponding mask.

[0026] The mask obtained in the above way can, on the one hand, extract information with the same distribution as the original training data set. On the other hand, since the data set for calculating the mask is small and the training set is large, the parameters selected by this scheme focus more on the differences and commonalities between different classes, so it can have better generalization when training larger data sets.

[0027] Furthermore, for the loss function, constraints are added to the traditional cross entropy loss function, including:

[0028] In order to ensure that the knowledge of the original pre-trained model can be retained during the fine-tuning process, an L2 norm constraint is added to the traditional cross entropy loss function during training, limiting the parameters in the fine-tuning update process to a smaller range of the original pre-trained model, thereby allowing the knowledge of the pre-trained model to be transferred to downstream tasks.

[0029] During fine-tuning, the parameters of the last few layers of the pre-trained model have a greater impact on the training results. Therefore, this constraint mainly acts on the last few layers of the pre-trained model. The uniform distribution of The input feature dimension is large, and this interval is a small interval near 0. Therefore, the L2 norm added to the classification head can reduce the complexity of the classification head parameters and serve as a regularization term for the optimization objective function to prevent the pre-trained model from being too complex to cater to the training set, resulting in overfitting, thereby improving the generalization ability of the pre-trained model. Therefore, the objective function includes the loss function and the constraint function. The final loss function is set to It is expressed as:

[0030]

[0031] in, is the final loss function, is the cross entropy loss function during training, λ is the constraint parameter of the L2 norm, is the set of layers consisting of the last L layers of the pre-trained model and the classification head, W i represents the pre-trained model parameters for the training update of the i-th layer, Represents the initial parameters of the pre-trained model of the i-th layer.

[0032] The present invention also provides a large model fine-tuning system, which includes: a module for executing the above-mentioned large model fine-tuning method.

[0033] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned large model fine-tuning method when executing the program.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the large model fine-tuning method described above is implemented.

[0035] The present invention also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the above-mentioned large model fine-tuning method is implemented.

[0036] The beneficial effects of the present invention are:

[0037] Efficiency: Compared with full fine-tuning, the large model fine-tuning method greatly reduces the training and updating cost of the model, and is suitable for large-scale model fine-tuning with multiple training sets and multiple parameters.

[0038] Accuracy: Through data processing and adding L2 norm regularization constraints, the knowledge of the pre-trained model is effectively transferred, and the generalization ability of the model is improved, which effectively solves the problem of model overfitting and improves model performance.

[0039] No latency: Since this method does not add additional training parameters to the model, it does not add any latency during the inference process.

[0040] The present invention greatly improves the computational efficiency and fine-tuning performance of large model fine-tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0042] Example 1: Figure 1 As shown, a large model fine-tuning method, the method comprising:

[0043] During the training update process, the selected parameters are updated through masking, and the unselected parameters are frozen, which greatly reduces the training update cost of the pre-trained model and is suitable for fine-tuning of large-scale pre-trained models with multiple training sets and multiple parameters.

[0044] For the loss function, constraints are added to the traditional cross entropy loss function, which effectively transfers the knowledge of the pre-trained model and improves the generalization ability of the pre-trained model, effectively solves the overfitting problem of the pre-trained model and improves the performance of the pre-trained model.

[0045] The mask is calculated as follows:

[0046] (1) Before starting training, the gradient is selected first. In order to prevent the randomly initialized classification head from affecting the gradient of the pre-trained model parameters, the Supervised Contrastive Learning (SCL) loss function is selected to calculate the gradient of the downstream task dataset;

[0047] (2) For each pre-trained model neuron, select the parameters corresponding to the largest first k gradients and update them in the subsequent training process to obtain the mask M. The elements in the mask M are:

[0048]

[0049] Among them, m ij and w ij are the elements of the i-th row and j-th column of the mask M and the pre-trained model parameters W, respectively.

[0050] The core of the present invention is the parameter selection based on gradient. In the process of training and updating, the gradient contains important information. In the updating process, the parameters with larger gradients have larger changes in the updating process, so they play a key role in the training of pre-trained models. Based on this, it can be considered that selecting parameters with larger gradient values ​​in training and updating these gradients can effectively achieve fine-tuning of pre-trained models.

[0051] Further, updating the selected parameters by masking includes:

[0052] In the back propagation update process, only the selected parameters are updated; for the t-th training cycle, the i-th layer of the pre-trained model, the pre-trained model parameters The update formula is:

[0053]

[0054] Among them, η is the learning rate, L is the loss function, ⊙ represents element-by-element multiplication, represents the pre-trained model parameters of the i-th layer in the t-th training cycle, M i Represents the mask corresponding to the i-th layer of the pre-trained model.

[0055] Furthermore, in the present invention, the mask is closely related to parameter selection, which affects the performance of pre-trained model fine-tuning. The determination of the mask is related to the data. Based on this, for downstream tasks, when the data set is large, it is particularly important to select valid data to calculate the mask. Before the mask calculation, the downstream task data is preprocessed:

[0056] For downstream task datasets Randomly divide it into s parts, recorded as After the s data sets are trained for one training cycle by the pre-trained model, the loss function and the corresponding pre-trained model gradient are calculated at this time; the data with the minimum loss function value among the s data sets are selected, and the pre-trained model gradient obtained by the data with the minimum loss function value among the s data sets is used to subsequently calculate the corresponding mask.

[0057] The mask obtained in the above way can, on the one hand, extract information with the same distribution as the original training data set. On the other hand, since the data set for calculating the mask is small and the training set is large, the parameters selected by this scheme focus more on the differences and commonalities between different classes, so it can have better generalization when training larger data sets.

[0058] Furthermore, for the loss function, constraints are added to the traditional cross entropy loss function, including:

[0059] In order to ensure that the knowledge of the original pre-trained model can be retained during the fine-tuning process, an L2 norm constraint is added to the traditional cross entropy loss function during training, limiting the parameters in the fine-tuning update process to a smaller range of the original pre-trained model, thereby allowing the knowledge of the pre-trained model to be transferred to downstream tasks.

[0060] During fine-tuning, the parameters of the last few layers of the pre-trained model have a greater impact on the training results. Therefore, this constraint mainly acts on the last few layers of the pre-trained model. The uniform distribution of The input feature dimension is large, and this interval is a small interval near 0. Therefore, the L2 norm added to the classification head can reduce the complexity of the classification head parameters and serve as a regularization term for the optimization objective function to prevent the pre-trained model from being too complex to cater to the training set, resulting in overfitting, thereby improving the generalization ability of the pre-trained model. Therefore, the objective function includes the loss function and the constraint function. The final loss function is set to It is expressed as:

[0061]

[0062] in, is the final loss function, is the cross entropy loss function during training, λ is the constraint parameter of the L2 norm, is the set of layers consisting of the last L layers of the pre-trained model and the classification head, W i represents the pre-trained model parameters for the training update of the i-th layer, Represents the initial parameters of the pre-trained model of the i-th layer.

[0063] The data processing of the present invention (i.e., step 1 in the figure) firstly processes the downstream task data set Randomly group them equally and record them as These data are used as inputs to the pre-trained model for one generation of training to calculate the gradient and mask. Grouping can effectively split a large data set into small parts, so that the parameters selected have better generalization than the entire data set. The mask is determined by selecting the weights corresponding to the largest gradient values ​​for each neuron and retaining these weights for subsequent training updates.

[0064] The present invention formally trains (i.e., step 2 shown in the figure). For pre-trained models (such as Vision Transformer (ViT)), during the update process of training, selected parameters are updated through masks, and unselected parameters are frozen. The loss function, taking into account the knowledge transfer of the pre-trained model and the regularization of the L2 norm, adds corresponding constraints after the traditional cross entropy loss function, so as to retain the knowledge of the pre-trained model as much as possible during the training process and obtain a better generalization effect.

[0065] The present invention also provides a large model fine-tuning system, the system comprising:

[0066] The update module is used to update the selected parameters through masks during the training update process, and freeze the unselected parameters;

[0067] The constraint module is used to add L2 norm constraints to the loss function during training.

[0068] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned large model fine-tuning method when executing the program.

[0069] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the large model fine-tuning method described above is implemented.

[0070] The present invention also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the above-mentioned large model fine-tuning method is implemented.

[0071] In order to verify the effect of the present invention, the present invention conducted the following experiments for verification. The verification contents of the data set, model, comparison benchmark, and experimental setting are as follows:

[0072] Dataset: The present invention is trained on five subtasks under the Fine-Grained Visual Classification (FGVC) dataset, including two bird datasets (Caltech-UCSD-Birds-200-2011, North America Birds), a flower classification dataset (Oxford Flower), a Stanford dog dataset (Stanford Dogs), and a Stanford vehicle dataset (Stanford Cars).

[0073] Model: We experimentally tested the ViT-B / 16 model pre-trained on ImageNet-21K. This is a visual transformer model for image classification. The model uses 16×16 image patches as input and is pre-trained on ImageNet-21k with a resolution of 224×224. By pre-training on the dataset, the model learns the internal representation of the image, which can be used to extract features for subsequent tasks.

[0074] Comparison benchmarks: Compare with various fine-tuning schemes including full fine-tuning, such as Low-rank Adaptation (LoRA), Visual Prompt Tuning (VPT), ​​Scaling and Shifting Tuning (SSF), etc.

[0075] Experimental settings: The present invention uses the Adaptive Moment Estimation (Adam) optimizer and uses cosine learning rate decay for fine-tuning. Each downstream task is trained 100 times, and a 10-cycle learning rate warm-up is added before the training iteration. The experiments of the present invention are all completed on an NVIDIA GeForce RTX4090 graphics card.

[0076] Ablation experiment: The present invention systematically studies the impact of different selection schemes and conducts comprehensive ablation experiments. For each data set, the present invention tests the impact of different norm constraint parameters, different modules, and different data groupings on the experimental results. The present invention finds that there is a performance peak for each data set.

[0077] As shown in Table 1, the ViT-B / 16 model pre-trained on ImageNet-21K was used as the pre-training model. The present invention was experimented on five different data sets and compared with other types of fine-tuning methods. It can be found that the scheme of the present invention is mostly superior to other fine-tuning methods, which fully proves that the method of the present invention is a simple and effective model fine-tuning method. On the five data sets, the scheme of the present invention outperforms other methods in four tasks, including full fine-tuning, and also achieved good results in another task, while using only 0.77% of trainable parameters.

[0078] Table 1 shows the experimental comparison results.

[0079]

[0080] The large model fine-tuning method of the present invention can be applied to multiple practical scenarios:

[0081] Natural language processing: machine translation, text generation, sentiment analysis and other application scenarios. Through fine-tuning, large models can be better applied to specific areas of natural language processing. Fine-tuning pre-trained models on datasets in specific areas can improve the performance of specific tasks and reduce computing and storage costs.

[0082] Multimodal intelligent question answering system: This solution has good transferability. Through fine-tuning, the pre-trained model is applied to specific downstream tasks, thereby obtaining a large multimodal intelligent question answering model with specific domain knowledge.

[0083] Automatic summary generation: Through model fine-tuning, the model can better understand the text content and has a certain ability to extract information, refine the text summary content, and summarize the text information well.

[0084] Computer Vision: Large model fine-tuning enables pre-trained models to be applied to a variety of image tasks, such as image classification, object detection, image generation, multimodal applications, etc. It can also be used for data enhancement and transfer learning, reducing the need for large-scale labeled data.

[0085] Autonomous driving: The large model is used in the field of autonomous driving to intelligently identify objects, achieve obstacle avoidance and automatic path finding. By fine-tuning the visual and voice models to process sensor data, the corresponding functions of intelligent driving can be realized.

[0086] Artificial Intelligence Generated Content (AIGC): uses deep learning and generative models to create content, such as text, images, videos, audio, etc. It can be widely used in content creation, advertising marketing, art creation, game development and other fields.

[0087] Financial and Business Analysis: Fine-tune language models to generate professional financial or business texts, and use time series data combined with large models for predictive analysis, thereby achieving financial risk assessment, market forecasting and other functions.

[0088] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A large model fine-tuning method, characterized by: The method comprises: During the training update process, the selected parameters are updated through the mask, and the unselected parameters are frozen; During the training process, an L2 norm constraint is added to the loss function.

2. A large model fine-tuning method according to claim 1, characterized in that: The mask is calculated as follows: (1) Before starting training, select the supervised contrastive learning loss function to calculate the gradient of the downstream task dataset; (2) For each pre-trained model neuron, select the parameters corresponding to the largest first k gradients and update them in the subsequent training process to obtain the mask M. The elements in the mask M are: Among them, m ij and w ij are the elements of the i-th row and j-th column of the mask M and the pre-trained model parameters W, respectively.

3. A large model fine-tuning method according to claim 1, characterized in that: The updating of the selected parameters by means of the mask comprises: In the back propagation update process, only the selected parameters are updated; for the t-th training cycle, the i-th layer of the pre-trained model, the pre-trained model parameters The update formula is: Among them, η is the learning rate, L is the loss function, ⊙ represents element-by-element multiplication, represents the pre-trained model parameters of the i-th layer in the t-th training cycle, M i Represents the mask corresponding to the i-th layer of the pre-trained model.

4. A large model fine-tuning method according to claim 1, characterized in that: It also includes preprocessing of downstream task data before mask calculation: For downstream task datasets Randomly divide it into s parts, recorded as After the s data sets are trained for one training cycle by the pre-trained model, the loss function and the corresponding gradient at this time are calculated; the data with the minimum loss function value among the s data sets are selected, and the gradient obtained by the data with the minimum loss function value among the s data sets is used to subsequently calculate the corresponding mask.

5. A large model fine-tuning method according to claim 1, characterized in that: In the training process, the L2 norm constraint is added to the loss function, including: Adding the L2 norm constraint to the traditional cross entropy loss function, the final loss function is set It is expressed as: in, is the final loss function, is the cross entropy loss function during training, λ is the constraint parameter of the L2 norm, is the set of layers consisting of the last L layers of the pre-trained model and the classification head, W i represents the pre-trained model parameters updated for layer i training, Represents the initial parameters of the pre-trained model of the i-th layer.

6. A large model fine-tuning system, characterized in that: The system comprises: a module for executing the method according to any one of claims 1-5.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, a large model fine-tuning method as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a large model fine-tuning method as described in any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, a large model fine-tuning method as described in any one of claims 1 to 5 is implemented.

Images