Progressive learning-based model fine tuning method and system

Abstract

The invention belongs to the field of language model processing, and relates to a model fine tuning method and system based on progressive learning. According to the method, in the fine tuning process of a pre-training model, according to different contributions of Transform blocks to the performance of the model, the Transform blocks with large contributions to the performance of the pre-training model are trained emphatically, and the Transform blocks with small contributions to the performance of the pre-training model are trained slightly. According to the method, the thought of progressive learning is introduced into model fine tuning, Transform blocks participating in updating are gradually reduced in the fine tuning process, consumption of computing resources is saved, meanwhile, overtraining of parameters is reduced, and waste of the computing resources and the overfitting phenomenon of training data can be reduced while the amount of updated parameters and the training time are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of language model processing, and specifically relates to a model fine-tuning optimization method. By introducing the idea of ​​progressive learning into model fine-tuning, the waste of computing resources and overfitting of training data can be reduced while reducing the amount of updated parameters and training time. Background Art

[0002] With the explosion of ChatGPT, a wave of big model technology has been set off around the world. After the big model is initialized, it is usually necessary to pre-train it on a general corpus to obtain a pre-trained model. The pre-trained model can be deployed directly or further fine-tuned on a task-specific data machine to obtain a more professional model and then deployed. Fine-tuning refers to the process of further adjusting and optimizing the parameters of the model on a task-specific dataset based on the pre-trained model to adapt to a specific task. However, fine-tuning requires updating all parameters of the model. With the development of big model technology, the size of the model is getting larger and larger, and has even grown exponentially in recent years, making it increasingly difficult to fine-tune the model.

[0003] In order to solve the challenge of huge model parameters and large computational and storage overhead required for fine-tuning the entire model, researchers proposed a parameter-efficient fine-tuning method. Parameter-efficient fine-tuning freezes most of the model's parameters and only updates a small number of trainable parameters to achieve a fine-tuning method that reduces computational and storage costs while maintaining model performance. The updated parameter amount is usually equivalent to about 0.1-10% of the model, but can produce results comparable to or even better than full fine-tuning. Parameter-efficient fine-tuning methods can be roughly divided into three categories:

[0004] Efficient fine-tuning methods based on adding parameters: The core idea of ​​this type of method is to freeze all parameters of the pre-trained model and add a small number of trainable parameters or modules. Representative works include adapter fine-tuning, which adds two adapter modules to each Transformer block and only updates the adapter parameters during training; there are also prompt micro-tagging and prefix fine-tuning, which have similar ideas and both guide the output of the language model through prompts. However, there are some problems with the addition-based method. For example, adapter fine-tuning destroys the structure of the pre-trained model because it introduces additional modules, and it will bring additional delays during inference; the prompt vector of prompt fine-tuning will occupy the input space, thereby affecting the performance of the model.

[0005] Selection-based efficient parameter fine-tuning methods: The core idea of ​​this type of method is to select a subset of model parameters for update and freeze most of the remaining parameters. Representative methods include BitFit, which has a very simple idea and can produce good results by only updating the bias term of the model; FAR uses evaluation criteria to select parameters that contribute more to the model and update them, and freeze the remaining parameters. Selection-based methods usually perform well when the model size is small, but as the model grows, the performance gap with full fine-tuning begins to emerge, and how to design a simple and effective parameter selection algorithm is still very challenging.

[0006] Efficient parameter fine-tuning methods based on reparameterization: The core idea of ​​this type of method is to reparameterize the model parameters or parameter updates to reduce the amount of parameter updates. Representative works include LoRA, which uses two low-rank matrices to approximate parameter updates and is currently the best performing efficient parameter fine-tuning method; and the KronA method replaces the low-rank decomposition in LoRA with the Kronecker product.

[0007] Hybrid method: This method is to use two or more of the above three methods. The representative method is UniPELT, which is a combination of LoRA, adapter fine-tuning and prefix fine-tuning. The advantages and disadvantages of this method are very obvious. The advantage is that the performance is better than that of a single method, and the disadvantage is that the implementation is more complicated.

[0008] However, fine-tuning and most parameter-efficient fine-tuning methods always update all trainable parameters during training, ignoring the contribution of different Transformer blocks to model performance and leading to suboptimal allocation of computing resources. Figure 1 From the simple example shown, we can see that: (1) different Transformer blocks contribute differently to the model performance; (2) high-index Transformer blocks perform better than low-index Transformer blocks. Therefore, during fine-tuning, computing resources should be selectively allocated to parameters that contribute more to the model, thereby optimizing the allocation of computing resources. Summary of the invention

[0009] In view of the above problems, the present invention provides a model fine-tuning method and system based on progressive learning.

[0010] The technical solution adopted by the present invention is as follows:

[0011] A model fine-tuning method based on progressive learning. During the fine-tuning process of the pre-trained model, the Transformer blocks involved in the update are gradually reduced, the Transformer blocks that contribute greatly to the performance of the pre-trained model are trained intensively, and the Transformer blocks that contribute little to the performance of the pre-trained model are trained in small quantities.

[0012] Furthermore, the fine-tuning process includes:

[0013] Divide the Transformer layer of the pre-trained model into τ parts evenly;

[0014] The τ parts obtained by division are combined to obtain τ overlapping but gradually shrinking stages;

[0015] In the i-th round of training, only the parameters of the i-th stage and embedding layer are updated.

[0016] Furthermore, the Transformer layer of the pre-trained model is evenly divided into τ parts, including:

[0017] Assuming a pre-trained model The model consists of an embedding layer E, L Transformer blocks, and a classification layer H. It is expressed as:

[0018]

[0019] Among them B i is a Transformer block, i∈{1,...,L};

[0020] Assume the number of fine-tuning rounds is τ, The Transformer layer of the model is evenly divided into τ parts, and P t Represents, where t∈{1,...,τ}, each part has Transformer blocks;

[0021] The divided parts are numbered from low to high, and the model is represented as follows:

[0022]

[0023] Furthermore, the τ overlapping but gradually shrinking stages are represented by S t Represents, where t∈{1,...,τ}, overlap means that each stage has the same part, and shrinking means that the number of Transformer blocks contained in each stage gradually decreases.

[0024] Furthermore, each stage consists of the corresponding index and all subsequent parts plus the classification layer H, and the stage is represented as:

[0025]

[0026] The first stage S1 consists of the part P1 with index 1 and all subsequent parts and the classification layer, that is, it contains all τ parts; the second stage S2 consists of the part P1 with index 2 12 And all subsequent parts and classification layers.

[0027] Furthermore, by only updating the parameters of the i-th stage and the embedding layer E in the i-th round of training, the Transformer blocks with high indexes get more training, and the Transformer blocks with low indexes only receive a small amount of training.

[0028] Furthermore, the division units are changed from Transformer blocks to trainable parameters, which are combined with efficient parameter fine-tuning methods to reduce computing resources.

[0029] A model fine-tuning system based on progressive learning, comprising:

[0030] A partitioning module is used to partition the Transformer layer of the pre-trained model, and evenly divide the Transformer layer of the pre-trained model into τ parts;

[0031] A combination module, used for combining the τ parts obtained by the division to obtain τ overlapping but gradually shrinking stages;

[0032] The training module is used to update only the parameters of the i-th stage and the embedding layer in the i-th round of training.

[0033] Compared with the existing related technologies, the present invention has the following significant positive effects:

[0034] (1) This patent combines the different contributions of Transformer blocks to model performance, and focuses training resources on parameters that contribute most to the task, training them intensively, while parameters with small contributions only require a small amount of training to reduce computing resources.

[0035] (2) The method disclosed in this patent introduces the idea of ​​progressive learning into model fine-tuning. During the fine-tuning process, the Transformer blocks involved in the update are gradually reduced, thereby saving computing resources. Compared with complete fine-tuning, the amount of updated parameters is reduced by about 26%. At the same time, due to the reduction of over-training of parameters and the occurrence of overfitting, the model performance has been slightly improved.

[0036] (3) The method disclosed in this patent is perfectly compatible with the efficient parameter fine-tuning method. It only needs to replace the division unit from the Transformer block with a trainable parameter, which can further reduce computing resources. When combined with adapter fine-tuning, it can even reduce the amount of updated parameters by 67% with only a slight decrease in performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the contribution of different Transformer blocks to model performance.

[0038] Figure 2 This is a schematic diagram of the Transformer layer division of the model.

[0039] Figure 3 It is a schematic diagram of the combination after model division.

[0040] Figure 4 is the updated parameter amount on the GLUE dataset. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.

[0042] This patent proposes a model fine-tuning method based on progressive learning. By introducing the idea of ​​progressive learning into model fine-tuning, not only can the amount of updated parameters be reduced, but also the occurrence of overfitting can be reduced, thereby improving model performance. Progressive learning was first proposed in the field of computer vision. The core idea is to first train the model on simple tasks, such as low-resolution images, small models, etc., and then gradually expand to difficult tasks, such as high-resolution images, larger models, etc. Such a training process can ensure the stability of the training stage. Some scholars have introduced the idea of ​​progressive learning into federated learning, which greatly reduces the computation and communication volume of federated learning.

[0043] First, the Transformer layer of the model is divided. Assume that the model The model consists of an embedding layer E, L Transformer blocks, and a classification layer H. It is expressed as:

[0044]

[0045] Among them B i is a Transformer block, i∈{1,...,L}.

[0046] Assume the number of fine-tuning rounds is τ, where The Transformer layer of the model is evenly divided into τ parts, and P tRepresents, where t∈{1,...,τ}, each part has Transformer blocks. The divided parts are numbered from low to high, and the model can be expressed as:

[0047]

[0048] like Figure 2 As shown, P1 contains two Transformer blocks, B1 and B2, P2 contains two Transformer blocks, B3 and B4, and so on.

[0049] The second step is to combine the τ parts obtained by division. Combining the τ parts obtained by division, we get τ overlapping but gradually shrinking stages, and use S t Represented by , where t∈{1,...,τ}. Overlap means that each stage has the same parts and overlaps with each other; gradually shrinking means that the number of Transformer blocks contained in each stage gradually decreases. Each stage consists of the corresponding index and all subsequent parts plus the classification layer H. The stage at this time can be expressed as:

[0050]

[0051] For example, the first stage S1 consists of the part P1 with index 1 and all subsequent parts and the classification layer, that is, it contains all τ parts; the second stage S2 consists of the part P2 with index 2 and all subsequent parts and the classification layer, which is equivalent to the first stage without the first part P1, such as Figure 3 shown.

[0052] Finally, in the i-th round of training, only the parameters of the i-th stage and the embedding layer E are updated. In this way, the Transformer blocks with high indexes can get more training, while the Transformer blocks with low indexes only need a small amount of training.

[0053] In addition, this method is highly scalable and can be perfectly combined with efficient parameter fine-tuning methods. When combined with efficient parameter fine-tuning methods, it only needs to change the divided units into trainable parameters, such as dividing the adapter module when combined with adapter fine-tuning.

[0054] Experimental verification:

[0055] In order to prove the innovation and practicality of this patent, a large number of experiments were carried out on the proposed method, and it was combined with an efficient parameter fine-tuning method to verify its scalability.

[0056] 1) Model and dataset:

[0057] BERT was used in the experiment BASE, BERT LARGE 、RoBERTa BASE Three model architectures, the model architecture is the same as stated in the original paper, BERT LARGE The performance on some datasets is not very stable, so they are excluded in the second half of the experiments.

[0058] The dataset used in the experiment is the GLUE dataset, which mainly examines natural language comprehension capabilities. It contains nine tasks, including natural language inference, text entailment, sentiment analysis, semantic similarity, and other tasks. The specific dataset quantity and evaluation indicators are shown in Table 1.

[0059] Table 1. GLUE dataset information

[0060]

[0061] The experiment follows the previous settings and excludes the WNLI task because the BERT model does not perform well on this task.

[0062] In addition, the experiment also includes the SQuAD dataset, which is a reading comprehension dataset with two versions, 1.1 and 2.0. The answer format is text span, which means a small paragraph in the original article is used as the answer to the question. Version 1.1 contains more than 100,000 question-answer pairs for more than 500 articles. Version 2.0 expands on this basis and adds more than 50,000 unanswerable questions, making the dataset more realistic. The main evaluation indicators are EM (Exact Match) and F1 value. Exact Match is a common evaluation standard for question-answering systems. It is used to evaluate the percentage of correct answers (ground truth answers) matched in the prediction.

[0063] 2) Experimental results:

[0064] The experiments are first compared with the fully fine-tuned method, and the experimental results are shown in Figure 4 As shown in Table 2. From the results, it can be seen that the method proposed in this patent can reduce the amount of updated parameters by about 26%, because the number of parameters involved in each round of update is getting smaller and smaller. As shown in Table 3, this method improves the average performance while reducing the amount of updated parameters and training time, which is due to the reduction of overfitting of training data.

[0065] Table 2. Experimental results on the GLUE dataset

[0066]

[0067] The experimental results on the SQuAD dataset show the same trend. The model performance is improved while reducing the amount of updated parameters, as shown in Table 3, which once again proves the effectiveness of the method proposed in this invention.

[0068] Table 3. Experimental results on the SQuAD dataset

[0069]

[0070] The experiment was continued on the GLUE dataset, combining the method proposed in the present invention with the efficient parameter fine-tuning method to verify the scalability of the method. Three methods were compared in the experiment, namely adapter fine-tuning, BitFit and LoRA. Their original methods were compared with the method proposed in the present invention. The experimental results are shown in Tables 4 and 5. From the results, it can be seen that the method proposed in the present invention can significantly reduce the amount of parameter updates after being combined with the efficient parameter fine-tuning method. In adapter fine-tuning, the amount of parameter updates can even be reduced by about 67%, while the performance is only slightly reduced. We think this is acceptable.

[0071] Table 4. Parameter update amount of the efficient fine-tuning method on the GLUE dataset

[0072]

[0073] Table 5. Experimental results of efficient parameter fine-tuning method on GLUE dataset

[0074]

[0075] Another embodiment of the present invention provides a model fine-tuning system based on progressive learning, comprising:

[0076] A partitioning module is used to partition the Transformer layer of the pre-trained model, and evenly divide the Transformer layer of the pre-trained model into τ parts;

[0077] A combination module, used for combining the τ parts obtained by the division to obtain τ overlapping but gradually shrinking stages;

[0078] The training module is used to update only the parameters of the i-th stage and the embedding layer in the i-th round of training.

[0079] The specific implementation process of each module refers to the above description of the method of the present invention.

[0080] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0081] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0082] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A model fine-tuning method based on progressive learning, characterized in that: During the fine-tuning process of the pre-trained model, the Transformer blocks involved in the update are gradually reduced, the Transformer blocks that contribute greatly to the performance of the pre-trained model are trained intensively, and the Transformer blocks that contribute little to the performance of the pre-trained model are trained in small quantities.

2. The method according to claim 1, characterized in that: The fine-tuning process includes: Divide the Transformer layer of the pre-trained model into τ parts evenly; The τ parts obtained by division are combined to obtain τ overlapping but gradually shrinking stages; In the i-th round of training, only the parameters of the i-th stage and embedding layer are updated.

3. The method according to claim 2, characterized in that The Transformer layer of the pre-trained model is evenly divided into τ parts, including: Assuming a pre-trained model The model consists of an embedding layer E, L Transformer blocks, and a classification layer H. It is expressed as: Among them B i is a Transformer block, i∈{1,...,L}; Assume the number of fine-tuning rounds is τ, The Transformer layer of the model is evenly divided into τ parts, and P t Represents, where h∈{1,...,τ}, each part has Transformer blocks; The divided parts are numbered from low to high, and the model is represented as follows:

4. The method according to claim 2, characterized in that: The τ overlapping but gradually shrinking stages are represented by S t Represents, where t∈{1,...,τ}, overlap means that each stage has the same part, and shrinking means that the number of Transformer blocks contained in each stage gradually decreases.

5. The method according to claim 4, characterized in that Each stage consists of the corresponding index and all subsequent parts plus the classification layer H, and the stage is represented as: The first stage S1 consists of the part P1 with index 1 and all subsequent parts and classification layers; the second stage S2 consists of the part P1 with index 2. 12 And all subsequent parts and classification layers.

6. The method according to claim 2, characterized in that By only updating the parameters of the i-th stage and the embedding layer E in the i-th round of training, the Transformer blocks with high indexes get more training, and the Transformer blocks with low indexes only get a small amount of training.

7. The method according to claim 1, characterized in that The division units are replaced by Transformer blocks into trainable parameters, which are combined with efficient parameter fine-tuning methods to reduce computational resources.

8. A model fine-tuning system based on progressive learning, characterized in that: include: A partitioning module is used to partition the Transformer layer of the pre-trained model, and evenly divide the Transformer layer of the pre-trained model into τ parts; A combination module, used for combining the τ parts obtained by the division to obtain τ overlapping but gradually shrinking stages; The training module is used to update only the parameters of the i-th stage and the embedding layer in the i-th round of training.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images