Large-model low-rank adaptive layer weight matrix fine tuning method

Abstract

The invention belongs to the technical field of large models of deep learning, and provides a large-model low-rank adaptive layer weight matrix fine tuning method in order to reduce calculation complexity in a low-rank weight matrix multiplication process in a low-rank adaptive method and optimize calculation power and memory requirements required by a training model. The two low-rank weight matrixes are copied, spliced and then added, or the two low-rank weight matrixes are divided into the sub-matrixes and then the sub-matrixes are added one by one, so that compared with a mode of multiplying the two low-rank weight matrixes, the calculation complexity of matrix low-rank decomposition can be reduced, meanwhile, the rank of the low-rank weight matrixes can be improved, and the calculation efficiency of matrix low-rank decomposition is improved. When different downstream tasks are finely adjusted, the corresponding weight parameters are independently trained and switched, so that the training efficiency of different fine adjustment weights is improved, and the weight memory requirement is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of large models for deep learning. Background Art

[0002] As a structure that uses the self-attention mechanism, Transformer is widely used in various language models and visual models, and has been extended to other technical fields such as computer vision. Large models based on Transformer have deeper layer structures and large-scale parameters, and can be trained on massive data. They are usually used in question-answering models and text-graph models, and have shown performance far exceeding that of ordinary-scale neural network models.

[0003] The large model is mainly a general model trained with general domain data. When it is migrated and applied to specific downstream tasks, the large model is used as a base model and fine-tuned using specific domain data. During fine-tuning, all parameters of the base model are updated. Due to the parameter scale of the large model, the computing platform has extremely high computing power and memory requirements. At the same time, if all parameters are fine-tuned for multiple downstream tasks, it means that each specific task needs to learn a new set of parameters. The scale of each set of parameters is consistent with the scale of the original base model parameters, which leads to certain difficulties in training, deployment and storage.

[0004] In response to the huge amount of computation and memory overhead when fine-tuning large models, the existing technology proposes an improvement plan: the low-rank adaptive method, which achieves the purpose of training by optimizing the low-rank decomposition weight matrix of the dense layer on the basis of fixing the pre-trained large model weights. During the training process, only the weights of the low-rank adaptive layer are adjusted, and the pre-trained large model weights and the low-rank adaptive layer weights are merged during deployment.

[0005] The pre-trained large model is built based on the Transformer structure and has a deep network structure, such as Figure 1 As shown, the weight matrix of the pre-trained large model is expressed as The low-rank adaptive layer weight matrix is ​​expressed as It contains two low-rank weight matrices A and B, where r represents the rank of W1, r<<min(m,n); the low-rank weight matrix A is initialized with a random Gaussian, and the low-rank weight matrix B is initialized with a zero matrix.

[0006] In the existing low-rank adaptive method, the low-rank adaptive layer weight matrix is ​​represented by the product of two low-rank weight matrices, that is, W1=B×A. The above matrix multiplication method combines the low-rank weight matrix, which increases the computational complexity, and the computing power and memory requirements required for training the model are still very high. Summary of the invention

[0007] In order to reduce the computational complexity of the low-rank weight matrix multiplication process in the low-rank adaptive method and optimize the computing power and memory requirements required for the training model, the present invention proposes a "large model low-rank adaptive layer weight matrix fine-tuning method".

[0008] A method for fine-tuning the weight matrix of a large model low-rank adaptive layer includes the following steps:

[0009] Step 1: Fix the weights of the pre-trained large model and initialize the weights of the low-rank adaptive layer: Pre-trained large model weight matrix Low-rank adaptive layer weight matrix It contains two low-rank weight matrices A and B, where the matrix matrix m is the number of rows of the low-rank adaptive layer weight matrix, n is the number of columns of the low-rank adaptive layer weight matrix, r represents the rank of W1, r<<min(m,n), the matrix A is initialized with a random Gaussian, and the matrix B is initialized to a zero matrix;

[0010] Step 2: Use domain-specific data to train and update the parameters of the low-rank decomposition weight matrix;

[0011] Step 3: Merge the two low-rank weight matrices A and B in step 1 into a low-rank adaptive layer weight matrix;

[0012] Step 4: Merge the pre-trained large model weights and the low-rank adaptive layer weights, and deploy the model for different downstream tasks;

[0013] The merging of the two low-rank weight matrices A and B in step three includes copying and splicing the matrix A and the matrix B respectively and then adding them together, or dividing the matrix A and the matrix B respectively into sub-matrices and then adding the sub-matrices one by one.

[0014] Technical effects:

[0015] 1. By copying two low-rank weight matrices in Example 1 and then splicing them into a low-rank adaptive layer weight matrix, compared with the method of multiplying two low-rank weight matrices, the computational complexity of matrix low-rank decomposition can be reduced, and the rank of the low-rank weight matrix can also be improved;

[0016] 2. By dividing the two low-rank weight matrices into sub-matrices and reusing features in Example 2, the computational complexity of low-rank decomposition of the matrix is ​​reduced, and the technical problem of high video memory consumption during large model training is further reduced;

[0017] 3. When fine-tuning for different downstream tasks, independently training and switching the corresponding weight parameters can improve the training efficiency of different fine-tuning weights and reduce the weight memory requirements.

[0018] In the existing low-rank adaptive method, W1=B×A, the amount of fine-tuning parameters is mr+rn, the computational complexity is rmn, and the rank of W1 is rank(W1)≤min(rank(B),rank(A)).

[0019] Compared with the existing low-rank adaptive method, the fine-tuning parameter amount in Example 1 and Example 2 of the present invention is mr+rn, the computational complexity is r(m+n), the rank of W'1 rank(W'1)≤rank(B)+rank(A), the rank of W″1 rank(W″1)≤rank(B)+rank(A), when the parameter amount remains unchanged, compared with the existing low-rank adaptive method, the computational complexity is reduced from rmn to r(m+n), and the rank of the low-rank weight matrix is ​​improved, so that the feature expression of the model has richer information carrying capacity.

[0020] In Example 2 of the present invention, two low-rank weight matrices are divided into sub-matrices and features are reused. Compared with Example 1, it can alleviate the technical problems of excessive memory consumption and frequent I / O reading and writing in the model training process, thereby effectively improving the efficiency of model training. On an NVIDIA A100 graphics card with a single video memory of 40GB, the ChatGLM model with 6 billion parameters is fine-tuned using specific domain data, and the training time efficiency is improved by 10 times compared with Example 1.

[0021] In summary, the present invention can reduce the computational complexity of multiplying the weight matrices of the low-rank adaptive layer, reduce the computing power required for training the model, reduce memory requirements, and improve the training efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the multiplication process of two low-rank weight matrices in the prior art.

[0023] Figure 2 It is a schematic diagram of the process of the low-rank adaptive layer weight matrix fine-tuning method in Example 1 of the present invention, and serves as a summary illustration.

[0024] Figure 3 It is a schematic diagram of the process of the low-rank adaptive layer weight matrix fine-tuning method according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in the field without creative work by adopting the embodiments of the present invention are within the scope of protection of the present invention.

[0026] The merging of the two low-rank weight matrices A and B in step 3 includes copying and splicing the matrix A and the matrix B respectively and then adding them together, specifically:

[0027] like Figure 2 As shown, in Example 1, the low-rank weight matrix B is copied as Group, then splice in the column direction The concatenated weight matrix B is denoted as B' = [B, B, ..., B].

[0028] Then copy the low-rank weight matrix A as Group, then splice in the row direction The group low-rank weight matrix A, the concatenated weight matrix is ​​recorded as

[0029] The result of adding the two concatenated weight matrices B' and A' is used as the weight matrix of the low-rank adaptive layer, that is, W1 = W'1 = B' + A'.

[0030] The merging of the two low-rank weight matrices A and B in step 3 includes dividing the matrix A and the matrix B into sub-matrices and then adding the sub-matrices one by one, specifically:

[0031] like Figure 3 As shown, in Example 2, the low-rank weight matrix B is divided into sub-matrices, represented as Each sub-matrix is ​​represented as in Divide the low-rank weight matrix A into sub-matrices, represented as Each sub-matrix is ​​represented as in New weight matrix Perform matrix block operation on W″1 and divide it into intervals, divided into intervals, divided into sub-matrices, each of which is represented by The value of each submatrix is ​​the result of adding the corresponding submatrix of weight matrix B to the submatrix of weight matrix A, expressed as c ij =b i +a j , thereby completing the weight update of the weight matrix W″1 as the low-rank adaptive layer weight matrix, that is, W1=W″1.

[0032] Specifically, step four is as follows: merging the fixed pre-trained large model weights and the low-rank adaptive layer weights trained using a specific domain as the low-rank matrix fine-tuning weights of the large model, i.e., W=W0+W1. The model inference process is expressed as h=W0x+W1x, where x represents input data and h represents output data. When the model is deployed for different downstream tasks, it is only necessary to replace the corresponding low-rank adaptive layer weight matrix and train the low-rank decomposition matrix of the low-rank adaptive layer weight matrix.

[0033] For example, Task 1 is a question-answering model in the field of speech recognition, and Task 2 is a question-answering model in the medical field. The weight W0 of the pre-trained large model is fixed, and only the corresponding W1 needs to be trained and replaced using speech recognition field data or medical field data. This can save training resources and time, reduce the number of model parameters and computational complexity, reduce the model's memory usage and computing requirements, make model training more efficient, and be suitable for resource-constrained devices or scenarios, thereby improving the generalization ability of the model.

[0034] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

Claims

1. A method for fine-tuning the weight matrix of a large model low-rank adaptive layer includes the following steps: Step 1: Fix the weights of the pre-trained large model and initialize the weights of the low-rank adaptive layer: Pre-trained large model weight matrix Low-rank adaptive layer weight matrix It contains two low-rank weight matrices A and B, where the matrix matrix m is the number of rows of the low-rank adaptive layer weight matrix, n is the number of columns of the low-rank adaptive layer weight matrix, r represents the rank of W1, r<<min(m,n), the matrix A is initialized with a random Gaussian, and the matrix B is initialized to a zero matrix; Step 2: Use domain-specific data to train and update the parameters of the low-rank decomposition weight matrix; Step 3: Merge the two low-rank weight matrices A and B in step 1 into a low-rank adaptive layer weight matrix; Step 4: Merge the pre-trained large model weights and the low-rank adaptive layer weights, and deploy the model for different downstream tasks; Features: In the step three, the merging of the two low-rank weight matrices A and B includes copying and splicing the matrix A and the matrix B respectively and then adding them together, or dividing the matrix A and the matrix B respectively into sub-matrices and then adding the sub-matrices one by one.

2. The large model low-rank adaptive layer weight matrix fine-tuning method according to claim 1 is characterized in that: The merging of the two low-rank weight matrices A and B in step 3 includes copying and splicing the matrix A and the matrix B respectively and then adding them together, specifically: Copy the low-rank weight matrix B as Group, then splice in the column direction The concatenated weight matrix B is denoted as B' = [B, B, ..., B]. Then copy the low-rank weight matrix A as Group, then splice in the row direction The group low-rank weight matrix A, the concatenated weight matrix is ​​recorded as The result of adding the two concatenated weight matrices B' and A' is used as the weight matrix of the low-rank adaptive layer, that is, "' W1=W1=B+A.

3. The large model low-rank adaptive layer weight matrix fine-tuning method according to claim 1 is characterized in that: In the step 3, the two low-rank weight matrices A and B are merged by dividing the matrix A and the matrix B into sub-matrices and then adding the sub-matrices one by one, specifically: Divide the low-rank weight matrix B into sub-matrices, represented as Each sub-matrix is ​​represented as in Divide the low-rank weight matrix A into sub-matrices, represented as Each sub-matrix is ​​represented as in New weight matrix Perform matrix block operation on W″1 and divide it into intervals, divided into intervals, divided into sub-matrices, each of which is represented by The value of each submatrix is ​​the result of adding the corresponding submatrix of the matrix B to the submatrix of the matrix A, expressed as c ij =b i +a j , thereby completing the weight update of the weight matrix W″1 as the low-rank adaptive layer weight matrix, that is, W1=W″1.

4. The large model low-rank adaptive layer weight matrix fine-tuning method according to claim 1 is characterized in that: The step 4 combines the pre-trained large model weights and the low-rank adaptive layer weights, specifically: combining the fixed pre-trained large model weights and the low-rank adaptive layer weights trained using a specific domain as the low-rank matrix fine-tuning weights of the large model, i.e., W=W0+W1. The model inference process is expressed as h=W0x+W1x, where x represents input data and h represents output data. When the model is deployed for different downstream tasks, it is only necessary to replace the corresponding low-rank adaptive layer weight matrix and train the low-rank decomposition matrix of the low-rank adaptive layer weight matrix.

Images