A low-rank adaptive layer weight matrix fine-tuning method based on large speech model

Abstract

The present invention belongs to the technical field of large models for deep learning. In order to reduce the computational complexity in the process of multiplying low-rank weight matrices in low-rank adaptive methods and optimize the computing power and memory requirements required for training models, the present invention proposes a "large model low-rank adaptive layer weight matrix fine-tuning method". By copying and splicing two low-rank weight matrices and then adding them together, or dividing the two low-rank weight matrices into sub-matrices and then adding the sub-matrices one by one, compared with the method of multiplying two low-rank weight matrices, the computational complexity of the low-rank decomposition of the matrix can be reduced, and the rank of the low-rank weight matrix can also be improved. When fine-tuning for different downstream tasks, the corresponding weight parameters are trained and switched independently, which improves the training efficiency of different fine-tuning weights and reduces the weight memory requirements.

Description

Technical Field

[0001] The present invention belongs to the technical field of large models in deep learning. Background Art

[0002] As a structure using the self-attention mechanism, Transformer is widely used in various language models and vision models, and extended to other technical fields such as computer vision. The construction of large models based on Transformer has a deeper layer structure and a large number of parameters, and can be trained on massive data. It is usually applied in question-answering models and text-to-image models, showing performance far exceeding that of ordinary-scale neural network models.

[0003] The large model is mainly a general model trained using general domain data. When it is migrated and applied to specific downstream tasks, as a base model, the large model is fine-tuned using specific domain data. When fine-tuning, all parameters of the base model are updated. With the parameter scale of the large model, it has extremely high computing power and memory requirements for the computing platform. At the same time, if full-parameter fine-tuning is performed for multiple downstream tasks, it means that each specific task needs to learn a new set of parameters, and the scale of each set of parameters is the same as that of the original base model parameters, resulting in certain difficulties in training, deployment, and storage.

[0004] To address the problem of huge computational volume and memory overhead during large model fine-tuning, the prior art proposed an improved solution: the low-rank adaptation method. On the basis of fixing the weights of the pre-trained large model, the training is achieved by optimizing the low-rank decomposition weight matrix of the dense layer. During training, only the weights of the low-rank adaptation layer are adjusted, and the weights of the pre-trained large model and the low-rank adaptation layer are merged during deployment.

[0005] The pre-trained large model is constructed based on the Transformer structure and has a deep network structure. As Figure 1 shown, the weight matrix of the pre-trained large model is represented as The weight matrix of the low-rank adaptation layer is represented as It contains two low-rank weight matrices A and B, where r represents the rank of W₁, r << min(m, n); the low-rank weight matrix A is initialized with random Gaussian values, and the low-rank weight matrix B is initialized as a zero matrix. <00照00020>In the existing low-rank adaptation method, the weight matrix of the low-rank adaptation layer is represented by the product of two low-rank weight matrices, that is, W₁ = B × A. The above method of multiplying matrices combines the low-rank weight matrices, increasing the computational complexity, and the computing power and memory requirements for training the model are still very high. Summary of the Invention<00000照2>

[0007] In order to reduce the computational complexity of the low-rank weight matrix multiplication process in the low-rank adaptive method and optimize the computing power and memory requirements required for the training model, the present invention proposes a low-rank adaptive layer weight matrix fine-tuning method based on a large speech model.

[0008] A method for fine-tuning a low-rank adaptive layer weight matrix based on a large speech model comprises the following steps:

[0009] Step 1: Fix the weights of the pre-trained large model and initialize the weights of the low-rank adaptive layer: Pre-trained large model weight matrix Low-rank adaptive layer weight matrix It contains two low-rank weight matrices A and B, where the matrix matrix m is the number of rows of the low-rank adaptive layer weight matrix, n is the number of columns of the low-rank adaptive layer weight matrix, r represents the rank of W1, r<<min(m,n), the matrix A is initialized with a random Gaussian, and the matrix B is initialized to a zero matrix;

[0010] Step 2: Use speech recognition data to train and update the parameters of the low-rank decomposition weight matrix;

[0011] Step 3: Merge the two low-rank weight matrices A and B in step 1 into a low-rank adaptive layer weight matrix;

[0012] Step 4: Merge the pre-trained large model weights and the low-rank adaptive layer weights and deploy the model for different downstream tasks;

[0013] The merging of the two low-rank weight matrices A and B in step three includes copying and splicing the matrix A and the matrix B respectively and then adding them together, or dividing the matrix A and the matrix B into sub-matrices respectively and then adding the sub-matrices one by one.

[0014] Technical effect:

[0015] 1. By copying two low-rank weight matrices in Example 1 and then splicing them into a low-rank adaptive layer weight matrix, compared to the method of multiplying two low-rank weight matrices, the computational complexity of matrix low-rank decomposition can be reduced, and the rank of the low-rank weight matrix can also be improved;

[0016] 2. By dividing the two low-rank weight matrices into sub-matrices and reusing features in Example 2, the computational complexity of matrix low-rank decomposition is reduced, and the technical problem of high video memory consumption during large model training is further reduced;

[0017] 3. When fine-tuning for different downstream tasks, independently training and switching the corresponding weight parameters can improve the training efficiency of different fine-tuning weights and reduce the weight memory requirements.

[0018] In the existing low-rank adaptive method, W1=B×A, the fine-tuning parameter amount is mr+rn, the computational complexity is rmn, and the rank of W1 is rank(W1)≤min(rank(B),rank(A)).

[0019] Compared with the existing low-rank adaptive method, the fine-tuning parameter amount in Example 1 and Example 2 of the present invention is mr+rn, the computational complexity is r(m+n), the rank of W'1 rank(W'1)≤rank(B)+rank(A), and the rank of W″1 rank(W″1)≤rank(B)+rank(A). When the parameter amount remains unchanged, compared with the existing low-rank adaptive method, the computational complexity is reduced from rmn to r(m+n), and the rank of the low-rank weight matrix is ​​improved, so that the feature expression of the model has richer information carrying capacity.

[0020] In Example 2 of the present invention, two low-rank weight matrices are divided into sub-matrices and features are reused. Compared with Example 1, this can alleviate the technical problems of excessive memory consumption and frequent I / O reading and writing during model training, thereby effectively improving the efficiency of model training. On an NVIDIA A100 graphics card with a single video memory of 40GB, the ChatGLM model with 6 billion parameters is fine-tuned using specific domain data, and the training time efficiency is improved by 10 times compared with Example 1.

[0021] In summary, the present invention can reduce the computational complexity of multiplying the weight matrices of the low-rank adaptive layer, reduce the computing power required for training the model, reduce memory requirements, and improve the training efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the multiplication process of two low-rank weight matrices in the prior art.

[0023] Figure 2 It is a schematic diagram of the process of the low-rank adaptive layer weight matrix fine-tuning method in Example 1 of the present invention, and serves as an abstract illustration.

[0024] Figure 3 This is a schematic diagram of the process of the low-rank adaptive layer weight matrix fine-tuning method according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work by adopting the embodiments of the present invention are within the scope of protection of the present invention.

[0026] In step 3, the two low-rank weight matrices A and B are merged by copying and splicing the matrix A and the matrix B respectively and then adding them together, specifically:

[0027] like Figure 2 As shown, in Example 1, the low-rank weight matrix B is copied as group, and then splice in the column direction The group low-rank weight matrix B, the concatenated weight matrix is ​​recorded as

[0028] Then copy the low-rank weight matrix A as Group, then splice in the row direction The group low-rank weight matrix A, the concatenated weight matrix is ​​recorded as

[0029] The result of adding the two concatenated weight matrices B' and A' is used as the weight matrix of the low-rank adaptive layer, that is, W1=W'1=B'+A'.

[0030] In step 3, the two low-rank weight matrices A and B are merged by dividing the matrix A and the matrix B into sub-matrices and then adding the sub-matrices one by one, specifically:

[0031] like Figure 3 As shown, in Example 2, the low-rank weight matrix B is divided into sub-matrices, represented as Each sub-matrix is ​​represented as in Divide the low-rank weight matrix A into sub-matrices, represented as Each sub-matrix is ​​represented as in Create a new weight matrix Perform matrix block operation on W″1 and divide it into intervals, divided into intervals, divided into sub-matrices, each sub-matrix is ​​represented as The value of each submatrix is ​​the result of adding the corresponding submatrix of weight matrix B to the submatrix of weight matrix A, expressed as c ij =b i +a j , thereby completing the weight update of the weight matrix W″1 as the low-rank adaptive layer weight matrix, that is, W1=W″1.

[0032] The specific steps in step four are: merging the fixed pre-trained large model weights and the low-rank adaptive layer weights trained using speech recognition data as the low-rank matrix fine-tuning weights of the large model, that is, W=W0+W1. The model inference process is expressed as h=W0x+W1x, where x represents input data and h represents output data. When deploying the model for different downstream tasks, it is only necessary to replace the corresponding low-rank adaptive layer weight matrix and train the low-rank decomposition matrix of the low-rank adaptive layer weight matrix.

[0033] For example, if Task 1 is a question-answering model in the field of speech recognition, and Task 2 is a question-answering model in the medical field, the weight W0 of the pre-trained large model is fixed, and only the corresponding W1 needs to be trained and replaced using speech recognition field data or medical field data; this saves training resources and training time, reduces the number of model parameters and computational complexity, reduces the model's memory usage and computing requirements, enables model training to be carried out more efficiently, and is suitable for resource-constrained devices or scenarios, thereby improving the model's generalization capabilities.

[0034] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the scope of protection of the present invention.

Claims

1. A method for fine-tuning a low-rank adaptive layer weight matrix based on a large speech model, comprising the following steps: Step 1: Fix the weights of the pre-trained large model and initialize the weights of the low-rank adaptive layer: Pre-trained large model weight matrix Low-rank adaptive layer weight matrix It contains two low-rank weight matrices A and B, where the matrix matrix m is the number of rows of the low-rank adaptive layer weight matrix, n is the number of columns of the low-rank adaptive layer weight matrix, r represents the rank of W1, r<<min(m,n), the matrix A is initialized with a random Gaussian, and the matrix B is initialized to a zero matrix; Step 2: Use speech recognition data to train and update the parameters of the low-rank decomposition weight matrix; Step 3: Merge the two low-rank weight matrices A and B in step 1 into a low-rank adaptive layer weight matrix; Step 4: Merge the pre-trained large model weights and the low-rank adaptive layer weights and deploy the model for different downstream tasks; It is characterized in that In the step 3, the two low-rank weight matrices A and B are merged by copying and splicing the matrix A and the matrix B respectively and then adding them together, or dividing the matrix A and the matrix B into sub-matrices and then adding the sub-matrices one by one; Copy and concatenate matrix A and matrix B separately and then add them together. Specifically: Copy the low-rank weight matrix B as group, and then splice in the column direction The group low-rank weight matrix B, the concatenated weight matrix is ​​recorded as Then copy the low-rank weight matrix A as Group, then splice in the row direction The group low-rank weight matrix A, the concatenated weight matrix is ​​recorded as The sum of the two concatenated weight matrices B' and A' is used as the weight matrix of the low-rank adaptive layer, that is, W1 = W'1 = B' + A'; Alternatively, divide matrix A and matrix B into sub-matrices and add the sub-matrices one by one, as follows: Divide the low-rank weight matrix B into sub-matrices, represented as Each sub-matrix is ​​represented as in Divide the low-rank weight matrix A into sub-matrices, represented as Each sub-matrix is ​​represented as in Create a new weight matrix Perform matrix block operation on W″1 and divide it into intervals, divided into intervals, divided into sub-matrices, each sub-matrix is ​​represented as The value of each submatrix is ​​the result of adding the corresponding submatrix of matrix B to the submatrix of matrix A, expressed as c ij =b i +a j , thereby completing the weight update of the weight matrix W″1 as the low-rank adaptive layer weight matrix, that is, W1=W″1.

2. A method for fine-tuning the low-rank adaptive layer weight matrix based on a large speech model according to claim 1, characterized in that In the step 4, the pre-trained large model weights and the low-rank adaptive layer weights are merged, specifically: the fixed pre-trained large model weights and the low-rank adaptive layer weights trained using speech recognition data are merged as the low-rank matrix fine-tuning weights of the large model, that is, W=W0+W1, and the model inference process is expressed as h=W0x+W1x, where x represents input data and h represents output data. When deploying the model for different downstream tasks, it is only necessary to replace the corresponding low-rank adaptive layer weight matrix and train the low-rank decomposition matrix of the low-rank adaptive layer weight matrix.

Images