Triangular decomposition-based high-efficiency fine tuning method for low-rank adaptive parameters of pre-training model

Abstract

The invention discloses a pre-training model low-rank adaptive parameter high-efficiency fine tuning method based on triangular decomposition, which comprises the following steps: for a pre-training large model, selecting levels needing parameter fine tuning from the large model according to actual needs, and respectively performing triangular decomposition on a weight matrix of each layer to be subjected to fine tuning, and taking the obtained scaling factor and the diagonal sub-matrix as trainable parameters, freezing other model parameters, inputting an input sample in the training sample set into the large model to update the trainable parameters, and reconstructing by adopting the updated trainable parameters to obtain a weight matrix, thereby completing efficient fine tuning of the pre-trained large model parameters. According to the method, the number of trainable parameters is remarkably reduced through three-solution decomposition, meanwhile, an orthogonal constraint mechanism is introduced, pre-training knowledge is kept, and meanwhile the performance level close to full-amount adjustment and optimization is achieved. The method not only reduces the consumption of computing resources, but also improves the optimization speed of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of large models. More specifically, it relates to a method for efficiently fine-tuning the parameters of a pre-trained model with low-rank adaptation based on triangular decomposition. Background Art

[0002] A large model refers to a machine learning model with a large number of parameters and a complex computational structure. With the rapid growth of the model scale, the need for efficiently fine-tuning large models on downstream data has become increasingly urgent. However, the computational resources required to fine-tune large models are huge, presenting significant challenges to research and applications. To address this issue, researchers have proposed various parameter-efficient fine-tuning techniques to reduce computational overhead while maintaining model performance improvement. Existing parameter-efficient tuning techniques are mainly divided into the following two categories:

[0003] The first category is the fine-tuning method based on low-rank decomposition. This type of method approximates the update process of the model's incremental weights by introducing the product of low-rank matrices, significantly reducing the number of trainable parameters. Among them, the most representative method is to approximately achieve efficient update of the incremental weight matrix by randomly initializing two low-rank matrices and optimizing them. This method not only improves the model's performance on downstream tasks but also greatly reduces computational resource consumption.

[0004] The second category is the fine-tuning method based on vector transformation. This type of method uses a fixed randomly initialized matrix and only trains the vector parameters for linear transformation. By reducing the scale of parameters to be optimized, the computational overhead is further reduced. At the same time, some improved methods effectively alleviate the model's memory of training data by reconstructing the input dimension and introducing a square matrix transformation layer, overcoming the limitations of traditional low-rank methods in terms of data overfitting.

[0005] The proposal of these technical solutions provides a feasible technical path for the efficient fine-tuning of large models, effectively solving the practical problem of limited computational resources and greatly promoting the wide application of large models in various fields.

[0006] However, the existing parameter-efficient fine-tuning methods for large models face many challenges in practical applications: (1) Compared with full-parameter fine-tuning, there is a significant performance gap, making it difficult to meet the requirements of high-precision tasks; (2) Although the number of fine-tuning parameters is reduced, the optimization overhead is still large, especially when dealing with large models; (3) Existing methods are difficult to effectively maintain the pre-trained knowledge of the model, affecting its generalization ability.

[0007] In addition, the existing parameter-efficient fine-tuning methods generally adopt a random initialization strategy, which has the following problems: (1) Random initialization leads to slow optimization speed and increases the computing power cost of fine-tuning; (2) Due to the limitations of the initialization method, the model may fall into a local optimal solution during training, reducing the optimization effect; (3) Random initialization is difficult to make full use of the knowledge in the weights of the original pre-trained model, resulting in limited model expression ability. Summary of the Invention

[0008] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a low-rank adaptive parameter-efficient fine-tuning method for pre-trained models based on triangular decomposition. By performing triangular matrix decomposition on the weight matrix of the layer to be fine-tuned, the number of trainable parameters is reduced, and at the same time, an orthogonal constraint mechanism is introduced to effectively improve the convergence performance and optimization effect of the model.

[0009] To achieve the above-mentioned invention purpose, the low-rank adaptive parameter-efficient fine-tuning method for pre-trained models based on triangular decomposition of the present invention includes the following steps:

[0010] S1: Obtain a training sample set for parameter fine-tuning according to actual needs. Each training sample includes an input sample and a label;

[0011] S2: For the pre-trained large model, select the layers that need to be fine-tuned from the large model according to actual needs. Denote the weight matrix of the layer to be fine-tuned as where k = 1, 2,..., K, K represents the number of layers to be fine-tuned, and M k ×N k represents the shape of the weight matrix of the layer to be fine-tuned;

[0012] S3: Perform layer weight decomposition based on triangular decomposition, which specifically includes the following steps:

[0013] S3.1: Use the LDU decomposition method to decompose the weight matrix W k of each layer to be fine-tuned to obtain W k = P k L k U k , where P k represents a permutation matrix, L k represents a lower triangular matrix, and U k represents an upper triangular matrix;

[0014] Extract the diagonal matrix D k of the upper triangular matrix U k , and then use the following formula to normalize the upper triangular matrix U k to obtain the normalized upper triangular matrix

[0015]

[0016] S3.2: Extract the lower triangular matrix L k 's first r k columns to obtain a lower triangular submatrix r k represents a preset rank; extract the diagonal matrix D k with the upper left corner size of r k ×r k submatrix to obtain a diagonal submatrix Extract the normalized upper triangular matrix 's first r k rows to obtain an upper triangular submatrix

[0017] S3.3: Preset the scaling factor α according to actual needs, and then calculate the scaling factor σ using the following formula k :

[0018] σ k = α / r k

[0019] Calculate the low-rank approximation matrix W using the following formula k,approx :

[0020]

[0021] Then calculate the residual matrix W using the following formula k,residual :

[0022] W k,residual = W k -W k,approx

[0023] S4: Freeze the permutation matrix P k corresponding to the weight matrix of each layer to be fine-tuned, the lower triangular submatrix the upper triangular submatrix and all the parameters of other layers in the large model, set the trainable parameter ρ k and set its initial value to σ k , set the trainable parameter and perform heuristic initialization on it according to the diagonal submatrix ;

[0024] S5: Input the input samples in the training sample set into the large model, calculate the loss function and update the trainable parameters ρ k and of each layer to be fine-tuned respectively to obtain the updated trainable parameters and

[0025] S6: According to the updated trainable parameters and Calculate the updated model weights for each layer to be fine-tuned

[0026]

[0027] S7: Determine whether the fine-tuning end condition is reached. If not, go to step S8; otherwise, the fine-tuning ends.

[0028] S8: Let the weight matrix Trainable parameters Return to step S5.

[0029] The efficient low-rank adaptive parameter fine-tuning method for pre-trained models based on triangular decomposition in the present invention selects the layers that need to be fine-tuned from the large pre-trained model according to actual needs, performs triangular decomposition on the weight matrix of each layer to be fine-tuned respectively, uses the obtained scaling factor and diagonal sub-matrix as trainable parameters, freezes other model parameters, inputs the input samples in the training sample set into the large model to update the trainable parameters, and reconstructs the weight matrix using the updated trainable parameters, thereby completing the efficient fine-tuning of the parameters of the large pre-trained model.

[0030] The present invention significantly reduces the number of trainable parameters through triangular matrix decomposition, and at the same time introduces an orthogonal constraint mechanism, achieving a performance level close to full-scale tuning while maintaining pre-trained knowledge. This method not only reduces the consumption of computing resources but also improves the optimization speed of the model. Description of the Drawings

[0031] Figure 1 is the flowchart of the specific implementation of the efficient low-rank adaptive parameter fine-tuning method for pre-trained models based on triangular decomposition in the present invention;

[0032] Figure 2 is the flowchart of the layer weight decomposition by triangular decomposition in the present invention;

[0033] Figure 3 is the comparison chart of the trainable parameters between the LoRA method and the present invention;

[0034] Figure 4 is the comparison chart of the change curves of the image classification accuracy with the number of trainable parameters after parameter fine-tuning between the present invention and the full-parameter fine-tuning method in this embodiment. Specific Embodiments

[0035] The following describes the specific embodiments of the present invention with reference to the drawings, so that those skilled in the art can better understand the present invention. It should be particularly noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0036] Embodiment

[0037] Figure 1 is the flowchart of the specific implementation of the method for efficiently fine-tuning the low-rank adaptive parameters of the pre-trained model based on triangular decomposition of the present invention. As Figure 1 shown, the specific steps of the method for efficiently fine-tuning the low-rank adaptive parameters of the pre-trained model based on triangular decomposition of the present invention include:

[0038] S101: Obtain the training sample set for efficient parameter fine-tuning:

[0039] Obtain the training sample set for efficient parameter fine-tuning according to actual needs. Each training sample includes an input sample and a label.

[0040] S102: Obtain the weight matrix to be fine-tuned:

[0041] For the pre-trained large model, select the layers that need to be efficiently fine-tuned from the large model according to actual needs. Denote the weight matrix of the layer to be fine-tuned as where k = 1, 2,..., K, K represents the number of layers to be fine-tuned, and M k ×N k represents the shape of the weight matrix of the layer to be fine-tuned. Generally, it is default to perform efficient parameter fine-tuning on all linear layers.

[0042] S103: Perform layer weight decomposition based on triangular decomposition:

[0043] To optimize the low-rank adaptation parameter fine-tuning, the present invention performs layer weight decomposition based on triangular decomposition. Figure 2 is the flowchart of the layer weight decomposition by triangular decomposition in the present invention. As Figure 2 shown, the specific steps of the layer weight decomposition by triangular decomposition in the present invention include:

[0044] S201: Triangular decomposition:

[0045] Use the LDU decomposition method to decompose the weight matrix W k of each layer to be fine-tuned to obtain W k = P k L k U k , where P k represents the permutation matrix, L k represents the lower triangular matrix, and U k represents the upper triangular matrix. LDU is a commonly used matrix decomposition method, and its specific process will not be elaborated here.

[0046] Extract the upper triangular matrix U k of the diagonal matrix D k , and then use the following formula to normalize the upper triangular matrix U k to obtain the normalized upper triangular matrix

[0047]

[0048] S202: Low-rank approximation:

[0049] Extract the first r k columns of the lower triangular matrix L k to obtain the lower triangular submatrix r k represents the preset rank. Extract the upper left submatrix of size r k ×r k in the diagonal matrix D k to obtain the diagonal submatrix Extract the first r rows of the normalized upper triangular matrix k to obtain the upper triangular submatrix

[0050] S203: Calculate the scaling factor and the residual matrix:

[0051] Pre-set the scaling coefficient α according to actual needs, and then calculate the scaling factor σ using the following formula k :

[0052] σ k =α / r k

[0053] Calculate the low-rank approximation matrix W using the following formula k,approx :

[0054]

[0055] Then calculate the residual matrix W using the following formula k,residual :

[0056] W k,residual =W k -W k,approx

[0057] S104: Trainable parameter setting and heuristic initialization:

[0058] In the present invention, in order to reduce the number of trainable parameters in model fine-tuning, the permutation matrix P corresponding to the weight matrix of each layer to be fine-tuned k , the lower triangular submatrix the upper triangular submatrix Freeze all the parameters of the large model and other layers, and set the trainable parameter ρ k and set its initial value to σ k , set the trainable parameter and perform heuristic initialization on it according to the diagonal submatrix . In this embodiment, the specific method of heuristic initialization of the trainable parameter is as follows: calculate the mean of the diagonal elements in the diagonal submatrix , or sample using a random uniform distribution or normal distribution, and then set the diagonal elements of the trainable parameter according to the calculation result or sampling result, and set the non-diagonal elements of the trainable parameter to 0, thus completing the initialization.

[0059] Figure 3 is a comparison chart of the trainable parameters of the LoRA method and the present invention. As Figure 3 shown, in the LoRA (Low-Rank Adaption) method, the two trainable low-rank matrices are A and B, and the weight change ΔW = BA. For each pre-trained weight matrix W, the number of trainable parameters in LoRA is r×(d in +d out ), where d in , d out respectively represent the input dimension and output dimension of the weight matrix W. In the present invention, the number of trainable parameters is the diagonal length of the diagonal submatrix plus a scaling factor, that is, r k +1. Since r k +1 is necessarily less than r×(d in +d out ), therefore, when fine-tuning the same number of layers, the number of training parameters of the present invention is much smaller than that of LoRA.

[0060] S105: Fine-tuning training:

[0061] Input the input samples in the training sample set into the large model, calculate the loss function, and update the trainable parameters ρ k and of each layer to be fine-tuned respectively, and obtain the updated trainable parameters and

[0062] S106: Update the large model parameters:

[0063] According to the updated trainable parameters and calculate the updated model weights of each layer to be fine-tuned

[0064]

[0065] S107: Determine whether the fine-tuning end condition is reached. If not, proceed to step S108; otherwise, the fine-tuning ends. The fine-tuning end condition can be set according to actual needs and usually includes two types: the maximum number of iterations and the convergence of model parameters.

[0066] S108: Update the fine-tuning parameters:

[0067] Let the weight matrix Trainable parameters Return to step S105.

[0068] Next, from a theoretical perspective, analyze the technical advantages of the present invention compared with other existing technologies. First, the present invention uses LDU decomposition to decompose the weight matrix of the layer to be fine-tuned, thereby setting the trainable parameters and initializing them, rather than using random initialization. This method has the following advantages:

[0069] 1) Orthogonality preservation: The lower triangular matrix and the upper triangular matrix in LDU decomposition naturally have orthogonal properties, which helps to preserve the pre-trained knowledge and enhance the generalization ability.

[0070] 2) Faster convergence: By initializing the trainable parameters using the LDU decomposition result, the present invention can achieve a faster convergence speed and avoid the convergence problems caused by random initialization.

[0071] In addition, the present invention only focuses on optimizing the diagonal submatrix, which also brings certain advantages:

[0072] 1) Parameter efficiency: Only O(n)-level parameters need to be optimized, which greatly reduces the number of trainable parameters compared with other PEFT methods that need to optimize O(n 2 )-level parameters.

[0073] 2) Scaling transformation: By optimizing the diagonal matrix, the present invention can achieve an effective scaling transformation and adjust the importance of features while maintaining the model structure.

[0074] The heuristic initialization method adopted by the present invention has the following characteristics:

[0075] 1) Constraining the initialization range: By restricting the range of initialization values, a more stable training process is achieved.

[0076] 2) Improving robustness: The heuristic initialization helps to improve the robustness of the model on different tasks and datasets.

[0077] The present invention can reduce the trainable parameters to 0.00025% of the original model while maintaining non-degrading performance. This breakthrough parameter reduction brings the following advantages: (1) Significantly reduce computational resource requirements: Greatly reduce the computational resources required for training, enabling effective model fine-tuning in resource-constrained environments. (2) Reduce storage overhead: Greatly reduce the storage requirements for model checkpoints, reducing storage costs. (3) Improve deployment efficiency: The substantial reduction in the parameter ratio makes the model easier to deploy on various devices, including resource-constrained edge devices.

[0078] To better illustrate the technical effects of the present invention, specific examples are used to experimentally verify the present invention.

[0079] In this embodiment, the large model uses an image classification model based on ViT (Vision Transformer). The training sample set uses the FGVC dataset and the StanfordCars dataset. The input samples are images, and the labels are the categories of the images. In this embodiment, four parameter-efficient fine-tuning methods in the prior art are selected as comparison methods, including the full-parameter fine-tuning method (FT), the FourierFT (discrete Fourier transform fine-tuning) method (the number of Fourier points n is 3000 and 10000 respectively), the LoRA algorithm (rank r = 16), and the linear probing method (Linear Probing, LP, only fine-tune the classification head). Then, the full-parameter fine-tuning method (85.8M parameters) and the linear probing method are used as the upper and lower limits of performance respectively. Figure 4 It is a comparison chart of the change curve of the image classification accuracy rate with the number of trainable parameters after parameter fine-tuning of the present invention and the full-parameter fine-tuning method in this embodiment. As Figure 4 shown, the present invention (denoted as LoLDU) shows superior parameter efficiency and performance compared to the linear probing method, FourierFT (n = {3000, 10000}), LoRA (r = 16), and the full-parameter fine-tuning method at ranks r = {1, 8, 16, 32, 64, 128, 512, 768}. When the rank r = 768, the present invention can achieve a performance exceeding that of the LoRA algorithm with rank r = 16 while reducing the training parameters by 96.837%. It is particularly noteworthy that the present invention can obtain a competitive score with only 24 trainable parameters at rank r = 1, and at r = 768, the present invention achieves the highest accuracy rate among all parameter-efficient fine-tuning methods: 42.15% for the FGVC task and 66.66% for the StanfordCars task, demonstrating the scalability and effectiveness of the present invention.

[0080] In this embodiment, the accuracy rates of the present invention and the comparative method after 10 rounds of parameter fine-tuning training on different image classification datasets are counted. Table 1 is a comparison table of the accuracy rates of the present invention and the comparative method after parameter fine-tuning training on different image classification datasets in this embodiment.

[0081]

[0082] Table 1

[0083] As shown in Table 1, the FourierFT method was evaluated with different trainable parameters for each layer. Δ baseline represents the performance gap between the LoLDU method of the present invention and the baseline method LoRA. Bold indicates the best result. It can be seen that compared with the baseline method LoRA, the present invention can achieve very significant improvements in the accuracy rates of some datasets, and there is also an obvious increase in the average accuracy rate of all data.

[0084] Although the above-described illustrative specific embodiments of the present invention have been described to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

Claims

1. A method for efficient fine-tuning of low-rank adaptive parameters of a pre-trained model based on triangular decomposition, characterized in that: The following steps are involved: S1: Obtain a training sample set for parameter fine-tuning according to actual needs, where each training sample includes an input sample and a label; S2: For the pre-trained large model, select the layer that needs to be fine-tuned from the large model according to actual needs. The weight matrix of the layer to be fine-tuned is Where k = 1, 2, ..., K, K represents the number of layers to be fine-tuned, M k ×N k Represents the shape of the weight matrix of the layer to be fine-tuned; S3: Layer weight decomposition based on triangular decomposition, specifically including the following steps: S3.1: Use LDU decomposition method to fine-tune the weight matrix W of each layer to be fine-tuned k Decompose to get W k =P k L k U k , where P k represents the permutation matrix, L k represents the lower triangular matrix, U k represents an upper triangular matrix; Extract the upper triangular matrix U k The diagonal matrix D k , and then use the following formula to calculate the upper triangular matrix U k Normalize and get the normalized upper triangular matrix S3.2: Extract the lower triangular matrix L k The first r k The lower triangular submatrix is ​​obtained by r k Represents the preset rank; extract the diagonal matrix D k The size of the upper left corner is r k × k Submatrix of , get the diagonal submatrix Extract the normalized upper triangular matrix The first r k Row, get the upper triangular submatrix S3.3: Preset the scaling factor α according to actual needs, and then calculate the scaling factor σ using the following formula k : s k =a / r k The low-rank approximation matrix W is calculated using the following formula: k,approx : Then the residual matrix W is calculated using the following formula k,residual : IN k,residual =In k -IN k,approx S4: The permutation matrix P corresponding to the weight matrix of each layer to be fine-tuned k , the lower triangular submatrix Upper triangular submatrix All parameters of other layers in the large model are frozen, and the trainable parameter ρ is set k And let its initial value be σ k , set the trainable parameters And according to the diagonal submatrix Initialize it with inspiration; S5: Input the input samples in the training sample set into the large model, calculate the loss function and the trainable parameter ρ of each layer to be fine-tuned k and Update them separately to get the updated trainable parameters and S6: Based on the updated trainable parameters and Calculate the updated model weights for each layer to be fine-tuned S7: Determine whether the fine-tuning end condition is met, if not, proceed to step S8, otherwise the fine-tuning ends; S8: Let the weight matrix Trainable parameters Return to step S5.

2. The method for efficient fine-tuning of low-rank adaptive parameters of a pre-trained model based on triangular decomposition according to claim 1, characterized in that: The trainable parameters in step S4 The specific method of heuristic initialization is: calculate the diagonal submatrix The mean of the diagonal elements in , or use random uniform distribution or normal distribution for sampling, and then set the trainable parameters based on the calculation results or sampling results The diagonal elements of The non-diagonal elements of are set to 0 to complete the initialization.

Images