A loRA fine-tuning method based on a parameter-free attention module

Abstract

The application relates to a LoRA fine-tuning method based on a parameter-free attention module, which comprises the following steps: obtaining a pre-training model and model parameters, and setting a parameter fine-tuning module at each layer of the pre-training model, wherein the parameter fine-tuning module is a LoRA model with a parameter-free attention module; obtaining a fine-tuning data set; loading and freezing the parameters of the pre-training model, and initializing the parameters of the parameter fine-tuning module; fine-tuning the parameters of the parameter fine-tuning module based on the fine-tuning data set to obtain a large language model after fine-tuning. The application has the beneficial effects that the application only trains the parameters of the LoRA module, does not need to fine-tune all parameters of the diffusion model, and the application is modified on the basis of the LoRA, an attention module without parameters is added before the LoRA parameters, so that the LoRA can better obtain semantic information of natural language in learning, and has better effects.

Description

Technical Field

[0001] This invention relates to the field of fine-tuning pre-trained model parameters, and more specifically, to a LoRA fine-tuning method based on a parameterless attention module. Background Technology

[0002] Generative pre-trained models have revolutionized the field of natural language processing, providing a powerful methodology for handling language understanding and generation tasks. These models learn rich language representations and capture complex structures at the lexical, syntactic, and semantic levels through pre-training on large-scale corpora. In particular, the GPT series of models has garnered widespread attention for its outstanding performance on various NLP tasks; its design based on autoregressive mechanisms and the Transformer architecture achieves high efficiency and accuracy in prediction tasks.

[0003] Several important techniques exist for fine-tuning pre-trained models to adapt them to specific tasks. Among them, Adapter, LoRA, and Prefix / Prompt are three commonly used approaches. They all play a crucial role in improving model adaptability and performance. LoRA is a popular method that adds two trainable side-path matrices, also known as low-rank separable matrices, to a specified layer. This method allows the model to adapt to a specific task by learning additional parameters. These matrices are typically designed to be low-rank, thus enabling significant model tuning with only a small number of added parameters without retraining the entire model.

[0004] While these fine-tuning methods have made significant progress in model adaptability and performance, they are typically only applicable to simple downstream tasks. Real-world applications often involve multi-task learning environments, requiring more flexible and efficient fine-tuning methods. Therefore, further research and innovation remain essential. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a LoRA fine-tuning method based on a parameterless attention module.

[0006] Firstly, a LoRA fine-tuning method based on a parameterless attention module is provided, including:

[0007] S1. Obtain the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module.

[0008] S2. Obtain the fine-tuning dataset and preprocess the fine-tuning dataset;

[0009] S3. Load and freeze the parameters of the pre-trained model, and initialize the parameters of the parameter fine-tuning module;

[0010] S4. Fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset to obtain the fine-tuned large language model.

[0011] Preferably, S2 includes:

[0012] S201. Set specific Prompt structures and content for different dataset tasks, and edit the dataset label results to convert them into text format.

[0013] S202, Shuffle and merge different datasets into a large dataset.

[0014] Preferably, S4 includes:

[0015] S401. Based on the fine-tuning dataset, forward propagation calculations were performed on the pre-trained model and the parameter fine-tuning module to obtain the results. base_model and Result lora ;

[0016] S402, Result base_model Add Result lora The output of the pre-trained model is used to generate the results.

[0017] S403. Calculate the loss based on cross-entropy and perform backpropagation based on the loss.

[0018] Preferably, in S1, the parameterless attention module calculates the linear separability between the target neuron and its surrounding neurons to obtain the importance of each neuron, and then uses the importance to adjust the weights of the neurons to achieve the attention mechanism; the following energy function is defined for each neuron:

[0019]

[0020] Where s′=w s s+b s And x′ i =w s x i +b s Where s refers to the target neuron, and x i It refers to other neurons, w s and b s It is s and x i The transformation weights and biases, where L is the sequence length and λw s 2 It is a regularization matrix;

[0021] The parameterless attention module is represented as follows:

[0022]

[0023] The sigmoid function is used to limit E. s The value is too large; X is the input sequence, and ⊙ represents the dot product.

[0024] Preferably, in the expression of the energy function, when s′ equals y, and all other x′s are equal, ... i When y equals y0, the equation reaches its minimum, where y and y0 are two distinct values. Minimizing this equation is equivalent to finding the linear separability between the target neuron s and all other neurons in the same channel. For simplification, binary labels are used to represent y and y0, and the final expression of the energy function is:

[0025]

[0026] Furthermore, there is a fast closed-form solution for the above formula:

[0027]

[0028]

[0029] in and Since the sum of the mean and variance calculated by all neurons in the formula is obtained on a single sequence, it is reasonable to assume that all pixels in a single channel follow the same distribution. Given this assumption, the mean and variance can be calculated on all neurons and reused across all neurons in that channel; substituting the above formula, we get:

[0030]

[0031] in This represents the variance of the input. λ represents the mean of the input, t represents the target neuron, and λ is a hyperparameter; the greater the difference between the target object and other objects, the more important the information contained in that object.

[0032] Preferably, in S401, the hidden layer information h is represented as follows after the parameter fine-tuning module is loaded:

[0033] h = W0X + BA*TkLoRA(X)

[0034] Where W0 is the initial weights for pre-training, X is the input sequence, and BA are two low-rank matrices obtained by the LoRA model by decomposing the change ΔW0 of the initial weights W0.

[0035] Preferably, in S1, the pre-trained model is a pre-trained large language model structure.

[0036] Secondly, a LoRA fine-tuning system based on a parameterless attention module is provided for performing any of the methods described in the first aspect, including:

[0037] The first acquisition module is used to acquire the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module.

[0038] The second acquisition module is used to acquire the fine-tuning dataset and preprocess the fine-tuning dataset;

[0039] The loading module is used to load and freeze the parameters of the pre-trained model and initialize the parameters of the parameter fine-tuning module.

[0040] The fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset, so as to obtain the fine-tuned large language model.

[0041] Thirdly, a computer storage medium is provided, wherein a computer program is stored therein; when the computer program is run on a computer, the computer executes the LoRA fine-tuning method based on a parameterless attention module as described in any of the first aspects.

[0042] Fourthly, an electronic device is provided, including a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor, characterized in that the processor, when executing the instructions or code, implements the LoRA fine-tuning method based on a parameterless attention module as described in any of the first aspects.

[0043] The beneficial effects of this invention are: the parameters trained by this invention are only the LoRA module, eliminating the need for full parameter fine-tuning of the diffusion model; furthermore, this invention modifies LoRA by adding a parameterless attention module before the LoRA parameters, enabling LoRA to better acquire semantic information of natural language during learning and achieve better results; at the same time, this module is simple and does not increase the inference time of the module. Attached Figure Description

[0044] Figure 1 This is a flowchart of a LoRA fine-tuning method based on a parameterless attention module;

[0045] Figure 2 A schematic diagram of LoRA with added parameterless attention module;

[0046] Figure 3This is a schematic diagram of a LoRA fine-tuning system based on a parameterless attention module. Detailed Implementation

[0047] The present invention will be further described below with reference to embodiments. The description of the embodiments below is only for the purpose of helping to understand the present invention. It should be noted that those skilled in the art can make several modifications to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0048] Example 1:

[0049] This application provides a LoRA fine-tuning method based on a parameterless attention module, which can be applied to electronic devices with instruction or program execution capabilities. These electronic devices can be, but are not limited to, various servers, workstations, personal computers, laptops, etc. The difference in execution between different computing devices is merely a difference in the execution entity; those skilled in the art will foresee that running the solution on different computing devices will produce the same technical effect. The invention will now be described in detail through specific embodiments.

[0050] Figure 1 A flowchart illustrating a LoRA fine-tuning method based on a parameterless attention module is provided in this application embodiment, including the following steps:

[0051] S1. Obtain the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module.

[0052] In S1, the parameterless attention module calculates the linear separability between the target neuron and its surrounding neurons to obtain the importance of each neuron, and then uses the importance to adjust the weights of the neurons to achieve the attention mechanism; the following energy function is defined for each neuron:

[0053]

[0054] Where s′=w s s+b s And x′ i =w s x i +b s Where s refers to the target neuron, and x i It refers to other neurons, w s and b s It is s and x i The transformation weights and biases, where L is the sequence length. It is a regularization matrix. When s′ equals y, and all other x′s are equal...i The equation reaches its minimum when y equals y0, where y and y0 are two distinct values. Minimizing this equation is equivalent to finding the linear separability between the target neuron s and all other neurons in the same channel. For simplicity, binary labels (i.e., 1 and -1) are used to represent y and y0, so the final expression of the energy function is:

[0055]

[0056] Furthermore, there is a fast closed-form solution for the above formula:

[0057]

[0058]

[0059]

[0060]

[0061] Since the sum of the mean and variance calculated by all neurons in the formula is obtained on a single sequence, it is reasonable to assume that all pixels in a single channel follow the same distribution. Given this assumption, the mean and variance can be calculated on all neurons and reused across all neurons in that channel. Therefore, substituting the above formula, we get:

[0062]

[0063] The above formula shows that the greater the difference between the target object and other objects, the more important the information contained in that object is.

[0064] Finally, the parameterless attention module is represented as:

[0065]

[0066] The sigmoid function is used to limit E. s A value that is too large will not affect the relative importance of each neuron, as the sigmoid function is monotonically increasing. (For text input sequences...) To transform the input sequence by utilizing the relationship between the sequences, and in order to compute the sequence attention module, this application performs an average pooling operation on the first dimension of the input sequence X to generate the sequence attention module. Then subtract X′ from each column of the original input sequence and take the absolute value, that is... Copy and expand Then the calculation yields...

[0067] X A =(XX′) 2=(X-AvgPool(X)) 2

[0068] Then X A Perform the following transformation on the formula:

[0069]

[0070] This application adds this module before each LoRA. Since the module in this application only involves the calculation of mean and variance, it is relatively easy to implement; only the calculation code needs to be added to the forward function of LoRA. In summary, this module is based on the basic theories of neuroscience. This application, based on research in computer vision, integrates this module into the field of natural language processing. Moreover, this module is easy to implement and can be integrated with most networks.

[0071] S2. Obtain the fine-tuning dataset and preprocess the fine-tuning dataset.

[0072] S2 includes:

[0073] S201. Set specific Prompt structures and content for different dataset tasks, and edit the dataset label results to convert them into text format.

[0074] S202, Shuffle and merge different datasets into a large dataset.

[0075] S3. Load and freeze the parameters of the pre-trained model, and initialize the parameters of the parameter fine-tuning module.

[0076] S4. Fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset to obtain the fine-tuned large language model.

[0077] Example 2:

[0078] Based on Example 1, Example 2 of this application provides a more specific LoRA fine-tuning method based on a parameterless attention module, including:

[0079] S1. Obtain the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module.

[0080] S2. Obtain the fine-tuning dataset and preprocess the fine-tuning dataset.

[0081] S3. Load and freeze the parameters of the pre-trained model, and initialize the parameters of the parameter fine-tuning module.

[0082] S4. Fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset to obtain the fine-tuned large language model.

[0083] S4 includes:

[0084] S401. Based on the fine-tuning dataset, forward propagation calculations were performed on the pre-trained model and the parameter fine-tuning module to obtain the results. base_model and Result lora .

[0085] In S401, the specific calculation method for the LoRA model with a parameterless attention module is as follows:

[0086] For a given linear layer, its forward propagation has the following characteristics:

[0087] h = W0X

[0088] LoRA is based on the assumption that updates made during fine-tuning exhibit a low "intrinsic rank." It incorporates two trainable side-path matrices, also known as low-rank separable matrices, into a specified layer. The product of these two low-rank matrices incrementally updates the pre-trained weights. For the initial pre-trained weights... LoRA uses To simulate the change in initial weight W0 during the fine-tuning of downstream tasks, ΔW0 is decomposed into two low-rank matrices BA. And r << min(d,k). Therefore, the model weights W′ after LoRA loading are expressed as:

[0089] W′=W0+△W0=W0+BA

[0090] The hidden layer information h is represented as follows after LoRA is loaded:

[0091] h=W′X=W0X+△W0X=W0X+BAX

[0092] like Figure 2 As shown, this application adds a parameterless attention module to LoRA, so the input X will pass through a sequence attention module without trained parameters, and then the LoRA structure is introduced. The final expression is:

[0093] h = W0X + BA*TkLora(X)

[0094] S402, Result base_model Add Result lora The output of the pre-trained model is used to generate the results.

[0095] S403. Calculate the loss based on cross-entropy and perform backpropagation based on the loss.

[0096] To verify its effectiveness, this invention selected ChatGLM3-6B as the pre-trained model in a real-world scenario and compared it with the following efficient fine-tuning methods: LoRA and TkLoRA. The datasets included GLUE and Super-GLUE. GLUE contained six datasets: CoLA, MNLI, QNLI, QQP, RTE, and SST-2; Super-GLUE contained four datasets: BoolQ, CB, WiC, and WSC.

[0097] The experimental results are shown in Tables 1 and 2 below. The results show that LoRA based on the parameterless attention module performs better than the original LoRA.

[0098] Table 1 shows the evaluation results on the GLUE dataset, with accuracy as the metric.

[0099]

[0100] Table 2 shows the evaluation results on the Super-GLUE dataset, with accuracy as the metric.

[0101]

[0102] As can be seen, this invention modifies LoRA by adding a parameterless attention module before the LoRA parameters, enabling LoRA to better acquire semantic information of natural language during learning. TkLoRA does not have additional training parameters when using the same hyperparameters as LoRA. At the same time, this module is simple and does not increase the inference time of the module.

[0103] Example 3:

[0104] Based on Embodiments 1 and 2, Embodiment 3 of this application provides a LoRA fine-tuning system based on a parameterless attention module, comprising:

[0105] The first acquisition module is used to acquire the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module.

[0106] The second acquisition module is used to acquire the fine-tuning dataset and preprocess the fine-tuning dataset;

[0107] The loading module is used to load and freeze the parameters of the pre-trained model and initialize the parameters of the parameter fine-tuning module.

[0108] The fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset, so as to obtain the fine-tuned large language model.

[0109] Specifically, the system provided in this embodiment is the system corresponding to the method provided in Embodiments 1 and 2. Therefore, the parts in this embodiment that are the same as or similar to those in Embodiments 1 and 2 can be referred to each other, and will not be repeated in this application.

Claims

1. A LoRA fine-tuning method based on a parameterless attention module, characterized in that, include: S1. Obtain the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module. In S1, the parameterless attention module calculates the linear separability between the target neuron and its surrounding neurons to obtain the importance of each neuron, and then uses the importance to adjust the weights of the neurons to achieve the attention mechanism; the following energy function is defined for each neuron: in and ,in It refers to the target neuron. It refers to other neurons. and It is s and The transformation weights and biases, where L is the sequence length. It is a regularization matrix; The parameterless attention module is represented as follows: The sigmoid function is used to limit... The value is too large; X is the input sequence. Dot product; S2. Obtain the fine-tuning dataset and preprocess the fine-tuning dataset; S3. Load and freeze the parameters of the pre-trained model, and initialize the parameters of the parameter fine-tuning module; S4. Fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset to obtain the fine-tuned large language model; S4 includes: S401. Based on the fine-tuning dataset, forward propagation calculations were performed on the pre-trained model and the parameter fine-tuning module to obtain the results. ; In S401, the hidden layer information h is represented as follows after the parameter fine-tuning module is loaded: in, Here, X represents the initial weights for pre-training, and X is the input sequence. Initialize weights for the LoRA model by decomposition Changes The two low-rank matrices obtained; S402, will add The output of the pre-trained model is used to generate the results. S403. Calculate the loss based on cross-entropy and perform backpropagation based on the loss.

2. The LoRA fine-tuning method based on a parameterless attention module according to claim 1, characterized in that, S2 include: S201. Set specific Prompt structures and content for different dataset tasks, and edit the dataset label results to convert them into text format. S202, Shuffle and merge different datasets into a large dataset.

3. The LoRA fine-tuning method based on a parameterless attention module according to claim 2, characterized in that, In the expression for the energy function, when equal And all of them equal When the equation reaches its minimum value, where and These are two different values; minimizing this equation is equivalent to finding the target neuron. And linear separability between all other neurons in the same channel; for simplicity, binary labels are used. and The final expression of the energy function is: Furthermore, there is a fast closed-form solution for the above formula: in and Since the sum of the mean and variance calculated by all neurons in the formula is obtained on a single sequence, it is assumed that all pixels in a single channel follow the same distribution; the mean and variance are calculated on all neurons and repeated in all neurons of that channel; substituting the closed-form solution, we get: in This represents the variance of the input. This represents the mean of the input. Represents the target neuron. It is a hyperparameter; the greater the difference between the target object and other objects, the more important the information contained in that object.

4. The LoRA fine-tuning method based on a parameterless attention module according to claim 3, characterized in that, In S1, the pre-trained model is a pre-trained large language model structure.

5. A LoRA fine-tuning system based on a parameterless attention module, characterized in that, For performing the method according to any one of claims 1 to 4, comprising: The first acquisition module is used to acquire the pre-trained model and model parameters, and set a parameter fine-tuning module in each layer of the pre-trained model to obtain the model to be fine-tuned; the parameter fine-tuning module is a LoRA model with a parameterless attention module. The second acquisition module is used to acquire the fine-tuning dataset and preprocess the fine-tuning dataset; The loading module is used to load and freeze the parameters of the pre-trained model and initialize the parameters of the parameter fine-tuning module. The fine-tuning module is used to fine-tune the parameters of the parameter fine-tuning module based on the fine-tuning dataset, so as to obtain the fine-tuned large language model.

6. A computer storage medium, characterized in that, The computer storage medium stores a computer program; when the computer program is run on the computer, it causes the computer to execute the LoRA fine-tuning method based on a parameterless attention module as described in any one of claims 1 to 4.

7. An electronic device, comprising a storage medium, a processor, and instructions or code stored on the storage medium and executable on the processor, characterized in that, When the processor executes the instructions or code, it implements the LoRA fine-tuning method based on a parameterless attention module as described in any one of claims 1 to 4.

Images