Large language model sparse method and dialogue generation method based on efficient fine tuning of parameters
By dynamically determining the sparse rate of large language models and allocating the rank of low-rank matrix, the problem of degradation of sparse models in the existing technology is solved, and the significant performance improvement of large language models in high-sparse configurations and the reduction of computing resources are achieved.
Patent Information
- Application Number
- CN202411039887.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-26
AI Technical Summary
The existing large language model sparse method causes the model accuracy to decline in high sparse configurations, the low-rank matrix and sparse LLM weights cannot be merged, the sparse rate and rank settings are uniform, ignoring the redundancy degree and reconstruction error differences of each layer, affecting the model performance .
Determine the weight importance by calculating the mutual information indicators of each layer, set the sparse rate dynamically, and allocate the rank of the low-rank matrix according to the reconstruction error sorting to ensure that the sparse mask is consistent, and only the low-rank matrix is fine-tuned to merge into the sparse large language model.
It significantly improves the performance of large language models in high sparse configurations, solves the problem of sparse weights and low-rank matrices that cannot be merged, achieves optimal sparse fine-tuning, improves the generation speed and accuracy of the model, and reduces computing resource consumption.
Smart Images

Figure CN119025640A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of large language models, and in particular to a large language model sparse method and a dialogue generation method based on efficient parameter fine-tuning. Background Art
[0002] In the field of natural language processing, large language models (LLMs) have been widely used in various language tasks. As the model size increases, the model performance has been significantly improved. However, this has also led to a sharp increase in the demand for computing resources for LLMs. To address this issue, researchers have developed a variety of optimization techniques, including sparsification and efficient parameter fine-tuning. These techniques not only significantly reduce the number of model parameters and computational complexity, but also significantly improve the performance of the model by fine-tuning a small number of parameters under limited resources, making the practical application of LLMs more efficient and feasible.
[0003] Model sparsification is a technique that reduces computational complexity and memory consumption by reducing the number of model parameters. In LLM, there are usually many redundant parameters that contribute little to the final model performance. Sparsification makes the model structure more concise and efficient by identifying and removing these redundant parameters. Post-training sparsification is a commonly used sparsification method that does not require a lot of retraining and can directly set some weights to zero to achieve sparsification with only a small amount of calibration data. However, in a high-ratio sparsity setting, post-training sparsification often leads to a significant decrease in model accuracy. One of the main reasons for this problem is that existing sparsification methods set the sparsity rate of each layer of LLM to a uniform value, ignoring the fact that the degree of redundancy of each layer of LLM is different. In addition, it becomes inevitable to fine-tune the sparse model to restore model accuracy.
[0004] Parameter-efficient fine-tuning is a method for optimizing large language models. It improves model performance by adjusting only a small number of parameters, thereby significantly reducing computational overhead and storage requirements. A common parameter-efficient fine-tuning method involves inserting a set of learnable low-rank matrices for each fully connected layer in a large model that needs to be sparse, and only fine-tuning these low-rank matrices to improve the performance of sparse LLM. After fine-tuning, the low-rank matrices can be merged into the LLM without introducing additional inference latency. However, directly inserting low-rank matrices into sparse LLMs will result in the low-rank matrices and sparse LLM weights being unable to be merged, increasing the inference latency of the sparse model. In addition, existing low-rank matrix fine-tuning methods set the ranks of all low-rank matrices to a uniform size, assigning the same amount of fine-tuning parameters to each layer of the sparse LLM, ignoring the differences in reconstruction errors of each layer in the sparse LLM, resulting in suboptimal fine-tuning accuracy.
[0005] In summary, the existing large language model processing mainly has the following problems: 1. Low-rank matrices and sparse LLM weights cannot be merged: Low-rank matrices are dense matrices, and LLM weights are sparse matrices, so they cannot be merged. If the low-rank matrix is kept unmerged, the inference latency of the sparse LLM will increase, thus undermining the acceleration advantage of the sparse LLM.
[0006] 2. Uniform setting of LLM sparsity rate: Existing sparse methods set the sparsity rate of each layer of LLM to the same value, ignoring the fact that the redundancy and importance of each layer of LLM are different, thus affecting the performance of the model.
[0007] 3. Uniform rank setting of low-rank matrices: The rank of a low-rank matrix determines the size of the fine-tuning parameters. More fine-tuning parameters often lead to better performance improvements. Existing parameter-efficient fine-tuning methods set the ranks of all low-rank matrices to the same value, resulting in uniform fine-tuning parameter distribution, ignoring the differences in sparse reconstruction errors at each layer, resulting in suboptimal fine-tuning performance. Summary of the invention
[0008] In view of the problems existing in the prior art, the purpose of the present invention is to provide a large language model sparsification method based on efficient parameter fine-tuning, which jointly optimizes sparsification and efficient parameter fine-tuning, and simultaneously sparsifies and fine-tunes the large language model, thereby improving the performance of the large language model under highly sparse configuration.
[0009] To achieve the above object, the present invention adopts the following technical solutions: A large language model sparse method based on efficient parameter fine-tuning, the method comprising the following steps: Step 1: Data preparation: Sample N segments from the training corpus, each segment contains L tokens; Step 2, set the iteration round and sparsity rate: set the iteration round of sparse fine-tuning to T, set the average sparsity rate of all layers after LLM fine-tuning to P, and for iteration round i (1≤i≤T), the average sparsity rate of all LLM layers is i / T*P; Step 3, determine the layer-by-layer sparsity rate: input the data obtained in step 1 into the large language model to obtain the layer-by-layer feature map of the large language model, then calculate the mutual information index of each layer, determine the weight importance of each layer according to the mutual information index, and set a lower sparsity rate for the layer with higher weight importance and a higher sparsity rate for the layer with lower weight importance while keeping the average sparsity rate unchanged, so as to obtain the layer-by-layer sparsity rate of the large language model; Step 4, determine the sparse mask: according to the layer-by-layer sparsity rate of the large language model, determine the weight of the large language model and the sparse mask of the low-rank matrix. The sparse mask of the large language model is the same as the sparse mask of the low-rank matrix, thereby obtaining a sparse large language model. Step 5, determine the rank of the low-rank matrix: calculate the reconstruction error of each layer of the sparse large language model, sort the reconstruction errors, assign a larger rank to the layer with a larger reconstruction error, and assign a smaller rank to the layer with a smaller reconstruction error; Step 6: Fine-tune the low-rank matrix: The weights of the sparse large language model remain frozen, and only the low-rank matrix is fine-tuned; Step 7: Merge the low-rank matrix into the large language model: Add the weights of the fine-tuned low-rank matrix and the sparse large language model to obtain the fine-tuned sparse large language model. Step 8: Determine whether the iteration round has been reached. If the iteration round has been reached, the final sparse large language model is obtained and the process goes to step 9; otherwise, continue with steps 3-8. Step 9: Model evaluation: Use different datasets to evaluate the sparse large language model and verify its performance improvement.
[0010] In step 3, a lower sparsity rate is set for a layer with higher weight importance, and a higher sparsity rate is set for a layer with lower weight importance. Specifically: The model contains L layers, and the weight importance S of the L layers is sorted from small to large; for the importance after sorting (1≤i≤L) The sparseness rate of the allocation is P+(i-1) , where the average sparsity rate is P+(L-1) , is a hyperparameter.
[0011] In step 5, a layer with a larger reconstruction error is assigned a larger rank, and a layer with a smaller reconstruction error is assigned a smaller rank. Specifically, The model contains L layers, and the reconstruction errors of the L layers are sorted from small to large, and the sorted reconstruction errors are divided into M parts, where L is a multiple of M; for the sorted reconstruction errors, the rank size assigned to the i-th (1≤i≤M) part is R+i-1, where R is a positive integer.
[0012] A method for generating a dialogue based on a sparse large language model comprises the following steps: Step 1: Get the dialogue text entered by the user; Step 2: using the sparse large language model obtained by the large language model sparse method based on efficient parameter fine-tuning as claimed in claim 1 to process the dialogue text input by the user to obtain a corresponding answer text; When the large language model performs sparse processing, the data used is a conversation dataset containing daily conversation fragments.
[0013] After adopting the above scheme, the present invention simultaneously sparses the LLM and the low-rank matrix according to the weight importance metric to ensure that the two have the same sparse mask and can be merged after fine-tuning. In each iteration, the importance of each layer is determined according to the mutual information index. While keeping the average sparsity rate unchanged, a lower sparsity rate is set for the more important layers, and a higher sparsity rate is set for the less important layers. During the fine-tuning process, the reconstruction error of each layer of the sparse LLM is calculated. Under the fine-tuning parameter budget constraint, more fine-tuning parameters are allocated to the layers with larger reconstruction errors, that is, larger ranks are allocated, and smaller ranks are allocated to the layers with smaller reconstruction errors. Therefore, the present invention can significantly improve the performance of large language models under highly sparse configurations, solve the problem in the prior art that sparse weights and low-rank matrix weights cannot be merged, and simultaneously determine the optimal LLM layer-by-layer sparsity rate and the optimal fine-tuning parameter allocation, and has wide applicability and significant beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0015] like Figure 1 As shown, the present invention discloses a large language model sparse method based on efficient parameter fine-tuning, which includes the following steps: Step 1: Data preparation: Sample N segments from the training corpus, each segment contains L tokens.
[0016] Step 2. Set the iteration round and sparsity rate: Set the iteration round of sparse fine-tuning to T, set the average sparsity rate of all layers after LLM fine-tuning to P, and for iteration round i (1≤i≤T), the average sparsity rate of all LLM layers is i / T*P.
[0017] Step 3, determine the layer-by-layer sparsity rate: input the data obtained in step 1 into the large language model to obtain the layer-by-layer feature map of the large language model, and then calculate the mutual information index of each layer, and determine the weight importance of each layer according to the mutual information index. While keeping the average sparsity rate unchanged, the layer with higher weight importance is set to a lower sparsity rate, and the layer with lower weight importance is set to a higher sparsity rate, so as to obtain the layer-by-layer sparsity rate of the large language model. The layers with higher weight importance are set with lower sparsity rate, and the layers with lower weight importance are set with higher sparsity rate. Specifically: The model contains L layers, and the weight importance S of the L layers is sorted from small to large. (1≤i≤L) The sparseness rate of the allocation is P+(i-1) , where the average sparsity rate is P+(L-1) , is a hyperparameter.
[0018] When setting the sparsity rate of each layer of a large language model, the following two conditions must be met: (1) keep the average sparsity rate unchanged; (2) the higher the weight importance, the lower the corresponding sparsity rate; the lower the weight importance, the higher the corresponding sparsity rate.
[0019] Step 4, determine the sparse mask: According to the layer-by-layer sparsity rate of the large language model, determine the weight of the large language model and the sparse mask of the low-rank matrix. The sparse mask of the large language model is the same as the sparse mask of the low-rank matrix, thereby obtaining a sparse large language model.
[0020] Step 5. Determine the rank of the low-rank matrix: Calculate the reconstruction error of each layer of the sparse large language model, sort the reconstruction errors, assign a larger rank to the layer with a larger reconstruction error, and assign a smaller rank to the layer with a smaller reconstruction error. The layers with larger reconstruction errors are assigned larger ranks, and the layers with smaller reconstruction errors are assigned smaller ranks. Specifically: The model contains L layers, and the reconstruction errors of the L layers are sorted from small to large, and the sorted reconstruction errors are divided into M parts, where L is a multiple of M. For the sorted reconstruction error, the rank size assigned to the i-th (1≤i≤M) part is R+i-1, where R is a positive integer.
[0021] Step 6: Fine-tune the low-rank matrix: The weights of the sparse large language model remain frozen, and only the low-rank matrix is fine-tuned.
[0022] Step 7: Merge the low-rank matrix into the large language model: Add the weights of the fine-tuned low-rank matrix and the sparse large language model to obtain the fine-tuned sparse large language model.
[0023] Step 8: Determine whether the iteration round has been reached. If the iteration round has been reached, the final sparse large language model is obtained and the process goes to step 9; otherwise, continue with steps 3-8.
[0024] Step 9: Model evaluation: Use different datasets to evaluate the sparse large language model and verify its performance improvement.
[0025] The present invention simultaneously sparses the LLM and the low-rank matrix according to the weight importance metric, ensuring that the two have the same sparse mask and can be merged after fine-tuning. In each iteration, the importance of each layer is determined according to the mutual information index. While keeping the average sparsity rate unchanged, a lower sparsity rate is set for the more important layers, and a higher sparsity rate is set for the less important layers. During the fine-tuning process, the reconstruction error of each layer of the sparse LLM is calculated. Under the fine-tuning parameter budget constraint, more fine-tuning parameters are allocated to the layers with larger reconstruction errors, that is, larger ranks are allocated, and smaller ranks are allocated to the layers with smaller reconstruction errors. Therefore, the present invention can significantly improve the performance of large language models under highly sparse configurations, solve the problem in the prior art that sparse weights and low-rank matrix weights cannot be merged, and simultaneously determine the optimal LLM layer-by-layer sparsity rate and the optimal fine-tuning parameter allocation, and has wide applicability and significant beneficial effects.
[0026] The present invention also discloses a method for generating a dialogue based on a sparse large language model, which comprises the following steps: Step 1: Get the dialogue text entered by the user; Step 2: Use the sparse large language model obtained by the above large language model sparse method to process the dialogue text input by the user to obtain the corresponding answer text.
[0027] When the large language model performs sparse processing, the data used is a conversation dataset containing daily conversation fragments.
[0028] In practical applications, the dialogue generation method based on the sparse large language model effectively improves the generation speed and accuracy of the answer text. For example, the BoolQ dataset is used. Each dialogue data in the BoolQ dataset contains English text and questions. The user inputs the dialogue data into the sparse large language model. The model judges whether the question is right or wrong based on the text input by the user, and outputs a "true" or "false" answer. The accuracy of the model on the BoolQ dataset is obtained based on the model's answer combined with the real label. The BoolQ dataset accuracy of the 70% sparse LLaMA-V2-7B model is 52.87%. After the 70% sparse LLaMA-V2-7B model is adjusted using the fine-tuning method proposed in the present invention, the accuracy is 64.28%. The text generation speed of the non-sparse LLaMA-V2-7B model on the CPU is 3.43 tokens / s, and the speed of the 70% sparse LLaMA-V2-7B model is 8.93 tokens / s, and the text generation speed is increased by 2.60 times.
[0029] The present invention has the following beneficial effects: 1. Joint sparseness of large language model and low-rank matrix: By simultaneously sparsely ...
[0030] 2. Dynamically determine the sparsity rate of LLM: The present invention determines the importance of each layer according to the mutual information index in each iteration, thereby dynamically determining the optimal layer-by-layer sparsity rate during the fine-tuning process.
[0031] 3. Dynamically assign the rank of the low-rank matrix: The present invention assigns appropriate fine-tuning parameters to each layer to minimize the output difference between dense and sparse large language models, thereby achieving optimal sparse fine-tuning.
[0032] Fourth, through the joint optimization method of sparsification and efficient parameter fine-tuning, the present invention significantly improves the performance of the model under highly sparse configurations. For example, the present invention reduces the perplexity of the 70% sparse LLaMA-V2-7B model by 68.38, and improves the zero-shot accuracy by 13.94%, achieving a 2.6-fold speedup on the CPU.
[0033] 5. Reduce computing resource consumption: By efficiently fine-tuning the parameters of sparse models, the consumption of computing resources is significantly reduced. For example, sparse fine-tuning the LLaMA2-7B model only takes about 2 hours on a single A800 80GB GPU.
[0034] 6. Wide applicability: The present invention shows significant performance improvement on different data sets and has wide applicability and versatility.
[0035] To summarize, this patent effectively solves the problems existing in the prior art, such as the inability to merge the weights of low-rank matrices and sparse large language models, the uniform setting of the sparsity rate of large language models, and the uniform setting of the rank of low-rank matrices, by introducing joint sparsity, dynamically determining the sparsity rate of large language models, and dynamically allocating the rank of low-rank matrices, thereby significantly improving the performance of sparse models.
[0036] The present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments. It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied. Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure. Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A large language model sparse method based on efficient parameter fine-tuning, characterized in that: The method comprises the following steps: Step 1: Data preparation: Sample N segments from the training corpus, each segment contains L tokens; Step 2, set the iteration round and sparsity rate: set the iteration round of sparse fine-tuning to T, set the average sparsity rate of all layers after LLM fine-tuning to P, and for iteration round i (1≤i≤T), the average sparsity rate of all LLM layers is i / T*P; Step 3, determine the layer-by-layer sparsity rate: input the data obtained in step 1 into the large language model to obtain the layer-by-layer feature map of the large language model, then calculate the mutual information index of each layer, determine the weight importance of each layer according to the mutual information index, and set a lower sparsity rate for the layer with higher weight importance and a higher sparsity rate for the layer with lower weight importance while keeping the average sparsity rate unchanged, so as to obtain the layer-by-layer sparsity rate of the large language model; Step 4, determine the sparse mask: according to the layer-by-layer sparsity rate of the large language model, determine the weight of the large language model and the sparse mask of the low-rank matrix. The sparse mask of the large language model is the same as the sparse mask of the low-rank matrix, thereby obtaining a sparse large language model. Step 5, determine the rank of the low-rank matrix: calculate the reconstruction error of each layer of the sparse large language model, sort the reconstruction errors, assign a larger rank to the layer with a larger reconstruction error, and assign a smaller rank to the layer with a smaller reconstruction error; Step 6: Fine-tune the low-rank matrix: The weights of the sparse large language model remain frozen, and only the low-rank matrix is fine-tuned; Step 7: Merge the low-rank matrix into the large language model: Add the weights of the fine-tuned low-rank matrix and the sparse large language model to obtain the fine-tuned sparse large language model. Step 8: Determine whether the iteration round has been reached. If the iteration round has been reached, the final sparse large language model is obtained and the process goes to step 9; otherwise, continue with steps 3-8. Step 9: Model evaluation: Use different datasets to evaluate the sparse large language model and verify its performance improvement.
2. A large language model sparse method based on efficient parameter fine-tuning according to claim 1, characterized in that: In step 3, a lower sparsity rate is set for a layer with higher weight importance, and a higher sparsity rate is set for a layer with lower weight importance. Specifically: The model contains L layers, and the weight importance S of the L layers is sorted from small to large; for the importance after sorting (1≤i≤L) The sparseness rate of the allocation is P+(i-1) , where the average sparsity rate is P+(L-1) , is a hyperparameter.
3. A large language model sparse method based on efficient parameter fine-tuning according to claim 1, characterized in that: In step 5, a layer with a larger reconstruction error is assigned a larger rank, and a layer with a smaller reconstruction error is assigned a smaller rank. Specifically, The model contains L layers, and the reconstruction errors of the L layers are sorted from small to large, and the sorted reconstruction errors are divided into M parts, where L is a multiple of M; for the sorted reconstruction errors, the rank size assigned to the i-th (1≤i≤M) part is R+i-1, where R is a positive integer.
4. A method for generating dialogues based on a sparse large language model, characterized in that: The following steps are involved: Step 1: Get the dialogue text entered by the user; Step 2: using the sparse large language model obtained by the large language model sparse method based on efficient parameter fine-tuning as described in any one of claims 1 to 3 to process the dialogue text input by the user to obtain a corresponding answer text; When the large language model performs sparse processing, the data used is a conversation dataset containing daily conversation fragments.
Citation Information
Patent Citations
Sparse training method of pre-training language model and deep language computing system
CN115222039A
Low-rank adaptation quantization fine tuning method and device for large language model
CN118035624A
Large language model acceleration method based on low-rank compression and implementation device
CN118396068A
Cited By
Method and device for performing parameter fine adjustment on pre-trained large model
CN119721176A
Improvement method of large language model, electronic equipment and storage medium
CN120688565A
Image information extraction method and device, electronic equipment and storage medium
CN121640472A
