Zoom adjustment method and dialogue generation method of low-rank matrix for efficient fine tuning of quantized large language model
By adopting the scaling adjustment method of low-rank matrix in large language models and the strategy of keeping the quantization format unchanged, the overfitting problems caused by underfitting caused by the quantization bit width and data duplication are solved, and the fine-tuning performance of the model is significantly improved and computational efficiency.
Patent Information
- Application Number
- CN202411037233.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-10
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing quantization and fine-tuning techniques have problems in large language models with low quantization bit width leading to underfitting, data duplication leading to overfitting, and difficulty in maintaining quantization formats, resulting in degradation in model performance.
Adopt the scaling adjustment method of low-rank matrix and the strategy of keeping the quantization format unchanged. By iterating batch by batch, iterating and using the low-rank matrix of the same granularity of the quantization network, avoid errors caused by the quantization bit width, keep the quantization format unchanged, and update Model weights to improve fine-tuning performance.
It effectively avoids underfitting and overfitting, improves the performance of the model under low bit quantization, ensures the stability of model fine-tuning and efficient use of computing resources, and significantly improves the performance of the model.
Smart Images
Figure CN119106117A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of large language models, and in particular to a scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model and a dialogue generation method. Background Art
[0002] In recent years, large language models have shown excellent performance in various fields and have attracted widespread attention. However, the increase in the number of parameters of these models makes training and fine-tuning increasingly challenging. This has led to a shift in research focus on how to efficiently enhance model performance in order to save resource overhead. Parameter efficient fine-tuning and quantization have become prominent methods for improving training efficiency and compressing models. Quantization is a technique that converts the weights and activation values of a model from high-precision floating-point numbers to low-bit fixed-point numbers. This method can reduce the size of the model and improve memory access efficiency, thereby speeding up the computation process. Therefore, the combination of quantization and fine-tuning techniques is widely used because it requires less resources and is efficient and fast. Although quantization and efficient fine-tuning techniques have made significant progress in efficient fine-tuning of large models, they have some major drawbacks when applied to large language models: 1. Low quantization bit width leads to underfitting: When processing large models, the impact of quantization bit width is crucial to the performance of large models, especially when quantized at extremely low bits, such as 2 bits, the accuracy of large models will drop seriously, which will lead to a decrease in the ability of the basic model and fine-tuning will tend to underfit.
[0003] 2. Overfitting caused by data duplication: When fine-tuning a large number of data sets, data duplication is often difficult to avoid, or a single small data set is repeated multiple times. In both cases, fine-tuning is likely to cause overfitting, resulting in degraded model performance.
[0004] 3. Difficulty in maintaining the quantization format: Existing methods cannot maintain the quantization format during fine-tuning, or it is difficult to effectively control the learning of the model while maintaining the quantization format.
[0005] In summary, the existing combined quantization and fine-tuning methods face significant challenges when applied to large model fine-tuning, mainly manifested in underfitting due to low quantization bit width, overfitting due to data duplication, and difficulty in maintaining the quantization format. These shortcomings need to be addressed through more sophisticated and optimized methods to achieve a more efficient combination of model quantization and model fine-tuning. Summary of the invention
[0006] The purpose of the present invention is to provide a low-rank matrix scaling adjustment method and a dialogue generation method for efficient fine-tuning of a quantized large language model, which solves the problems of improper scaling adjustment of low-rank matrices under different quantization bit widths in the prior art and improves the performance of fine-tuning of large language models by introducing a low-rank matrix scaling adjustment strategy and a method for keeping the quantization format unchanged.
[0007] To achieve the above object, the technical solution adopted by the present invention is: A scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model, the method comprising the following steps: Step 1: Data preparation: Select a data set from the training corpus and segment the data in the data set into tokens. After segmentation, the length of the question part of each segment is limited to 1024 tokens, and the length of the label part is limited to 256 tokens. Step 2: Weight quantization: quantize the weight W of the large language model by N bits to obtain the quantized weight Wq, and determine the quantization step size Scale and zero point ZeroPoint; Step 3: Generate low-rank matrix: Generate granularity and quantize network parameters Low-rank matrices of the same granularity; Step 4: Scaling adjustment processing: Calculate the scaling ratio according to the bit width N of the quantization weight Wq; Step 5: Apply fine-tuning: Apply the low-rank matrix and the data processed in step 1 for forward calculation and back propagation, and keep the backbone network parameters Update the low-rank matrix parameters for fine-tuning in the frozen state; Step 6: Determine whether the preset number of iterations has been reached. If not, continue iterating, apply the fine-tuned low-rank matrix and repeat steps 5-6; otherwise, update the weights of the large language model based on the fine-tuned low-rank matrix and quantized weights. , according to the updated weight Get a new large language model and execute step 7; Step 7: Model evaluation: The large language model obtained in step 4 is evaluated using different datasets mainly based on MMLU to verify its performance improvement.
[0008] In step 2, the quantization weight Wq, the quantization step size Scale and the zero point ZeroPoint are calculated as follows: ZeroPoint=min(W),
[0009] In step 4, the scaling ratio is calculated as follows: , Among them, alpha is a hyperparameter; rank is the intermediate dimension of the low-rank matrix, which is adjusted as a hyperparameter; Related to the bit width N of the quantization weight, .
[0010] In step 6, the updated large language model weight as follows:
[0011] Among them, A', B' are corresponding Fine-tuned low-rank matrix, A, B corresponds to Fine-tuned low-rank matrix.
[0012] A method for generating a dialogue based on a large language model, the method comprising the following steps: Step 1: Get the dialogue text entered by the user; Step 2: Using the large language model obtained by the above scaling adjustment method to process the dialogue text input by the user to obtain the corresponding answer text; When the large language model performs scaling and adjustment processing, the data used is a conversation data set containing daily conversation fragments.
[0013] A scaling and adjustment system for a low-rank matrix for efficient fine-tuning of a quantized large language model, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a scaling and adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model as described above.
[0014] A computer-readable storage medium having a computer program / instruction stored thereon, wherein when the computer program / instruction is executed by a processor, the steps of a scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model as described above are implemented.
[0015] A computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of a scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model as described above are implemented.
[0016] After adopting the above scheme, the present invention will iterate all data batch by batch during the process of weight fine-tuning, and the iterative process is used to calculate forward calculation and reverse calculation to optimize the model. And the influence of the quantization bit width is considered in each iteration to avoid underfitting and overfitting problems caused by errors caused by different bit quantization. At the same time, in each iteration, this patent uses a low-rank matrix with the same granularity as the quantization network, so that the quantization format can be kept unchanged after the quantization fine-tuning is completed. Therefore, the problems such as improper scaling and adjustment of low-rank matrices under different quantization bit widths in the prior art are solved, and the performance of fine-tuning of large language models is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1This is a performance comparison chart of the large language model obtained by the present invention and the existing large language model. DETAILED DESCRIPTION
[0018] The present invention discloses a scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model, which comprises the following steps: Step 1: Data preparation: Select a data set from the training corpus and segment the data in the data set into tokens. After segmentation, the length of the question part of each segment is limited to 1024 tokens, and the length of the label part is limited to 256 tokens.
[0019] In this embodiment, a data set is selected from the Flan-v2 or Alpaca training corpus.
[0020] Flan-v2 is a dataset developed by Google Research to optimize instruction tuning of language models. Flan-v2 contains multiple sub-datasets that are formatted using a variety of templates to improve the performance of models on various tasks. Alpaca is an open source dataset released by Stanford University that aims to improve the performance of language models on different instruction tasks. It is fine-tuned based on OpenAI's GPT-3 model and uses about 50,000 instruction-response pairs generated by GPT-3.
[0021] In this embodiment, 320k data are sampled from Flan-v2, or all 52k data sets are directly selected from Alpaca. After the data in the data set is tokenized, the length of the question part of each segment is limited to 1024 tokens, and the length of the tag part is limited to 256 tokens.
[0022] Step 2: Weight quantization: quantize the weight W of the large language model by N bits to obtain the quantized weight Wq, determine the quantization step size Scale and zero point ZeroPoint, and the calculation formula is: ZeroPoint=min(W),
[0023] In the quantization process, the GPTQ method (or other PTQ methods) is used in this embodiment. PTQ is a technology that quantizes the model after it is trained. Its main goal is to convert floating-point weights and activation functions into low-precision representations (such as 4-bit integers) to reduce the storage space and computational complexity of the model. GPTQ is a variant of PTQ that optimizes the quantization process by introducing gradient information, thereby reducing the error caused by quantization.
[0024] Step 3: Generate low-rank matrix: Generate granularity and quantize network parameters Low-rank matrices of the same granularity.
[0025] The low-rank matrix is generated randomly, and the requirement is that the granularity of the low-rank matrix is the same as the network parameters. Just keep the same particle size.
[0026] Step 4: Scaling adjustment processing: Calculate the scaling ratio according to the bit width N of the quantization weight Wq. The formula is: , where alpha is a hyperparameter and is usually not adjusted; rank is the middle dimension of the low-rank matrix and is adjusted as a hyperparameter, and its value is usually 64 or 128; Related to the bit width N of the quantization weight, .
[0027] Step 5: Apply fine-tuning: Apply the low-rank matrix and the data processed in step 1 for forward calculation and back propagation, and keep the backbone network parameters In the frozen case, the low-rank matrix parameters are updated for fine-tuning.
[0028] Step 6: Determine whether the preset number of iterations has been reached. If not, continue iterating, apply the fine-tuned low-rank matrix and repeat steps 5-6; otherwise, update the weights of the large language model based on the fine-tuned low-rank matrix and quantized weights. , according to the updated weight Get the new large language model and execute step 7.
[0029] Updated large language model weights as follows:
[0030] Among them, A', B' are corresponding Fine-tuned low-rank matrix, A, B are corresponding The fine-tuned low-rank matrix, Wq, Scale, and ZeroPoint are obtained in step 2, and scaling is obtained in step 3.
[0031] Step 7: Model evaluation: The large language model obtained in step 4 is evaluated using different datasets mainly based on MMLU to verify its performance improvement.
[0032] like Figure 1 As shown, compared with the existing large language model, the performance of the large language model obtained by the method of the present invention can be effectively improved. The figure shows the performance of Llama-7b under 2 bits.
[0033] The present invention also discloses a dialogue generation method based on a large language model, which comprises the following steps: Step 1: Get the dialogue text entered by the user; Step 2: Use the large language model obtained by the above-mentioned scaling adjustment method to process the dialogue text input by the user to obtain the corresponding answer text.
[0034] When the large language model is scaling and adjusting, the data used is a conversation dataset containing daily conversation fragments.
[0035] In practical applications, the dialogue generation method based on a large language model obtains answer texts with higher accuracy and better fits dialogue data.
[0036] In summary, the present invention has the following beneficial effects: First, by considering the fine-tuning error caused by the quantization bit width in each fine-tuning, the accumulation of quantization error and fine-tuning error under low-bit quantization is avoided, thereby improving the overall performance of the model.
[0037] 2. The present invention adjusts the scaling of the low-rank matrix according to the quantization bit width in each iteration, further adjusts the convergence curve of the model, and ensures the stability of model fine-tuning.
[0038] 3. The present invention significantly improves the performance of the model under low-bit quantization through a scaling adjustment strategy for low-rank matrices and a fine-tuning method that keeps the quantization format unchanged.
[0039] 4. The present invention significantly reduces the consumption of computing resources through a large-scale fine-tuning low-rank matrix process, making model compression more efficient.
[0040] 5. The method of the present invention shows significant performance improvement on different models and data sets, and has wide applicability and versatility.
[0041] In summary, the present invention effectively solves the problems of error accumulation and improper scaling of low-rank matrices existing in the prior art by introducing a scaling adjustment strategy for low-rank matrices and a method for keeping the quantization format unchanged, thereby significantly improving the fine-tuning effect and model performance.
[0042] The present invention also provides a scaling and adjustment system for low-rank matrices for efficient fine-tuning of quantized large language models, which includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the scaling and adjustment method for low-rank matrices for efficient fine-tuning of quantized large language models as described above.
[0043] The present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device.
[0044] The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.
[0045] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0046] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0047] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.
[0048] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A scaling method for low-rank matrices for efficient fine-tuning of quantized large language models, characterized in that: The method comprises the following steps: Step 1: Data preparation: Select a data set from the training corpus and segment the data in the data set into tokens. After segmentation, the length of the question part of each segment is limited to 1024 tokens, and the length of the label part is limited to 256 tokens. Step 2: Weight quantization: quantize the weight W of the large language model by N bits to obtain the quantized weight Wq, and determine the quantization step size Scale and zero point ZeroPoint; Step 3: Generate low-rank matrix: Generate granularity and quantize network parameters Low-rank matrices of the same granularity; Step 4: Scaling adjustment processing: Calculate the scaling ratio according to the bit width N of the quantization weight Wq; Step 5: Apply fine-tuning: Apply the low-rank matrix and the data processed in step 1 for forward calculation and back propagation, and keep the backbone network parameters Update the low-rank matrix parameters for fine-tuning in the frozen state; Step 6: Determine whether the preset number of iterations has been reached. If not, continue iterating, apply the fine-tuned low-rank matrix and repeat steps 5-6; otherwise, update the weights of the large language model based on the fine-tuned low-rank matrix and quantized weights. , according to the updated weight Get a new large language model and execute step 6; Step 7: Model evaluation: The large language model obtained in step 4 is evaluated using different datasets mainly based on MMLU to verify its performance improvement.
2. The method for scaling a low-rank matrix for efficient fine-tuning of a quantized large language model according to claim 1, characterized in that: In step 2, the quantization weight Wq, the quantization step size Scale and the zero point ZeroPoint are calculated as follows: ZeroPoint=min(W), 3. The method for scaling a low-rank matrix for efficient fine-tuning of a quantized large language model according to claim 1, characterized in that: In step 4, the scaling ratio is calculated as follows: ， Among them, alpha is a hyperparameter; rank is the intermediate dimension of the low-rank matrix, which is adjusted as a hyperparameter; Related to the bit width N of the quantization weight, .
4. The method for scaling a low-rank matrix for efficient fine-tuning of a quantized large language model according to claim 1, characterized in that: In step 6, the updated large language model weight as follows: Among them, A', B' are corresponding Fine-tuned low-rank matrix; A, B are corresponding Fine-tuned low-rank matrix.
5. A method for generating dialogue based on a large language model, characterized in that: The method comprises the following steps: Step 1: Get the dialogue text entered by the user; Step 2: Process the dialogue text input by the user using the large language model obtained by the scaling adjustment method of the low-rank matrix for efficient fine-tuning of the quantized large language model as described in any one of claims 1 to 4 to obtain a corresponding answer text; When the large language model performs scaling and adjustment processing, the data used is a conversation data set containing daily conversation fragments.
6. A low-rank matrix scaling system for efficient fine-tuning of quantized large language models, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a scaling adjustment method for a low-rank matrix for efficient fine-tuning of a quantized large language model as described in any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by the processor, the steps of a method for scaling and adjusting a low-rank matrix for efficient fine-tuning of a quantized large language model as described in any one of claims 1-4 are implemented.
8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the steps of a method for scaling and adjusting a low-rank matrix for efficient fine-tuning of a quantized large language model as described in any one of claims 1-4 are implemented.