Instruction fine tuning method for quantized multi-modal large model, electronic equipment and storage medium

Through quantitative perception scale learning and uniform quantization technology, the problem of high resource consumption in the process of fine-tuning of visual language instructions by multimodal large models is solved, and the effect of maintaining high task performance while reducing computing and storage needs is achieved.

CN119089967AActive Publication Date: 2024-12-06XIAMEN UNIV

Patent Information

Application Number
CN202411002426.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-12-06
Estimated Expiration
2044-07-25

AI Technical Summary

Technical Problem

The existing multimodal large models have a high computational and memory burden during the fine-tuning of visual language instructions, resulting in excessive consumption of training resources, especially in the case of limited resources and real-time task adjustments, which is difficult to adapt quickly.

Method used

Using quanti-aware scale learning methods, quantization errors are adaptively reduced by creating learnable scale factors and uniform quantization techniques, and mixed data sets are gradually introduced during the iteration process to avoid overfitting and reduce computing and storage requirements.

Benefits of technology

It effectively reduces the calculation and storage requirements in the fine-tuning process of multimodal instruction, while maintaining the original performance, especially when dealing with outlier characteristics, reduces quantization errors and improves resource utilization efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089967A_ABST
    Figure CN119089967A_ABST
Patent Text Reader

Abstract

The invention discloses an instruction fine tuning method for a quantized multi-mode large model. The instruction fine tuning method comprises the following steps: step 1, scale learning of quantized perception; 2, a multi-mode warm-up strategy used for scale learning; and step 3, carrying out specific task learning by using the quantized multi-modal large model. According to the method, scale learning of quantization perception is adopted, quantization errors of abnormal positions can be minimized in a self-adaptive manner, and particularly, the quantization errors in each quantization group can be effectively reduced under the condition that certain position activation represents abnormal value features; therefore, the problem that quantization errors caused by outliers cannot be effectively processed in the prior art is solved; according to the method, through a uniform quantification method, weight cutting is adopted to relieve quantification difficulty occurring in a language task; according to the method, after iteration reaches a certain number of times, a multi-modal instruction data set is replaced with a mixed data set, so that language data is gradually introduced while multi-modal task performance is maintained, and overfitting is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an instruction fine-tuning method, electronic device and storage medium for a quantized multi-modal large model. Background Art

[0002] The excellent performance of Large Language Models (LLMs) has been fully demonstrated in various fields, which has also aroused researchers' interest in multimodal large language models (hereinafter referred to as "multimodal large models") and promoted substantial progress in many visual language tasks.

[0003] In the existing technology, the multimodal large model adopts a modular architecture, which converts the visual features extracted by the visual encoder into the word embedding space of the large language model in advance, and then uses the multimodal instruction data to guide and adjust the large language model, so that the large language model has the ability to understand images. This innovative method enables the large language model to perform multimodal tasks in an autoregressive manner. A well-known method of this type of technology is LLaVA, which bridges the feature space gap between the visual encoder and the large language model by adopting a linear projection layer as a modular architecture.

[0004] However, although great progress has been made in large multimodal models, the current fine-tuning process of visual language instructions for large multimodal models has considerable redundancy in terms of computation and memory burden, mainly due to the inherent large size of the large language model itself. In some prior art studies, although the training overhead has been reduced by introducing more efficient adapters and freezing large language models, the adjustment of visual language within the framework still requires a large amount of memory usage and computing resources, which poses a huge challenge to the rapid adaptation of large language models to cross-modal tasks, especially when training resources are limited and immediate, task-specific adjustments are required. In order to solve the above problems, the present invention explores the potential of parameter quantization for large multimodal models, aiming to alleviate the huge training requirements encountered in the fine-tuning process of multimodal instructions while maintaining the original performance. Summary of the invention

[0005] The purpose of the present invention is to provide an instruction fine-tuning method, electronic device and storage medium for a quantized multimodal large model, to solve the problems existing in the prior art, and to reduce the training requirements in the multimodal instruction fine-tuning process while maintaining the original performance, thereby reducing the computing load and storage requirements.

[0006] In order to achieve the above object, one of the solutions of the present invention is:

[0007] A method for fine-tuning instructions for a quantized multi-modal large model, comprising the following steps:

[0008] Step 1: Quantization-aware scale learning

[0009] Create a learnable scale factor s and initialize it to 1; define the weight adjustment formula of the linear layer as Where w represents the linear layer weight, Represents the adjusted linear layer weight; the weight quantization formula of the linear layer is defined as in Represents the quantized linear layer weights, and clamp() means truncating the input data to 0 to 2 k -1, k represents the number of quantization bits. represents the rounding operation, Δ and zp represent the quantization step size and zero point parameter respectively;

[0010] Step 2: Multimodal warm-up strategy for scale learning

[0011] For a given multimodal instruction dataset D mm , mixed data set D hybrid , and the preset multimodal warm-up iteration number η, scale learning iteration number σ, perform σ scale learning iterations:

[0012] Step 2-1: When learning at the i-th scale, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into the quantized multimodal large model to calculate the loss, otherwise hybrid Extract image-text pairs or plain text data and feed them into the quantized multimodal large model to calculate the loss; where i∈[1,2,…,σ];

[0013] Step 2-2: Perform back propagation and use the gradient descent algorithm to update the scale factor s and the modular architecture θ of the multimodal large model. a ;

[0014] Step 2-3: Use the weight adjustment formula and weight quantization formula of the linear layer to re-quantize the multimodal large model to update the quantized linear layer weights;

[0015] Step 3: Use the quantized multimodal large model for specific task learning

[0016] For the number of instruction fine-tuning iterations T, at the tth instruction fine-tuning, the loss is calculated according to the downstream task type, and the modular architecture θ of the multimodal large model is trained using the gradient descent algorithm a ; where t∈[1,2,...,T];

[0017] Step 4: Output the quantized multimodal large model where θ IRepresents a pre-trained image encoder.

[0018] Preferably, in step 2-1, for the input image-text pair, a pre-trained image encoder θ is used. I Image features are extracted and input into the quantized multimodal large model together with the text data in the image-text pair to calculate the cross entropy loss.

[0019] The second solution of the present invention is:

[0020] An electronic device comprises a processor, a memory and an application program; the application program is stored in the memory and is configured to be executed by the processor to perform the instruction fine-tuning method for a quantized multimodal large model as claimed in claim 1 or 2.

[0021] The third solution of the present invention is:

[0022] A computer-readable storage medium storing a computer program; when the computer program is executed in the computer, the computer executes the instruction fine-tuning method for a quantized multimodal large model as claimed in claim 1 or 2.

[0023] After adopting the above technical solution, the present invention has the following technical effects:

[0024] The present invention adopts quantization-aware scale learning instead of using a low-rank adapter to fine-tune the quantized multimodal large model, which can adaptively minimize the quantization error of abnormal positions, especially when the activation of certain positions exhibits abnormal value characteristics, it can effectively reduce the quantization error within each quantization group, thereby solving the problem that the prior art cannot effectively handle the quantization error caused by outliers; the present invention uses a uniform quantization method and weight clipping to alleviate the quantization difficulties occurring in language tasks, instead of relying solely on the probability density of weights like NormalFormat; after a certain number of iterations, the present invention replaces the multimodal instruction data set with a mixed data set, which helps to gradually introduce language data while maintaining multimodal task performance and avoid overfitting; the present invention finally returns to the quantized multimodal large model, while reducing resource consumption and maintaining high task performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an algorithm flow chart of a specific embodiment of the present invention;

[0026] Figure 2 A data diagram showing a comparison of resource expenditures between the present invention and the prior art;

[0027] Figure 3 This is a data diagram comparing the task performance of the present invention and the prior art. DETAILED DESCRIPTION

[0028] In order to further explain the technical solution of the present invention, the present invention is described in detail below through specific embodiments.

[0029] "Quantization" is a network compression technique that converts full-precision weights into low-bit representations. It has been adopted in instruction fine-tuning of large language models. Its famous work is QLoRA. Specifically, QLoRA quantizes the weights of each linear layer to a 4-bit NormalFormat data type and uses a low-rank adapter for fine-tuning. Combined with quantization technology, large language models have the advantages of lightweight parameters and low-rank adapters with only a small number of trainable parameters, allowing QLoRA to perform LLaMA-65B fine-tuning on a single 48GB GPU without losing chat performance. Unfortunately, although QLoRA can achieve performance comparable to full-precision performance in instruction fine-tuning tasks for large language models, it causes considerable performance losses in multimodal tasks, such as almost a 4% reduction in accuracy on ScienceQA.

[0030] In order to solve the huge performance loss problem in multimodal tasks, the present invention focuses on language and image multimodal data, studies the activation distribution of the intermediate layer corresponding to the multimodal large model, and finds that a significant proportion of activations in multimodal inputs appear as outliers, which poses a major challenge to the quantization of multimodal large models. This is because small weight quantization errors may accumulate and interact with the activation values ​​of these activation anomalies through matrix multiplication, causing the output to be irreversibly distorted. In addition, the density and frequency of activation anomalies in multimodal inputs are significantly increased compared to unimodal language inputs. This observation explains why the previous method QLoRA has degraded performance in instruction fine-tuning of multimodal large models, because the NormalFormat data type it adopts only seeks to make the number of values ​​in all quantization intervals equal, without considering the influence of activation value outliers, and leads to serious information loss, making it difficult for QLoRA to recover from the information loss caused by quantization and migrate to downstream tasks.

[0031] See also Figure 1 The present invention discloses a method for fine-tuning instructions for a quantized multi-modal large model, comprising the following steps:

[0032] Step 1: Quantization-aware scale learning

[0033] Create a learnable scale factor s and initialize it to 1; define the weight adjustment of the linear layer as dividing the weight of the linear layer into multiple quantization groups, each quantization group is assigned a learnable scale factor s, then the weight adjustment formula of the linear layer is Where w represents the linear layer weight, Represents the adjusted linear layer weight; then, the adjusted linear layer weight is quantized using uniform quantization, and the weight quantization formula of the linear layer is in Represents the quantized linear layer weights, and clamp() means truncating the input data to 0 to 2 k -1, k represents the number of quantization bits. represents the rounding operation, Δ and zp represent the quantization step size and zero point parameter respectively;

[0034] Step 2: Multimodal warm-up strategy for scale learning

[0035] For a given multimodal instruction dataset D mm , mixed data set D hybrid , and the preset multimodal warm-up iteration number η, scale learning iteration number σ, perform σ scale learning iterations:

[0036] Step 2-1: When learning at the i-th scale, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into the quantized multimodal large model to calculate the loss, otherwise hybrid Extract image-text pairs or plain text data and feed them into the quantized multimodal large model to calculate the loss; where i∈[1,2,…,σ];

[0037] Step 2-2: Perform back propagation and use the gradient descent algorithm to update the scale factor s and the modular architecture θ of the multimodal large model. a ;

[0038] Step 2-3: Use the weight adjustment formula and weight quantization formula of the linear layer to re-quantize the multimodal large model to update the quantized linear layer weights and reduce the loss caused by quantization error;

[0039] Step 3: Use the quantized multimodal large model for specific task learning

[0040] For the number of instruction fine-tuning iterations T, at the tth instruction fine-tuning, the loss is calculated according to the downstream task type, and the modular architecture s of the multimodal large model is trained using the gradient descent algorithm a ; where t∈[1,2,…,T];

[0041] Step 4: Output the quantized multimodal large model where θ I represents the pre-trained image encoder, θ a These are the final results obtained from step 2 and step 3 respectively.

[0042] Through the above scheme, the present invention adopts quantization-aware scale learning instead of using a low-rank adapter to fine-tune the quantized multimodal large model, which can adaptively minimize the quantization error of abnormal positions, especially when the activation of certain positions exhibits abnormal value characteristics, it can effectively reduce the quantization error within each quantization group, thereby solving the problem that the prior art cannot effectively handle the quantization error caused by outliers; the present invention uses a uniform quantization method and weight clipping to alleviate the quantization difficulties occurring in language tasks, instead of relying solely on the probability density of weights like NormalFormat; after a certain number of iterations, the present invention replaces the multimodal instruction data set with a mixed data set, which helps to gradually introduce language data while maintaining multimodal task performance and avoid overfitting; the present invention finally returns to the quantized multimodal large model, reducing resource consumption while maintaining high task performance.

[0043] Specific embodiments of the present invention are shown below.

[0044] In step 2-1 above, for the input image-text pair, the pre-trained image encoder θ is used I Image features are extracted and input into the quantized multimodal large model together with the text data in the image-text pair to calculate the cross entropy loss.

[0045] In addition, the present invention also discloses an electronic device, including a processor, a memory and an application program; the application program is stored in the memory and is configured to execute the above method by the processor.

[0046] The present invention also discloses a computer-readable storage medium, which stores a computer program; when the computer program is executed in a computer, the computer executes the above method. The storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0047] The following are experimental data of specific embodiments of the present invention:

[0048] From the efficiency point of view, scale learning makes VL-guided adjustment of MLLMs more parameter-efficient and memory-efficient. For example, for a group size of 128, see Figure 2 Compared with the original full-precision instruction fine-tuning, the time and memory overhead of scale learning are reduced by 1.4 times, and the number of parameters introduced is only 16.83% of the number of parameters that need to be fine-tuned using LoRA; at the same time, the performance on downstream tasks can also remain consistent with the original full-precision, which is 4% higher than the performance of fine-tuning using LoRA.

[0049] In addition, to more fully verify the effectiveness of our method, we selected two types of MLLM: LLaVA with a two-stage full fine-tuning strategy and LaVIN with a single-stage parameter efficient fine-tuning strategy for downstream task experiments, see Figure 3 , the scale learning method is better than the method using LoRA for fine-tuning. For example, on LLaVA, the scale learning method achieved 91.04%, while LoRA only had 86.96% performance. On LaVIN, the scale learning method achieved 89.79% while LoRA only had 88.27%.

[0050] The above embodiments do not limit the product form and style of the present invention. Any appropriate changes or modifications made by ordinary technicians in the relevant technical field should be deemed to be within the patent scope of the present invention.

Claims

1. A method for fine-tuning instructions for a quantized multi-modal large model, characterized in that The following steps are involved: Step 1: Quantization-aware scale learning Create a learnable scale factor s and initialize it to 1; The weight adjustment formula for the linear layer is defined as Where w represents the linear layer weight, Represents the adjusted linear layer weight; the weight quantization formula of the linear layer is defined as in Represents the quantized linear layer weights, and clamp() means truncating the input data to 0 to 2 k -1, k represents the number of quantization bits. represents the rounding operation, Δ and zp represent the quantization step size and zero point parameter respectively; Step 2: Multimodal warm-up strategy for scale learning For a given multimodal instruction dataset D mm , mixed data set D hybrid , and the preset multimodal warm-up iteration number η, scale learning iteration number σ, perform σ scale learning iterations: Step 2-1: When learning at the i-th scale, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into the quantized multimodal large model to calculate the loss, otherwise hybrid Extract image-text pairs or plain text data and feed them into the quantized multimodal large model to calculate the loss; where i∈[1,2,...,σ]; Step 2-2: Perform back propagation and use the gradient descent algorithm to update the scale factor s and the modular architecture θ of the multimodal large model. a ; Step 2-3: Use the weight adjustment formula and weight quantization formula of the linear layer to re-quantize the multimodal large model to update the quantized linear layer weights; Step 3: Use the quantized multimodal large model for specific task learning For the number of instruction fine-tuning iterations T, at the tth instruction fine-tuning, the loss is calculated according to the downstream task type, and the modular architecture θ of the multimodal large model is trained using the gradient descent algorithm a ; where t∈[1,2,...,T]; Step 4: Output the quantized multimodal large model where θ I Represents a pre-trained image encoder.

2. The instruction fine-tuning method for a quantized multi-modal large model according to claim 1, characterized in that: In step 2-1, for the input image-text pair, use the pre-trained image encoder θ I Image features are extracted and input into the quantized multimodal large model together with the text data in the image-text pair to calculate the cross entropy loss.

3. An electronic device, characterized in that The system comprises a processor, a memory and an application program; the application program is stored in the memory and is configured to be executed by the processor as claimed in claim 1 or 2 to execute the instruction fine-tuning method for a quantized multimodal large model.

4. A computer-readable storage medium, characterized in that The storage medium stores a computer program; when the computer program is executed in the computer, the computer is caused to execute the instruction fine-tuning method for a quantized multimodal large model as described in claim 1 or 2.

Citation Information

Patent Citations

  • Method for improving efficiency of adapting large language model to multi-modal task

    CN117194989A

  • Memory efficient and quantitative perception fine tuning method and device for large language model

    CN118350420A

  • Multi-modal prompt learning for representation transfer on image recognition tasks

    US20240220722A1

Cited By

  • Visual language large model hybrid quantification method and device and electronic equipment

    CN119441513A