Instruction fine-tuning method for quantized multi-modal large model, electronic device and storage medium

Abstract

This invention discloses a method for fine-tuning instructions in a quantized multimodal large model, comprising: step 1, quantization-aware scale learning; step 2, a multimodal warm-up strategy for scale learning; and step 3, using the quantized multimodal large model for task-specific learning. This invention employs quantization-aware scale learning, which adaptively minimizes quantization errors at outlier locations. Especially when activations at certain locations exhibit outlier features, it effectively reduces quantization errors within each quantization group, thus solving the problem of existing technologies' inability to effectively handle quantization errors caused by outliers. This invention uses uniform quantization and weight pruning to alleviate quantization difficulties in language tasks. After a certain number of iterations, this invention replaces the multimodal instruction dataset with a mixed dataset, which helps to gradually introduce language data while maintaining multimodal task performance, avoiding overfitting.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for fine-tuning instructions, an electronic device, and a storage medium for quantizing large multimodal models. Background Technology

[0002] The excellent performance of large language models (LLMs) has been fully demonstrated in various fields, which has also aroused researchers' interest in multimodal large language models (hereinafter referred to as "multimodal large models") and promoted substantial progress in many visual language tasks.

[0003] In existing technologies, large multimodal language models employ a modular architecture. Visual features extracted by the visual encoder are pre-transformed into the word embedding space of the large language model. Then, multimodal instruction data is used to guide and adjust the large language model, enabling it to understand images. This innovative approach allows the large language model to perform multimodal tasks in an autoregressive manner. A well-known method in this category is LLaVA, which uses a linear projection layer as a modular architecture to bridge the feature space gap between the visual encoder and the large language model.

[0004] However, despite significant progress in large multimodal models, the fine-tuning of visual language instructions for these models still suffers from considerable computational and memory overhead, primarily due to the inherently large size of the models themselves. While some existing research has reduced training overhead by introducing more efficient adapters and freezing large language models, adjusting visual language within the framework still requires substantial memory and computational resources. This poses a significant challenge to the rapid adaptation of large language models to cross-modal tasks, especially when training resources are limited and immediate, task-specific adjustments are required. To address these issues, this invention explores the potential of parameter quantization for large multimodal models, aiming to alleviate the massive training demands encountered during multimodal instruction fine-tuning while maintaining original performance. Summary of the Invention

[0005] The purpose of this invention is to provide an instruction fine-tuning method, electronic device, and storage medium for quantized multimodal large models, which solves the problems existing in the prior art and reduces the training requirements in the multimodal instruction fine-tuning process while maintaining the original performance, thereby reducing the computational load and storage requirements.

[0006] To achieve the above objectives, one solution of the present invention is:

[0007] A method for fine-tuning instructions for quantized multimodal large models includes the following steps:

[0008] Step 1: Scale learning of quantified perception

[0009] Create a learnable scaling factor s and initialize it to 1; define the weight adjustment formula for the linear layer as follows: Where w represents the linear layer weight. This represents the adjusted linear layer weights; the weight quantization formula for the linear layer is defined as follows: in This represents the quantized linear layer weights; `clamp()` truncates the input data to between 0 and 2. k In the range of -1, k represents the number of bits quantized. This indicates rounding operation, where Δ and zp represent the quantization step size and zero-point parameter, respectively.

[0010] Step 2: Multimodal warm-up strategy for scale learning

[0011] Given a multimodal instruction dataset D mm Mixed dataset D hybrid And the preset number of multimodal warm-up iterations η and the number of scale learning iterations σ, perform σ scale learning iterations:

[0012] Step 2-1: During the i-th scale learning, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into a quantized multimodal large model to compute the loss; otherwise, extract them from a mixed dataset D. hybrid Image-text pairs or plain text data are extracted and fed into a quantized multimodal large model to calculate the loss; where i∈[1,2,…,σ];

[0013] Step 2-2: Perform backpropagation and update the scaling factor s and the modular architecture θ of the multimodal large model using the gradient descent algorithm. a ;

[0014] Steps 2-3: Using the linear layer weight adjustment formula and weight quantization formula, requantize the multimodal large model to update the quantized linear layer weights;

[0015] Step 3: Use the quantized multimodal large model for task-specific learning.

[0016] For the number of instruction fine-tuning iterations T, at the t-th instruction fine-tuning iteration, the loss is calculated based on the downstream task type, and the modular architecture θ of the multimodal large model is trained using the gradient descent algorithm. a Where t∈[1,2,...,T];

[0017] Step 4: Output the quantized multimodal large model Where θ IThis represents a pre-trained image encoder.

[0018] Preferably, in step 2-1, for the input image-text pair, a pre-trained image encoder θ is used. I Extract image features and input them together with the text data in the image-text pair into a quantized multimodal large model to calculate the cross-entropy loss.

[0019] The second solution of the present invention is:

[0020] An electronic device includes a processor, a memory, and an application program; the application program is stored in the memory and configured to be executed by the processor as described in claim 1 or 2, using an instruction fine-tuning method for quantizing a multimodal large model.

[0021] The third solution of the present invention is:

[0022] A computer-readable storage medium storing a computer program; when executed in the computer, the computer program causes the computer to perform the instruction fine-tuning method for quantization of a multimodal large model as described in claim 1 or 2.

[0023] After adopting the above technical solution, the present invention has the following technical effects:

[0024] This invention employs quantization-aware scale learning, rather than using a low-rank adapter to fine-tune the quantized multimodal large model. This adaptively minimizes quantization errors at outlier locations, effectively reducing quantization errors within each quantization group, especially when activations at certain locations exhibit outlier features. This addresses the problem of existing technologies' inability to effectively handle quantization errors caused by outliers. The invention uses uniform quantization and weight pruning to alleviate quantization difficulties in language tasks, unlike NormalFormat which relies solely on the probability density of weights. After a certain number of iterations, the invention replaces the multimodal instruction dataset with a mixed dataset, helping to gradually introduce language data while maintaining multimodal task performance and avoiding overfitting. Finally, the invention returns a quantized multimodal large model, maintaining high task performance while reducing resource consumption. Attached Figure Description

[0025] Figure 1 This is a flowchart of the algorithm in a specific embodiment of the present invention;

[0026] Figure 2 This is a comparison chart of resource consumption between the present invention and existing technologies;

[0027] Figure 3 This is a performance comparison chart between the present invention and existing technologies. Detailed Implementation

[0028] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0029] Quantization is a network compression technique that converts full-precision weights into low-bit representations. It has been adopted in instruction fine-tuning of large language models, with QLoRA being a prominent example. Specifically, QLoRA quantizes the weights of each linear layer into a 4-bit NormalFormat data type and uses a low-rank adapter for fine-tuning. Combining quantization with this technique allows large language models to benefit from lightweight parameters and the small number of trainable parameters of the low-rank adapter. This enables QLoRA to perform LLaMA-65B fine-tuning on a single 48GB GPU without sacrificing chat performance. Unfortunately, while QLoRA can achieve performance comparable to full-precision performance in instruction fine-tuning tasks of large language models, it leads to a significant performance penalty in multimodal tasks, such as a nearly 4% reduction in accuracy on ScienceQA.

[0030] To address the significant performance loss in multimodal tasks, this invention focuses on language and image multimodal data, investigating the activation distribution of intermediate layers in large multimodal models. It reveals that a significant proportion of activations in multimodal inputs are outliers, posing a major challenge to the quantization of large multimodal models. This is because small weighted quantization errors can accumulate and interact with these outlier activations through matrix multiplication, leading to irreversible output distortion. Furthermore, the density and frequency of activation outliers are significantly increased in multimodal inputs compared to unimodal language inputs. This observation clarifies the performance degradation of the previous method QLoRA in instruction fine-tuning of large multimodal models. QLoRA's NormalFormat data type only aims to ensure the number of values ​​across all quantization intervals is equal, neglecting the impact of outliers and resulting in severe information loss. This makes it difficult for QLoRA to recover from the information loss caused by quantization and migrate to downstream tasks.

[0031] See Figure 1 This invention discloses a method for fine-tuning instructions for quantized multimodal large models, comprising the following steps:

[0032] Step 1: Scale learning of quantified perception

[0033] Create a learnable scaling factor s and initialize it to 1; define the weight adjustment of the linear layer as dividing the weights of the linear layer into multiple quantization groups, assigning a learnable scaling factor s to each quantization group, then the formula for the weight adjustment of the linear layer is: Where w represents the linear layer weight. This represents the adjusted linear layer weights; then, uniform quantization is used to quantize the adjusted linear layer weights, resulting in the linear layer weight quantization formula: in This represents the quantized linear layer weights; `clamp()` truncates the input data to between 0 and 2. k In the range of -1, k represents the number of bits quantized. This indicates rounding operation, where Δ and zp represent the quantization step size and zero-point parameter, respectively.

[0034] Step 2: Multimodal warm-up strategy for scale learning

[0035] Given a multimodal instruction dataset D mm Mixed dataset D hybrid And the preset number of multimodal warm-up iterations η and the number of scale learning iterations σ, perform σ scale learning iterations:

[0036] Step 2-1: During the i-th scale learning, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into a quantized multimodal large model to compute the loss; otherwise, extract them from a mixed dataset D. hybrid Image-text pairs or plain text data are extracted and fed into a quantized multimodal large model to calculate the loss; where i∈[1,2,…,σ];

[0037] Step 2-2: Perform backpropagation and update the scaling factor s and the modular architecture θ of the multimodal large model using the gradient descent algorithm. a ;

[0038] Steps 2-3: Using the weight adjustment formula and weight quantization formula of the linear layer, requantize the multimodal large model to update the weights of the quantized linear layer and reduce the loss caused by quantization error.

[0039] Step 3: Use the quantized multimodal large model for task-specific learning.

[0040] For the number of instruction fine-tuning iterations T, at the t-th instruction fine-tuning iteration, the loss is calculated based on the downstream task type, and the modular architecture s of the multimodal large model is trained using the gradient descent algorithm. a Where t∈[1,2,…,T];

[0041] Step 4: Output the quantized multimodal large model Where θ I This represents a pre-trained image encoder. θ a These are the final results obtained in steps 2 and 3, respectively.

[0042] Through the above scheme, this invention employs quantization-aware scale learning instead of using a low-rank adapter to fine-tune the quantized multimodal large model. This adaptively minimizes quantization errors at outlier locations, effectively reducing quantization errors within each quantization group, especially when activations at certain locations exhibit outlier features. This solves the problem of existing technologies being unable to effectively handle quantization errors caused by outliers. This invention uses uniform quantization and weight pruning to alleviate quantization difficulties in language tasks, rather than relying solely on the probability density of weights like NormalFormat. After a certain number of iterations, this invention replaces the multimodal instruction dataset with a mixed dataset, helping to gradually introduce language data while maintaining multimodal task performance and avoiding overfitting. Finally, this invention returns a quantized multimodal large model, maintaining high task performance while reducing resource consumption.

[0043] The following illustrates specific embodiments of the present invention.

[0044] In step 2-1 above, for the input image-text pair, a pre-trained image encoder θ is used. I Extract image features and input them together with the text data in the image-text pair into a quantized multimodal large model to calculate the cross-entropy loss.

[0045] Furthermore, the present invention also discloses an electronic device, including a processor, a memory, and an application program; the application program is stored in the memory and configured to be executed by the processor using the methods described above.

[0046] The present invention also discloses a computer-readable storage medium storing a computer program; when the computer program is executed in a computer, it causes the computer to perform the above-described method. The storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0047] The following are experimental data from specific embodiments of the present invention:

[0048] From an efficiency perspective, scale learning makes VL-guided tuning of MLLMs more parameter- and memory-efficient. For example, for a group size of 128, see [link to relevant documentation]. Figure 2 Scale learning reduces time and memory overhead by 1.4 times compared to the original full-precision instruction fine-tuning, and the number of parameters introduced is only 16.83% of the number of parameters that need to be fine-tuned using LoRA; at the same time, the performance in downstream tasks can be kept consistent with the original full-precision, and is 4% higher than the performance of fine-tuning using LoRA.

[0049] Furthermore, to more comprehensively validate the effectiveness of our method, we selected two types of MLLMs: LLaVA, which employs a two-stage fully fine-tuning strategy, and LaVIN, which employs a single-stage parameter-efficient fine-tuning strategy, for downstream task experiments. See [link to relevant documentation]. Figure 3 Scale learning methods outperformed methods that used LoRA for fine-tuning. For example, on LLaVA, scale learning achieved 91.04% performance, while LoRA only achieved 86.96%. On LaVIN, scale learning achieved 89.79% performance, while LoRA only achieved 88.27%.

[0050] The above embodiments are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. A method for fine-tuning instructions for quantized multimodal large models, characterized in that... Includes the following steps: Step 1: Scale learning of quantified perception Create a learnable scaling factor s and initialize it to 1; The weight adjustment formula for the linear layer is defined as follows: Where w represents the linear layer weight. This represents the adjusted linear layer weights; the weight quantization formula for the linear layer is defined as follows: in This represents the quantized linear layer weights; `clamp()` truncates the input data to between 0 and 2. k In the range of -1, k represents the number of bits quantized. This indicates rounding operation, where Δ and zp represent the quantization step size and zero-point parameter, respectively. Step 2: Multimodal warm-up strategy for scale learning Given a multimodal instruction dataset D mm Mixed dataset D hybrid And the preset number of multimodal warm-up iterations η and the number of scale learning iterations σ, perform σ scale learning iterations: Step 2-1: During the i-th scale learning, if i < η, from the multimodal instruction dataset D mm Extract image-text pairs and feed them into a quantized multimodal large model to compute the loss; otherwise, extract them from a mixed dataset D. hybrid Image-text pairs or plain text data are extracted and fed into a quantized multimodal large model to calculate the loss; where i∈[1,2,...,σ]; Step 2-2: Perform backpropagation and update the scaling factor s and the modular architecture θ of the multimodal large model using the gradient descent algorithm. a ; Steps 2-3: Using the linear layer weight adjustment formula and weight quantization formula, requantize the multimodal large model to update the quantized linear layer weights; Step 3: Use the quantized multimodal large model for task-specific learning. For the number of instruction fine-tuning iterations T, at the t-th instruction fine-tuning iteration, the loss is calculated based on the downstream task type, and the modular architecture θ of the multimodal large model is trained using the gradient descent algorithm. a Where t∈[1,2,...,T]; Step 4: Output the quantized multimodal large model Where θ I This represents a pre-trained image encoder.

2. The instruction fine-tuning method for quantization of a large multimodal model as described in claim 1, characterized in that: In step 2-1, for the input image-text pair, a pre-trained image encoder θ is used. I Extract image features and input them together with the text data in the image-text pair into a quantized multimodal large model to calculate the cross-entropy loss.

3. An electronic device, characterized in that... It includes a processor, memory, and an application program; the application program is stored in the memory and configured to be executed by the processor as described in claim 1 or 2, using the instruction fine-tuning method for quantizing a multimodal large model.

4. A computer-readable storage medium, characterized in that... The storage medium stores a computer program; when the computer program is executed in the computer, it causes the computer to perform the instruction fine-tuning method for quantization of a multimodal large model as described in claim 1 or 2.

Images