A continuous visual command fine-tuning method based on separable mixture low-rank adaptation

Abstract

The present invention discloses a continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation, which relates to the field of model deep learning and continuous learning technology; S1, creating a pre-training model; S2, inputting a continuous visual instruction fine-tuning data set sequence into the pre-training model; S3, selecting a suitable low-rank adaptation matrix according to the continuous visual instruction fine-tuning data set and performing fine-tuning; S4, using separable routing technology to obtain S3's visual understanding routing selection and instruction following routing selection; S5, performing weighted operation on the separable routing result of S4 and its corresponding adaptability matrix to obtain two corresponding outputs; S6, using adaptive fusion technology to obtain the final output of the low-rank adaptation module; the present invention adopts the above-mentioned continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation to ensure that the model does not interfere when processing visual and instruction tasks, effectively avoiding double catastrophic forgetting.

Description

Technical Field

[0001] The present invention relates to the field of model deep learning and continuous learning technology, and in particular to a continuous visual instruction fine-tuning method based on separable mixed low-rank adaptation. Background Art

[0002] Existing technologies have problems such as serious resource consumption, flexible usage scenarios, and weak ability to solve forgetting, mainly including the following points: regularization methods have weak ability to solve forgetting, and for models with larger parameters, huge training resources are required, making it difficult to effectively achieve reliable continuous instruction fine-tuning; memory replay methods need to save part of the data of each past task, which greatly increases the memory resource usage and is not feasible in actual application scenarios; hybrid low-rank adaptation usually uses the calculation of the routing network to assign different low-rank adaptation matrices to the input. In the scenario of continuous visual instruction fine-tuning, the continuous dynamic changes of the task make it impossible for the routing network to effectively assign the most reasonable low-rank adaptation matrix to the input, resulting in catastrophic forgetting.

[0003] To address the above shortcomings, we first noticed two catastrophic forgetting phenomena in the continuous visual instruction fine-tuning scenario: visual understanding forgetting, that is, the model cannot correctly recognize the content of the input image; instruction following forgetting, that is, the model cannot correctly give the corresponding correct answer format for the input text instruction. Therefore, it is urgent to effectively alleviate the double forgetting problem in continuous visual instruction fine-tuning. Summary of the Invention

[0004] The purpose of this invention is to provide a continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation, which utilizes hybrid low-rank adaptation and separable routing technology to efficiently solve the double catastrophic forgetting problem without sacrificing the model's prior learning ability.

[0005] To achieve the above objectives, the present invention provides a method for continuous visual instruction fine-tuning based on separable hybrid low-rank adaptation, comprising the following steps:

[0006] S1. Create a pre-trained model M and several adaptability matrices W i Low-rank adaptation module;

[0007] S2, fine-tune the continuous visual instruction dataset sequence T = {t1, t2, ..., t n-1 ,t n} Input to the pre-trained model M, n∈N;

[0008] S3. Fine-tune dataset t based on continuous visual instructions i Select a suitable low-rank adaptation matrix and perform fine-tuning, i∈n;

[0009] S4, using separable routing technology to obtain the visual understanding of S3 routing selection G vu (Z uv ) and instruction follow routing G if (Z if );

[0010] S5, through S4 in G vu (Z uv ) and G if (Z if ) and its corresponding adaptability matrix W i Perform weighted operations to obtain two corresponding outputs

[0011] S6. Use adaptive fusion technology to obtain the final output of the low-rank adaptation module

[0012]

[0013] Preferably, the adaptability matrix W of the low-rank adaptation module in S1 is i Updated by the following formula:

[0014]

[0015] Where G(z) k represents the weight assigned by the routing network to the kth adaptability matrix based on the input z.

[0016] Preferably, the continuous visual instruction fine-tuning dataset sequence in S2 includes an image feature dataset, a video feature dataset, and a text feature dataset.

[0017] Preferably, the visual understanding routing in S4 is G vu (Z uv ) is calculated as follows:

[0018] G vu (Z uv )=softmax(top k (R uv ·Avg(x l-1 )));(2)

[0019] where R uv represents the trainable routing matrix stored in the visual understanding module, x l-1 is the input of the module, softmax is the normalization function, l is the current layer number of the model, Z uv Input for visual understanding routing, top k Only the first k largest values ​​are taken, and the other values ​​are set to 0. Avg is the average value.

[0020] Preferably, the instruction in S4 follows the routing selection G if (Z if ) is calculated as follows:

[0021] G if (Z if )=softmax(top k (R if ·f(X ins )));(3)

[0022] where X ins is the instruction for the current visual instruction fine-tuning task, Z if The input for the instruction following the routing selection, R if is the weight matrix and f is the embedding operation.

[0023] Preferably, the output in S5 The calculation process is as follows:

[0024]

[0025] Preferably, the output in S5 The calculation process is as follows:

[0026]

[0027] Preferably, the adaptive fusion technology fusion algorithm in S6 is as follows:

[0028]

[0029] Where α and β are weight coefficients learned during training.

[0030] Therefore, the present invention adopts a continuous visual instruction fine-tuning method based on separable mixed low-rank adaptation, which has the following advantages compared with the prior art:

[0031] 1. By dividing the task into two independent modules, visual understanding and instruction following, interference between tasks is avoided and each module can focus on optimizing a specific task, effectively solving the catastrophic forgetting problem in both aspects.

[0032] 2. By dynamically selecting and allocating low-rank adaptation modules, the model's adaptability is adjusted according to the needs of different tasks. This provides high flexibility and efficiency, not only reducing the waste of computing resources but also enabling efficient adaptation to multiple tasks, ensuring that the model maintains efficient and stable performance across multiple tasks.

[0033] 3. Through a specific model architecture and routing optimization strategy, the double catastrophic forgetting problem is effectively alleviated, ensuring that the model can continue to learn new tasks while retaining the knowledge and performance of old tasks.

[0034] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is an overall framework diagram of a continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation of the present invention;

[0036] Figure 2 This is a separable hybrid low-rank adaptation module diagram of a continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation of the present invention. DETAILED DESCRIPTION

[0037] Example

[0038] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limiting the present invention.

[0039] like Figure 1-Figure 2 As shown, a continuous visual instruction fine-tuning method based on separable mixed low-rank adaptation of the present invention includes the following steps:

[0040] S1. Create a pre-trained model M and several adaptability matrices W i Low-rank adaptation module; Adaptation matrix W i Updated by the following formula:

[0041]

[0042] Where G(z) k represents the weight assigned by the routing network to the kth adaptability matrix based on the input z; the adaptability matrix W i As the pre-trained model M changes, the dynamic nature of the task requires a more adaptive routing strategy to avoid knowledge loss.

[0043] S2, fine-tune the continuous visual instruction dataset sequence T = {t1, t2, ..., t n-1 ,t n Input to the pre-trained model M, n∈N, N is a positive integer collection, and the continuous visual instruction fine-tuning dataset sequence includes image feature dataset, video feature dataset, and text feature dataset; responsible for processing visual input and extracting information related to the visual task;

[0044] S3. Fine-tune dataset t based on continuous visual instructions i Select a suitable low-rank adaptation matrix and perform fine-tuning, i∈n;

[0045] S4, using separable routing technology to obtain the visual understanding of S3 routing selection G vu (Z uv ) and instruction follow routing G if (Z if ); Separate hybrid low-rank adaptation handles differences between tasks through separable routing technology. In visual understanding routing, the average features of the input are used to dynamically select the appropriate low-rank adaptation module; in instruction following routing, the embedding of the current task instruction is used as input to select the appropriate low-rank adaptation module for tasks that require instruction interpretation;

[0046] Visual Understanding Routing G vu (Z uv ) is calculated as follows:

[0047] G vu (Z uv )=softmax(top k (R uv ·Avg(x l-1 )));(2)

[0048] where R uv represents the trainable routing matrix stored in the visual understanding module, x l-1 is the input of the module, softmax is the normalization function, l is the current layer number of the model, Z uv Input for visual understanding routing, top k To take only the first k largest values, set the other values ​​to 0, and Avg is the average value;

[0049] Instruction Follow Routing G if (Z if ) is calculated as follows:

[0050] G if (Z if )=softmax(top k (R if ·f(X ins )));(3)

[0051] where X ins is the instruction for the current visual instruction fine-tuning task, Z if The input for the instruction following the routing selection, R if is the weight matrix, f is the embedding operation;

[0052] S5, through S4 in G vu(Z uv ) and G if (Z if ) and its corresponding adaptability matrix W i Perform weighted operations to obtain two corresponding outputs

[0053] in The calculation process is as follows:

[0054]

[0055] in The calculation process is as follows:

[0056]

[0057] S6. Use adaptive fusion technology to obtain the final output of the low-rank adaptation module The fusion algorithm of adaptive fusion technology is as follows:

[0058]

[0059] Among them, α and β are the weight coefficients learned during the training process. After the routing is completed, the output results of the visual understanding module and the instruction following module are will be adaptively fused, which assigns a weight to the output of each module to ensure that the most relevant output is used first;

[0060] The specific implementation process assumes that there is a pre-trained model M and multiple adaptability matrices W in the low-rank adaptation module. i Continuous visual instruction fine-tuning dataset sequence T = {t1, t2, ..., t n-1 ,t n};When the i-th instruction fine-tunes the dataset t i When it arrives, it is necessary to select a suitable low-rank adaptation matrix and fine-tune it to meet the requirements of the current task; use the separable routing technology to obtain the visual understanding routing selection G of the current task vu (Z uv ) and instruction follow routing G if (Z if );

[0061] By G vu (Z uv ) and G if (Z if ) and the corresponding adaptability matrix W i Perform weighted operations to obtain the outputs of the two modules The final output of the low-rank adaptation module is obtained by using adaptive fusion technology. The low-rank adaptation module is optimized through back-propagation, which effectively solves the double catastrophic forgetting problem without sacrificing the model's previous learning ability.

[0062] Therefore, the present invention adopts the above-mentioned continuous visual instruction fine-tuning method based on separable hybrid low-rank adaptation, and utilizes hybrid low-rank adaptation and separable routing technology to efficiently solve the double catastrophic forgetting problem without sacrificing the model's previous learning ability.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for continuous visual command fine-tuning based on separable mixture low-rank adaptation, characterized by: S1. Create a pre-trained model and contains several adaptability matrices Low-rank adaptation module; S2. Fine-tune the dataset sequence with continuous visual instructions Input to pre-trained model , N ; S3. Fine-tuning the dataset based on continuous visual instructions Select a suitable low-rank adaptation matrix and perform fine-tuning, ; S4, using separable routing technology to obtain visual understanding routing selection of S3 and directives follow routing ; Among them, visual understanding routing The calculation is as follows: ；（2） in represents the trainable routing matrix stored in the visual understanding module, is the input of this module, is the normalization function, is the current layer number of the model, Input for visual understanding routing, To take only the front k The largest value, other values ​​are set to 0, is the average value; S5, through S4 and The corresponding adaptability matrix Perform weighted operations to obtain two corresponding outputs , ; where instructions follow routing The calculation is as follows: ；（3） in Fine-tune the instructions for the task for the current visual instructions, Input for command follow routing selection, is the weight matrix, is the embedded operation; S6. Use adaptive fusion technology to obtain the final output of the low-rank adaptation module .

2. The method for continuous visual command fine-tuning based on separable mixture low-rank adaptation according to claim 1, characterized in that: The adaptability matrix of the low-rank adaptation module described in S1 Updated by the following formula: ；（1） in Indicates that the routing network is based on the input For the first The weights assigned to the adaptability matrix.

3. The method for continuous visual command fine-tuning based on separable mixture low-rank adaptation according to claim 2, characterized in that: The continuous visual instruction fine-tuning dataset sequence in S1 includes image feature dataset, video feature dataset, and text feature dataset.

4. The method for continuous visual command fine-tuning based on separable mixture low-rank adaptation according to claim 3, characterized in that: Output as described in S5 The calculation process is as follows: ；（4）。 5. The method for continuous visual command fine-tuning based on separable mixture low-rank adaptation according to claim 4, characterized in that: Output as described in S5 The calculation process is as follows: ；（5）。 6. The method for continuous visual command fine-tuning based on separable mixture low-rank adaptation according to claim 5, characterized in that: The adaptive fusion technology fusion algorithm in S6 is as follows: ；（6） in and are the weight coefficients learned during the training process.

Images