A data-free model fusion method based on orthogonal and projection dual-space optimization

Abstract

The present invention discloses a data-free model fusion method based on orthogonal and projection dual-space optimization, which belongs to the field of data model fusion technology and is used for data-free model fusion. The method comprises performing singular value decomposition on a shared orthogonal subspace, removing redundant vectors to obtain a non-redundant subspace, projecting key parameters into an orthogonal subspace parameter module through orthogonal space projection, an orthogonal subspace optimizer performing gradient information update and then feeding it back to the orthogonal subspace parameter module, a projection subspace optimizer performing gradient update and feeding it back to a dual-space constraint, and performing model fusion on the pre-trained model, the output of the projection subspace optimizer, and the renormalized vector. The present invention is suitable for merging multiple expert models in multi-task scenarios, does not require reliance on additional data or retraining, and significantly reduces computational costs and privacy risks. The method further improves the overall performance and applicability of data-free model fusion by maximizing task shared information and minimizing parameter conflicts in the subspace at the same time.

Description

Technical Field

[0001] The invention discloses a data-free model fusion method based on orthogonal and projection dual-space optimization, belonging to the technical field of data model fusion. Background Art

[0002] With the widespread application of the pre-training-fine-tuning paradigm in various AI tasks, the number of expert models targeting different downstream tasks has increased dramatically. Although each fine-tuned model has achieved significant performance improvements on its corresponding task, deploying multiple fine-tuned models individually can lead to a significant increase in storage resources and operational costs. To address this issue, model fusion technology has emerged. Model fusion combines multiple fine-tuned models at the parameter level to form a unified model that can meet the needs of multiple tasks, thereby enabling unified management and deployment of multiple expert models. Compared to integrating or reordering the results of each model during inference, model fusion solutions directly merge the results in the parameter space, significantly reducing the total model storage size and avoiding the computational overhead of multiple model calls during inference.

[0003] Currently, mainstream model fusion methods can be divided into two categories: test-time adaptation and data-free model fusion. The former usually requires access to the original or approximate datasets of each task to compensate or calibrate the fusion results. However, the need for additional data access limits its applicability in scenarios where data is unavailable or security and privacy are important. To address this limitation, data-free model fusion methods have emerged. By utilizing only the parameters of pre-trained and fine-tuned models, without any additional data or retraining, multiple models can be directly merged in the parameter space. Currently, the main data-free model fusion methods are as follows:

[0004] 1) Linear interpolation methods: This type of method quickly preserves key information from each model by taking a weighted average of the model parameter values ​​according to preset weights. However, simple interpolation cannot effectively resolve parameter conflicts, often resulting in a decline in the overall performance of the fusion model. When the parameters of the models involved in the fusion vary significantly, the averaging operation is prone to introducing conflicts. Moreover, the effectiveness of this method is highly dependent on the diversity and quality of the selected fine-tuning models, limiting the upper limit of its performance improvement.

[0005] 2) Weighted task arithmetic methods: This type of method first obtains each task vector by subtracting the difference between the fine-tuned model weights and the pre-trained model weights. Then, arithmetic operations such as weighted addition or subtraction are performed on the task vectors to control the resulting model behavior. Simple addition and subtraction operations are difficult to fundamentally alleviate the conflict problem between task vectors, and the allocation of weights for each task vector lacks a systematic design, which can easily destroy the original beneficial parameters and thus affect the performance of the fused model on certain tasks.

[0006] 3) Subspace-based task arithmetic methods: This type of method maps the parameters of each model to orthogonal or projected subspaces for fusion, and reduces the interference between tasks through subspace coordinate transformation. However, it focuses on the characterization of a single subspace and ignores the effective features in other potential subspaces. It is difficult to balance information sharing and conflict elimination in multiple dimensions. Summary of the Invention

[0007] The purpose of the present invention is to provide a data-free model fusion method based on orthogonal and projection dual-space optimization to solve the problem of parameter conflict that the data-free model fusion method in the prior art is prone to.

[0008] A data-free model fusion method based on orthogonal and projective dual-space optimization, including orthogonal subspaces under computational constraints, dual-space constraint merging, and fusion of the final model;

[0009] The orthogonal subspace under the computational constraint includes performing a subtraction operation on the fine-tuning model set and the pre-training model to obtain a task vector, performing singular value decomposition on the task vector, splicing the decomposition result into a shared orthogonal subspace, performing singular value decomposition on the shared orthogonal subspace, removing redundant vectors to obtain a non-redundant subspace, and projecting key parameters into an orthogonal subspace parameter module through orthogonal space projection; the fine-tuning model set is provided with a branch, adaptively scaled, adjusted according to the layer scaling coefficient, forming a reshaped vector, and then inputted into an orthogonal subspace optimizer, the orthogonal subspace parameter module outputs gradient information, performs task vector preprocessing on the task vector for fusion, and then inputs into the orthogonal subspace optimizer, the orthogonal subspace optimizer updates the gradient information and feeds back to the orthogonal subspace parameter module;

[0010] Decomposing the fine-tuning model set into task parameters, element parameters, and layer parameters, the dual-space constraint merging includes inputting the reshaped vector, orthogonal subspace parameters, task parameters, element parameters, and layer parameters into a dual-space constraint, and then inputting them into a projected subspace optimizer, which performs gradient update and feeds back to the dual-space constraint;

[0011] The fusion final model includes model fusion of the pre-trained model, the output of the projection subspace optimizer, and the reshaped vector, and finally obtains a fusion model to complete the data-free model fusion.

[0012] The subtraction operation includes:

[0013] ;

[0014] Where, It is Task vectors, It is a pre-trained model, which is the initial model parameters pre-trained based on a large-scale general dataset. is the first model parameters fine-tuned for different downstream tasks.

[0015] Singular values ​​of the task vector include:

[0016] ;

[0017] Where, is the left singular matrix after decomposition of the task vector, is the singular value matrix after the task vector is decomposed, is the right singular matrix after decomposition of the task vector, Represents matrix transpose.

[0018] Splicing the decomposition results into a shared orthogonal subspace includes retaining Before Column, extract the main spatial features in the task vector , all Stitch together to construct a shared orthogonal subspace :

[0019] ;

[0020] Where, Represents a splicing operation, express The maximum value of .

[0021] Performing singular value decomposition on a shared orthogonal subspace involves:

[0022] ;

[0023] Where, is the left singular matrix after the shared orthogonal subspace decomposition, is the singular value matrix after the shared orthogonal subspace decomposition, It is the right singular matrix after the decomposition of the shared orthogonal subspace, storing the key parameters into the main spatial features in the shared orthogonal subspace , forming a non-redundant subspace.

[0024] Orthographic space projections include:

[0025] ;

[0026] Where, is a matrix of arbitrary dimension, yes exist projection;

[0027] The orthogonal subspace optimizer performs gradient information updates including:

[0028] ;

[0029] Where, is the gradient of each layer, Is the gradient information updated .

[0030] Orthogonal subspace parameters for:

[0031] ;

[0032] Where, For the After the gradient information is updated , is the layer-by-layer scaling factor;

[0033] Loss function in gradient information update for:

[0034] .

[0035] Reshaped vector for:

[0036] ;

[0037] ;

[0038] Where, Amplification factor that controls the zoom factor.

[0039] The orthogonal subspace optimizer performs iterative optimization layer by layer until convergence, and calculates the gradient loss of each layer :

[0040] ;

[0041] Where, For the first All elements in the layer can learn to adjust the element parameters, is the constraint parameter, For the layer , For the layer Update adjustment number ,dynamic equilibrium and The contribution weight of For the layer .

[0042] Fusion Model for:

[0043] ;

[0044] Where, for The maximum value of .

[0045] Compared with the existing technology, the present invention has the following beneficial effects: the present invention is suitable for the merging of multiple expert models in multi-task scenarios, without relying on additional data or retraining, significantly reducing computational costs and privacy risks; by maximizing task shared information and minimizing parameter conflicts in subspaces at the same time, the overall performance and applicability of data-free model fusion are further improved; through dual-space joint optimization and adaptive parameter fusion mechanism, efficient model merging is achieved without the need for additional data or training; in the process of multi-task fusion, the mutual interference between different tasks is suppressed to the greatest extent, significantly improving the overall performance of the fusion model in multimodal tasks such as vision and language, reaching the leading level of current technology, and effectively overcoming the information loss problem caused by traditional fusion strategies based on a single subspace. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall flow chart of the present invention.

[0047] Figure 2 is the average accuracy of different fusion methods on ViT-B / 32;

[0048] Figure 3 is the influence of constraint parameter P on different tasks. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] A data-free model fusion method based on orthogonal and projective dual-space optimization, including orthogonal subspaces under computational constraints, dual-space constraint merging, and fusion of the final model;

[0051] The orthogonal subspace under the computational constraint includes performing a subtraction operation on the fine-tuning model set and the pre-training model to obtain a task vector, performing singular value decomposition on the task vector, splicing the decomposition result into a shared orthogonal subspace, performing singular value decomposition on the shared orthogonal subspace, removing redundant vectors to obtain a non-redundant subspace, and projecting key parameters into an orthogonal subspace parameter module through orthogonal space projection; the fine-tuning model set is provided with a branch, adaptively scaled, adjusted according to the layer scaling coefficient, forming a reshaped vector, and then inputted into an orthogonal subspace optimizer, the orthogonal subspace parameter module outputs gradient information, performs task vector preprocessing on the task vector for fusion, and then inputs into the orthogonal subspace optimizer, the orthogonal subspace optimizer updates the gradient information and feeds back to the orthogonal subspace parameter module;

[0052] Decomposing the fine-tuning model set into task parameters, element parameters, and layer parameters, the dual-space constraint merging includes inputting the reshaped vector, orthogonal subspace parameters, task parameters, element parameters, and layer parameters into a dual-space constraint, and then inputting them into a projected subspace optimizer, which performs gradient update and feeds back to the dual-space constraint;

[0053] The fusion final model includes model fusion of the pre-trained model, the output of the projection subspace optimizer, and the reshaped vector, and finally obtains a fusion model to complete the data-free model fusion.

[0054] The subtraction operation includes:

[0055] ;

[0056] Where, It is Task vectors, It is a pre-trained model, which is the initial model parameters pre-trained based on a large-scale general dataset. is the first model parameters fine-tuned for different downstream tasks.

[0057] Singular values ​​of the task vector include:

[0058] ;

[0059] Where, is the left singular matrix after decomposition of the task vector, is the singular value matrix after the task vector is decomposed, is the right singular matrix after decomposition of the task vector, Represents matrix transpose.

[0060] Splicing the decomposition results into a shared orthogonal subspace includes retaining Before Column, extract the main spatial features in the task vector , all Stitch together to construct a shared orthogonal subspace :

[0061] ;

[0062] Where, Represents a splicing operation, express The maximum value of .

[0063] Performing singular value decomposition on a shared orthogonal subspace involves:

[0064] ;

[0065] Where, is the left singular matrix after the shared orthogonal subspace decomposition, is the singular value matrix after the shared orthogonal subspace decomposition, It is the right singular matrix after the decomposition of the shared orthogonal subspace, storing the key parameters into the main spatial features in the shared orthogonal subspace , forming a non-redundant subspace.

[0066] Orthographic space projections include:

[0067] ;

[0068] Where, is a matrix of arbitrary dimension, yes exist projection;

[0069] The orthogonal subspace optimizer performs gradient information updates including:

[0070] ;

[0071] Where, is the gradient of each layer, Is the gradient information updated .

[0072] Orthogonal subspace parameters for:

[0073] ;

[0074] Where, For the After the gradient information is updated , is the layer-by-layer scaling factor;

[0075] Loss function in gradient information update for:

[0076] .

[0077] Reshaped vector for:

[0078] ;

[0079] ;

[0080] Where, Amplification factor that controls the zoom factor.

[0081] The orthogonal subspace optimizer performs iterative optimization layer by layer until convergence, and calculates the gradient loss of each layer :

[0082] ;

[0083] Where, For the first All elements in the layer can learn to adjust the element parameters, is the constraint parameter, For the layer , For the layer Update adjustment number ,dynamic equilibrium and The contribution weight of For the layer .

[0084] Fusion Model for:

[0085] ;

[0086] Where, for The maximum value of .

[0087] The technical process of the present invention is as follows Figure 1As shown. The present invention has been experimentally verified in two main areas: visual tasks and natural language processing tasks. In visual tasks, experiments were carried out on two classic visual Transformer architectures (ViT-B / 32 and ViT-L / 14) based on the CLIP framework, covering eight image recognition scenarios, including fine-grained classification (StanfordCars), remote sensing image analysis (RESISC45 and EuroSAT), street scene digit recognition (SVHN), traffic sign recognition (GTSRB), scene classification (SUN397), texture recognition (DTD) and handwritten digit benchmark (MNIST). The experimental results are evaluated using top-1 accuracy (the unit is percentage, and the units in the table of this invention are all percentages). In natural language processing tasks, the base and large versions of the Flan-T5 series were selected to perform performance verification on the eight core tasks (CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST2, STSB) in the GLUE multi-task evaluation system.

[0088] The multi-task performance results when merging the ViT-B / 32 model on eight core task vision benchmarks are shown in Table 1.

[0089] Table 1 Multi-task performance results when merging the ViT-B / 32 model on eight core task vision benchmarks

[0090] ;

[0091] In Table 1, the English words in the method column represent various methods in the prior art.

[0092] The multi-task performance results when merging the ViT-L / 14 model on eight core task vision benchmarks are shown in Table 2.

[0093] Table 2 Multi-task performance results when merging the ViT-L / 14 model on eight core task vision benchmarks

[0094] ;

[0095] The multi-task performance results when merging the Flan-T5-base (LoRA fine-tuning) model on all eight language tasks are shown in Table 3. The English words in the method column are the English representations of various methods in the prior art.

[0096] Table 3 Multi-task performance results when merging the Flan-T5-base (LoRA fine-tuned) model on all eight language tasks

[0097] ;

[0098] The multi-task performance results when merging the Flan-T5-large (LoRA fine-tuned) model on all eight language tasks are shown in Table 4.

[0099] Table 4 Multi-task performance results when merging the Flan-T5-large (LoRA fine-tuned) model on all eight language tasks

[0100] ;

[0101] For a large number of tasks, the average accuracy of different fusion methods on ViT-B / 32 is shown in Table 5.

[0102] Table 5 Average accuracy of different fusion methods on ViT-B / 32

[0103] ;

[0104] When merging the ViT-B / 32 model on six tasks, the generalization results on two unseen tasks are shown in Table 6.

[0105] Table 6 Generalization results on two unseen tasks when merging the ViT-B / 32 model on six tasks

[0106] ;

[0107] The impact of different components on task accuracy is shown in Table 7.

[0108] Table 7 Effects of different components on task accuracy

[0109] ;

[0110] The present invention also lists the average accuracy of different fusion methods on ViT-B / 32 for different numbers of tasks. Figure 2 As shown, the influence of constraint parameter P on different tasks is as follows Figure 3 shown. Figure 2 In the figure, the vertical axis is the average accuracy (in percentage), the horizontal axis is the task number (without unit), ORION (Ours) represents the results of the present invention, and the rest are the results of other existing technologies. Figure 3 In the figure, the left vertical axis is the average accuracy of the visual task, in percentage, the right vertical axis is the average accuracy of the natural language processing task, in percentage, and the horizontal axis is the different values ​​of the constraint parameter P, without unit.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data-free model fusion method based on orthogonal and projection dual space optimization, characterized in that: Including orthogonal subspaces under computational constraints, merging of dual space constraints, and fusion of the final model; The orthogonal subspace under the computational constraint includes performing a subtraction operation on the fine-tuning model set and the pre-training model to obtain a task vector, performing singular value decomposition on the task vector, splicing the decomposition result into a shared orthogonal subspace, performing singular value decomposition on the shared orthogonal subspace, removing redundant vectors to obtain a non-redundant subspace, and projecting key parameters into an orthogonal subspace parameter module through orthogonal space projection; the fine-tuning model set is provided with a branch, adaptively scaled, adjusted according to the layer scaling coefficient, forming a reshaped vector, and then inputted into an orthogonal subspace optimizer, the orthogonal subspace parameter module outputs gradient information, performs task vector preprocessing on the task vector for fusion, and then inputs into the orthogonal subspace optimizer, the orthogonal subspace optimizer updates the gradient information and feeds back to the orthogonal subspace parameter module; Decomposing the fine-tuning model set into task parameters, element parameters, and layer parameters, the dual-space constraint merging includes inputting the reshaped vector, orthogonal subspace parameters, task parameters, element parameters, and layer parameters into a dual-space constraint, and then inputting them into a projected subspace optimizer, which performs gradient update and feeds back to the dual-space constraint; The fusion final model includes fusing the pre-trained model, the output of the projection subspace optimizer, and the reshaped vector to finally obtain a fusion model and complete the data-free model fusion; The fusion model is used for natural language processing to obtain processing results for different tasks.

2. The data-free model fusion method based on orthogonal and projection dual space optimization according to claim 1 is characterized in that: The subtraction operation includes: ； Where, It is Task vectors, It is a pre-trained model, which is the initial model parameters pre-trained based on a large-scale general dataset. is the first model parameters fine-tuned for different downstream tasks.

3. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 2 is characterized in that: Singular values ​​of the task vector include: ； Where, is the left singular matrix after decomposition of the task vector, is the singular value matrix after the task vector is decomposed, is the right singular matrix after decomposition of the task vector, Represents matrix transpose.

4. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 3 is characterized in that: Splicing the decomposition results into a shared orthogonal subspace includes retaining Before Column, extract the main spatial features in the task vector , all Stitch together to construct a shared orthogonal subspace : ； Where, Represents a splicing operation, express The maximum value of .

5. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 4 is characterized in that: Performing singular value decomposition on a shared orthogonal subspace involves: ； Where, is the left singular matrix after the shared orthogonal subspace decomposition, is the singular value matrix after the shared orthogonal subspace decomposition, It is the right singular matrix after the decomposition of the shared orthogonal subspace, storing the key parameters into the main spatial features in the shared orthogonal subspace , forming a non-redundant subspace.

6. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 5 is characterized in that: Orthographic space projections include: ； Where, is a matrix of arbitrary dimension, yes exist projection; The orthogonal subspace optimizer performs gradient information updates including: ； Where, is the gradient of each layer, Is the gradient information updated .

7. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 6 is characterized in that: Orthogonal subspace parameters for: ； Where, For the After the gradient information is updated , is the layer-by-layer scaling factor; Loss function in gradient information update for: 。 8. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 7 is characterized in that: Reshaped vector for: ； ； Where, Amplification factor that controls the zoom factor.

9. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 8 is characterized in that: The orthogonal subspace optimizer performs iterative optimization layer by layer until convergence, and calculates the gradient loss of each layer : ； Where, For the first All elements in the layer can learn to adjust the element parameters, is the constraint parameter, For the layer , For the layer Update adjustment number ,dynamic equilibrium and The contribution weight of For the layer .

10. The data-free model fusion method based on orthogonal and projection dual-space optimization according to claim 9 is characterized in that: Fusion Model for: ； Where, for The maximum value of .

Images