Expert model fusion method

Abstract

The invention provides an expert model fusion method. The method comprises the steps of obtaining a pre-training basic model; determining at least more than two homologous fine tuning expert models based on the pre-trained basic model; and determining a fusion expert model based on the pre-trained basic model and the at least two homologous fine adjustment expert models. The method provided by the invention is clear in structure and high in universality, can be suitable for multiple types of neural network architectures such as a large language model and a multi-modal model, and provides an efficient and stable parameter fusion mechanism for model integration and knowledge migration in a multi-task scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence model integration, in particular to an expert model fusion method. BACKGROUND

[0002] With the rapid development of deep learning and multi-task learning (MTL), large-scale pre-training models (such as BERT, GPT, etc.) have become mainstream. However, in the multi-task scenario, different task expert models are usually fine-tuned based on the same pre-training model, resulting in significant parameter differences. How to balance task performance and reduce mutual interference when fusing these expert models has become a research focus.

[0003] Existing methods mostly use parameter weighted averaging or simple summation, but do not fully consider the conflict between tasks. For example, inconsistent task vector directions can cause knowledge cancellation, affecting the fusion effect. To this end, researchers have proposed improved methods such as TIES-Merging (symbolic election and pruning) and Activation-Informed Merging (activation amplitude weighting) to alleviate conflicts and improve fusion performance.

[0004] However, most of these methods still rely on training data or single-step fusion, making it difficult to deal with dynamic relationships between complex tasks. With the rise of multi-modal learning and privacy computing, there is an urgent need for a training-free, low-interference, and multi-scene adaptable fusion mechanism to efficiently integrate parameters while preserving task features. SUMMARY

[0005] The present application provides an expert model fusion method, in order to solve the above technical problems, the present application adopts the following technical method: In a first aspect, the present application provides an expert model fusion method, comprising: obtaining a pre-training base model; determining at least two or more homologous fine-tuned expert models based on the pre-training base model; determining a fusion expert model based on the pre-training base model and the at least two or more homologous fine-tuned expert models.

[0006] Optionally, the pre-training base model is obtained through an open source community.

[0007] Optionally, determining at least two or more homologous fine-tuned expert models based on the pre-training base model comprises: adapting the pre-training base model to different tasks through fine-tuning to determine at least two or more homologous fine-tuned expert models.

[0008] Optionally, the determining the fusion expert model based on the pre-training base model and the at least two same-homologous fine-tuning expert models comprises: determining a plurality of same parameter difference task vectors in the pre-training base model and the at least two same-homologous fine-tuning expert models based on the pre-training base model and the at least two same-homologous fine-tuning expert models; determining a parameter pruning task vector based on the plurality of same parameter difference task vectors; determining a consensus set of parameter index positions based on the parameter pruning task vector; determining a fusion parameter task vector based on the parameters in the consensus set; determining the fusion expert model based on the fusion parameter task vector and parameters of the pre-training base model.

[0009] Optionally, the determining the parameter pruning task vector based on the plurality of same parameter difference task vectors comprises: sorting the plurality of same parameter difference task vectors in descending order of parameter amplitude absolute value to obtain descending parameter task vectors; assigning a drop probability positively correlated with ranking to the descending parameter task vectors, pruning the descending parameter task vectors after the assignment of ranking by using Bernoulli distribution sampling to obtain the parameter pruning task vector.

[0010] Optionally, the determining the consensus set of parameter index positions based on the parameter pruning task vector comprises: calculating a contribution weight of each parameter at a corresponding position for the parameter pruning task vector; determining an amplitude weighted sum of each parameter based on the contribution weight; performing consensus direction screening on the parameters at each position in the parameter pruning task vector based on the amplitude weighted sum to determine the consensus set of parameter index positions.

[0011] Optionally, the determining the fusion parameter task vector based on the parameters in the consensus set comprises: performing weighted fusion on the parameters in the consensus set to obtain the fusion parameter task vector, wherein a weight of the weighted fusion is determined by Dirichlet distribution sampling.

[0012] Optionally, the determining the fusion expert model based on the fusion parameter task vector and the parameters of the pre-training base model comprises: adding the fusion parameter task vector and the parameters of the pre-training base model to determine the fusion expert model.

[0013] In a second aspect, the present application also provides a system comprising: a memory for storing instructions executable by the processor; a processor for executing the instructions to implement the method according to any one of the first aspect.

[0014] In a third aspect, the present application also provides a computer readable medium storing computer program code, which, when executed by a processor, implements the method according to any one of the first aspect.

[0015] The present application has the following beneficial effects: The method and structure of the present application are clear and versatile, and can be applied to various neural network architectures such as large language models and multi-modal models, providing an efficient and stable parameter fusion mechanism for model integration and knowledge transfer in multi-task scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of an expert model fusion method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] To facilitate the understanding of those skilled in the art, the present application will be further described below in conjunction with the embodiments and drawings, and the content mentioned in the embodiments is not a limitation of the present application.

[0018] To solve the above technical problems, as shown in the specification, Figure 1 The present application provides an expert model fusion method, comprising: Step S101: obtaining a pre-trained base model; The pre-trained base model is obtained from an open source community. The model needs to adapt to the technical framework and application scenario of the target task, and its core use is as a homologous fine-tuning base model for generating simple expert models adapted to specific tasks.

[0019] Step S102: determining at least two or more homologous fine-tuning expert models based on the pre-trained base model; The pre-trained model is fine-tuned to adapt to different tasks, thereby determining at least two or more homologous fine-tuning expert models. The parameter structure of the homologous fine-tuning expert model is completely consistent with that of the pre-trained base model (satisfying the homology requirement), which needs to be strictly controlled from the aspects of parameter structure definition and consistency verification to ensure that the parameters can be calculated element by element to obtain the task vector. At the same time, a test set corresponding to the adapted task is obtained for evaluation of the fusion expert ability.

[0020] The similarities include the following: Overall model architecture: the network type and module order are completely identical (e.g., both are Transformer decoder-only architectures); Embedding layer parameters: the vocabulary size (vocab_size) and embedding dimension (hidden_size) are the same; Number of Transformer layers: the number of encoder / decoder layers (num_hidden_layers) are completely identical; Hidden dimensions: the hidden state dimension (hidden_size) and intermediate layer dimension (intermediate_size) are the same; Attention mechanism parameters: the number of attention heads (num_attention_heads) and the dimension of each head (head_dim) are the same; Normalization layer parameters: the layer normalization type (e.g., RMSnorm, LayerNorm) and eps value are the same; Activation function: the activation function type of the feedforward network (FFN) is the same (e.g., SwiGLU, ReLU); Output layer parameters: the output projection matrix dimension and whether the embedding layer weights are shared (tie_word_embeddings) are the same.

[0021] Step S103: Determine the fusion expert model based on the pre-trained base model and the at least two homologous fine-tuned expert models.

[0022] First, based on the pre-trained base model and the at least two homologous fine-tuned expert models, it is necessary to determine multiple identical parameter difference task vectors in the pre-trained base model and the at least two homologous fine-tuned expert models. The specific process is shown in the following formula: (1) in, Indicates the first The parameters of an expert model, The parameters representing the base model, Indicates the first The difference between the model and the initial parameters, i.e. the .... The task vector of each model.

[0023] Then, based on multiple task vectors with the same parameter difference, the parameter pruning task vector is determined, as follows: By row level (e.g., by row of the Q matrix in a Transformer layer), for multiple task vectors with the same parameter difference. After sorting the parameters in descending order according to their absolute magnitudes, we obtain the descending parameter task vector: (2) in, = 0 indicates the most important (largest amplitude) parameter.

[0024] Further, assign a drop probability positively correlated with the rank to each descending parameter task vector, (lower amplitude parameters are more likely to be dropped): Probability change increment: (3) Drop probability definition: (4) Where, , represents the control drop range, is the average drop probability, is the total number of parameters in the current layer, the drop probability finally assigned to the th parameter. This is done to assign differentiated drop probabilities to parameters of different importance: the later the rank, the larger the amplitude) parameter, the larger, the higher, the easier to be pruned. Avoid “one-size-fits-all” pruning: both retain high-amplitude core task parameters and targetedly eliminate low-amplitude redundant / noise parameters, balancing “retaining task ability” and “reducing invalid interference”.

[0025] Finally, prune each descending parameter task vector to which a drop probability has been assigned to obtain a parameter pruned task vector, that is, for each descending parameter task vector, sample from a Bernoulli distribution (1 indicates drop): (5) (6) In the formula, is the pruning mask (1 indicates drop, 0 indicates retention), subject to a Bernoulli distribution (random sampling according to probability ), is element-wise multiplication, is the pruned parameter.

[0026] “Flexible pruning” is achieved through random sampling: instead of fixedly eliminating parameters below a certain threshold, low-importance parameters are dropped according to probability, avoiding excessive pruning that leads to loss of task ability.

[0027] Scale the retained parameters to ensure that the expected output distribution remains unchanged, and the retained parameters are amplified according to the retention probability: (7)Scale the retained parameters after pruning to ensure that the “expected output distribution of the parameters remains unchanged”.

[0028] After obtaining the parameter pruning task vector, the consensus set of the index positions of each dimension of the parameters can be determined based on the parameter pruning task vector. The specific process is as follows: For the parameter pruning task vector, calculate the position of each parameter at the corresponding location. Contribution weight : (8) Then, based on this contribution weight, the magnitude-weighted sum of each parameter is determined: (9) By combining the weighted opinions of all experts at the k-th parameter position, considering both the parameter direction and the overall situation, the analysis can be considered. (Positive and negative), while also taking into account the contribution of experts. This avoids the problem of ordinary summation ignoring the differences in expert reliability.

[0029] Based on magnitude-weighted summation, consensus direction filtering is performed on the parameters at each position in the parameter pruning task vector to determine the consensus set for each dimension index position of the parameters: The main task here is to determine the dominant sign and then retain the set of experts with consistent signs. (10) dominant symbol : Most experts believe the parameters should be increased; Most experts believe it should be reduced; :represent The locations conflict.

[0030] Set of experts that retains consistent symbols This is the consensus set of the index positions of each dimension of the parameters, excluding models with opposite directions and reducing conflicts. (11) After obtaining the consensus set, the fusion parameter task vector can be determined based on the parameters in the consensus set. The specific process is as follows: The parameters in the consensus set are weighted and fused to obtain a fused parameter task vector. The weights of the weighted fusion are determined by sampling through a Dirichlet distribution. This step is as follows: Preset concentration parameters ;from (dimension is) Sampling weights ,satisfy .

[0031] The weighted fusion parameter task vector is added to the parameters of the pre-trained base model to generate a fused expert model. : (12) Finally, the fused parameter task vector is added to the parameters of the pre-trained base model to generate a fused expert model. (13) In the formula, is the fused expert model.

[0032] The fused expert model is evaluated on the tasks corresponding to each expert using the test set obtained in the foregoing, and the "retention ability" and "interference reduction" effects are verified by comparing the single expert model and other merging methods.

[0033] To sum up, the method provided in the application has clear structure and strong universality, and can be applied to large language models, multi-modal models and other types of neural network architectures, thereby providing an efficient and stable parameter fusion mechanism for model integration and knowledge transfer in a multi-task scenario.

[0034] In some embodiments, the application also provides a computer device including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above-mentioned method embodiments when executing the computer program.

[0035] The application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the above-mentioned methods in the embodiments of the application. For brevity, the details are not repeated here.

[0036] The above embodiments are the preferred implementation schemes of the application. In addition to this, the application can also be implemented in other ways, and any obvious substitutions do not deviate from the technical solution concept of the application and are within the protection scope of the application.

[0037] In order to make those skilled in the art more easily understand the improvements of the present application over the prior art, some of the drawings and descriptions of the present application have been simplified, and some other elements have been omitted from the present application file for the sake of clarity. Those skilled in the art should be aware that these omitted elements can also constitute the content of the present application.

Claims

1. An expert model fusion method, characterized in that, The method comprises: obtaining a pre-training base model; determining at least two or more homologous fine-tuning expert models based on the pre-training base model; determining a fusion expert model based on the pre-training base model and the at least two or more homologous fine-tuning expert models.

2. The method of claim 1, wherein, The pre-training base model is obtained from an open source community.

3. The method of claim 2, wherein, The determination of the at least two or more homologous fine-tuning expert models based on the pre-training base model comprises: adapting the pre-training base model to different tasks through fine-tuning to determine the at least two or more homologous fine-tuning expert models.

4. The method of claim 3, wherein, The determination of the fusion expert model based on the pre-training base model and the at least two or more homologous fine-tuning expert models comprises: determining a plurality of same parameter difference value task vectors of the pre-training base model and the at least two or more homologous fine-tuning expert models based on the pre-training base model and the at least two or more homologous fine-tuning expert models; determining a parameter pruning task vector based on the plurality of same parameter difference value task vectors; determining a consensus set of parameter index positions in each dimension based on the parameter pruning task vector; determining a fusion parameter task vector based on the parameters in the consensus set; determining a fusion expert model based on the fusion parameter task vector and the parameters of the pre-training base model.

5. The method of claim 4, wherein, The determination of the parameter pruning task vector based on the plurality of same parameter difference value task vectors comprises: sorting the plurality of same parameter difference value task vectors in descending order according to the absolute values of the parameter amplitudes to obtain descending parameter task vectors; assigning a drop probability positively correlated with the ranking to the descending parameter task vectors, sampling according to a Bernoulli distribution, pruning the descending parameter task vectors after the ranking is assigned, and obtaining a parameter pruning task vector.

6. The method of claim 4, wherein, The determination of the consensus set of parameter index positions in each dimension based on the parameter pruning task vector comprises: calculating the contribution weight of each parameter at the corresponding position based on the parameter pruning task vector; determining the amplitude weighted sum of each parameter based on the contribution weight; performing consensus direction filtering on the parameters at each position in the parameter pruning task vector based on the amplitude weighted sum to determine the consensus set of parameter index positions in each dimension.

7. The method of claim 4, wherein, The determination of the fusion parameter task vector based on the parameters in the consensus set comprises: performing weighted fusion on the parameters in the consensus set to obtain a fusion parameter task vector; and the weight of the weighted fusion is determined by sampling according to a Dirichlet distribution.

8. The method of claim 4, wherein, The determination of the fusion expert model based on the fusion parameter task vector and the parameters of the pre-training base model comprises: adding the fusion parameter task vector and the parameters of the pre-training base model to determine a fusion expert model.

9. A system, characterized by The method comprises: a memory for storing instructions executable by a processor; a processor for executing the instructions to implement the method of any one of claims 1-8.

10. A computer readable medium characterized by A computer program code is stored, and the computer program code, when executed by a processor, implements the method of any one of claims 1-8.

Images