Large language model knowledge fusion method and device based on parameter restriction

Abstract

The invention provides a large language model knowledge fusion method and device based on parameter restriction, and relates to the technical field of knowledge fusion. The method comprises the steps of obtaining a plurality of task models; performing redundant parameter suppression on the task vectors in the plurality of task models by adopting a nonlinear activation function to obtain the suppression degree of redundant parameters; calculating the similarity of the same position parameters in the initial task vector by adopting a nonlinear activation function; obtaining a parameter importance score in the task vector according to the suppression degree and similarity of the redundant parameters; according to the parameter importance score in the task vector, obtaining a final combined task vector; and the finally combined task vectors are zoomed again, and combined task model parameters are generated. By adopting the method, the big language model merging performance can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge fusion, and in particular to a large language model knowledge fusion method and device based on parameter constraints. Background Art

[0002] Pre-trained models (PTMs) have become very common in many real-world applications. Typically, using PTMs requires fine-tuning them to be specialized for a specific task, which can improve performance with less task-specific labeled data. The above advantages have led to the release of thousands of fine-tuned checkpoints based on popular PTMs. However, preparing a fine-tuned model for each task also has disadvantages: (1) Each new application requires storing and deploying a separate model, which increases costs; (2) Models trained in isolation cannot leverage information from related tasks to improve in-domain performance or out-of-domain generalization capabilities. Multi-task learning can address the above issues, but it requires expensive training costs and access to all tasks at the same time. In addition, determining how to best mix datasets to ensure that multi-task training benefits all tasks can be complex and resource-intensive.

[0003] As data privacy and resource conservation issues become increasingly prominent, deep model fusion has attracted widespread attention. Model fusion has many potential applications, such as multi-task learning, federated learning, and model distillation. Among them, model fusion can be divided into three categories: the first category is ensemble learning, which improves prediction performance and robustness by combining the outputs of multiple different models. However, ensemble learning requires the parallel deployment of multiple models, which increases resource overhead; the second category is model connection alignment, which achieves better average effect by bringing the solutions of different models closer; modal connection connects different models by optimizing paths to solve local optimization problems, while alignment reduces differences by matching model units and averaging them. The third category is weight averaging, which is the mainstream method for deep model fusion; the above methods face problems of computational complexity, parameter tuning, and memory burden in practice. In previous studies, when multiple models are fine-tuned from the same pre-training initialization, averaging their weights can improve the performance of single tasks, different tasks, and out-of-distribution generalization. In addition, "fusion" by averaging the weights of fine-tuned models can provide a better starting point for new downstream tasks. The main methods include: (1) using the Fisher information matrix to identify the importance of individual parameters and weighting them according to these weights when fusing models. However, this method has limited effectiveness when fusing multiple checkpoints and requires calculating gradients, resulting in high memory costs; (2) The parameters of the fused model are obtained by solving a local linear regression problem for each linear layer; however, additional data statistics of the same size as the model need to be transmitted, and additional inference steps are required to calculate these statistics; (3) The concept of "task vector" is introduced, showing that merging task vectors through arithmetic operations can effectively support multi-task learning; the above method only fuses parameters with consistent signs and uses the same fusion coefficient for all task vectors, which limits the flexibility of task vector combinations and requires complex training and unlabeled test datasets. Summary of the invention

[0004] In order to solve the technical problems of low model fusion efficiency and poor fusion model performance in the prior art, the embodiment of the present invention provides a large language model knowledge fusion method and device based on parameter constraints. The technical solution is as follows:

[0005] On the one hand, a large language model knowledge fusion method based on parameter constraints is provided, the method is implemented by a large language model knowledge fusion device based on parameter constraints, and the method includes:

[0006] S1. Obtain the same task model as multiple initial pre-training models, and obtain a corresponding initial task vector by performing differential processing with the initial pre-processing model; use a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters;

[0007] S2, using a nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector;

[0008] S3. Obtaining a parameter importance score in the initial task vector according to the suppression degree of the redundant parameters and the similarity;

[0009] S4. Obtain a final merged task vector according to the parameter importance scores in the initial task vector; rescale the final merged task vector to generate a final merged task model.

[0010] Optionally, the step of S3 obtaining a parameter importance score in the initial task vector according to the suppression degree of the redundant parameters and the similarity includes:

[0011] S31, constructing an importance score for each task model according to the suppression degree of the redundant parameters and the similarity;

[0012] S32, constructing a mask according to the importance score of each task model;

[0013] S33. Obtain the parameter importance scores in the initial task vector based on the mask and the importance scores of each task model.

[0014] Optionally, the mask is used to select the highest scoring element from each element in the importance score of each task model.

[0015] Optionally, the step S4 of rescaling the finally merged task vector to generate a finally merged task model includes:

[0016] The magnitude of the task vector is proportionally adjusted according to the final merged task vector, integrated with the initial parameter value, and the merged task model parameters are generated by setting the scaling hyperparameters. According to the merged task model parameters, the final merged task model is generated.

[0017] Optionally, after the step of obtaining a final merged task vector according to the parameter importance scores in the initial task vector and rescaling the final merged task vector to generate a final merged task model in S4, the method further includes:

[0018] The intelligent optimization algorithm and the covariance matrix adaptive evolution strategy are used to optimize the scaling hyperparameters to obtain the optimal scaling hyperparameters and improve the verification accuracy of the maximized merged model.

[0019] On the other hand, a large language model knowledge fusion device based on parameter constraints is provided, and the device is applied to a large language model knowledge fusion method based on parameter constraints, and the device comprises:

[0020] A first acquisition unit is used to acquire the same task model as a plurality of initial pre-training models, and obtain a corresponding initial task vector by performing differential processing with the initial pre-processing model; and suppress redundant parameters of the initial task vector using a nonlinear activation function to obtain a suppression degree of redundant parameters;

[0021] A calculation unit, used for calculating the similarity of parameters at the same position in the initial task vector using a nonlinear activation function;

[0022] A second acquisition unit, configured to obtain a parameter importance score in an initial task vector according to the suppression degree of the redundant parameters and the similarity;

[0023] A generating unit is used to obtain a final merged task vector according to the parameter importance scores in the initial task vector; and rescale the final merged task vector to generate a final merged task model.

[0024] Optionally, the second acquiring unit is used to:

[0025] constructing an importance score for each task model according to the degree of suppression of the redundant parameters and the similarity;

[0026] Construct a mask based on the importance score of each task model;

[0027] According to the mask and the importance scores of each task model, the parameter importance scores in the initial task vector are obtained.

[0028] Optionally, the mask is used to select the highest scoring element from each element in the importance score of each task model.

[0029] Optionally, rescaling the final merged task vector to generate a final merged task model includes:

[0030] The magnitude of the task vector is proportionally adjusted according to the final merged task vector, integrated with the initial parameter value, and the merged task model parameters are generated by setting the scaling hyperparameters. According to the merged task model parameters, the final merged task model is generated.

[0031] Optionally, after the step of obtaining a final merged task vector according to the parameter importance scores in the initial task vector and rescaling the final merged task vector to generate a final merged task model, the method further includes:

[0032] The intelligent optimization algorithm and the covariance matrix adaptive evolution strategy are used to optimize the scaling hyperparameters to obtain the optimal scaling hyperparameters and improve the verification accuracy of the maximized merged model.

[0033] On the other hand, a large language model knowledge fusion device based on parameter constraints is provided, and the large language model knowledge fusion device based on parameter constraints includes: a processor; a memory, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, any one of the large language model knowledge fusion methods based on parameter constraints described above is implemented.

[0034] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned large language model knowledge fusion methods based on parameter constraints.

[0035] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0036] The embodiment of the present invention first obtains the same task model as multiple initial pre-training models, obtains the corresponding initial task vector by performing differential processing with the initial preprocessing model; uses a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters; uses a nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector; secondly, obtains the parameter importance score in the initial task vector according to the degree of suppression and similarity of redundant parameters; finally, obtains the final merged task vector according to the parameter importance score in the initial task vector; rescales the final merged task vector to generate the final merged task model. The present invention aims at the shortcomings of existing methods in dealing with potential conflicts and complex associations between tasks, especially in the adjustment of parameter levels, which makes it a challenge to effectively balance parameter competition among multiple tasks. In order to solve the above problems, the present invention proposes a parameter competition balancing method, which is a lightweight and training-free technology that can achieve efficient model merging by adjusting the coefficient of each parameter. The method measures the importance of parameters in a single task through internal balance and evaluates the similarity of parameters between different tasks through external balance. Parameters with low importance scores are eliminated and the remaining parameters are readjusted to form the final merged model; the method is evaluated in multiple merging scenarios across tasks, domains, training configurations, and out-of-domain generalization. The method significantly improves the model merging performance in multiple modalities, domains, model sizes, number of tasks, fine-tuning forms, and large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 This is a flow chart of a large language model knowledge fusion method based on parameter constraints provided by an embodiment of the present invention;

[0039] Figure 2 is a performance comparison diagram of a model fusion method representing different modal tasks provided by an embodiment of the present invention;

[0040] Figure 3 This is an average performance comparison of an embodiment of the present invention on different large language model tasks provided by an embodiment of the present invention;

[0041] Figure 4 is a performance diagram comparing different methods when the model is integrated with different training environment nodes provided by an embodiment of the present invention;

[0042] Figure 5 is a diagram of ablation experiment results for different components of the present invention provided by an embodiment of the present invention;

[0043] Figure 6 It is a block diagram of a large language model knowledge fusion device based on parameter constraints provided by an embodiment of the present invention;

[0044] Figure 7 It is a structural diagram of a large language model knowledge fusion device based on parameter constraints provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0046] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0047] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0048] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0049] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0050] The embodiment of the present invention provides a large language model knowledge fusion method based on parameter constraints, which can be implemented by a large language model knowledge fusion device based on parameter constraints, and the large language model knowledge fusion device based on parameter constraints can be a terminal or a server. Figure 1 The flowchart of the large language model knowledge fusion method based on parameter constraints is shown in the figure. The processing flow of the method may include the following steps:

[0051] S1. Obtain the same task model as multiple initial pre-trained models, obtain the corresponding initial task vector by performing differential processing with the initial pre-processing model; use a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters.

[0052] Among them, the task model is a large language model that is suitable for different tasks and different scenarios.

[0053] The initial task vector is the vector obtained by subtracting the pre-training weight from the fine-tuned weight, which can be expressed by the following formula (1):

[0054] (1)

[0055] in, represents the i-th task vector; Indicates the task model ordinal number; represents the weight of the i-th fine-tuning model; Represents the weights of the pre-trained model.

[0056] The degree of suppression of redundant parameters can be expressed by the following formula (2):

[0057] (2)

[0058] in, Indicates the degree of suppression of redundant parameters corresponding to the i-th task vector; Indicates the number of tasks; Indicates normalization processing.

[0059] Among them, by adopting nonlinear activation functions for self-perception, important parameters can be emphasized and redundant parameters can be suppressed to a certain extent; when the number of fusion tasks increases, the competition between parameters will intensify, and the number of tasks is used to adjust the degree of suppression of redundant parameters.

[0060] S2. Use nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector.

[0061] The process of calculating the similarity of the parameters at the same position in the initial task vector can be expressed by the following formula (3):

[0062] (3)

[0063] in, Represents the similarity between the i-th task vector and the parameters at the same position of each task vector; represents the jth task vector;

[0064] Among them, cross-task perception is achieved by calculating the similarity of parameters at the same position in different task vectors, so that the parameters in the task group can interact with each other and resolve potential conflicts and complex associations between tasks.

[0065] S3. Obtain the parameter importance score in the initial task vector based on the suppression degree and similarity of redundant parameters.

[0066] Optionally, the specific implementation steps of S3 may include S31-S33:

[0067] S31, constructing an importance score for each task model according to the degree of suppression of redundant parameters and the similarity;

[0068] The importance score of each task model can be expressed by the following formula (4):

[0069] (4)

[0070] in, Represents the importance score of the i-th task model.

[0071] S32, constructing a mask according to the importance score of each task model;

[0072] Optionally, a mask is used to select the highest scoring element from each element in the importance score for each task model.

[0073] In a feasible implementation, the mask can be expressed by the following formula (5):

[0074] (5)

[0075] in, represents the parameter of the dth position in the ith mask; d represents the rank corresponding to the position parameter in the importance score; represents the parameter of the dth position in the i-th importance score; represents the sorting function; represents the mask ratio, 0 < r < 1; D represents the number of elements.

[0076] S33. Obtain the parameter importance scores in the initial task vector based on the mask and the importance scores of each task model.

[0077] Among them, the parameter importance score in the initial task vector can be expressed by the following formula (6):

[0078] (6)

[0079] in, Represents the parameter importance score in the task vector.

[0080] S4. Obtain a final merged task vector according to the parameter importance scores in the initial task vector; rescale the final merged task vector to generate a final merged task model.

[0081] In a feasible implementation, the final merged task vector can be expressed by the following formula (7):

[0082] (7)

[0083] in, Represents the final merged task vector.

[0084] Optionally, S4 rescales the final merged task vector to generate a final merged task model, including:

[0085] The magnitude of the task vector is proportionally adjusted according to the final merged task vector, integrated with the initial parameter value, and the merged task model parameters are generated by setting the scaling hyperparameters. According to the merged task model parameters, the final merged task model is generated.

[0086] In a feasible implementation, the generated merged task model parameters can be expressed by the following formula (8):

[0087] (8)

[0088] in, Indicates the generated merged task model parameters; Represents the pre-trained model parameters; represents the scaling hyperparameter, also known as the merging coefficient.

[0089] Among them, the initial parameter value is set to 1.

[0090] Optionally, after obtaining the final merged task vector according to the parameter importance scores in the initial task vector in S4; and rescaling the final merged task vector to generate the final merged task model step, the method further includes:

[0091] The intelligent optimization algorithm and the covariance matrix adaptive evolution strategy are used to optimize the scaling hyperparameters to obtain the optimal scaling hyperparameters and improve the verification accuracy of the maximized merged model.

[0092] Among them, the intelligent optimization algorithm can be: ant colony algorithm, particle swarm optimization algorithm and genetic algorithm.

[0093] Among them, the model merging method based on task vectors is more sensitive to the merging coefficient. Even if a suitable uniform merging coefficient is selected, to further improve the fusion performance, it is necessary to perform a network search on the merging coefficient of each task vector. The search process is complex and tedious. Especially when dealing with a large number of tasks, an inappropriate merging coefficient may result in performance lower than the average weighted value, or an unacceptably low accuracy.

[0094] In a feasible implementation, the present invention uses an intelligent optimization algorithm to search for mixing coefficients, and obtains greater improvements by using uniform coefficients. In the optimization process, the best set of merging coefficients is found to improve accuracy.

[0095] In a feasible implementation, by searching for the best merging coefficient, the final merged task vector can be expressed by the following formula (9):

[0096] (9)

[0097] Among them, the present invention adopts a covariance matrix adaptive evolution strategy for search; wherein, the covariance matrix adaptive evolution strategy is a probability-based population optimization algorithm, the covariance matrix adaptive evolution strategy dynamically adjusts the search distribution defined by the covariance matrix, and systematically updates the mean and covariance of the distribution in each iteration to learn and utilize the potential structure of the search space and improve the optimization efficiency.

[0098] In a feasible implementation, the covariance matrix adaptive evolution strategy uses Gaussian distribution to simulate the sampling process of candidate solutions, and gradually updates the covariance matrix of the Gaussian distribution so that the search direction and range can be adaptively adjusted according to the feedback in the optimization process, and the mean and covariance matrix are gradually updated according to the evaluation results of the candidate solutions. The update of the covariance matrix introduces the variation information between successful individuals, so that subsequent searches are more inclined to the variation direction that is conducive to success, so that the covariance matrix adaptive evolution strategy can automatically select and adjust the search step size and direction, thereby effectively exploring complex optimization space.

[0099] In a feasible implementation, the present invention proposes a merging method capable of managing the competition of internal parameters of tasks, which exhibits self-awareness, and a method capable of balancing the competition of parameters between tasks, which is cross-task awareness. The present invention establishes a method for parameter scaling balance matrix that has both self-awareness and cross-task awareness, a model merging method that does not require training and data, uses internal balance to measure the importance of internal parameters of tasks, and evaluates the similarity of parameters between tasks through external balance, discards parameters with lower scores, and rescales the remaining parameters. Finally, the adjusted task vector is merged into the pre-trained model to obtain the final merged model.

[0100] In a feasible implementation, in order to truly demonstrate the effectiveness of the present invention, extensive experiments were conducted to compare the method of the present invention with the existing model merging methods. The superiority of the present invention is demonstrated from four aspects: cross-task merging, cross-domain merging, cross-training configuration, and out-of-domain generalization. In cross-task merging, the present invention uses models such as T5, ViT, and Llama2 in multiple NLP and visual tasks, and tests its ability to fuse multiple PEFT adapters. All experiments showed significant improvements, especially in the T5-base model, which achieved a 4.3% performance improvement.

[0101] Among them, the performance of different methods for merging five domain-specific models in sentiment classification tasks was compared. Roberta-base and T5-base models were adopted, and multiple sets of preprocessed datasets were used. In order to train a single model, 5 high-resource datasets were selected, and 5 low-resource datasets were selected to evaluate the out-of-domain generalization ability. The average accuracy on the in-domain dataset and the average accuracy on the out-of-domain dataset were calculated using various model merging techniques. In addition, experiments were conducted under different random seeds, and the average results of five seeds were reported. Among them, Figure 2 The performance comparison chart of model fusion methods for different modality tasks is shown; Figure 3 It shows an average performance comparison of an embodiment of the present invention on different large language model tasks; Figure 4 It shows the performance comparison of different methods when the model is integrated with different training environment nodes; Figure 5 It is a graph of ablation experiment results for different components of the present invention provided by the embodiment of the present invention. It can be seen from the above comparison results that the method proposed by the present invention improves the performance of Roberta-base and T5-base by 1.1% and 1.3% respectively compared with the strongest baseline method, and the generalization ability in coping with domain migration is improved by 0.8% and 0.7% respectively.

[0102] Among them, in cross-domain merging, the method proposed in this paper successfully merged multiple domain-specific models in tasks such as sentiment classification, demonstrating its effectiveness in processing diverse domain data. In addition, multiple models from different training environments were merged on a single task, highlighting the flexibility and robustness of the method. Finally, the performance of multi-task and multi-domain fusion was evaluated on a domain migration dataset, verifying its generalization ability under various frameworks.

[0103] The embodiment of the present invention first obtains the same task model as multiple initial pre-training models, obtains the corresponding initial task vector by performing differential processing with the initial preprocessing model; uses a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters; uses a nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector; secondly, according to the degree of suppression and similarity of redundant parameters, obtains the parameter importance score in the task vector; finally, according to the parameter importance score in the task vector, obtains the final merged task vector; rescales the final merged task vector to generate merged task model parameters. The present invention aims at the shortcomings of existing methods in dealing with potential conflicts and complex associations between tasks, especially in the adjustment of parameter levels, which makes it a challenge to effectively balance parameter competition among multiple tasks. In order to solve the above problems, the present invention proposes a parameter competition balancing method, which is a lightweight and training-free technology that can achieve efficient model merging by adjusting the coefficient of each parameter. The method measures the importance of parameters in a single task through internal balance and evaluates the similarity of parameters between different tasks through external balance. Parameters with low importance scores are eliminated and the remaining parameters are readjusted to form the final merged model; the method is evaluated in multiple merging scenarios across tasks, domains, training configurations, and out-of-domain generalization. The method significantly improves the model merging performance in multiple modalities, domains, model sizes, number of tasks, fine-tuning forms, and large language models.

[0104] Figure 6 The block diagram of a large language model knowledge fusion device based on parameter constraints according to an exemplary embodiment is shown. The device is used in a large language model knowledge fusion method based on parameter constraints. Figure 6, the device includes a first acquisition unit 610, a calculation unit 620, a second acquisition unit 630 and a generation unit 640. Wherein:

[0105] The first acquisition unit 610 acquires the same task model as the multiple initial pre-training models, and obtains a corresponding initial task vector by performing differential processing with the initial pre-processing model; and suppresses redundant parameters of the initial task vector using a nonlinear activation function to obtain a degree of suppression of redundant parameters;

[0106] A calculation unit 620, used for calculating the similarity of the same position parameters in the initial task vector using a nonlinear activation function;

[0107] A second acquisition unit 630 is used to obtain a parameter importance score in the task vector according to the suppression degree of the redundant parameters and the similarity;

[0108] The generating unit 640 is used to obtain a final merged task vector according to the parameter importance scores in the task vector; and rescale the final merged task vector to generate merged task model parameters.

[0109] Optionally, the second acquiring unit 630 is configured to:

[0110] constructing an importance score for each task model according to the degree of suppression of the redundant parameters and the similarity;

[0111] Construct a mask based on the importance score of each task model;

[0112] Based on the mask and the importance scores of each task model, the parameter importance scores in the task vector are obtained.

[0113] Optionally, the mask is used to select the highest scoring element from each element in the importance score of each task model.

[0114] Optionally, rescaling the final merged task vector to generate merged task model parameters includes:

[0115] The magnitude of the task vector is proportionally adjusted according to the final merged task vector, integrated with the initial parameter value, and the merged task model parameters are generated by setting the scaling hyperparameters. According to the merged task model parameters, the final merged task model is generated.

[0116] Optionally, after the step of obtaining a final merged task vector according to the parameter importance scores in the task vector and rescaling the final merged task vector to generate merged task model parameters, the method further includes:

[0117] The intelligent optimization algorithm and the covariance matrix adaptive evolution strategy are used to optimize the scaling hyperparameters to obtain the optimal scaling hyperparameters and improve the verification accuracy of the maximized merged model.

[0118] The embodiment of the present invention first obtains the same task model as multiple initial pre-training models, obtains the corresponding initial task vector by performing differential processing with the initial preprocessing model; uses a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters; uses a nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector; secondly, according to the degree of suppression and similarity of redundant parameters, obtains the parameter importance score in the task vector; finally, according to the parameter importance score in the task vector, obtains the final merged task vector; rescales the final merged task vector to generate merged task model parameters. The present invention aims at the shortcomings of existing methods in dealing with potential conflicts and complex associations between tasks, especially in the adjustment of parameter levels, which makes it a challenge to effectively balance parameter competition among multiple tasks. In order to solve the above problems, the present invention proposes a parameter competition balancing method, which is a lightweight and training-free technology that can achieve efficient model merging by adjusting the coefficient of each parameter. The method measures the importance of parameters in a single task through internal balance and evaluates the similarity of parameters between different tasks through external balance. Parameters with low importance scores are eliminated and the remaining parameters are readjusted to form the final merged model; the method is evaluated in multiple merging scenarios across tasks, domains, training configurations, and out-of-domain generalization. The method significantly improves the model merging performance in multiple modalities, domains, model sizes, number of tasks, fine-tuning forms, and large language models.

[0119] Figure 7 is a schematic diagram of the structure of a large language model knowledge fusion device based on parameter constraints provided by an embodiment of the present invention, such as Figure 7 As shown, the large language model knowledge fusion device based on parameter constraints may include the above Figure 6 The large language model knowledge fusion device based on parameter constraints is shown. Optionally, the large language model knowledge fusion device based on parameter constraints 610 may include a first processor 2001.

[0120] Optionally, the large language model knowledge fusion device 710 based on parameter constraints may also include a memory 2002 and a transceiver 2003 .

[0121] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0122] Combine the following Figure 7The components of the large language model knowledge fusion device 710 based on parameter constraints are specifically introduced as follows:

[0123] The first processor 2001 is the control center of the parameter-constrained large language model knowledge fusion device 710, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0124] Optionally, the first processor 2001 can perform various functions of the large language model knowledge fusion device 710 based on parameter constraints by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0125] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 are shown in FIG.

[0126] In a specific implementation, as an embodiment, the large language model knowledge fusion device 710 based on parameter constraints may also include multiple processors, such as Figure 7 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0127] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0128] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0129] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0130] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 7 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0131] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and communicate with the first processor 2001 through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0132] It should be noted that Figure 7 The structure of the large language model knowledge fusion device 710 based on parameter constraints shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0133] In addition, the technical effects of the large language model knowledge fusion device 710 based on parameter constraints can refer to the technical effects of the large language model knowledge fusion method based on parameter constraints described in the above method embodiment, and will not be repeated here.

[0134] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0135] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0136] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0137] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0138] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0139] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0140] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0145] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0146] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A large language model knowledge fusion method based on parameter constraints, characterized in that: The method comprises: S1. Obtain the same task model as multiple initial pre-training models, and obtain a corresponding initial task vector by performing differential processing with the initial pre-processing model; use a nonlinear activation function to suppress redundant parameters of the initial task vector to obtain the degree of suppression of redundant parameters; S2, using a nonlinear activation function to calculate the similarity of parameters at the same position in the initial task vector; S3. Obtaining a parameter importance score in the initial task vector according to the suppression degree of the redundant parameters and the similarity; S4. Obtain a final merged task vector according to the parameter importance scores in the initial task vector; rescale the final merged task vector to generate a final merged task model.

2. The large language model knowledge fusion method based on parameter constraints according to claim 1 is characterized in that: The step S3 obtains the parameter importance score in the initial task vector according to the suppression degree of the redundant parameters and the similarity, including: S31, constructing an importance score for each task model according to the suppression degree of the redundant parameters and the similarity; S32, constructing a mask according to the importance score of each task model; S33. Obtain the parameter importance scores in the initial task vector based on the mask and the importance scores of each task model.

3. The large language model knowledge fusion method based on parameter constraints according to claim 2 is characterized in that: The mask is used to select the highest scoring element from each element in the importance score of each task model.

4. The large language model knowledge fusion method based on parameter constraints according to claim 1 is characterized in that: The step S4 rescales the final merged task vector to generate a final merged task model, including: The magnitude of the task vector is proportionally adjusted according to the final merged task vector, integrated with the initial parameter value, and the merged task model parameters are generated by setting the scaling hyperparameters. According to the merged task model parameters, the final merged task model is generated.

5. The large language model knowledge fusion method based on parameter constraints according to claim 1 is characterized in that: The step S4 obtains a final merged task vector according to the parameter importance scores in the initial task vector; After rescaling the final merged task vector to generate the final merged task model step, the method further includes: The intelligent optimization algorithm and the covariance matrix adaptive evolution strategy are used to optimize the scaling hyperparameters to obtain the optimal scaling hyperparameters and improve the verification accuracy of the maximized merged model.

6. A large language model knowledge fusion device based on parameter constraints, the large language model knowledge fusion device based on parameter constraints is used to implement the large language model knowledge fusion method based on parameter constraints as described in any one of claims 1-5, characterized in that: The device comprises: A first acquisition unit is used to acquire the same task model as a plurality of initial pre-training models, and obtain a corresponding initial task vector by performing differential processing with the initial pre-processing model; and suppress redundant parameters of the initial task vector using a nonlinear activation function to obtain a suppression degree of redundant parameters; A calculation unit, used for calculating the similarity of parameters at the same position in the initial task vector using a nonlinear activation function; A second acquisition unit, configured to obtain a parameter importance score in an initial task vector according to the suppression degree of the redundant parameters and the similarity; A generating unit is used to obtain a final merged task vector according to the parameter importance scores in the initial task vector; and rescale the final merged task vector to generate a final merged task model.

7. The large language model knowledge fusion device based on parameter constraints according to claim 6 is characterized in that: The second acquiring unit is used for: constructing an importance score for each task model according to the degree of suppression of the redundant parameters and the similarity; Construct a mask based on the importance score of each task model; According to the mask and the importance scores of each task model, the parameter importance scores in the initial task vector are obtained.

8. The large language model knowledge fusion device based on parameter constraints according to claim 6 is characterized in that: The mask is used to select the highest scoring element from each element in the importance score of each task model.

9. A large language model knowledge fusion device based on parameter constraints, characterized in that: The large language model knowledge fusion device based on parameter constraints includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images