Large language model merging method and device, electronic equipment and storage medium

Abstract

The invention provides a big language model merging method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a density hyper-parameter and a model merging coefficient hyper-parameter used for model merging based on a task target of a merging model in a downstream task; generating a mask matrix based on the density hyper-parameter, and determining task vectors of the two to-be-merged models based on the mask matrix, the respective model parameters of the two to-be-merged models and the model parameters of the basic model; based on the parameter vector angle of the corresponding neuron between the two to-be-merged models, the task vectors of the two to-be-merged models and the model merging coefficient hyper-parameter, determining the task vector of the merged model; and obtaining the merging model based on the task vector of the merging model and the model parameters of the basic model. By adopting the technical scheme, the problem that the performance of a merging model obtained by an existing model merging method is poor is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of model merging, and in particular to a large language model merging method, device, electronic device and storage medium. Background Art

[0002] Model merging technology uses parameter averaging, interpolation and other methods to fuse two or more models with the same structure and the same initialization weights into a model with better performance.

[0003] Large language models refer to models with a large number of model parameters, such as Llama2-13B and other models. As the training cost of large language models increases, model merging technology has become an important solution to improve model performance. Merging models can effectively combine the advantages of different models, thereby improving model performance, reducing model complexity, or achieving better generalization capabilities.

[0004] In traditional model merging methods, the model parameters are usually directly weighted averaged. However, the model merging methods in the prior art may cause conflicts in the parameter space, and the performance of the merged model is poor. Summary of the invention

[0005] The present invention provides a large language model merging method, device, electronic device and storage medium, which are used to solve the problem of poor performance of the merged model obtained by the existing model merging method.

[0006] In a first aspect, the present invention provides a large language model merging method, comprising: determining a density hyperparameter and a model merging coefficient hyperparameter used for model merging based on the task objectives of the merged model in the downstream tasks. Generate a mask matrix based on the density hyperparameter, and determine the task vectors of the two models to be merged based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model. Determine the task vector of the merged model based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vector of the two models to be merged, and the model merging coefficient hyperparameter. Obtain a merged model based on the task vector of the merged model and the model parameters of the base model.

[0007] Optionally, based on the parameter vector angles of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameters, the task vector of the merged model is determined, including: based on the angle interval of the parameter vector angles of the neurons at the corresponding positions between the two models to be merged, and the task vectors of the two models to be merged and the model merging coefficient hyperparameters, the task vector of the merged model is determined. The angle intervals include [0°, 60°], (60°, 120°], and (120°, 180°].

[0008] Optionally, based on the angle interval of the parameter vector angle of the neurons at the corresponding positions between the two models to be merged, as well as the task vectors of the two models to be merged and the model merging coefficient hyperparameters, the task vector of the merged model is determined, including: when the parameter vector angle of the neurons at the first position between the two models to be merged is in the interval [0°, 60°], based on the elements corresponding to the neurons at the first position in the task vectors of the two models to be merged and the model merging coefficient hyperparameters, the elements corresponding to the neurons at the first position in the task vector of the merged model are determined. When the parameter vector angle of the neurons at the first position between the two models to be merged is in the interval (60°, 120°], based on the elements corresponding to the neurons at the first position in the task vectors of the two models to be merged and the parameter vector angle of the neurons at the first position between the two models to be merged, the elements corresponding to the neurons at the first position in the task vector of the merged model are determined. When the parameter vector angle of the neurons at the first position between the two models to be merged is in the interval (120°, 180°], the elements corresponding to the neurons at the first position in the task vector of the merged model are determined to be 0.

[0009] Optionally, based on the elements corresponding to the first position neurons in the task vectors of the two models to be merged and the model merging coefficient hyperparameters, the elements corresponding to the first position neurons in the task vector of the merged model are determined, including: based on the model merging coefficient hyperparameters, weighted averaging the elements corresponding to the first position neurons in the task vectors of the two models to be merged to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0010] Optionally, based on the elements corresponding to the first position neurons in the task vectors of the two models to be merged and the parameter vector angle of the first position neurons between the two models to be merged, the elements corresponding to the first position neurons in the task vector of the merged model are determined, including: based on the parameter vector angle and the interpolation factor of the first position neurons between the two models to be merged, the elements corresponding to the first position neurons in the task vectors of the two models to be merged are interpolated and merged to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0011] Optionally, based on the mask matrix, the model parameters of each of the two models to be merged and the model parameters of the base model, the task vectors of the two models to be merged are determined respectively, including: subtracting the model parameter matrix of the base model from the model parameter matrix of any model to be merged, multiplying the result with the mask matrix to obtain the task vector of the model to be merged.

[0012] In a second aspect, the present invention provides a large language model merging device, comprising: a first determination module, for determining a density hyperparameter and a model merging coefficient hyperparameter used for model merging based on the task objectives of the merged model in the downstream tasks. A second determination module, for generating a mask matrix based on the density hyperparameter, and for determining the task vectors of the two models to be merged based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model. A third determination module, for determining the task vector of the merged model based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter. A processing module, for obtaining the merged model based on the task vector of the merged model and the model parameters of the base model.

[0013] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the large language model merging methods described above is implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the large language model merging methods described above.

[0015] In a fifth aspect, the present invention further provides a computer program product, including a computer program, which implements any of the above-mentioned large language model merging methods when executed by a processor.

[0016] The large language model merging method, device, electronic device and storage medium provided by the present invention determine the density hyperparameter and model merging coefficient hyperparameter used for model merging by merging the model's task objectives in downstream tasks. A mask matrix is ​​generated based on the density hyperparameter, and the task vectors of the two models to be merged are determined respectively using the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model. The task vector of the merged model is determined by calculating the parameter vector angle of the corresponding neurons between the two models to be merged, combining the task vectors of the two models to be merged and the model merging coefficient hyperparameters, to ensure that the merged model can give full play to the advantages of each model in the downstream tasks. A merged model is obtained based on the task vector of the merged model and the model parameters of the base model. With the help of the parameter vector angle and the task vector, it is ensured that the merged model can achieve the best performance in a specific application scenario. The problem of poor performance of the merged model obtained by the existing model merging method is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the large language model merging method provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the large language model merging device provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Figure 1 Schematic diagram of the process of merging large language models provided by the present invention. Figure 1 As shown, the method includes the following steps S101 to S104.

[0023] Step S101: Based on the task objectives of the merged model in the downstream tasks, determine the density hyperparameters and model merging coefficient hyperparameters used for model merging.

[0024] As an optional embodiment, model merging requires at least two models to be merged. For the sake of convenience, the following embodiment takes two models to be merged as an example. The present invention does not further limit the specific number of models to be merged.

[0025] As an optional embodiment, to complete the model merging, the models to be merged are required to meet the conditions of the same model structure and initialization weight. For example, GAIR / Abel-7B-001 (mathematical domain model) and AdaptLLM / finance-chat (financial domain model) are selected. Both models are based on the Llama2 architecture and are fine-tuned versions of the Llama2-7B pre-trained model, with a size of 7B. The task goal of the downstream task can be to improve the performance of the AdaptLLM / finance-chat model in mathematical reasoning ability through model merging, thereby improving the indicators on the test set.

[0026] As an optional embodiment, the task goal of the merged model in the downstream task is the actual performance requirement of the merged model in the final application scenario. For example, if the goal of the downstream task is image classification, then when merging the models, it is necessary to ensure that the final model can better recognize and classify images. If the goal is sentiment analysis in natural language processing, when merging the models, it is necessary to focus on retaining and strengthening the characteristics of sentiment understanding in the two models. Therefore, during the merging process, the density hyperparameters and the model merging coefficient hyperparameters are set with the help of the task goals so that the merged model can perform well in specific tasks.

[0027] As an optional embodiment, the density hyperparameter determines the ratio of parameters selected from two models to be merged when merging models. The merging coefficient hyperparameter controls the fusion ratio of task vectors of two models to be merged when merging.

[0028] As an optional embodiment, when determining the density hyperparameter and the model merging coefficient hyperparameter used for model merging, it is necessary to determine the objective function according to the task objectives of the merged model in the downstream task. The expression of the objective function is:

[0029]

[0030] In the above formula, is the accuracy of the combined model on the evaluation dataset, D is the evaluation dataset of the combined model, is the model input, is the output of the combined model, is the density hyperparameter, Represents the model merging coefficient hyperparameter. Represents a merged model.

[0031] As an optional embodiment, when determining the density hyperparameter and the model merging coefficient hyperparameter used for model merging, the hyperparameters may be set based on manual experience or determined with the help of a Bayesian-particle swarm algorithm.

[0032] As an optional embodiment, when using the Bayesian-particle swarm algorithm to determine the density hyperparameter and the model merging coefficient hyperparameter, there are mainly the following steps.

[0033] Step 1: Determine the objective function based on the task objectives of the merged model in the downstream tasks. The expression of the objective function has been described in the above embodiment and will not be repeated here.

[0034] Step 2: Set the hyperparameter search space and initialize the hyperparameters (including density hyperparameters and model merging coefficient hyperparameters).

[0035] Step 3: Use the Bayesian algorithm to model the objective function, update the posterior mean and variance, and calculate the acquisition function. Repeat multiple iterations to obtain the initialization hyperparameters with better performance.

[0036] Step 4: Use the initialization hyperparameters (including density hyperparameters and model merging coefficient hyperparameters) obtained in step 3 as the initialization particle swarm of the particle swarm algorithm to update the particle speed and position.

[0037] Step 5: When the maximum number of iterations is reached, the iteration is stopped and the density hyperparameter and model merging coefficient hyperparameter at this time are output.

[0038] As an optional embodiment, the present invention does not further limit the method for obtaining the density hyperparameter and the model merging coefficient hyperparameter used for model merging.

[0039] Step S102: Generate a mask matrix based on the density hyperparameter, and determine the task vectors of the two models to be merged based on the mask matrix, the model parameters of the two models to be merged, and the model parameters of the base model.

[0040] As an optional embodiment, the elements in the mask matrix are composed of 0 and 1. First, a matrix with the same dimension as the parameter vector and all elements are 1 is initialized, and the ratio is randomly set to the density hyperparameter The elements of are set to 0, and finally a binary matrix containing only 0 and 1 is obtained, and the binary matrix is ​​the mask matrix.

[0041] As an optional embodiment, the model parameters of the two models to be merged and the model parameters of the base model are parameter matrices, each neuron in the model to be merged corresponds to a parameter matrix, and the parameter matrices of all neurons constitute the model parameters of the models to be merged. Since the architecture and size of the two models to be merged are the same as the base model.

[0042] As an optional embodiment, the models to be merged are GAIR / Abel-7B-001 (model 1) and AdaptLLM / finance-chat (model 2). Both models are Llama2 architectures and are fine-tuned versions of the Llama2-7B pre-trained model. The base model is Llama2-7B.

[0043] As an optional embodiment, the density hyperparameter It is used to determine the ratio of parameters selected from the two models to be merged when merging models. Through the density hyperparameter, a mask matrix can be generated , the mask matrix indicates which model's parameters are preferred during the merging process. For example, with a density hyperparameter of 0.7, the matrix mask will select 70% of the parameters from GAIR / Abel-7B-001 and the remaining 30% from AdaptLLM / finance-chat.

[0044] As an optional embodiment, the mask matrix determines which parameters are preferred when merging model parameters. The mask matrix is ​​used to extract the parts that need to be retained from the model parameters of the two models to be merged, and the parts are combined with the model parameters of the base model to generate task vectors for the two models to be merged. The task vectors represent the features that the models retain and play in a specific task.

[0045] In an exemplary embodiment, based on the mask matrix, the model parameters of the two models to be merged, and the model parameters of the base model, respectively determining the task vectors of the two models to be merged includes: Subtract the model parameter matrix of the basic model from the model parameter matrix of any model to be merged, and multiply the result with the mask matrix to obtain the task vector of the model to be merged.

[0046] As an optional embodiment, the task vectors of the two models to be merged can be respectively expressed as.

[0047]

[0048] in, and is the model parameter matrix of the two models to be merged, is the task vector, i.e., the model parameter matrix of the model to be merged minus the model parameter matrix With the mask matrix The dot product of is the density hyperparameter. The task vector corresponding to the first model to be merged, represents the task vector of the second model to be merged. As an optional embodiment, It can be understood as the mission vector of GAIR / Abel-7B-001. It can be understood as the task vector of AdaptLLM / finance-chat. These are the model parameters of Llama2-7B.

[0049] In the above embodiment, a mask matrix is ​​used to extract a task vector from two models to be merged. Through this selective parameter retention and fusion, the merged new model can effectively combine the advantages of the two models to be merged, and improve the performance of the merged model in downstream tasks.

[0050] Step S103: Determine the task vector of the merged model based on the parameter vector angles of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameters.

[0051] As an optional embodiment, since the size and architecture of the two models to be merged are exactly the same, the neurons between the two models to be merged are one-to-one corresponding, so the parameter vector angle can be determined by combining the parameter matrix of the corresponding neurons between the two models to be merged with the cosine operation formula. The task vector of the merged model is determined based on the parameter vector angle, the task vector and the model merging coefficient hyperparameter.

[0052] As an optional embodiment, the parameter vector angle of the corresponding neurons between the two models to be merged can be calculated by the following formula.

[0053]

[0054] in, The corresponding parameter matrix of the neuron in the first position in the model parameters of model 1 is represented. Represents the parameter matrix of model 2 corresponding to the neurons at the corresponding positions in model 1.

[0055] As an optional embodiment, before calculating the parameter vector angle, it is necessary to and Normalization is performed, and the cosine value of the angle between the parameter vectors is obtained by dot product calculation, and then the parameter vector angle is derived.

[0056] As an optional embodiment, since there are multiple neurons corresponding to model 1 and model 2, and corresponding to parameter matrices of multiple neurons, multiple parameter vector angles can be obtained. Different merging strategies are needed according to the angle of the parameter vector angle to determine the task vector corresponding to each parameter vector angle, and the task vector of the final merged model is determined based on each task vector.

[0057] Step S104: Obtain a merged model based on the task vector of the merged model and the model parameters of the basic model.

[0058] As an optional embodiment, since the merged model is obtained by merging two models to be merged that have the same size and architecture as the base model, only the weight parameters of the model merging are changed during the merging process, and the structure of the model will not be changed. Therefore, the architecture of the merged model and the base model remains the same. Therefore, the model parameters of the base model are a parameter matrix, and the corresponding task vector is also a vector matrix that corresponds one-to-one to the model parameters of the base model. Therefore, the task vector of the merged model can be added to the model parameters of the original base model through operations between matrices to obtain the above-mentioned merged model.

[0059] In the above steps S101 to S104, the density hyperparameters and model merging coefficient hyperparameters used for model merging are determined by merging the task objectives of the models in the downstream tasks. A mask matrix is ​​generated based on the density hyperparameters, and the task vectors of the two models to be merged are determined respectively using the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model. The task vector of the merged model is determined by calculating the parameter vector angle of the corresponding neurons between the two models to be merged, combining the task vectors of the two models to be merged and the model merging coefficient hyperparameters, to ensure that the merged model can give full play to the advantages of each model in the downstream tasks. The merged model is obtained based on the task vector of the merged model and the model parameters of the base model. With the help of the parameter vector angle and the task vector, it is ensured that the merged model can achieve the best performance in a specific application scenario. The problem of poor performance of the merged model obtained by the existing model merging method is solved.

[0060] In an exemplary embodiment, the above step S103 can be implemented by the following step S11.

[0061] Step S11, determining the task vector of the merged model based on the angle interval of the parameter vector angles of the neurons at corresponding positions between the two models to be merged, as well as the task vectors of the two models to be merged and the model merging coefficient hyperparameters.

[0062] The angle ranges include [0°, 60°], (60°, 120°], and (120°, 180°].

[0063] As an optional embodiment, in the traditional model merging method, simply weighting or averaging the parameters may introduce conflicts, especially when the characteristics of the two models to be merged are quite different. The present invention analyzes the angle of the parameter vector. If the angle between the two parameter vectors is small ([0°, 60°]), it indicates that the characteristics of the two neurons at the corresponding positions are relatively similar or synergistic. If the parameter vector angle is between (60°, 120°], it means that the two parameter vectors have both similarities and certain differences. If the angle is between (120°, 180°], it indicates that there are large differences in the parameter characteristics of the two neurons. The task vector of the final merged model is determined according to the angles of different parameter vector angles, avoiding invalid parameter superposition. The stability of the merged model is enhanced while retaining beneficial features.

[0064] As an optional embodiment, the elements corresponding to each parameter vector angle are obtained by determining the angle of each parameter vector angle and the task vector and model merging coefficient hyperparameters, and then the task vector of the merged model is obtained based on the addition of the elements of each parameter vector angle.

[0065] In the above embodiment, different merging strategies are adopted according to the task vector angle to obtain the task vector of the final merged model. This ensures that when merging models, more accurate fusion decisions can be made based on feature similarity. This avoids conflicts caused by simple parameter averaging, and ultimately ensures that the merged model has stronger adaptability and performance in downstream tasks.

[0066] In an exemplary embodiment, based on the angle interval of the parameter vector angles of neurons at corresponding positions between the two models to be merged, as well as the task vectors of the two models to be merged and the model merging coefficient hyperparameters, determining the task vector of the merged model includes: When the parameter vector angle of the first position neuron between the two models to be merged is in the interval [0°, 60°], the element corresponding to the first position neuron in the task vector of the merged model is determined based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the model merging coefficient hyperparameters.

[0067] In an exemplary embodiment, based on the elements corresponding to the first position neurons in the task vectors of the two models to be merged and the model merging coefficient hyperparameter, determining the elements corresponding to the first position neurons in the task vector of the merged model includes: Based on the model merging coefficient hyperparameter, the elements corresponding to the first position neurons in the task vectors of the two models to be merged are weighted averaged to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0068] As an optional embodiment, when the angle is between 0 and 60 degrees, it is considered that the elements of the parameter vector angle have the same direction and are directly weighted averaged to obtain the element corresponding to the neuron at the first position in the task vector of the merged model.

[0069] When the parameter vector angle of the first position neuron between the two models to be merged is in the interval of (60°, 120°], the element corresponding to the first position neuron in the task vector of the merged model is determined based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the parameter vector angle of the first position neuron between the two models to be merged.

[0070] In an exemplary embodiment, determining the element corresponding to the first position neuron in the task vector of the merged model based on the element corresponding to the first position neuron in the task vectors of the two models to be merged and the parameter vector angle of the first position neuron between the two models to be merged includes: Based on the parameter vector angle and interpolation factor of the first position neuron between the two models to be merged, the elements corresponding to the first position neurons in the task vectors of the two models to be merged are interpolated and merged to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0071] As an optional embodiment, interpolation merging is performed when the angle is between 60 and 120 degrees.

[0072] When the parameter vector angle of the first position neuron between the two models to be merged is in the interval of (120°, 180°], it is determined that the element corresponding to the first position neuron in the task vector of the merged model is 0.

[0073] As an optional embodiment, the above embodiment can be expressed by a formula.

[0074]

[0075] In the above formula, represents the model merging coefficient hyperparameter, represents the task vector, represents the parameter vector angle, is the interpolation factor, The value of is usually a constant. Represents the task vectors obtained at different angle ranges.

[0076] In the above embodiment, different elements are determined for different parameter vector angles, which can effectively retain useful features, eliminate conflicts, and finely control feature fusion when models are merged.

[0077] In an exemplary embodiment, since the size and architecture of the model to be merged are consistent with the base model, the task vector of the merged model is added to the model parameters of the base model to obtain the final merged model. It can be expressed as:

[0078]

[0079] in, To merge the models, is the task vector of the combined model, which is composed of the task vectors calculated by the above three angle ranges and is a Matrices of the same size, Represents the model parameters of the base model.

[0080] The large language model merging device provided by the present invention is described below. The large language model merging device described below and the large language model merging method described above can be referenced to each other.

[0081] Figure 2 Schematic diagram of the structure of the large language model merging device provided by the present invention. Figure 2 As shown, the device includes.

[0082] The first determination module 210 is used to determine the density hyperparameter and the model merging coefficient hyperparameter used for model merging based on the task objectives of the merged model in the downstream tasks.

[0083] The second determination module 220 is used to generate a mask matrix based on the density hyperparameter, and determine the task vectors of the two models to be merged based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model.

[0084] The third determination module 230 is used to determine the task vector of the merged model based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter.

[0085] The processing module 240 is used to obtain the merged model based on the task vector of the merged model and the model parameters of the basic model.

[0086] The above-mentioned device determines the density hyperparameter and model merging coefficient hyperparameter used for model merging by merging the task objectives of the models in the downstream tasks. A mask matrix is ​​generated based on the density hyperparameter, and the task vectors of the two models to be merged are determined respectively using the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model. By calculating the parameter vector angle of the corresponding neurons between the two models to be merged, combining the task vectors of the two models to be merged and the model merging coefficient hyperparameters, the task vector of the merged model is determined to ensure that the merged model can give full play to the advantages of each model in the downstream tasks. Based on the task vector of the merged model and the model parameters of the base model, a merged model is obtained. With the help of the parameter vector angle and the task vector, it is ensured that the merged model can achieve the best performance in a specific application scenario. The problem of poor performance of the merged model obtained by the existing model merging method is solved.

[0087] In some embodiments, the third determination module 230 is further used to determine the task vector of the merged model based on the angle interval of the parameter vector angle of the neurons at the corresponding positions between the two models to be merged, as well as the task vectors of the two models to be merged and the model merging coefficient hyperparameters. The angle intervals include [0°, 60°], (60°, 120°], and (120°, 180°].

[0088] In some embodiments, the third determination module 230 is further used to determine the element corresponding to the first position neuron in the task vector of the merged model based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the model merging coefficient hyperparameters when the parameter vector angle of the first position neuron between the two models to be merged is in the interval [0°, 60°]. When the parameter vector angle of the first position neuron between the two models to be merged is in the interval (60°, 120°], the element corresponding to the first position neuron in the task vector of the merged model is determined based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the parameter vector angle of the first position neuron between the two models to be merged. When the parameter vector angle of the first position neuron between the two models to be merged is in the interval (120°, 180°], the element corresponding to the first position neuron in the task vector of the merged model is determined to be 0.

[0089] In some embodiments, the third determination module 230 is also used to perform weighted averaging of the elements corresponding to the first position neurons in the task vectors of the two models to be merged based on the model merging coefficient hyperparameter to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0090] In some embodiments, the third determination module 230 is also used to interpolate and merge the elements corresponding to the first position neurons in the task vectors of the two models to be merged based on the parameter vector angle and interpolation factor of the first position neurons between the two models to be merged, so as to obtain the elements corresponding to the first position neurons in the task vector of the merged model.

[0091] In some embodiments, the second determination module 220 is further used to subtract the model parameter matrix of the basic model from the model parameter matrix of any model to be merged, and multiply the result with the mask matrix to obtain the task vector of the model to be merged.

[0092] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the large language model merging method, which includes.

[0093] Based on the task objectives of the merged model in the downstream tasks, the density hyperparameter and model merging coefficient hyperparameter used for model merging are determined.

[0094] A mask matrix is ​​generated based on the density hyperparameter, and the task vectors of the two models to be merged are determined based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model.

[0095] The task vector of the merged model is determined based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter.

[0096] Based on the task vector of the merged model and the model parameters of the base model, a merged model is obtained.

[0097] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0098] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large language model merging method provided by the above methods, which includes.

[0099] Based on the task objectives of the merged model in the downstream tasks, the density hyperparameter and model merging coefficient hyperparameter used for model merging are determined.

[0100] A mask matrix is ​​generated based on the density hyperparameter, and the task vectors of the two models to be merged are determined based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model.

[0101] The task vector of the merged model is determined based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter.

[0102] Based on the task vector of the merged model and the model parameters of the base model, a merged model is obtained.

[0103] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the large language model merging method provided by the above methods, and the method includes.

[0104] Based on the task objectives of the merged model in the downstream tasks, the density hyperparameter and model merging coefficient hyperparameter used for model merging are determined.

[0105] A mask matrix is ​​generated based on the density hyperparameter, and the task vectors of the two models to be merged are determined based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model.

[0106] The task vector of the merged model is determined based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter.

[0107] Based on the task vector of the merged model and the model parameters of the base model, a merged model is obtained.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0109] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large language model merging method, characterized in that: include: Based on the task objectives of the merged model in the downstream tasks, determine the density hyperparameters and model merging coefficient hyperparameters used for model merging; Generate a mask matrix based on the density hyperparameter, and determine the task vectors of the two models to be merged based on the mask matrix, the model parameters of the two models to be merged, and the model parameters of the base model; Determine the task vector of the merged model based on the parameter vector angles of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter; The merged model is obtained based on the task vector of the merged model and the model parameters of the base model.

2. The large language model merging method according to claim 1, characterized in that: The determining the task vector of the merged model based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter includes: Determine the task vector of the merged model based on the angle interval of the parameter vector angles of the neurons at corresponding positions between the two models to be merged, as well as the task vectors of the two models to be merged and the model merging coefficient hyperparameter; The angle intervals include [0°, 60°], (60°, 120°], and (120°, 180°].

3. The large language model merging method according to claim 2, characterized in that: The step of determining the task vector of the merged model based on the angle interval of the parameter vector angles of neurons at corresponding positions between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameters includes: When the parameter vector angle of the first position neuron between the two models to be merged is in the interval [0°, 60°], based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the model merging coefficient hyperparameter, determining the elements corresponding to the first position neuron in the task vector of the merged model; When the parameter vector angle of the first position neuron between the two models to be merged is in the interval of (60°, 120°], based on the elements corresponding to the first position neuron in the task vectors of the two models to be merged and the parameter vector angle of the first position neuron between the two models to be merged, determining the elements corresponding to the first position neuron in the task vector of the merged model; When the parameter vector angle of the first position neuron between the two models to be merged is in the interval of (120°, 180°], it is determined that the element corresponding to the first position neuron in the task vector of the merged model is 0.

4. The large language model merging method according to claim 3, characterized in that: The step of determining the element corresponding to the first position neuron in the task vector of the merged model based on the element corresponding to the first position neuron in the task vector of the two models to be merged and the model merging coefficient hyperparameter includes: Based on the model merging coefficient hyperparameter, weighted averaging is performed on the elements corresponding to the first position neuron in the task vectors of the two models to be merged to obtain the elements corresponding to the first position neuron in the task vector of the merged model.

5. The large language model merging method according to claim 3, characterized in that: The step of determining the element corresponding to the first position neuron in the task vector of the merged model based on the element corresponding to the first position neuron in the task vectors of the two models to be merged and the parameter vector angle of the first position neuron between the two models to be merged, comprises: Based on the parameter vector angle and interpolation factor of the first position neuron between the two models to be merged, the elements corresponding to the first position neuron in the task vectors of the two models to be merged are interpolated and merged to obtain the elements corresponding to the first position neuron in the task vector of the merged model.

6. The large language model merging method according to any one of claims 1 to 5, characterized in that: The step of determining the task vectors of the two models to be merged based on the mask matrix, the model parameters of the two models to be merged, and the model parameters of the base model, respectively, includes: The model parameter matrix of any model to be merged is subtracted from the model parameter matrix of the basic model, and the result is multiplied by the mask matrix to obtain the task vector of the model to be merged.

7. A large language model merging device, characterized in that: include: A first determination module is used to determine a density hyperparameter and a model merging coefficient hyperparameter used for model merging based on a task goal of the merged model in a downstream task; A second determination module is used to generate a mask matrix based on the density hyperparameter, and determine the task vectors of the two models to be merged based on the mask matrix, the model parameters of each of the two models to be merged, and the model parameters of the base model; A third determination module is used to determine the task vector of the merged model based on the parameter vector angle of the corresponding neurons between the two models to be merged, the task vectors of the two models to be merged, and the model merging coefficient hyperparameter; A processing module is used to obtain the merged model based on the task vector of the merged model and the model parameters of the basic model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large language model merging method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model merging method as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the large language model merging method as claimed in any one of claims 1 to 6 is implemented.

Images