Model merging device and method, and computer program

Abstract

The present invention provides a model merging device and method, and a computer program.SOLUTION: The model merging device includes a comparison unit configured to compare a difference between each model of a plurality of models and a base model, the plurality of models being models obtained based on the base model, and a merging unit configured to merge the plurality of models based on a comparison result of the comparison unit to obtain a merged model capable of implementing functions of the plurality of models.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technical field of information processing, and more particularly to an apparatus, method and computer program for model merging. [Background technology]

[0002] Model merging is a promising method for developing models (e.g., LLMs (Large Language Models)). It can combine multiple models into one single powerful model and inherit the advantages of each model. Model merging does not require training or fine-tuning, is highly efficient, and saves resources. Currently, most of the models at the top of the evaluation lists of open source platforms (e.g., the hugging face platform) are models obtained by model merging. Therefore, the model merging method is very important, as it can directly affect the overall performance of the merged model. Summary of the Invention [Problem to be solved by the invention]

[0003] An object of the present invention is to provide a model merging apparatus, method, and computer program for improving the performance of a merged model. [Means for solving the problem]

[0004] According to one aspect of the present invention, there is provided a model merging apparatus, comprising: a comparison unit that compares the difference between each model of a plurality of models and a base model, the plurality of models being models obtained based on the base model; and The system further includes a merging unit that merges the plurality of models based on the comparison result of the comparison unit to obtain a merged model that can realize the functions of the plurality of models.

[0005] According to another aspect of the present invention, there is provided a method for model merging, comprising: comparing the difference between each model of the plurality of models and a base model, the plurality of models being models derived based on the base model; and The method further includes merging the plurality of models based on the comparison result to obtain a merged model that can realize the functions of the plurality of models.

[0006] According to another aspect of the present invention, there is provided a machine-readable storage medium carrying a program product including machine-readable instruction code which, when read and executed by a computer, causes the computer to perform the model merging method according to the present invention.

[0007] According to another aspect of the invention, a computer program is provided which enables a computer to carry out the model merging method according to the invention. [Effects of the Invention]

[0008] By using the model merging device, model merging method, and computer program according to the present invention, it is possible to take into account the differentiated changes (difference changes) of each model relative to the base model during model merging, thereby enabling the merged model to maintain knowledge of different fields (domains). [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of a model merging device according to an embodiment of the present invention; [Figure 2] FIG. 10 is a block diagram showing the configuration of a model merging device according to another embodiment of the present invention. [Figure 3] FIG. 1 illustrates the merging of three models according to the present invention. [Figure 4] FIG. 1 is a diagram (part 1) showing the merging of two models according to the present invention. [Figure 5] FIG. 2 is a diagram (part 2) showing the merging of two models according to the present invention. [Figure 6]FIG. 3 is a diagram (part 3) showing the merging of two models according to the present invention. [Figure 7] 1 is a flowchart of a method for merging models in an embodiment of the present invention. [Figure 8] 1 is a block diagram showing an exemplary configuration of a general-purpose personal computer capable of implementing a model merging apparatus and method according to an embodiment of the present invention. MODES FOR CARRYING OUT THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Note that the following embodiments are merely illustrative and are not intended to limit the scope of the present invention.

[0011] Traditional model merging methods primarily focus on parameter space. One method is to perform weighted averaging, i.e., linear interpolation, of multiple models (e.g., [1] Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, 2022 ICML). Spherical linear interpolation (SLERP) (e.g., [2] Shoemake, K. (1985). Animating rotation with quaternion curves. ACM SIGGRAPH Computer Graphics, 19(3), 245-254) is also widely used to merge parameters from different models. Other related methods exist, such as randomly setting parameters to zero and rescaling the remaining parameters (e.g., [3] Yu L, Yu B, Yu H, et al. Language models are super mario: Absorbing abilities from homologous models as a free lunch [C] / / Forty-first International Conference on Machine Learning, 2024).

[0012] Current model merging methods typically perform the same processing on all layers of a model and merge them using the same set of hyperparameters (e.g., weights). However, different training or fine-tuning methods can result in different knowledge contained in each layer of the model. For example, when a model is fine-tuned using the low rank adaptation (LoRA) method to acquire specific domain knowledge, the domain knowledge may be concentrated in several model layers designated for fine-tuning. In such cases, when a LoRA model is merged with a base model or a full parameter fine-tuning model, layer differences should not be ignored; that is, a model merging method that performs differentiated processing on each layer of the model is required. The objective of the present invention is to provide such a layer-differentiated model merging method and apply it to models of different domains (e.g., LLMs) so that the merged model can maintain the knowledge of the different domains.

[0013] 1 is a diagram illustrating the configuration of a model merging device according to an embodiment of the present invention. As shown in FIG. 1, a model merging device 100 according to an embodiment of the present invention may include a comparison unit 110 and a merging unit 120.

[0014] The comparison unit 110 can compare the difference between each of the multiple merged models and the base model. Here, all of the multiple models are models obtained based on the base model. In other words, the configurations of the multiple models are the same as or similar to the base model. For example, the multiple models may be obtained by directly training or fine-tuning the base model, or by training or fine-tuning a model obtained directly based on the base model. For example, if a first model is obtained by training or fine-tuning the base model and a second model is obtained by training the first model, the second model also has the same configuration as the base model, so the second model is also a model obtained based on the base model. Alternatively, one of the multiple models may be the base model itself. The base model in the present invention is constructed using a neural network, and the base model and the merged model all include multiple layers with the same or similar configurations, but the parameters of the corresponding layers may be different.

[0015] For example, the base model may be the Mistral-7B model, and multiple models may be domain models based on the Mistral-7B model. Mistral-7B has nearly 7 billion parameters and is a pure decoder-based transformer-based LLM with a dictionary size of 32,000. It consists of 32 transformer modules and one fully connected layer. The transformer module contains a multi-head self-attention structure and a multilayer perceptron (MLP) structure. The self-attention structure contains a qkvo layer, and the multilayer perceptron structure contains a gate / up / down layer.

[0016] As can be understood by those skilled in the art, the base model is not limited to the above, and may be any of various base models in the art. Hereinafter, the multiple layers constituting the model will also be referred to as model constituting units.

[0017] The merging unit 120 can merge multiple models based on the comparison results of the comparison unit 110 to obtain a merged model that can realize the functions of the multiple models. The comparison unit 110 can obtain parameter differences between the model to be merged and the base model by comparing them. Such parameter differences can embody (reflect) which layers of the model to be merged have changed compared to the corresponding layers of the base model and which layers have not changed. Therefore, the merging unit 120 can perform model merging that takes layer differences into account.

[0018] As a result, the model merging device 100 in an embodiment of the present invention can take into account the differentiated changes (difference changes) of each layer of each model during model merging, so that the merged model can maintain knowledge from different fields.

[0019] 2 is a diagram showing the configuration of a model merging device in another embodiment of the present invention. As shown in FIG. 2, a model merging device 200 in another embodiment of the present invention may include a comparison unit 210, a change determination unit 230, a weight determination unit 240, and a merging unit 220. The comparison unit 210 and the merging unit 220 shown in FIG. 2 correspond to the comparison unit 110 and the merging unit 120 shown in FIG. 1, and therefore the above description also applies to this embodiment and will be omitted here.

[0020] The comparison unit 210 can compare the difference between each model of the plurality of models of the combined town and the base model. For example, the base model may include a first set of model constituent units, and each model of the plurality of models may include a second set of model constituent units corresponding to the first set of model constituent units.

[0021] The change determination unit 230 can determine model configuration units in the first set of model configuration units that have not changed for the multiple models based on the difference in parameters of corresponding model configuration units between the first set of model configuration units calculated by the comparison unit 210 and each second set of model configuration units among the multiple second set of model configuration units.

[0022] The weight determination unit 240 can determine, based on the parameter differences, weights to be assigned to each model construction unit in the second set of model construction units that correspond to the model construction units in the first set of model construction units that have been determined to be changing when corresponding model construction units of multiple models are merged.

[0023] For example, for model construction units in the second set of model construction units corresponding to the model construction units in the first set of model construction units determined to have changed, the weight determination unit 240 can set the weight of the model construction units in the multiple models whose parameter difference is less than or equal to a first threshold to 0, and determine the weights of the other corresponding model construction units based on the magnitude (value) of the parameter difference. For example, the larger the parameter difference, the more knowledge the corresponding model construction unit has in the corresponding field, and therefore the greater the weight is given. For example, the first threshold can be the average value of the parameter differences of the corresponding model construction units in the multiple models.

[0024] By setting the weights of model constituent units below the first threshold to 0, it is possible to alleviate (reduce) the interference problem in the parameter fusion process of different models. In addition, by assigning weights that take into account parameter differences, it is possible to selectively fuse models based on the differences in knowledge contained in the model layers, so that the knowledge of each field (also known as domain knowledge) of the different models before fusion can be retained to the maximum extent possible in the fused model.

[0025] For example, for a model configuration unit in the second set of model configuration units that corresponds to a model configuration unit in the first set of model configuration units that has been determined to have changed, the weight determination unit 240 can set the weight of the model configuration unit with the largest parameter difference among the corresponding model configuration units of the multiple models to 1, and set the weights of the other corresponding model configuration units to 0.

[0026] By selecting the most advantageous model-configuration unit among the corresponding model-configuration units and discarding the other corresponding model-configuration units, interference can be reduced to the maximum extent possible. In addition, by assigning weights taking into account parameter differences, selective fusion can be performed on models based on the differences in knowledge contained in the model layers, so that the domain knowledge of each of the different models before fusion can be retained to the maximum extent possible in the fused model.

[0027] In addition, the weight determination unit 240 can determine the weights independently of the parameter differences. For example, the weight determination unit 240 can assign the same weight to corresponding model configuration units of multiple models, where the sum of the weights of corresponding model configuration units of multiple models is 1. For example, the weight determination unit 240 can further assign a higher weight to a model configuration unit corresponding to a function related to a requirement (demand) according to the needs of the actual application. By taking the needs of the actual application into consideration, a merged model that satisfies the requirements to the greatest extent possible can be obtained, thereby realizing model customization.

[0028] The merging unit 220 can obtain a merged model by merging corresponding model-construction units in a plurality of second set of model-construction units of a plurality of models. Specifically, when merging a plurality of models, the merging unit 220 copies the parameters of the unchanged (unchanged) model-construction units as the parameters of the corresponding model-construction units in the merged model, and for model-construction units in the second set of model-construction units corresponding to model-construction units in the first set of model-construction units that have been determined to have changed, the merging unit 220 can merge the parameters of the corresponding model-construction units of the plurality of models as the parameters of the corresponding model-construction units in the merged model. In addition, the merging unit 220 can perform the merging using the weights of each model-construction unit provided by the weight determination unit 240.

[0029] By identifying unchanged model-building units and directly copying the parameters of unchanged model-building units, the amount of computational work can be reduced, saving time and cost. This is because assignment operations are faster than multiplications and additions. For example, direct copying is an assignment operation, whereas multiplications and additions are required when parameters of different model-building units need to be combined.

[0030] When parameters of model construction units are fused, linear interpolation or spherical linear interpolation can be used to perform the fusion. When linear interpolation is used to perform the fusion, parameters of multiple model construction units can be fused simultaneously, and when spherical linear interpolation is used to perform the fusion, parameter fusion of multiple model construction units can be achieved by performing pairwise fusion multiple times.

[0031] As a result, the model merging device 200 in an embodiment of the present invention can take into account the differentiated changes (difference changes) of each layer of each model during model merging, so that the merged model can maintain knowledge from different fields.

[0032] The model merging apparatus 100, 200 according to the present invention can be applied to various scenarios, including but not limited to the following:

[0033] Application scenario 1: Dialect machine Most current LLMs are proficient in English. For example, each region in China has its own dialect. A model merging device according to the present invention can merge various single-dialect models to form a single model proficient in multiple dialects. The dialect model obtained by model merging can be used to effectively process information (e.g., text or images) in multiple dialects. For example, in a family consisting of members from different regions, some family members, such as elderly people, may be from Sichuan and Henan provinces, respectively, and may only speak their own dialects. In this case, dialect models can be embedded in different home robots, such as a music player, a conversation robot, and a cleaning robot, allowing the elderly people to easily operate these robots using their own dialects. This can improve the safety and efficiency of users (e.g., elderly people in the family) operating machines with various functions.

[0034] Application Scenario 2: Multi-style Writing and Conversation Model There are many interesting writing and conversation styles, such as teen drama writing, Qing court drama writing, and French writing. A specific LLM for a particular style can be obtained by fine-tuning a base model (LLM) using a large corpus of that style. For example, if a single model is needed that can write in both teen drama and Qing court drama styles, there is no need to collect a large corpus of these two styles and fine-tune the model again, and no data is required. Instead, the fine-tuned open-source teen drama writing model and Qing court drama writing model can be merged using the model merging method. The writing and conversation model obtained by model merging can be used to provide multi-style creative text based on input information (e.g., text or images) about the scenario. For example, a multi-style conversation model can be incorporated into a social media app. When users do not want to manually write a specific comment or reply to a specific comment, they can simply click and select the style icon to automatically comment or reply using that style. This can improve the efficiency and convenience for users to process data (e.g., text in various styles), thereby improving the user experience of human-computer interaction.

[0035] Application scenario 3: Medical professional model Different hospitals may have pre-trained medical expert models for different subspecialties (e.g., oral, skin, tumor, etc.) based on their respective medical characteristics. Merging models from different subspecialties can yield a comprehensive medical model for disease analysis and classification. The advantage of model merging is that it effectively protects the privacy of each hospital's data and does not require data transmission; it only requires fusing model parameters. For example, by inputting a patient's medical images or case history into the comprehensive medical model obtained by model merging, a comprehensive analysis of the patient's condition can be performed. This comprehensive medical model is particularly necessary for elderly people, who may have problems in various parts of their body. Alternatively, a comprehensive medical model can be used for a specific user, who may have various physical problems that cannot be solved by a single specialized model. In this case, the comprehensive medical model is also very efficient. This can improve the efficiency and accuracy of processing user medical data.

[0036] As can be seen from the above description of specific application scenarios, the model merging technology proposal in the embodiments of the present invention can improve the efficiency and accuracy of users processing data or information in various specific fields.

[0037] Furthermore, data collection and model fine-tuning are very time-consuming, labor-intensive, and expensive tasks, whereas model merging has the advantages of data isolation, speed (a few minutes), and low cost.

[0038] The operation of the model merging apparatus 100, 200 will now be described in more detail in conjunction with Figures 3 to 6. Figure 3 is a diagram illustrating the merging of three models according to the present invention.

[0039] 3 shows the merging of three models A, B, and C as an example, but of course the present invention is not limited to the merging of three models. The model merging device of the present invention can also merge more than three models.

[0040] As an example, the base model includes a first set of model-constituting units O1, O2, O3, and O4, the combined town model A includes a second set of model-constituting units A1, A2, A3, and A4, the combined town model B includes a second set of model-constituting units B1, B2, B3, and B4, and the combined town model C includes a second set of model-constituting units C1, C2, C3, and C4. Note that although Figure 3 shows each set including four model-constituting units, the present invention is not limited to this and may include more or fewer model-constituting units.

[0041] Since models A, B, and C are models obtained based on a base model, model-composing unit O1 corresponds to model-composing units A1, B1, and C1, model-composing unit O2 corresponds to model-composing units A2, B2, and C2, model-composing unit O3 corresponds to model-composing units A3, B3, and C3, and model-composing unit O4 corresponds to model-composing units A4, B4, and C4. Note that "corresponding" here means that the structure of the model-composing units (or neural network layers) is the same, and only the parameters may differ.

[0042] The comparison units 110, 210 compare the differences between models A, B, and C and the base model, thereby obtaining the parameter differences of model A, model B, and model C. As shown in FIG. 3, since a model includes multiple model constituent units, the parameter differences can be represented by a set (set) of parameter differences. The set of parameter differences of model A is (0, D_A2, D_A3, 0), the set of parameter differences of model B is (0, D_B2, 0, 0), and the set of parameter differences of model C is (0, D_C2, D_C3, D_C4). A parameter difference of 0 indicates that the model constituent units have not changed relative to the base model.

[0043] Based on the above-mentioned set of parameter differences, the change determination unit 230 can determine that the model-constructing unit O1 in the first set of model-constructing units has not changed for all three models, and determine that the model-constructing units O2, O3 and O4 have changed.

[0044] The weight determination unit 240 can determine weights to be used when merging models. For example, the weight determination unit 240 can determine weights to be used when merging corresponding model-constituting units of three models, i.e., model-constituting unit A2 of model A, model-constituting unit B2 of model B, and model-constituting unit C2 of model C, based on parameter differences D_A2, D_B2, and D_C2. Similarly, the weight determination unit 240 can determine weights to be used when merging other corresponding model-constituting units based on parameter differences D_A3, 0, D_C3 and parameter differences 0, 0, D_C4. Note that, since several weight determination methods have already been described above, detailed description thereof will be omitted here.

[0045] When merging models, the merging unit 120, 220 can copy the parameters of the model-building unit O1 of the base model as the parameters of the first model-building unit in the merged model. Of course, the merging unit may copy the parameters of any one of the model-building units A1, B1, and C1. For the second model-building unit M2 of the merged model, the merging unit can combine the parameters of the model-building units A2, B2, and C2. For the third model-building unit M3 of the merged model, the merging unit can combine the parameters of the model-building units A3, B3, and C3. For the fourth model-building unit M4 of the merged model, the merging unit can combine the parameters of the model-building units A4, B4, and C4.

[0046] This allows for a merged model including model-constituting units O1, M2, M3, and M4 to be obtained.

[0047] 4 to 6 show different embodiments of the merging of two models according to the present invention.

[0048] Model A and Model B in Figure 4 are similar to the models in Figure 3, but for simplicity, only three model-building units are shown here, not including other model-building units that may exist and are unchanged from the base model.

[0049] Model A (first field) and model B (second field) may represent models of two different fields that have been trained or fine-tuned. In the process of fusing models of two different fields, for each model component of each model, first determine which field's advantage model component the model component belongs to. If the model component belongs to the advantage model component of the first field, the parameters of the corresponding model component of the merged model are from model A; if the model component belongs to the advantage model component of the second field, the parameters of the corresponding model component of the merged model are from model B; and for other model component units, the parameters of the two models are merged to become the parameters of the merged model. Below, a judgment method is given, which uses the difference in parameters of each model component of the two models with respect to the base model as the judgment criterion.

[0050] The comparison units 110 and 210 can calculate a set of first parameter differences (D_A1, D_A2, D_A3) between model A and the base model, and a set of second parameter differences (D_B1, D_B2, D_B3) between model B and the base model. Here, the difference between each parameter in the set of parameter differences may be the absolute value of the difference. In the case of FIG. 4, all parameters of the three model construction units are changing. If the difference between the parameter differences of two corresponding model construction units in the first parameter difference set and the second parameter difference set is greater than a second threshold, the weight determination unit 240 can set the weight of one of the two corresponding model construction units with a relatively large parameter difference to 1 and the weight of the other model construction unit to 0. When the difference between the parameter differences of two corresponding model construction units in the first parameter difference set and the second parameter difference set is equal to or less than a second threshold, the weight determination unit 240 may set the weights of the two corresponding model construction units based on the magnitude (value) of the parameter difference, or may set the weights of the two corresponding model construction units to 0.5, or may assign a higher weight to the model construction unit corresponding to the function related to the requirement according to the needs of the actual application. For example, the larger the parameter difference, the higher the weight of the corresponding model construction unit may be assigned.

[0051] 4, if the difference between D_A1 and D_B1 is greater than the second threshold, the parameters of model-composition unit A1 of model A may be selected as the parameters of model-composition unit M1. For example, if the difference between D_B3 and D_A3 is greater than the second threshold, the parameters of model-composition unit B3 of model B may be selected as the parameters of model-composition unit M3. If the difference between D_A2 and D_B2 is equal to or less than the second threshold, the parameters of model-composition unit A2 and the parameters of model-composition unit B2 may be combined to be used as the parameters of model-composition unit M2.

[0052] This allows us to obtain a merged model that includes model-constituting units M1, M2, and M3.

[0053] Figure 5 shows a more specific example of Figure 4. Using the same base model, full-parameter fine-tuning is employed for the first domain, changing the parameters of all model-building units (all layers). For the second domain, the LoRA fine-tuning method is employed, changing the parameters of only some layers (intermediate model-building units) of the self-attention module. In other words, knowledge from the first domain is distributed across all model-building units (gray areas) in the full-parameter fine-tuning model, while knowledge from the second domain is stored in only some model-building units (gray areas) in the LoRA model. In this case, the model-building units of the two models are not equal, and it is clearly unreasonable to perform the same merger across all layers of the model, as in conventional methods. Therefore, to preserve the knowledge from the two domains, the present invention performs different processing based on the differences between each layer.

[0054] The method of differentially merging different model-building units of two models in the present invention can maximize the advantages of each of the two models. As shown in FIG. 5, for the all-parameter fine-tuning model, all model-building units contain knowledge from the first domain, so the parameters of all model-building units should be reflected in the merged model. For the LoRA fine-tuning model, only some model-building units (gray parts) contain knowledge from the second domain, so only the parameters of the gray parts need to be passed to the merged model. Therefore, as shown in FIG. 5, the gray parts of the second domain model are merged with the corresponding gray parts of the first domain model, and the parameters of the all-parameter fine-tuning model are directly copied to the remaining model-building units of the final model. In other words, for each model-building unit of a model, if the model-building unit contains knowledge from only one domain, the parameters of the model-building unit containing knowledge from this domain are directly copied to the final model. If the model-building unit contains knowledge from two domains, the parameters from the models in the two domains are merged for the model-building unit, and then passed to the final corresponding model-building unit.

[0055] FIG. 6 illustrates a case where one of two models is a base model and the other is a trained or fine-tuned model. The merging unit can merge the two models based on the difference in parameters between the trained or fine-tuned model and the base model. Specifically, the merging unit can copy the parameters of unchanged model-composition units in the base model as the parameters of corresponding model-composition units in the merged model, and set the parameters of the corresponding model-composition units in the merged model as the weighted sum of the parameters of the changed model-composition units in the base model and the parameters of the corresponding model-composition units in the other model. For example, the weight determination unit can set the weights of the two corresponding model-composition units to 0.5, or the weight determination unit can assign a higher weight to a model-composition unit corresponding to a function related to a requirement according to the needs of an actual application.

[0056] By performing fusion with the base model, the forgetting problem of the merged model can be effectively reduced.

[0057] For example, as shown in Figure 6, since only the parameters of the middle model configuration unit (gray part) in the LoRA fine-tuning model are changed, only the parameters of the middle model configuration unit are fused, while the parameters of the other two model configuration units are unchanged, so there is no need to fused them. In this way, the forgetting problem caused by LoRA fine-tuning can be effectively alleviated.

[0058] The multi-layer model merging method of the present invention allows multiple models to be integrated into one single powerful model, which can incorporate knowledge from multiple domains. This technique effectively complements existing model merging methods. It takes into account that different models contain knowledge from different domains at different layers, and therefore, each layer is treated differently in the merging process.

[0059] The model merging method according to the embodiment of the present invention will be described below in conjunction with FIG.

[0060] 7, the model merging method in the embodiment of the present invention starts in step S710. In step S710, the difference between each model among the plurality of models and a base model is compared, where the plurality of models are different models obtained based on the base model.

[0061] Next, in step S720, the multiple models are merged based on the comparison result to obtain a merged model that can realize the functions of the multiple models.

[0062] This ends the process.

[0063] It should be noted that the various specific implementations of the above steps of the model merging method in the embodiment of the present invention have been described in detail, and therefore, detailed description thereof will be omitted here.

[0064] In this invention, a new model merging method is proposed, which selectively merges the differences in knowledge contained in the model layers, thereby retaining the maximum amount of knowledge in each field of the different models before merging in the merged model.

[0065] The effectiveness of the method for merging models with layer differences according to the present invention has been verified through experiments.

[0066] (1) Our method can obtain the best performance when merging the full-parameter fine-tuning model and the LoRA model.

[0067] The base model used in the experiment was Mistral-7b-Instruct-v0.1. The inventors merged one full-parameter fine-tuning model in the mathematics field and one LoRA model in the medical field. The numbers in the table represent the test results for the corresponding model under a specific test set. For example, the numbers can represent the accuracy rate of the model's answers. As an example, the test set consisted of 1,000 question-answer pairs. The questions were input to the large-scale model, which then output predicted answers. The predicted answers were compared with the actual answers to confirm the accuracy of the predicted answers, and the percentage of correct answers to the total number of answers in the test set was calculated. The experimental results are shown in Table 1. The proposed method for merging models with layer differences achieved the best average performance (the average value was the largest), retaining the knowledge of both fields to the maximum extent possible. This method outperforms other methods, such as those described in [2] and [3].

[0068] [Table 1] (2) Our layer-difference model merging method can improve the performance of various fusion methods.

[0069] The inventors of the present invention have further verified that the layer difference model merging method of the present invention is not limited to a specific merging technology, but is effective for both linear interpolation merging and spherical linear interpolation merging, as shown in Table 2, and is superior to each of the conventional merging methods.

[0070] [Table 2] (3) The merging of models with different layers in the present invention can help the LoRA model to prevent forgetting.

[0071] In experiments, the inventors discovered that merging models with different layers can help prevent forgetting in LoRA models. As shown in Table 3, the LoRA model's performance improved in a specific domain (e.g., the medical domain here), but its general ability (general ability) significantly decreased, i.e., it exhibited catastrophic forgetting. The merged model obtained after fusing the LoRA model and the base model using the merging method with different layers in the present invention can not only maintain performance in the general domain (general domain), but also improve performance in specific domains, and in particular, the average performance is significantly improved.

[0072] [Table 3] As can be seen, the novel layer-difference model merging method proposed in this invention integrates several LLMs to obtain a single powerful LLM, allowing the model to incorporate knowledge from multiple domains. This technology effectively complements existing model merging methods. It takes into account that different LLMs contain knowledge from different domains at different layers, so each layer is treated differently during the merging process. Experiments have verified that the method proposed in this invention is superior to traditional methods. Given the many customization requirements for domain models in the real world, the method proposed in this invention can facilitate the development of domain models.

[0073] Of course, each operation process of the model merging method according to the present invention may be realized by a computer-executable program stored in various machine-readable storage media.

[0074] The object of the present invention may also be realized in the following manner: a storage medium storing the executable program code is provided directly or indirectly to a system or device, and the program code is read and executed by a computer or central processing unit (CPU) in the system or device. In this case, as long as the system or device has a function capable of executing a program, the embodiment of the present invention is not limited to a program, and the program may be in any format, such as an object-oriented program, a program executed by an interpreter, or a script program provided to an OS.

[0075] Such machine-readable storage media include, but are not limited to, various memory devices and storage units, semiconductor devices, magnetic disk drives such as magnetic and magneto-optical disks, and other media suitable for storing information.

[0076] In addition, the technical solution of the present invention can also be realized by connecting a computer to a corresponding website on the Internet, downloading and installing the computer program code of the present invention on the computer, and then running the program.

[0077] Furthermore, each component or unit in the above-described device may be configured using software, firmware, hardware, or a combination thereof. Since the specific means and methods used for configuration are well known to those skilled in the art, detailed explanations thereof will be omitted here. When implemented using software or firmware, the programs constituting the software are installed from a storage medium or a network onto a computer having a dedicated hardware structure (e.g., the general-purpose computer shown in FIG. 8), and the computer can realize various functions when various programs are installed.

[0078] FIG. 8 is a hardware configuration diagram of a personal computer that can implement the device and method according to the present invention.

[0079] 8, a central processing unit (CPU) 801 performs various processes based on a program stored in a ROM 802 or a program loaded from a storage device 808 to a RAM 803. The RAM 803 can also store data required by the CPU 801 to perform various processes, depending on the needs. The CPU 801, ROM 802, and RAM 803 are connected to one another via a bus 804. An input / output interface 805 is also connected to the bus 804.

[0080] The input / output interface 805 is further connected to the following components: an input device 806 including a keyboard; an output device 807 including a display such as a liquid crystal display (LCD) and a speaker; a storage device 808 including a hard disk; and a communication device 809 including a network interface card such as a LAN card or a modem. The communication device 809 performs communication processing via a network such as the Internet or a LAN. A drive 810 may be connected to the input / output interface 805 as needed. A removable medium 811, such as a semiconductor memory, can be inserted into the drive 810 as needed, allowing a computer program read from the medium to be installed in the storage device 808.

[0081] The present invention also provides a program product including machine-readable instruction codes, which, when read and executed by a machine, can perform the methods of the above-described embodiments of the present invention. Accordingly, various storage media for carrying such program products, such as magnetic disks (including floppy disks (registered trademark)), optical disks (including CD-ROMs and DVDs), magneto-optical disks (including MDs (registered trademark)), and semiconductor storage devices, are also included in the present invention.

[0082] The storage medium may include, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory device, etc., but is not limited to these.

[0083] Moreover, each operation (process / step) in the above-described method can also be realized in the form of a computer-executable program stored in various machine-readable storage media.

[0084] Furthermore, the following supplementary notes are disclosed regarding the above-mentioned embodiments.

[0085] (Appendix 1) A model merging device, a comparison unit that compares the difference between each model of a plurality of models and a base model, the plurality of models being models obtained based on the base model; and a merging unit that merges the plurality of models based on the comparison result of the comparison unit and obtains a merged model that can realize the functions of the plurality of models.

[0086] (Appendix 2) 2. The model merging device of claim 1, comprising: The base model includes a first set of model construction units, and each model among the plurality of models includes a second set of model construction units corresponding to the first set of model construction units, and the merging unit merges corresponding model construction units in a plurality of second set of model construction units of the plurality of models to obtain the merged model; and The model merging device further includes a change determination unit that determines model construction units in the first set of model construction units that have not changed for the plurality of models based on the difference in parameters of corresponding model construction units between the first set of model construction units and each second set of model construction units among the plurality of second set of model construction units calculated by the comparison unit.

[0087] (Appendix 3) 3. The model merging device of claim 2, further comprising: When merging the plurality of models, the merging unit copies the parameters of the unchanged model-construction units as parameters of the corresponding model-construction units in the merged model; and The model merging device further includes a weight determining unit, which determines a weight for a model building unit in the second set of model building units corresponding to a model building unit in the first set of model building units that is determined to have changed. determining a weight to be assigned to each model-building unit when merging corresponding model-building units of the plurality of models based on the parameter differences; or Giving the same weight to corresponding model-constituting units of the plurality of models, wherein the sum of the weights of the corresponding model-constituting units of the plurality of models is 1; or Depending on the needs of the actual application, higher weights are given to model configuration units that correspond to the features related to the requirements.

[0088] (Appendix 4) 4. The model merging device of claim 3, The weight determination unit determines whether a model structuring unit in the second set of model structuring units corresponds to a model structuring unit in the first set of model structuring units that has been determined to have changed. Among the corresponding model configuration units of the plurality of models, the weight of a model configuration unit whose parameter difference is equal to or less than a first threshold is set to 0, and the weights of other corresponding model configuration units are determined based on the magnitude of the parameter difference.

[0089] (Appendix 5) 5. The model merging device of claim 4, The first threshold value is an average value of differences in parameters of corresponding model constituent units of the plurality of models.

[0090] (Appendix 6) 4. The model merging device of claim 3, The weight determination unit determines whether a model structuring unit in the second set of model structuring units corresponds to a model structuring unit in the first set of model structuring units that has been determined to have changed. Among the corresponding model configuration units of the plurality of models, the weight of the model configuration unit having the largest difference in the parameters is set to 1, and the weights of the other corresponding model configuration units are set to 0.

[0091] (Appendix 7) 4. The model merging device of claim 3, the plurality of models includes a first model and a second model; and the comparators respectively calculate a set of first parameter differences representing differences between the first model and the base model and a set of second parameter differences representing differences between the second model and the base model; and When the difference between the parameter differences of the two corresponding model construction units in the first parameter difference set and the second parameter difference set is greater than a second threshold, the weight determination unit sets the weight of one of the two corresponding model construction units with a larger parameter difference to 1, and sets the weight of the other model construction unit to 0; otherwise, The weight determining unit sets weights of two corresponding model-constructing units based on the magnitude of the difference between the parameters; or The weight determination unit sets the weights of the two corresponding model-constructing units to 0.5; or The weight determining unit assigns a higher weight to the model configuration unit corresponding to the function related to the requirement according to the needs of the actual application.

[0092] (Appendix 8) 2. The model merging device of claim 1, comprising: The plurality of models includes a third model and the base model, and the merging unit merges the third model and the base model based on a difference between parameters of the third model and the base model.

[0093] (Appendix 9) 9. The model merging apparatus of claim 8, further comprising: The merging unit copies the parameters of the unchanged model-constituting units in the base model as the parameters of the corresponding model-constituting units in the merged model, and sets the weighted sum of the parameters of the changed model-constituting units in the base model and the parameters of the corresponding model-constituting units in the third model as the parameters of the corresponding model-constituting units in the merged model; The weight determination unit sets the weights of the two corresponding model-constructing units to 0.5; or The weight determining unit assigns a higher weight to the model configuration unit corresponding to the function related to the requirement according to the needs of the actual application.

[0094] (Appendix 10) 1. A method for model merging, comprising: comparing the difference between each model of the plurality of models and a base model, the plurality of models being models derived based on the base model; and merging the plurality of models based on the comparison result to obtain a merged model capable of realizing the functions of the plurality of models.

[0095] (Appendix 11) 11. The method of model merging of claim 10, further comprising: The base model includes a first set of model construction units, and each model among the plurality of models includes a second set of model construction units corresponding to the first set of model construction units, and corresponding model construction units in a plurality of second set of model construction units of the plurality of models are merged to obtain the merged model; and The method further includes determining model construction units in the first set of model construction units that are all unchanged for a plurality of models based on parameter differences of corresponding model construction units between the first set of model construction units and each second set of model construction units among a plurality of second set of model construction units.

[0096] (Appendix 12) 12. The method of model merging according to claim 11, further comprising: When merging the plurality of models, copying the parameters of the unchanged model-building units as parameters of the corresponding model-building units in the merged model; and The method further comprises: for a model building unit in the second set of model building units that corresponds to a model building unit in the first set of model building units that has been determined to have changed; determining a weight to be assigned to each model-building unit when merging corresponding model-building units of the plurality of models based on the parameter differences; or Giving the same weight to corresponding model-constituting units of the plurality of models, wherein the sum of the weights of the corresponding model-constituting units of the plurality of models is 1; or This involves giving higher weight to model configuration units that correspond to requirements-related features according to the needs of the actual application.

[0097] (Appendix 13) 13. The method of model merging of claim 12, further comprising: For a model construction unit in the second set of model construction units corresponding to a model construction unit in the first set of model construction units that has been determined to have changed, Among the corresponding model configuration units of the plurality of models, the weight of a model configuration unit whose parameter difference is equal to or less than a first threshold is set to 0, and the weights of other corresponding model configuration units are determined based on the magnitude of the parameter difference.

[0098] (Appendix 14) 14. The method of model merging according to claim 13, further comprising: The first threshold value is an average value of differences in parameters of corresponding model constituent units of the plurality of models.

[0099] (Appendix 15) 13. The method of model merging of claim 12, further comprising: For a model construction unit in the second set of model construction units corresponding to a model construction unit in the first set of model construction units that has been determined to have changed, Among the corresponding model configuration units of the plurality of models, the weight of the model configuration unit having the largest difference in the parameters is set to 1, and the weights of the other corresponding model configuration units are set to 0.

[0100] (Appendix 16) 13. The method of model merging of claim 12, further comprising: the plurality of models includes a first model and a second model; and The method includes calculating a set of first parameter differences representing differences between the first model and the base model and a set of second parameter differences representing differences between the second model and the base model, respectively; and If the difference between the parameter differences of the two corresponding model configuration units in the first parameter difference set and the second parameter difference set is greater than a second threshold, set the weight of one of the two corresponding model configuration units with the greater parameter difference to 1, and set the weight of the other model configuration unit to 0; otherwise, Setting weights for two corresponding model-building units based on the magnitude of the difference between the parameters; or Set the weights of the two corresponding model units to 0.5, or This involves giving higher weight to model configuration units that correspond to requirements-related features according to the needs of the actual application.

[0101] (Appendix 17) 11. The method of model merging of claim 10, further comprising: The plurality of models includes a third model and the base model, and the third model and the base model are merged based on a difference in parameters between the third model and the base model.

[0102] (Appendix 18) 18. The model merging method of claim 17, further comprising: Copying the parameters of the unchanged model-building units in the base model as the parameters of the corresponding model-building units in the merged model, and setting the weighted sum of the parameters of the changed model-building units in the base model and the parameters of the corresponding model-building units in the third model as the parameters of the corresponding model-building units in the merged model; and Set the weights of the two corresponding model units to 0.5, or Depending on the needs of the actual application, higher weights are given to model configuration units that correspond to the features related to the requirements.

[0103] (Appendix 19) 18. The model merging method of claim 17, further comprising: The third model is a LoRA fine-tuning model.

[0104] (Appendix 20) 1. A machine-readable storage medium, comprising: A program product of machine-readable instruction code, The instruction code, when read and executed by a computer, can cause the computer to perform the model merging method described in any one of Appendixes 10-19.

[0105] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to this embodiment, and any modification to the present invention falls within the technical scope of the present invention as long as it does not depart from the spirit of the present invention.

Claims

1. 1. An apparatus for merging models, comprising: a comparison unit that compares the difference between each model of a plurality of models and a base model, the plurality of models being models obtained based on the base model; and a merging unit that merges the plurality of models based on a comparison result of the comparing unit to obtain a merged model that can realize the functions of the plurality of models.

2. 10. The apparatus of claim 1, the base model includes a first set of model constituent units, and each model among the plurality of models includes a second set of model constituent units corresponding to the first set of model constituent units; the merging unit merges corresponding model constituting units in a plurality of second set model constituting units of the plurality of models to obtain the merged model; The apparatus further includes a change determination unit, which determines model construction units in the first set of model construction units that are not changed for the plurality of models based on the difference in parameters of corresponding model construction units between the first set of model construction units and each second set of model construction units among the plurality of second set of model construction units calculated by the comparison unit.

3. 3. The apparatus of claim 2, When merging the plurality of models, the merging unit copies the parameters of the unchanged model-constituting units as parameters of the corresponding model-constituting units in the merged model; The apparatus further includes a weight determiner; The weight determination unit For a model construction unit in the second set of model construction units corresponding to a model construction unit in the first set of model construction units that has been determined to have changed, determining a weight to be assigned to each model-building unit when merging corresponding model-building units of the plurality of models based on the parameter differences; or giving the same weight to corresponding model-building units of the plurality of models, the sum of the weights of corresponding model-building units of the plurality of models being 1; or The device assigns higher weights to model configuration units corresponding to requirements-related features according to the needs of the actual application.

4. 4. The apparatus of claim 3, The weight determination unit For a model construction unit in the second set of model construction units corresponding to a model construction unit in the first set of model construction units that has been determined to have changed, setting a weight of a model configuration unit of the plurality of models, the difference of which is equal to or less than a first threshold, to 0; The apparatus determines weights of corresponding other model-constructing units based on the parameter difference values.

5. 5. The apparatus of claim 4, The apparatus, wherein the first threshold is an average value of differences in parameters of corresponding model constituent units of the plurality of models.

6. 4. The apparatus of claim 3, The weight determination unit For a model construction unit in the second set of model construction units corresponding to a model construction unit in the first set of model construction units that has been determined to have changed, Among the corresponding model configuration units of the plurality of models, a weight of the model configuration unit having the largest difference in the parameter is set to 1; The device sets the weights of the corresponding other model configuration units to 0.

7. 4. The apparatus of claim 3, the plurality of models includes a first model and a second model; each of the comparison units calculates a first set of parameter differences representing differences between the first model and the base model, and a second set of parameter differences representing differences between the second model and the base model; The weight determination unit When the difference between the parameter differences of two corresponding model-constructing units in the first parameter difference set and the second parameter difference set is greater than a second threshold value, Of the two corresponding model configuration units, the weight of one with a larger parameter difference is set to 1, and the weight of the other model configuration unit is set to 0; If not, setting weights of two corresponding model-building units based on the parameter difference values; or Set the weights of the two corresponding model-building units to 0.5, or The device assigns higher weights to model configuration units corresponding to requirements-related features according to the needs of the actual application.

8. 10. The apparatus of claim 1, the plurality of models includes a third model and the base model; the merging unit merges the third model and the base model based on a difference in parameters between the third model and the base model; The merging unit copying parameters of unchanged model-building units in the base model as parameters of corresponding model-building units in the merged model; The parameter of the corresponding model-building unit in the merged model is determined by weighting the parameter of the model-building unit in the base model and the parameter of the corresponding model-building unit in the third model; The apparatus further includes a weight determiner; The weight determination unit Set the weights of the two corresponding model-building units to 0.5, or The device assigns higher weights to model configuration units corresponding to requirements-related features according to the needs of the actual application.

9. 1. A method for merging models, comprising: comparing the difference between each model of the plurality of models and a base model, the plurality of models being models obtained based on the base model; and merging the plurality of models based on a comparison result to obtain a merged model capable of realizing the functionality of the plurality of models.

10. A program for causing a computer to execute the method according to claim 9.

Images