Model merging apparatus, model merging method, and machine readable storage medium

Abstract

The invention relates to a model merging device, a model merging method and a machine readable storage medium. The model merging apparatus includes: a comparison section that compares differences between each of a plurality of models, which are models obtained based on a pedestal model, and the pedestal model; and a merging unit that merges the plurality of models on the basis of the comparison results of the comparison unit to obtain a merged model capable of implementing the functions of the plurality of models. The model merging device can consider the differentiation change of each model relative to the base model in model merging, so that the merged model can keep knowledge in different fields.

Description

Technical Field

[0001] This disclosure relates to the field of information processing, and more particularly to apparatus, methods and machine-readable storage media for model merging. Background Technology

[0002] This section provides background information relating to this disclosure, which is not necessarily prior art.

[0003] Model merging is a promising approach for developing models (such as Large Language Models, LLMs). It combines multiple models into a single, powerful model, inheriting the strengths of each individual model. Model merging requires no training or fine-tuning, making it highly efficient and resource-saving. Currently, most of the top-performing models on open-source platforms (such as Hugging Face) are obtained through model merging. Therefore, the model merging method is crucial, as it directly impacts the overall performance of the merged model. Summary of the Invention

[0004] This section provides a general overview of this disclosure, rather than a full disclosure of its entire scope or all its features.

[0005] The purpose of this disclosure is to provide a model merging apparatus, a model merging method, and a machine-readable storage medium for improving the performance of merged models.

[0006] According to one aspect of this disclosure, a model merging apparatus is provided, comprising: a comparison unit that compares the differences between each of a plurality of models and a base model, wherein the plurality of models are models obtained based on the base model; and a merging unit that merges the plurality of models based on the comparison result of the comparison unit to obtain a merged model capable of realizing the functions of the plurality of models.

[0007] According to another aspect of this disclosure, a model merging method is provided, comprising: comparing the differences between each of a plurality of models and a base model, wherein the plurality of models are different models obtained based on the base model; and merging the plurality of models based on the comparison results to obtain a merged model capable of realizing the functions of the plurality of models.

[0008] According to another aspect of this disclosure, a machine-readable storage medium is provided that carries a program product including machine-readable instruction code stored thereon, wherein the instruction code, when read and executed by a computer, enables the computer to perform the model merging method according to this disclosure.

[0009] Using the model merging apparatus, model merging method, and machine-readable storage medium according to this disclosure, the merged model can retain knowledge from different domains by taking into account the differences between each model and the base model during model merging.

[0010] Further applicability will become apparent from the description provided herein. The descriptions and specific examples in this summary are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0011] The accompanying drawings described herein are for illustrative purposes only and not for all possible implementations, and are not intended to limit the scope of this disclosure. In the drawings:

[0012] Figure 1 This is a block diagram illustrating the structure of a model merging apparatus according to an embodiment of the present disclosure;

[0013] Figure 2 This is a block diagram illustrating the structure of a model merging apparatus according to another embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram illustrating the merging of the three models according to this disclosure;

[0015] Figures 4 to 6 This is a schematic diagram illustrating the merging of the two models according to this disclosure;

[0016] Figure 7 A flowchart illustrating a model merging method according to an embodiment of the present disclosure; and

[0017] Figure 8 This is a block diagram of an exemplary structure of a general-purpose personal computer in which the model merging apparatus and method according to embodiments of the present disclosure can be implemented.

[0018] While this disclosure is readily subject to various modifications and substitutions, specific embodiments thereof have been shown by way of example in the accompanying drawings and are described in detail herein. However, it should be understood that the description of specific embodiments herein is not intended to limit this disclosure to the specific forms disclosed, but rather, this disclosure is intended to cover all modifications, equivalents, and substitutions falling within the spirit and scope of this disclosure. It should be noted that throughout the drawings, corresponding reference numerals indicate corresponding parts. Detailed Implementation

[0019] Examples of this disclosure will now be described more fully with reference to the accompanying drawings. The following description is merely exemplary and is not intended to limit the disclosure, its application, or its uses.

[0020] Example embodiments are provided so that this disclosure will become exhaustive and will fully convey its scope to those skilled in the art. Numerous specific details, such as examples of particular components, apparatus, and methods, are set forth to provide a detailed understanding of embodiments of this disclosure. It will be apparent to those skilled in the art that the specific details are not required, and that the example embodiments may be implemented in many different forms, none of which should be construed as limiting the scope of this disclosure. In some example embodiments, well-known processes, well-known structures, and well-known techniques are not described in detail.

[0021] Traditional model merging methods mainly focus on the parameter space. One approach is to perform a weighted average of multiple models, i.e., linear interpolation (see, for example, reference [1], Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, 2022 ICML). In addition, spherical linear interpolation (SLERP) (see, for example, reference [2], Shoemake, K. (1985). Animating rotation with quaternion curves. ACM SIGGRAPH Computer Graphics, 19(3), 245-254) is also widely used to merge the parameters of different models. Other related methods also exist, such as random zeroing and rescaling of the remaining parameters (see, for example, reference [3], Yu L, Yu B, Yu H, et al. Language models are super mario: Absorbing abilities from homologous models as a free lunch [C] / / Forty-first International Conference on Machine Learning. 2024).

[0022] Current model merging methods typically treat all layers of a model similarly and use the same set of hyperparameters (such as weights) for merging. However, depending on the training or fine-tuning methods, the knowledge contained in each layer of the model may differ. For example, when using low-rank adaptation (LoRA) to fine-tune a model to acquire domain-specific knowledge, this domain knowledge is concentrated in a few layers that are specifically fine-tuned. In this case, when merging a LoRA model into a base model or a fully parameterized fine-tuned model, layer differences should not be ignored; that is, a model merging method that handles the differences between the various layers of the model is needed. This application aims to propose such a layer-differentiated model merging method and apply it to models from different domains (e.g., large language models), so that the merged model can retain knowledge from different domains.

[0023] Figure 1 The diagram illustrates the structure of a model merging apparatus according to an embodiment of the present disclosure. For example... Figure 1 As shown, the model merging apparatus 100 according to an embodiment of the present disclosure may include a comparison unit 110 and a merging unit 120.

[0024] The comparison unit 110 can compare the differences between each of the multiple models to be merged and the base model. Here, all the multiple models are models obtained based on the base model. That is, the structure of the multiple models is the same as or similar to the base model. For example, the multiple models can be obtained by directly training or fine-tuning the base model, or by training or fine-tuning a model directly obtained based on the base model. For example, if a first model is obtained by training or fine-tuning the base model and a second model is obtained by training the first model, the structure of the second model is also the same as the base model, so the second model is also a model obtained based on the base model. Furthermore, the models among the multiple models can also be the base model itself. The base model in this application is composed of a neural network, and both the base model and the models to be merged include multiple layers with the same or similar structure, but the parameters of the corresponding layers may be different.

[0025] As an example, the base model could be the Mistral-7B model, while multiple models could be domain-specific models using the Mistral-7B model as their base. Mistral-7B has nearly 7 billion parameters and is a large-scale, transformer-based language model with a pure decoder structure and a dictionary size of 32,000. It consists of 32 transformer modules and one fully connected layer. The transformer modules internally include multi-head self-attention structures and multi-layer perceptron (MLP) structures. The self-attention structure includes qkvo layers, and the multi-layer perceptron structure includes gate / up / down layers.

[0026] Those skilled in the art will understand that the base model is not limited to this, and can be any base model in the art. In the following text, the multiple layers constituting the model will also be referred to as model building units.

[0027] The merging unit 120 can merge multiple models based on the comparison results of the comparison unit 110 to obtain a merged model that can realize the functions of multiple models. The comparison unit 110 obtains the parameter differences between the model to be merged and the base model by comparing them. These parameter differences can reflect which layers of the model to be merged have changed relative to the corresponding layers of the base model and which layers have not changed. Therefore, the merging unit 120 can perform model merging that takes into account layer differences.

[0028] Therefore, the model merging apparatus 100 according to the embodiments of the present disclosure can take into account the differential changes of each layer of each model in the model merging process, so that the merged model can maintain knowledge of different domains.

[0029] Figure 2 The diagram illustrates the structure of a model merging apparatus according to another embodiment of the present disclosure. For example... Figure 2 As shown, a model merging apparatus 200 according to another embodiment of the present disclosure may include a comparison unit 210, a change determination unit 230, a weight determination unit 240, and a merging unit 220. Figure 2 The comparison unit 210 and the merging unit 220 shown correspond to Figure 1 The comparison section 110 and the merging section 120 shown above are also applicable to this embodiment, and the description will be omitted appropriately.

[0030] The comparison unit 210 can compare the differences between each of the multiple models to be merged and the base model. For example, the base model may include a first set of model building units, and each of the multiple models may include a second set of model building units corresponding to the first set of model building units.

[0031] The change determination unit 230 can determine the model building units in the first group of model building units that have not been changed for all multiple models based on the parameter differences between the corresponding model building units in the first group of model building units and each of the multiple second group of model building units calculated by the comparison unit 210.

[0032] The weight determination unit 240 can determine the weights assigned to each model building unit when merging the corresponding model building units of multiple models, based on parameter differences, for the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed.

[0033] For example, the weight determination unit 240 can, for the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that were determined to have changed, set the weights of model building units whose parameter differences among the corresponding model building units of multiple models are below a first threshold to 0, and determine the weights of the corresponding other model building units based on the magnitude of the parameter differences. For example, model building units with larger parameter differences may have more knowledge in the corresponding domain, and therefore are given greater weights. For example, the first threshold may be the average of the parameter differences among the corresponding model building units of multiple models.

[0034] By setting the weights of model building units below the first threshold to 0, the interference problem of parameters from different models during the fusion process can be mitigated. Furthermore, assigning weights based on parameter differences allows for selective fusion of models based on the differences in knowledge contained within the model layers, thereby maximizing the retention of the respective domain knowledge of the different models before fusion in the fused model.

[0035] For example, the weight determination unit 240 can, for the model building units in the second group of model building units that are determined to have changed in the first group of model building units, set the weight of the model building unit with the largest parameter difference among the corresponding model building units of multiple models to 1, and set the weight of the corresponding other model building units to 0.

[0036] By selecting the most advantageous model building unit from the corresponding model building units and discarding the others, interference can be minimized. Furthermore, by considering parameter differences when assigning weights, selective fusion of models can be performed based on the knowledge differences contained in the model layers, thereby maximizing the retention of the respective domain knowledge of the different models before fusion in the fused model.

[0037] Furthermore, the weight determination unit 240 can also determine weights independently of parameter differences. As an example, the weight determination unit 240 can assign the same weight to corresponding model building units of multiple models, where the sum of the weights of the corresponding model building units of multiple models is 1. As another example, the weight determination unit 240 can also assign higher weights to model building units corresponding to the functions involved in the requirements, based on actual application needs. By considering actual application needs, a merged model that best meets the requirements can be obtained, achieving model customization.

[0038] The merging unit 220 can obtain a merged model by merging corresponding model building units from multiple second-group model building units of multiple models. Specifically, when merging multiple models, the merging unit 220 can copy the parameters of unchanged model building units as the parameters of corresponding model building units in the merged model, and for model building units in the second group that correspond to model building units in the first group that are determined to have changed, the merging unit 220 can fuse the parameters of corresponding model building units from multiple models as the parameters of corresponding model building units in the merged model. Furthermore, the merging unit 220 can use the weights of each model building unit provided by the weight determination unit 240 to perform the fusion.

[0039] By identifying unchanged model building blocks and directly copying their parameters, the amount of computer processing can be reduced, thus saving time and cost. This is because assignment operations are faster than multiplication and addition operations. For example, direct copying is an assignment operation; if it is necessary to merge the parameters of different fusion model building blocks, multiplication and addition operations will be involved.

[0040] When fusing the parameters of model building units, linear interpolation or spherical linear interpolation can be used. When using linear interpolation, parameters from multiple model building units can be fused simultaneously. When using spherical linear interpolation, multiple pairwise fusions can be performed to achieve the fusion of parameters from multiple model building units.

[0041] Therefore, the model merging apparatus 200 according to the embodiments of the present disclosure can take into account the differential changes of each layer of each model in the model merging, so that the merged model can maintain knowledge of different domains.

[0042] The model merging devices 100 and 200 according to this application can be applied to various scenarios, including but not limited to the following scenarios.

[0043] Application Scenario 1: Dialect Machine

[0044] Most current large-scale language models are proficient in English. However, different regions in China have their own dialects. The model merging device described in this application can be used to merge individual dialect models to form a model proficient in multiple dialects. The dialect language model obtained through model merging can be used to efficiently process information involving multiple dialects (such as text or images). For example, in a family composed of people from different regions, some family members, such as the elderly, may be from Sichuan and Henan respectively. The elderly may only speak their dialect. In this case, the dialect language model can be integrated into different home robots, such as music players, conversational robots, and robot vacuum cleaners, allowing the elderly to easily operate these robots using their dialect. This improves the safety and efficiency for users (e.g., elderly family members) operating various functional machines.

[0045] Application Scenario 2: Dialogue Model for Writing in Multiple Styles

[0046] In writing and dialogue scenarios, many interesting styles exist, such as youth-themed melancholy literature, the "Empresses in the Palace" style, and the "Buddhist" style. A large corpus of specific styles can be used to fine-tune a large-scale base language model, resulting in a specific large-scale language model for that style. When a model capable of writing both youth-themed melancholy literature and the "Empresses in the Palace" style is desired, there's no need to collect large corpora of both styles for re-fine-tuning. No data is required; simply merge the already fine-tuned open-source youth-themed melancholy literature model and the "Empresses in the Palace" style model using a model merging method. The resulting writing and dialogue model can be used to provide multi-style creative text based on context-specific information (such as text or images). For example, a multi-style dialogue model can be integrated into online social applications. When we don't want to write certain comments or reply to certain comments manually, we can directly select a style to achieve automatic replies using that style. This improves the efficiency and convenience of users processing data (such as text in various styles), thereby enhancing the user experience of human-computer interaction.

[0047] Application Scenario 3: Medical Expert Model

[0048] Different hospitals may have pre-trained medical expert models in different sub-domains (e.g., dentistry, dermatology, oncology) based on their own medical specialties. By merging these different sub-domain models, a general practice model can be obtained for disease analysis and classification. The benefits of model merging include effectively protecting the data privacy of each hospital, eliminating the need for data transmission, and requiring only the fusion of model parameters. For example, inputting a patient's medical images or medical records into a general practice model obtained through model merging allows for comprehensive analysis of the patient's condition. This is particularly necessary for an elderly person whose body may have problems in various parts. Alternatively, a general practice model can be tailored to specific users whose health issues are diverse and cannot be addressed by a single specialist model; therefore, a general practice model is also highly efficient. This improves the efficiency and accuracy of processing user medical data.

[0049] As can be seen from the above description of specific application scenarios, the model merging technology solution according to the embodiments of this application is beneficial to improving the efficiency and accuracy of users in processing various specific domain data or information.

[0050] In addition, since collecting data and fine-tuning the model is a very time-consuming, labor-intensive, and expensive task, model merging has advantages such as data isolation, speed (within minutes), and low cost.

[0051] The following is for reference. Figures 3 to 6 To explain in more detail the operation of model merging devices 100 and 200. Figure 3 This is a schematic diagram illustrating the merging of the three models according to this disclosure.

[0052] As an example, Figure 3 The illustration shows the merging of three models A, B, and C; however, this application is not limited to merging only three models. The model merging device of this application can merge more than three models.

[0053] As an example, the base model includes a first set of model building units O1, O2, O3, and O4; the model to be merged, A, includes a second set of model building units A1, A2, A3, and A4; the model to be merged, B, includes a second set of model building units B1, B2, B3, and B4; and the model to be merged, C, includes a second set of model building units C1, C2, C3, and C4. Although Figure 3 The diagram shows that each group includes four model building units, but this application is not limited to this and may include more or fewer model building units.

[0054] Models A, B, and C are derived from the base model. Therefore, model building unit O1 corresponds to model building units A1, B1, and C1; model building unit O2 corresponds to model building units A2, B2, and C2; model building unit O3 corresponds to model building units A3, B3, and C3; and model building unit O4 corresponds to model building units A4, B4, and C4. In this paper, "correspondence" means that the structure of the model building units (or layers of the neural network) is the same, only the parameters may differ.

[0055] Comparison units 110 and 210 can compare the differences between models A, B, and C and the base model, thereby obtaining the parameter differences of model A, model B, and model C. For example... Figure 3 As shown, since the model comprises multiple model building units, parameter differences can be represented by parameter difference sets. The parameter difference set for model A is (0, D_A2, D_A3, 0), the parameter difference set for model B is (0, D_B2, 0, 0), and the parameter difference set for model C is (0, D_C2, D_C3, D_C4). A parameter difference of 0 indicates that the model building unit has not changed relative to the base model.

[0056] The change determination unit 230 can determine the model building unit O1 in the first group of model building units that has not been changed for all three models based on the above parameter difference set, and can determine that model building units O2, O3 and O4 have been changed.

[0057] The weight determination unit 240 can determine the weights used when merging models. As an example, the weight determination unit 240 can determine the weights used when fusing the corresponding model building units of three models—model building unit A2 of model A, model building unit B2 of model B, and model building unit C2 of model C—based on parameter differences D_A2, D_B2, and D_C2. Similarly, the weight determination unit 240 can determine the weights used when fusing other corresponding model building units based on parameter differences D_A3, 0, D_C3 and parameter differences 0, 0, D_C4. Various weight determination methods have been described above and will not be repeated here.

[0058] When merging models, merging units 120 and 220 can copy the parameters of model building unit O1 of the base model as the parameters of the first model building unit in the merged model. Of course, the merging units can also copy the parameters of any one of model building units A1, B1, and C1. For the second model building unit M2 of the merged model, the merging unit can fuse the parameters of model building units A2, B2, and C2. For the third model building unit M3 of the merged model, the merging unit can fuse the parameters of model building units A3, B3, and C3. For the fourth model building unit M4 of the merged model, the merging unit can fuse the parameters of model building units A4, B4, and C4.

[0059] Thus, a merged model including model building units O1, M2, M3, and M4 can be obtained.

[0060] Figures 4 to 6 This is a schematic diagram illustrating different embodiments of the merging of two models according to the present disclosure.

[0061] Figure 4 Models A and B in Figure 3 The model is similar to the one shown in the image, but for simplicity, only three model building units are shown here. It should be noted that other model building units that may exist and remain unchanged relative to the base model are not shown here.

[0062] Model A (first domain) and Model B (second domain) can represent two trained or fine-tuned models from different domains. During the fusion of these two models, for each building block of each model, the process first determines which domain's dominant building block it belongs to. If it belongs to the first domain, the parameters of the corresponding building block in the merged model are derived from Model A; if it belongs to the second domain, the parameters are derived from Model B. For other building blocks, the parameters of the merged model are the parameters of the combined model. One method for this determination is shown below, which uses the parameter differences between each building block of the two models relative to the base model as the criterion.

[0063] Comparison units 110 and 210 can calculate the first set of parameter differences (D_A1, D_A2, D_A3) between model A and the base model, and the second set of parameter differences (D_B1, D_B2, D_B3) between model B and the base model. The differences in each parameter within these sets can be absolute values. Figure 4In the case where the parameters of all three model building units change, if the difference in parameter differences between two corresponding model building units in the first and second parameter difference sets is greater than a second threshold, the weight determination unit 240 can set the weight of the model building unit with the larger parameter difference to 1 and the weight of the other model building unit to 0. If the difference in parameter differences between two corresponding model building units in the first and second parameter difference sets is less than or equal to the second threshold, the weight determination unit 240 can set the weights of the two corresponding model building units according to the magnitude of the parameter difference, or set the weights of both corresponding model building units to 0.5, or assign a higher weight to the model building unit corresponding to the function involved in the requirement according to the actual application needs. For example, the model building unit with a larger parameter difference can be assigned a larger weight.

[0064] For example, in Figure 4 In this process, if the difference between D_A1 and D_B1 is greater than the second threshold, the parameters of model building unit A1 of model A can be selected as the parameters of model building unit M1. Similarly, if the difference between D_B3 and D_A3 is greater than the second threshold, the parameters of model building unit B3 of model B can be selected as the parameters of model building unit M3. If the difference between D_A2 and D_B2 is less than or equal to the second threshold, the parameters of model building unit A2 and model building unit B2 are combined as the parameters of model building unit M2.

[0065] Thus, a merged model including model building units M1, M2, and M3 can be obtained.

[0066] Figure 5 It shows Figure 4 In a more specific case, using the same base model, full-parameter fine-tuning was employed for the first domain, changing the parameters of all model building units (all layers); for the second domain, LoRA fine-tuning was used, typically changing the parameters of only a subset of layers (intermediate model building units) in the self-attention module. In other words, knowledge from the first domain is distributed across all model building units (gray areas) of the fully-parameter-fine-tuned model, while knowledge from the second domain is stored only in a subset of model building units (gray areas) of the LoRA model. In this situation, the individual model building units of the two models are asymmetrical, and it is clearly unreasonable to perform similar merging of all layers of the model as in traditional methods. Therefore, to preserve knowledge from both domains, this application processes each layer differently based on their differences.

[0067] The method of differentially merging different model building blocks of two models in this application can retain the advantages of each model to the greatest extent. For example... Figure 5As shown, for the fully parameterized fine-tuning model, all model building units contain knowledge from the first domain, so the parameters of all model building units should be reflected in the merged model. However, for the LoRA fine-tuning model, only some model building units (the gray parts) contain knowledge from the second domain, so only the parameters of the gray parts need to be passed to the merged model. Therefore, as... Figure 5 As shown, the gray parts of the second domain model and the corresponding gray parts of the first domain model are merged. For the remaining model building units of the final model, the parameters of the fully parameterized fine-tuned model are directly copied. In other words, for each model building unit, if the model building unit contains knowledge from only one domain, the parameters of the model building unit containing domain knowledge are directly copied to the final model. If the model building unit contains knowledge from two domains, the parameters of the model building unit from the two domain models are merged and then assigned to the corresponding model building unit in the final model.

[0068] Figure 6 This illustrates a scenario where one model is a base model and the other is a trained or fine-tuned model. The merging unit can merge the two models based on the parameter differences between the trained or fine-tuned model and the base model. Specifically, the merging unit can copy the parameters of the unchanged model building units in the base model as the parameters of the corresponding model building units in the merged model, and use the weighted sum of the parameters of the modified model building units in the base model and the parameters of the corresponding model building units in the other model as the parameters of the corresponding model building units in the merged model. For example, the weight determination unit can set the weights of both corresponding model building units to 0.5, or it can assign higher weights to the model building units corresponding to the functions involved in the application requirements, depending on the actual application needs.

[0069] By fusing with the base model, the problem of forgetting in the merged model can be effectively alleviated.

[0070] For example, such as Figure 6 As shown, since the LoRA fine-tuning model only changes the parameters of the middle model building unit (gray area), only the parameters of the middle model building unit are fused, while the parameters of the other two model building units remain unchanged and do not need to be fused. This can effectively alleviate the forgetting problem caused by LoRA fine-tuning.

[0071] The layer-differentiated model merging method proposed in this application can combine multiple models into a single, powerful model, enabling this model to possess knowledge from multiple domains. This technique is an effective supplement to existing model merging methods. It takes into account that different models contain different domain knowledge at different layers, and therefore treats each layer differently during the merging process.

[0072] The following is combined with Figure 7 To describe a model merging method according to embodiments of the present disclosure.

[0073] like Figure 7 As shown, the model merging method according to an embodiment of the present disclosure begins at step S710. In step S710, the differences between each of the plurality of models and the base model are compared, wherein the plurality of models are different models obtained based on the base model.

[0074] Next, in step S720, multiple models are merged based on the comparison results to obtain a merged model that can realize the functions of multiple models.

[0075] After that, the process ended.

[0076] Various specific implementations of the above steps of the model merging method according to embodiments of this disclosure have been described in detail above and will not be repeated here.

[0077] This application proposes a novel model merging method that selectively merges models based on the differences in knowledge contained in the model layers, thereby preserving the domain knowledge of the different models before merging to the maximum extent possible in the merged model.

[0078] The following describes experimental verification of the effectiveness of the layer differentiation model merging method according to this disclosure.

[0079] (1) The method of the present invention achieves the best performance when merging the fully parameter fine-tuning model and the LoRA model.

[0080] The base model used in the experiment was Mistral-7b-Instruct-v0.1. The inventors merged a fully parameterized fine-tuned model in the field of mathematics and a LoRA model in the field of medicine. The numbers in the table represent the test results of the corresponding models under a specific test set. For example, the numbers can represent the accuracy of the model's answers. As an example, the test set can consist of 1000 question-answer pairs. The questions are input into the large model, the large model outputs the predicted answers, and the predicted answers are compared with the actual answers to calculate the proportion of correct answers to the total number of test sets. The experimental results are shown in Table 1. The layer differentiation model merging method proposed in this application has achieved the best average performance (maximum average value), and can retain the knowledge of the two fields to the greatest extent, which is better than other methods such as those described in references [2] and [3].

[0081] Table 1

[0082]

[0083] (2) The layer differentiation model merging of the present invention can improve the performance of various fusion methods.

[0084] The inventors of this application have further verified that the layer differentiation model merging method of this application is not limited to a specific fusion technique, and is effective in both linear interpolation fusion and spherical linear interpolation fusion, outperforming their respective traditional merging methods, as shown in Table 2.

[0085] Table 2

[0086]

[0087] (3) According to the layer differentiation model merging in this application, the LoRA model can help prevent forgetting.

[0088] In their experiments, the inventors discovered that layer-specific model merging helps the LoRA model prevent forgetting. As shown in Table 3, while the LoRA model improves performance in specific domains (e.g., the medical field here), its generalizability declines drastically, exhibiting catastrophic forgetting. By fusing the LoRA model and the base model using the differential model merging method of this invention, the resulting merged model retains performance in the general domain while improving performance in specific domains, resulting in a significant improvement in average performance.

[0089] Table 3

[0090]

[0091] As can be seen, the novel layer-differentiated model merging method proposed in this invention can combine several large language models into a single, powerful large language model, enabling this model to possess knowledge from multiple domains. This technique is an effective supplement to existing model merging methods. It takes into account that different large language models contain different domain knowledge at different layers, and therefore treats each layer differently during the merging process. Experiments have demonstrated that the method of this application is superior to current methods. Since there are many customized needs for domain models in the real world, the method according to this application will contribute to the development of domain models.

[0092] Obviously, the various operational processes of the model merging method according to this disclosure can be implemented as computer executable programs stored in various machine-readable storage media.

[0093] Furthermore, the objective of this disclosure can also be achieved by providing a storage medium storing the aforementioned executable program code directly or indirectly to a system or device, and having a computer or central processing unit (CPU) in the system or device read and execute the aforementioned program code. In this case, as long as the system or device has the function of executing a program, the implementation of this disclosure is not limited to a program, and the program can be in any form, such as an object program, a program executed by an interpreter, or a script program provided to an operating system.

[0094] The aforementioned machine-readable storage media include, but are not limited to: various memories and storage units, semiconductor devices, disk units such as optical, magnetic and magneto-optical disks, and other media suitable for storing information.

[0095] Alternatively, the technical solution of this disclosure can also be implemented by connecting to a corresponding website on the Internet, downloading and installing the computer program code according to this disclosure onto the computer, and then executing the program.

[0096] Figure 8 This is a block diagram of an exemplary structure of a general-purpose personal computer in which the model merging apparatus and method according to embodiments of the present disclosure can be implemented.

[0097] like Figure 8 As shown, CPU 801 executes various processes based on programs stored in read-only memory (ROM) 802 or programs loaded into random access memory (RAM) 803 from storage device 808. RAM 803 also stores data required as needed when CPU 801 executes various processes, etc. CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output interface 805 is also connected to bus 804.

[0098] The following components are connected to input / output interface 805: input device 806 (including keyboard, mouse, etc.), output device 807 (including display, such as cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.), storage device 808 (including hard disk, etc.), and communication device 809 (including network interface card, such as LAN card, modem, etc.). Communication device 809 performs communication processing via a network such as the Internet. Drive 810 may also be connected to input / output interface 805 as needed. Removable media 811, such as disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 810 as needed, so that computer programs read from them can be installed into storage device 808 as needed.

[0099] When the above series of processes are implemented through software, the program constituting the software is installed from a network such as the Internet or a storage medium such as removable media 811.

[0100] Those skilled in the art will understand that such storage media are not limited to Figure 8The illustration shows a removable medium 811 containing a program, distributed separately from the device to provide the program to the user. Examples of removable media 811 include magnetic disks (including floppy disks (registered trademark)), optical disks (including optical disc read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including mini-disk (MD) (registered trademark)), and semiconductor memory. Alternatively, the storage medium may be ROM 802, a hard disk included in storage device 808, etc., containing programs and distributed to the user along with the device containing them.

[0101] In the systems and methods of this disclosure, it is apparent that the components or steps can be decomposed and / or recombined. Such decomposition and / or recombination should be considered equivalent to those disclosed. Furthermore, the steps performing the above series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order. Some steps can be performed in parallel or independently of each other.

[0102] While embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, it should be understood that the embodiments described above are merely illustrative and do not constitute a limitation thereof. Those skilled in the art can make various modifications and alterations to the above embodiments without departing from the spirit and scope of the present disclosure. Therefore, the scope of the present disclosure is defined only by the appended claims and their equivalents.

[0103] The various techniques described in this specification can be performed independently of each other unless there is a contradiction. Of course, any of the various techniques can be performed in combination. In one example, some or all of the techniques described in another embodiment can be combined to perform some or all of the techniques described in any embodiment. Additionally, any part or all of the techniques described above can be combined with another technique not described above.

[0104] Regarding the implementation methods including the above embodiments, the following notes are also disclosed:

[0105] Appendix 1. A model merging device, comprising:

[0106] The comparison section compares the differences between each of a plurality of models and the base model, wherein the plurality of models are models obtained based on the base model; and

[0107] The merging unit merges the multiple models based on the comparison results of the comparison unit to obtain a merged model that can realize the functions of the multiple models.

[0108] Appendix 2. According to the model merging apparatus described in Appendix 1, the base model includes a first set of model building units, each of the plurality of models includes a second set of model building units corresponding to the first set of model building units, and the merging unit obtains the merged model by merging the corresponding model building units in the plurality of second sets of model building units of the plurality of models.

[0109] The model merging device further includes a change determination unit, which determines the model building units in the first group of model building units that have not been changed for all the multiple models based on the parameter differences between the corresponding model building units in the first group of model building units and each of the multiple second group of model building units calculated by the comparison unit.

[0110] Appendix 3. According to the model merging apparatus described in Appendix 2, when merging the plurality of models, the merging unit copies the parameters of the unchanged model building units as the parameters of the corresponding model building units in the merged model, and

[0111] The model merging device further includes: a weight determination unit, which targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed.

[0112] The weights assigned to each model building unit when merging the corresponding model building units of the multiple models are determined based on the parameter differences, or

[0113] The corresponding model building units of the plurality of models are assigned the same weight, wherein the sum of the weights of the corresponding model building units of the plurality of models is 1, or

[0114] Based on the actual application needs, assign higher weights to the model building units corresponding to the functions involved in the requirements.

[0115] Appendix 4. The model merging apparatus according to Appendix 3, wherein the weight determination unit targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed:

[0116] The weights of model building units whose parameter differences are below a first threshold are set to 0, and the weights of other model building units are determined based on the magnitude of the parameter differences.

[0117] Note 5. The model merging apparatus according to Note 4, wherein the first threshold is the average value of the parameter differences of the corresponding model building units of the plurality of models.

[0118] Appendix 6. The model merging apparatus according to Appendix 3, wherein the weight determination unit targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed:

[0119] The weight of the model building unit with the largest parameter difference among the corresponding model building units of the multiple models is set to 1, and the weights of the other corresponding model building units are set to 0.

[0120] Appendix 7. The model merging apparatus according to Appendix 3, wherein the plurality of models includes a first model and a second model, and

[0121] The comparison section calculates a first set of parameter differences representing the difference between the first model and the base model, and a second set of parameter differences representing the difference between the second model and the base model.

[0122] If the difference in parameter differences between two corresponding model building units in the first parameter difference set and the second parameter difference set is greater than a second threshold, the weight determination unit sets the weight of the model building unit with the larger parameter difference to 1 and sets the weight of the other model building unit to 0; otherwise,

[0123] The weight determination unit sets the weights of the two corresponding model building units based on the magnitude of the parameter differences, or

[0124] The weight determination unit sets the weights of both corresponding model building units to 0.5, or

[0125] The weight determination unit assigns higher weights to the model building units corresponding to the functions involved in the requirements, based on the actual application needs.

[0126] Appendix 8. The model merging apparatus according to Appendix 1, wherein the plurality of models includes a third model and the base model, and the merging unit merges the third model and the base model based on the parameter differences between the third model and the base model.

[0127] Appendix 9. According to the model merging apparatus described in Appendix 8, the merging unit copies the parameters of the unchanged model building units in the base model as the parameters of the corresponding model building units in the merged model, and uses the weighted sum of the parameters of the changed model building units in the base model and the parameters of the corresponding model building units in the third model as the parameters of the corresponding model building units in the merged model.

[0128] The weight determination unit sets the weights of the two corresponding model building units to 0.5, or

[0129] The weight determination unit assigns higher weights to the model building units corresponding to the functions involved in the requirements, based on the actual application needs.

[0130] Appendix 10. A model merging method, comprising:

[0131] Compare the differences between each of the multiple models and the base model, wherein the multiple models are different models derived from the base model; and

[0132] Based on the comparison results, the multiple models are merged to obtain a merged model that can achieve the functions of the multiple models.

[0133] Appendix 11. According to the model merging method described in Appendix 10, the base model includes a first set of model building units, each of the plurality of models includes a second set of model building units corresponding to the first set of model building units, and the merged model is obtained by merging the corresponding model building units in the plurality of second sets of model building units of the plurality of models.

[0134] The method further includes: determining, based on the parameter differences between the corresponding model building units in the first group of model building units and each of the multiple second group of model building units, the model building units in the first group of model building units that have not been changed for all the multiple models.

[0135] Appendix 12. According to the model merging method described in Appendix 11, when merging the multiple models, the parameters of the unchanged model building units are copied as the parameters of the corresponding model building units in the merged model, and

[0136] The method further includes: targeting the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed.

[0137] The weights assigned to each model building unit when merging the corresponding model building units of the multiple models are determined based on the parameter differences, or

[0138] The corresponding model building units of the plurality of models are assigned the same weight, wherein the sum of the weights of the corresponding model building units of the plurality of models is 1, or

[0139] Based on the actual application needs, assign higher weights to the model building units corresponding to the functions involved in the requirements.

[0140] Appendix 13. According to the model merging method described in Appendix 12, wherein, for the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed:

[0141] The weights of model building units whose parameter differences are below a first threshold are set to 0, and the weights of other model building units are determined based on the magnitude of the parameter differences.

[0142] Note 14. According to the model merging method described in Note 13, the first threshold is the average of the parameter differences of the corresponding model building units of the plurality of models.

[0143] Note 15. According to the model merging method described in Note 12, wherein, for the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed:

[0144] The weight of the model building unit with the largest parameter difference among the corresponding model building units of the multiple models is set to 1, and the weights of the other corresponding model building units are set to 0.

[0145] Note 16. The model merging method according to Note 12, wherein the plurality of models includes a first model and a second model, and

[0146] The method includes: calculating a first set of parameter differences representing the difference between the first model and the base model, and a second set of parameter differences representing the difference between the second model and the base model, respectively.

[0147] If the difference in parameter differences between two corresponding model building units in the first and second parameter difference sets is greater than a second threshold, the weight of the model building unit with the larger parameter difference is set to 1, and the weight of the other model building unit is set to 0; otherwise,

[0148] The weights of the two corresponding model building units are set according to the magnitude of the parameter differences, or

[0149] Set the weights of both corresponding model building units to 0.5, or

[0150] Based on the actual application needs, assign higher weights to the model building units corresponding to the functions involved in the requirements.

[0151] Note 17. The model merging method according to Note 10, wherein the plurality of models includes a third model and the base model, and the third model and the base model are merged based on the parameter differences between the third model and the base model.

[0152] Appendix 18. According to the model merging method described in Appendix 17, the parameters of the unchanged model building units in the base model are copied as the parameters of the corresponding model building units in the merged model, and the result of a weighted sum of the parameters of the changed model building units in the base model and the parameters of the corresponding model building units in the third model is used as the parameters of the corresponding model building units in the merged model.

[0153] In this case, the weights of the two corresponding model building units are both set to 0.5, or

[0154] Based on the actual application needs, assign higher weights to the model building units corresponding to the functions involved in the requirements.

[0155] Note 19. The model merging method described in Note 17, wherein the third model is a LoRA fine-tuning model.

[0156] Note 20. A machine-readable storage medium carrying a program product including machine-readable instruction code stored thereon, wherein, when read and executed by a computer, the instruction code enables the computer to perform the model merging method according to any one of Notes 10-19.

Claims

1. A model merging device, comprising: The comparison section compares the differences between each of a plurality of models and the base model, wherein the plurality of models are models obtained based on the base model; and The merging unit merges the multiple models based on the comparison results of the comparison unit to obtain a merged model that can realize the functions of the multiple models.

2. The model merging apparatus according to claim 1, wherein, The base model includes a first set of model building units, and each of the plurality of models includes a second set of model building units corresponding to the first set of model building units. The merging unit obtains the merged model by merging the corresponding model building units in the plurality of second sets of model building units of the plurality of models. The model merging device further includes a change determination unit, which determines the model building units in the first group of model building units that have not been changed for all the multiple models based on the parameter differences between the corresponding model building units in the first group of model building units and each of the multiple second group of model building units calculated by the comparison unit.

3. The model merging apparatus according to claim 2, wherein, When merging the multiple models, the merging unit copies the parameters of the unchanged model building units as the parameters of the corresponding model building units in the merged model, and The model merging device further includes: a weight determination unit, which targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed. The weights assigned to each model building unit when merging the corresponding model building units of the multiple models are determined based on the parameter differences, or The corresponding model building units of the plurality of models are assigned the same weight, wherein the sum of the weights of the corresponding model building units of the plurality of models is 1, or Based on the actual application needs, assign higher weights to the model building units corresponding to the functions involved in the requirements.

4. The model merging apparatus according to claim 3, wherein, The weight determination unit targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed: The weights of model building units whose parameter differences are below a first threshold are set to 0, and the weights of other model building units are determined based on the magnitude of the parameter differences.

5. The model merging apparatus according to claim 4, wherein, The first threshold is the average of the parameter differences of the corresponding model building units of the plurality of models.

6. The model merging apparatus according to claim 3, wherein, The weight determination unit targets the model building units in the second group of model building units that correspond to the model building units in the first group of model building units that have been determined to have changed: The weight of the model building unit with the largest parameter difference among the corresponding model building units of the multiple models is set to 1, and the weights of the other corresponding model building units are set to 0.

7. The model merging apparatus according to claim 3, wherein, The multiple models include a first model and a second model, and The comparison section calculates a first set of parameter differences representing the difference between the first model and the base model, and a second set of parameter differences representing the difference between the second model and the base model. If the difference in parameter differences between two corresponding model building units in the first parameter difference set and the second parameter difference set is greater than a second threshold, the weight determination unit sets the weight of the model building unit with the larger parameter difference to 1 and sets the weight of the other model building unit to 0; otherwise, The weight determination unit sets the weights of the two corresponding model building units based on the magnitude of the parameter differences, or The weight determination unit sets the weights of both corresponding model building units to 0.5, or The weight determination unit assigns higher weights to the model building units corresponding to the functions involved in the requirements, based on the actual application needs.

8. The model merging apparatus according to claim 1, wherein, The plurality of models includes a third model and the base model, and the merging unit merges the third model and the base model based on the parameter differences between the third model and the base model. Specifically, the merging unit copies the parameters of the unchanged model building units in the base model as the parameters of the corresponding model building units in the merged model, and uses the weighted sum of the parameters of the changed model building units in the base model and the parameters of the corresponding model building units in the third model as the parameters of the corresponding model building units in the merged model. The weight determination unit sets the weights of the two corresponding model building units to 0.5, or The weight determination unit assigns higher weights to the model building units corresponding to the functions involved in the requirements, based on the actual application needs.

9. A model merging method, comprising: Compare the differences between each of the multiple models and the base model, wherein the multiple models are different models derived from the base model; and Based on the comparison results, the multiple models are merged to obtain a merged model that can achieve the functions of the multiple models.

10. A machine-readable storage medium carrying a program product including machine-readable instruction code stored thereon, wherein, When the instruction code is read and executed by a computer, it enables the computer to perform the method according to claim 9.

Images