Pre-training model fine tuning method and related equipment

Abstract

The invention provides a pre-training model fine tuning method and related equipment. The pre-training model fine tuning method is applied to first computing equipment corresponding to a model provider. The plurality of data providers hold private data corresponding to different reasoning tasks. The method comprises the following steps: sending a pre-training model to be finely tuned to a plurality of second computing devices corresponding to a plurality of data providers, so that the plurality of second computing devices carry out fine tuning training on the pre-training model based on held private data; constructing an MoE gating network corresponding to the pre-training model, and training the MoE gating network based on a plurality of fine-tuned pre-training models obtained by a plurality of second computing devices; and the MoE gating network is used for determining the weight of the plurality of pre-training models after fine tuning according to the correlation between a to-be-executed reasoning task and the plurality of pre-training models after fine tuning, and performing weighted fusion on a plurality of reasoning results output by executing the reasoning task by the plurality of pre-training models after fine tuning according to the determined weight.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of pre-trained model technology, and in particular, to a pre-trained model fine-tuning method and related equipment. Background Art

[0002] The pre-trained model is a language model constructed by a deep neural network with more than tens of billions of parameters. It uses self-supervised learning methods to master many language phenomena through pre-training on large and diverse public data sets.

[0003] In order to apply the pre-trained model to various reasoning tasks related to different application scenarios, it is necessary to further fine-tune the pre-trained model through data sets related to different application scenarios. However, the data sets used in the fine-tuning stage often contain private data held by each participant involved in the fine-tuning. If each participant directly shares the data sets used in the fine-tuning stage to centralize all data to fine-tune the pre-trained model, it is very easy to cause the leakage of private data and infringe on the interests of users. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide a pre-training model fine-tuning method and related equipment.

[0005] In a first aspect, this specification provides a pre-trained model fine-tuning method, which is applied to a first computing device corresponding to a model provider, wherein the model provider holds a pre-trained model to be fine-tuned; a plurality of data providers connected to the model provider hold private data corresponding to different types of reasoning tasks; the method comprises:

[0006] Sending the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively perform fine-tuning training on the pre-trained model based on the private data held by them, to obtain a fine-tuned pre-trained model;

[0007] Constructing a MoE gating network corresponding to the pre-training model, and training the MoE gating network based on a plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein,

[0008] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0009] In a second aspect, the present specification provides a pre-trained model fine-tuning method, which is applied to any second computing device among a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of reasoning tasks; the method comprises:

[0010] Obtaining a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider;

[0011] The pre-trained model is fine-tuned based on the private data held to obtain a fine-tuned pre-trained model, so that the first computing device trains the MoE gating network corresponding to the pre-trained model constructed by the first computing device based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein,

[0012] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0013] In a third aspect, the present specification provides a pre-trained model fine-tuning device, which is applied to a first computing device corresponding to a model provider, wherein the model provider holds a pre-trained model to be fine-tuned; a plurality of data providers connected to the model provider hold private data corresponding to different types of reasoning tasks; the device comprises:

[0014] A fine-tuning unit, configured to send the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively perform fine-tuning training on the pre-trained model based on the private data held by them, to obtain a fine-tuned pre-trained model;

[0015] A gating network training unit, used to construct a MoE gating network corresponding to the pre-training model, and train the MoE gating network based on a plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein,

[0016] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0017] In a fourth aspect, the present specification provides a pre-trained model fine-tuning device, applied to any second computing device among a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of reasoning tasks; the device comprises:

[0018] An acquisition unit, configured to acquire a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider;

[0019] A gated network training unit, configured to fine-tune the pre-trained model based on the private data held, to obtain a fine-tuned pre-trained model, so that the first computing device trains the MoE gated network corresponding to the pre-trained model constructed by the first computing device based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein,

[0020] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0021] Accordingly, the present specification also provides a computing device, comprising: a memory and a processor; the memory stores computer programs / instructions that can be executed by the processor; when the processor executes the computer program / instructions, it executes the pre-trained model fine-tuning method described in the first aspect or the second aspect above.

[0022] Accordingly, the present specification also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the pre-trained model fine-tuning method as described in the first aspect or the second aspect above is executed.

[0023] Accordingly, the present specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the pre-trained model fine-tuning method described in the first aspect or the second aspect above is executed.

[0024] In summary, each data provider in this application can use the private data it holds to fine-tune the pre-trained model to be fine-tuned locally. Then, the model provider can build a MoE gating network corresponding to the pre-trained model, and the pre-trained models fine-tuned locally by each data provider can be regarded as experts in the MoE architecture. Furthermore, the pre-trained models fine-tuned locally by each data provider can be used to train the MoE gating network. The trained MoE gating network can dynamically adjust the weights of the pre-trained models fine-tuned by each data provider for different reasoning tasks, and based on the weights, perform weighted fusion on the reasoning results output by the pre-trained models fine-tuned by each party to perform reasoning tasks, and obtain the final global model output. In this way, while meeting privacy protection, the global model can perform excellent performance in reasoning tasks in various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic diagram of a system architecture provided by an exemplary embodiment;

[0026] Figure 2 It is a flowchart of a pre-training model fine-tuning method provided by an exemplary embodiment;

[0027] Figure 3 It is a structural schematic diagram of a pre-training model fine-tuning device provided by an exemplary embodiment;

[0028] Figure 4 is a structural schematic diagram of another pre-training model fine-tuning device provided by an exemplary embodiment;

[0029] Figure 5 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0031] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0032] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0033] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] First, some terms in this specification are explained to facilitate understanding by those skilled in the art.

[0035] (1) Large Language Models (LLMs), also known as pre-trained models, are language models built using deep neural networks with more than tens of billions of parameters. They use self-supervised learning methods to master many language phenomena through pre-training on large and diverse public datasets.

[0036] (2) Federated Learning (FL) is a distributed machine learning framework that allows various participants (usually referred to as clients) to collaboratively train a shared global model without centralizing the private data held by each client in one place. In other words, there is no need for each client to share their own private data, thereby allowing learning and improving the model's capabilities from multiple data sources while protecting data privacy and security.

[0037] In federated learning, each client can use its own private data to train the global model locally and generate updated model parameters or gradient information, and then send these updated model parameters or gradient information to a central server. Correspondingly, the central server can aggregate the updated model parameters or gradient information sent by multiple clients and update the global model based on the aggregated data. Furthermore, after the global model is updated, the new model parameters can be distributed to each client so that each client can use its own private data to train the updated global model, and so on, and the above process is repeated until the predetermined training goal is achieved.

[0038] (3) Mixture of Experts (MoE) is a machine learning architecture that consists of multiple sub-models (called experts) and a gating network. Each expert can be used to process a specific part or pattern in the data. The gating network is used to decide which experts should be used to process given input data and how the outputs of these experts should be combined.

[0039] Exemplarily, the operation process of MoE is generally as follows:

[0040] 1. Transmit input data to all experts.

[0041] 2. The gating network evaluates the relevance or utility of each expert to the current input data and assigns a corresponding weight to each expert based on this.

[0042] 3. Each expert processes the input data independently and outputs the processing results (such as prediction results or features, etc.).

[0043] 4. According to the weights assigned by the gating network, the processing results of all experts’ outputs are weighted fused (such as simple weighted summation, etc.) to obtain the final model output.

[0044] It should be understood that although the pre-trained model has demonstrated excellent capabilities in understanding and generating natural language, in order to apply the pre-trained model to various reasoning tasks related to different application scenarios, it is necessary to further fine-tune the pre-trained model through data sets related to different application scenarios. However, the data sets used in the fine-tuning stage often contain private data held by each participant involved in the fine-tuning. If each participant directly shares the data sets used in the fine-tuning stage to centralize all data to fine-tune the pre-trained model, it is very easy to cause the leakage of private data and infringe on the interests of users.

[0045] As mentioned above, federated learning, as an emerging distributed machine learning paradigm, allows multiple participants to collaboratively train a shared global model without sharing data. Based on this, federated learning can be used to fine-tune the pre-trained model with multiple participants. Specifically, each participant can fine-tune the pre-trained model locally using the private data it holds, and then upload the fine-tuning training results (such as updated model parameters or gradient information, etc.) to the central server for aggregation. The central server then updates the pre-trained model based on the aggregated data, and so on, thereby obtaining a pre-trained model that is fine-tuned based on the private data of multiple parties. This not only protects the data privacy of each participant, but also can utilize a wide range of data from different sources and different application scenarios to enhance the generalization ability of the pre-trained model.

[0046] Although federated learning provides an effective privacy protection mechanism, due to the heterogeneity of the data distribution of private data held by each participant, the global model obtained by directly aggregating the fine-tuning training results of all parties may be difficult to meet the personalized needs of all participants, resulting in the final global model failing to perform well in reasoning tasks in different application scenarios.

[0047] Based on this, this specification provides a technical solution that uses the MoE architecture to determine the weights of the pre-trained models (which can be regarded as experts) fine-tuned by each participant for different reasoning tasks through the MoE gating network, and performs weighted fusion on the reasoning results output by each party, so that while meeting privacy protection, the global model can perform well in a variety of application scenarios.

[0048] In implementation, the present application can be applied to a first computing device corresponding to a model provider, which holds a pre-trained model to be fine-tuned; and multiple data providers connected to the model provider hold private data corresponding to different types of reasoning tasks. In an illustrated embodiment, the first computing device may first send the pre-trained model to be fine-tuned to multiple second computing devices corresponding to the multiple data providers. Further, the multiple second computing devices may respectively perform fine-tuning training on the pre-trained model based on the private data held, and each obtain a fine-tuned pre-trained model. Further, the first computing device may construct a MoE gating network corresponding to the pre-trained model, and train the MoE gating network based on multiple fine-tuned pre-trained models obtained by multiple second computing devices. Among them, the trained MoE gating network can be used to: determine the weights of multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be performed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models to perform the reasoning task according to the determined weights.

[0049] In the above technical scheme, each data provider in this application can use the private data it holds to fine-tune the pre-trained model to be fine-tuned locally. Then, the model provider can build a MoE gating network corresponding to the pre-trained model, and the pre-trained models fine-tuned locally by each data provider can be regarded as experts in the MoE architecture. Furthermore, the pre-trained models fine-tuned locally by each data provider can be used to train the MoE gating network. The trained MoE gating network can dynamically adjust the weights of the pre-trained models fine-tuned by each data provider for different reasoning tasks, and based on the weights, perform weighted fusion on the reasoning results output by the pre-trained models fine-tuned by each party to perform reasoning tasks, and obtain the final global model output. In this way, while meeting privacy protection, the global model can perform excellent performance in reasoning tasks in various application scenarios.

[0050] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a system architecture provided by an exemplary embodiment. Figure 1 As shown, the system may include a first computing device 100 corresponding to a model provider, and multiple computing devices corresponding to multiple data providers, such as a second computing device 200a, a second computing device 200b, and a second computing device 200c. In an illustrated embodiment, the first computing device 100, and the second computing devices 200a, 200b, and 200c may establish a communication connection in any possible manner, and this specification does not specifically limit this.

[0051] Among them, the model provider holds a pre-trained model that has been publicly trained, and hopes that the pre-trained model can successfully perform different reasoning tasks related to various application scenarios through further fine-tuning training.

[0052] Among them, multiple data providers hold private data corresponding to different types of reasoning tasks. Exemplarily, the above-mentioned different types of reasoning tasks may include, for example, asset analysis, traffic planning, user interface development, image generation, chart analysis, and question-answering tasks, and this specification does not make specific limitations on this. Exemplarily, the above-mentioned private data may include, for example, math data corresponding to asset analysis tasks, code data corresponding to user interface development tasks, and image data corresponding to image generation tasks, etc., and this specification does not make specific limitations on this.

[0053] In an illustrated embodiment, the first computing device 100 corresponding to the model provider may send the pre-trained model to be fine-tuned held by it to the second computing device 200a, the second computing device 200b, the second computing device 200c, etc. corresponding to multiple data providers. In an illustrated embodiment, the first computing device 100 may broadcast the pre-trained model to be fine-tuned to the second computing device 200a, the second computing device 200b, the second computing device 200c, etc. by broadcasting.

[0054] Accordingly, the second computing device 200a, the second computing device 200b, and the second computing device 200c may receive the pre-trained model to be fine-tuned sent by the first computing device 100. Furthermore, the second computing device 200a, the second computing device 200b, and the second computing device 200c may respectively perform fine-tuning training on the pre-trained model locally based on the private data held by each, to obtain a fine-tuned pre-trained model.

[0055] It should be noted that this specification does not specifically limit the specific implementation method of fine-tuning training. In one illustrated embodiment, the second computing device 200a, the second computing device 200b, and the second computing device 200c can freeze the parameters of the pre-trained model, and use the parameter-efficient fine-tuning (PEFT) method or the full-scale fine-tuning (Fine-Tuning) method to fine-tune the pre-trained model. Exemplarily, the PEFT method may include LoRA (Low-Rank Adaptation of LLMs), prefix tuning (Prefix Tuning) and prompt word tuning (Prompt Tuning), etc., and this specification does not specifically limit this.

[0056] In addition, in order to ensure that the fine-tuning parameters obtained after fine-tuning by each party can act on the same position in the pre-trained model, such as in the same dimension of the same model parameter, the second computing device 200a, the second computing device 200b and the second computing device 200c need to adopt the same fine-tuning method, such as all using LoRA, or all using Prefix Tuning, etc. This specification does not make specific limitations on this.

[0057] Furthermore, the first computing device 100 may construct a MoE gating network corresponding to the pre-trained model. The network structure of the gating network may include a fully connected layer (linear) or a self-attention mechanism (Self-Attention) layer, etc., which is not specifically limited in this specification.

[0058] Further, the first computing device 100 can train the MoE gating network based on the fine-tuned pre-trained models obtained by the second computing device 200a, the second computing device 200b, and the second computing device 200c. In an embodiment shown, a federated learning method can be used to train the MoE gating network by combining multiple fine-tuned pre-trained models. For details, see the following Figure 2 The description of the corresponding embodiments will not be elaborated in detail here.

[0059] Among them, the trained MoE gating network can be used to: determine the weights of multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and according to the determined weights, perform weighted fusion on the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task.

[0060] In an illustrated embodiment, the first computing device 100 is the central server in federated learning, and the second computing device 200a, the second computing device 200b, and the second computing device 200c are multiple participants (or clients) in federated learning. In an illustrated embodiment, the first computing device 100, and the second computing device 200a, the second computing device 200b, and the second computing device 200c can be a smart wearable device, a smart phone, a tablet computer, a laptop computer, a desktop computer, a server, or a server cluster or a cloud computing center composed of multiple servers, etc., which is not specifically limited in this specification.

[0061] See also Figure 2 , Figure 2 is a flowchart of a method for fine-tuning a pre-trained model provided by an exemplary embodiment. The method can be applied to Figure 1 The system architecture shown can be applied to Figure 1 The first computing device corresponding to the model provider is shown. Figure 2 As shown, the method may specifically include the following steps S201-S202.

[0062] Step S201, sending the pre-trained model to be fine-tuned to multiple second computing devices corresponding to multiple data providers, so that the multiple second computing devices respectively fine-tune the pre-trained model based on the private data they hold, to obtain the fine-tuned pre-trained model.

[0063] First of all, it should be noted that the complete technical solution provided in this application may include the fine-tuning stage of the pre-trained model, the training stage of the MoE gating network, and the reasoning stage in practical applications. These three stages will be explained in turn below.

[0064] (1) Fine-tuning of the pre-trained model

[0065] As mentioned above, the model provider holds the pre-trained model to be fine-tuned, and multiple data providers hold private data corresponding to different types of inference tasks. For details, please refer to the above Figure 1 The description of the corresponding embodiments will not be repeated here.

[0066] In an illustrated embodiment, a first computing device corresponding to a model provider may first send a pre-trained model to be fine-tuned held by the first computing device to multiple second computing devices corresponding to multiple data providers.

[0067] Correspondingly, multiple second computing devices can receive the pre-trained model and fine-tune the pre-trained model locally based on the private data they hold.

[0068] In one illustrated embodiment, multiple second computing devices can freeze the parameters of the pre-trained model before fine-tuning training. Based on this, after the multiple second computing devices perform fine-tuning training locally, they can respectively obtain some newly added parameters (i.e., fine-tuning parameters), but the parameters of the pre-trained model will not be updated.

[0069] In one illustrated embodiment, the pre-trained model can be regarded as a backbone large model, whose model parameters can be used to provide basic reasoning capabilities, and the fine-tuning parameters obtained by multiple second computing devices can be used to provide personalized reasoning capabilities corresponding to their private data.

[0070] In an illustrated embodiment, the model parameters of the pre-trained model and the fine-tuning parameters obtained by each second computing device can constitute a fine-tuned pre-trained model, that is, in the fine-tuning stage of the pre-trained model, multiple second computing devices obtain a total of multiple fine-tuned pre-trained models. In an illustrated embodiment, the fine-tuning parameters obtained by each second computing device can be used as an expert in the MoE, or the fine-tuned pre-trained model composed of the model parameters of the pre-trained model and the fine-tuning parameters obtained by each second computing device can be used as an expert, and so on. This specification does not make specific limitations on this.

[0071] It should be noted that this specification does not specifically limit the specific implementation method of the above-mentioned fine-tuning training. In an embodiment shown in FIG. 1 , multiple second computing devices can use the PEFT method or the full-scale fine-tuning method to fine-tune the pre-trained model. For details, please refer to the above-mentioned Figure 1The description of the corresponding embodiment will not be repeated here. It should be understood that if multiple second computing devices all use the full fine-tuning method to fine-tune the pre-trained model, the newly added parameters obtained by each second computing device after the fine-tuning training can be the difference between the parameters of the fine-tuned pre-trained model and the parameters of the original pre-trained model, and this specification does not make specific limitations on this.

[0072] Step S202, constructing a MoE gating network corresponding to the pre-trained model, and training the MoE gating network based on multiple fine-tuned pre-trained models obtained by multiple second computing devices; wherein the trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the inference task to be executed and the multiple fine-tuned pre-trained models, and according to the determined weights, weightedly fuse the multiple inference results output by the multiple fine-tuned pre-trained models when executing the inference task.

[0073] (2) Training phase of the gated network

[0074] Furthermore, after the plurality of second computing devices complete fine-tuning training locally respectively, the first computing device may construct a MoE gating network corresponding to the pre-trained model.

[0075] In an illustrated embodiment, the pre-trained model may include multiple computing layers, and accordingly, corresponding MoE gating networks may be constructed for at least some of the multiple computing layers according to a preset gating strategy.

[0076] In an illustrated embodiment, the at least part of the computing layer may include the last computing layer for outputting the inference result, i.e., the output layer. Accordingly, the MoE gating network corresponding to the output layer may be used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the inference task to be performed and the multiple fine-tuned pre-trained models, and perform weighted fusion on the multiple inference results output by the multiple fine-tuned pre-trained models when performing the inference task according to the determined weights, so as to obtain the final output result of the model.

[0077] In an illustrated embodiment, the at least part of the computing layer may also include several target computing layers in addition to the output layer. Exemplarily, the pre-trained model includes 32 layers, and the target computing layer may be the 3rd layer, the 10th layer or the 20th layer, etc., which is not specifically limited in this specification. Accordingly, the MoE gating network corresponding to the target computing layer can be used to: determine the weights of multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and perform weighted fusion on the calculation results output by the target computing layer in the multiple fine-tuned pre-trained models according to the determined weights to obtain a fused calculation result. Furthermore, the first computing device may send the fused calculation result to multiple second computing devices respectively, and accordingly, the multiple second computing devices may input the fused calculation result to the next computing layer connected to the target computing layer in each fine-tuned pre-trained model for subsequent calculation.

[0078] In summary, this specification does not specifically limit the number of gated networks to be constructed and the corresponding positions. In one embodiment shown, considering that the inference results output by multiple fine-tuned training models need to be weighted and fused in the end, it is necessary to construct a MoE gated network for the output layer of the pre-trained model. Furthermore, the corresponding MoE gated network can be adaptively constructed for the remaining several computing layers according to the actual situation and computing requirements. In addition, it should be noted that the computing layer targeted by the multiple second computing devices during fine-tuning training has nothing to do with the computing layer corresponding to the construction of the MoE gated network. Exemplarily, multiple second computing devices can be fine-tuned for the 4th layer of the pre-trained model respectively, and accordingly, the multiple fine-tuning parameters obtained can be located in the 4th layer. Further, the first computing device can construct a corresponding MoE gated network for the 4th layer of the pre-trained model, or the first computing device can also construct a corresponding MoE gated network for the 5th or 6th layer of the pre-trained model, and so on. This specification does not specifically limit this.

[0079] Furthermore, after the MoE gating network is constructed, the MoE gating network can be trained based on multiple fine-tuned pre-trained models obtained by multiple second computing devices, so that the MoE gating network can dynamically and accurately determine the weights of multiple fine-tuned pre-trained models according to different reasoning tasks.

[0080] In an illustrated embodiment, a first computing device and a plurality of second computing devices may train the MoE gating network in a federated learning manner. Exemplarily, the MoE gating network may be trained in a horizontal federated learning manner. Furthermore, it should be noted that this specification does not specifically limit the parameter aggregation method in horizontal federated learning, and may be any parameter aggregation method such as federated averaging, adaptive averaging, or an enhanced security aggregation method such as differential privacy may also be used.

[0081] Specifically, the first computing device may first obtain a training sample set, which may include a plurality of reasoning tasks of different types, and each reasoning task corresponds to a preset reasoning result, which is equivalent to a sample label.

[0082] Furthermore, the first computing device can send the target reasoning task in the training sample set to multiple second computing devices. Accordingly, the multiple second computing devices can freeze the pre-trained model and the local fine-tuning parameters, which is equivalent to freezing the pre-trained model after local fine-tuning. Then, each first computing device in the multiple second computing devices can input the target reasoning task into the pre-trained model after local fine-tuning, and the fine-tuned pre-trained model executes the target reasoning task and outputs the corresponding reasoning result.

[0083] Furthermore, the first computing device can obtain multiple inference results obtained by the multiple second computing devices, and input the multiple inference results into a pre-built MoE gating network, which performs weighted fusion on the multiple inference results and outputs the weighted fusion inference results.

[0084] Furthermore, the first computing device may adjust and optimize the parameters of the MoE gating network according to the difference between the reasoning result after the weighted fusion and the reasoning result preset for the target reasoning task.

[0085] Furthermore, the first computing device can also send other reasoning tasks in the training sample set to multiple second computing devices to further adjust the parameters of the MoE gating network, and repeatedly perform the above steps until the output of the MoE gating network meets expectations (for example, the difference between the preset reasoning result and the preset value is less than the preset value), thereby completing the federated training of the MoE gating network. Among them, the trained MoE gating network can dynamically and accurately determine the weights of multiple fine-tuned pre-training models corresponding to multiple second computing devices for different reasoning tasks input into the pre-training model.

[0086] (3) Reasoning stage

[0087] After completing the local fine-tuning of the above pre-trained model and the federated training of the MoE gating network, the trained global model can be applied to actual reasoning tasks.

[0088] Next, we will take the construction of the corresponding MoE gating network for the target computing layer and output layer in the pre-trained model as an example to illustrate the reasoning process of the trained global model in practical applications.

[0089] First, in response to a user's request, the first computing device may send an inference task corresponding to the request to multiple second computing devices. Accordingly, the multiple second computing devices may input the inference task into their respective fine-tuned pre-trained models, and the fine-tuned pre-trained models may perform the inference task.

[0090] Specifically, during the calculation process of executing the inference task, the target computing layer contained in each fine-tuned pre-trained model can output the corresponding calculation result, and then each second computing device can send the calculation result output by the target computing layer to the first computing device. Accordingly, the first computing device can obtain multiple calculation results sent by multiple second computing devices, and input the multiple calculation results into the pre-built MoE gating network corresponding to the target computing layer, and the MoE gating network performs weighted fusion on the multiple calculation results according to the determined weights to obtain the fused calculation result. Exemplarily, weighted fusion of multiple calculation results may specifically include weighted summation of the multiple calculation results.

[0091] Furthermore, the first computing device can return the fusion calculation result output by the MoE gating network to multiple second computing devices. Correspondingly, each second computing device can input the received fusion calculation result into the next calculation layer connected to the target calculation layer included in the fine-tuned pre-trained model for subsequent calculation.

[0092] Furthermore, as the calculation proceeds layer by layer, the multiple fine-tuned pre-trained models corresponding to the multiple second computing devices finally output the reasoning results corresponding to the reasoning task. Accordingly, the first computing device can obtain the multiple reasoning results obtained by the multiple second computing devices, and input the multiple reasoning results into the pre-built MoE gating network corresponding to the output layer, and the MoE gating network performs weighted fusion on the multiple reasoning results according to the determined weights to obtain the final output result of the model. Furthermore, the first computing device can return the output result to the user, etc., and this specification does not make specific limitations on this.

[0093] Exemplarily, the above-mentioned reasoning task may be a user interface development task. The MoE gating network may determine, based on the correlation between the user interface development task and a plurality of fine-tuned pre-trained models, that the weight of the fine-tuned pre-trained model (equivalent to programming capability) in the second computing device holding code data may be 0.5, the weight of the fine-tuned pre-trained model (equivalent to image generation capability) in the second computing device holding image data may be 0.4, the weight of the fine-tuned pre-trained model (equivalent to mathematical analysis calculation) in the second computing device holding mathematical data may be 0.1, and so on. This specification does not make any specific limitation on this.

[0094] Exemplarily, the above-mentioned reasoning task can be a graph analysis task, then the MoE gating network can determine, based on the correlation between the graph analysis task and multiple fine-tuned pre-trained models, that the weight of the fine-tuned pre-trained model (equivalent to having programming capability) in the second computing device holding code data can be 0, the weight of the fine-tuned pre-trained model (equivalent to having image generation capability) in the second computing device holding image data can be 0.3, the weight of the fine-tuned pre-trained model (equivalent to having mathematical analysis calculation) in the second computing device holding mathematical data can be 0.7, and so on. This specification does not make specific limitations on this.

[0095] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides a pre-training model fine-tuning device. Figure 3 , Figure 3 is a schematic diagram of a pre-training model fine-tuning device provided by an exemplary embodiment. The device 30 can be applied to Figure 1 The system architecture shown can be applied to Figure 1 The first computing device corresponding to the model provider shown, wherein the model provider holds a pre-trained model to be fine-tuned, and multiple data providers connected to the model provider hold private data corresponding to different types of reasoning tasks. Figure 3 As shown, the device 30 includes:

[0096] A fine-tuning unit 301 is used to send the pre-trained model to be fine-tuned to multiple second computing devices corresponding to the multiple data providers, so that the multiple second computing devices respectively fine-tune the pre-trained model based on the private data they hold, to obtain a fine-tuned pre-trained model;

[0097] The gated network training unit 302 is used to construct a MoE gated network corresponding to the pre-trained model, and train the MoE gated network based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein,

[0098] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0099] In an illustrated embodiment, the device 30 further includes a first weighted fusion unit 303, which is configured to:

[0100] Sending the inference task to be executed to the plurality of second computing devices, so that the plurality of second computing devices input the inference task into the fine-tuned pre-trained model, the fine-tuned pre-trained model executes the inference task, and outputs the corresponding inference result;

[0101] Acquire multiple inference results obtained by the multiple second computing devices, and input the multiple inference results into the MoE gating network, wherein the MoE gating network determines the weights of the multiple fine-tuned pre-training models according to the correlation between the inference task and the multiple fine-tuned pre-training models, and performs weighted fusion on the multiple inference results output by the multiple fine-tuned pre-training models according to the determined weights.

[0102] In an illustrated embodiment, the gated network training unit 302 is specifically configured to:

[0103] Based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices, the MoE gating network is trained in a federated learning manner.

[0104] In an illustrated embodiment, the gated network training unit 302 is specifically configured to:

[0105] Acquire a training sample set, wherein the training sample set includes a plurality of reasoning tasks of different types, and each reasoning task corresponds to a preset reasoning result;

[0106] Sending the target reasoning task in the training sample set to the plurality of second computing devices, so that the plurality of second computing devices input the target reasoning task into the fine-tuned pre-trained model, and the fine-tuned pre-trained model executes the target reasoning task and outputs a corresponding reasoning result;

[0107] Acquire multiple inference results obtained by the multiple second computing devices, and input the multiple inference results into the MoE gating network, and the MoE gating network performs weighted fusion on the multiple inference results, and outputs the weighted fusion inference results;

[0108] According to the difference between the weighted fused reasoning result output by the MoE gating network and the reasoning result preset by the target reasoning task, the parameters of the MoE gating network are adjusted.

[0109] In an illustrated embodiment, the plurality of second computing devices adopt the same parameter fine-tuning method to respectively perform fine-tuning training on the pre-trained model based on the private data held by each of them.

[0110] In an illustrated embodiment, the parameter fine-tuning method is a high-efficiency parameter fine-tuning PEFT method or a full-quantity fine-tuning method.

[0111] In an illustrated embodiment, the pre-trained model includes multiple computing layers; the gated network training unit 302 is specifically used to:

[0112] For at least some of the multiple computing layers included in the pre-trained model, corresponding MoE gating networks are respectively constructed; wherein the at least some of the computing layers include the last computing layer for outputting the inference result.

[0113] In an illustrated embodiment, the at least part of the calculation layers further include a target calculation layer except the last calculation layer; the apparatus 30 further includes a second weighted fusion unit 304, which is used to:

[0114] Obtaining multiple calculation results output by the target calculation layer contained in the multiple fine-tuned pre-trained models;

[0115] Inputting the plurality of calculation results into the MoE gating network corresponding to the target calculation layer, and the MoE gating network performs weighted fusion on the plurality of calculation results to obtain a fused calculation result;

[0116] The fused calculation result is sent to the multiple second computing devices, and the multiple second computing devices input the fused calculation result into the next calculation layer connected to the target calculation layer contained in the respective fine-tuned pre-trained models.

[0117] Accordingly, the embodiments of this specification also provide another pre-training model fine-tuning device. Figure 4 , Figure 4 40 is a schematic diagram of another pre-training model fine-tuning device provided by an exemplary embodiment. Figure 1 The system architecture shown can be applied to Figure 1 Any second computing device among the plurality of second computing devices corresponding to the plurality of data providers shown, wherein the plurality of data providers hold private data corresponding to different types of reasoning tasks. Figure 4 As shown, the device 40 includes:

[0118] An acquisition unit 401 is used to acquire a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider;

[0119] The gated network training unit 402 is used to fine-tune the pre-trained model based on the private data held to obtain a fine-tuned pre-trained model, so that the first computing device can train the MoE gated network corresponding to the pre-trained model constructed by the first computing device based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein,

[0120] The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

[0121] The implementation process of the functions and effects of each unit in the above-mentioned device 30 and device 40 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 30 and device 40 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0122] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0123] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0124] Corresponding to the above method embodiment, the embodiment of this specification also provides a computing device. Figure 5 , Figure 5 is a schematic diagram of a computing device provided by an exemplary embodiment. The computing device may be Figure 1 The first computing device corresponding to the model provider shown, or any second computing device among multiple second computing devices corresponding to multiple data providers. The model provider holds a pre-trained model to be fine-tuned; multiple data providers hold private data corresponding to different types of reasoning tasks. Figure 5 As shown, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Figure 5 As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be executed by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the pre-trained model fine-tuning method in the embodiment of this specification.

[0125] For a detailed description of each step of the above pre-training model fine-tuning method, please refer to the previous content and will not be repeated here.

[0126] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, which stores a computer program, and when these computer programs are executed by the processor, they execute the various steps of the pre-training model fine-tuning method in the embodiment of this specification. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0127] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0128] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0129] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0130] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0131] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A pre-training model fine-tuning method, characterized in that: applied to a first computing device corresponding to a model provider, the model provider holding a pre-trained model to be fine-tuned; A plurality of data providers connected to the model provider hold private data corresponding to different types of reasoning tasks; the method comprises: Sending the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively perform fine-tuning training on the pre-trained model based on the private data held by them, to obtain a fine-tuned pre-trained model; Constructing a MoE gating network corresponding to the pre-training model, and training the MoE gating network based on a plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein, The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

2. The method according to claim 1, characterized in that: The method further comprises: Sending the inference task to be executed to the plurality of second computing devices, so that the plurality of second computing devices input the inference task into the fine-tuned pre-trained model, the fine-tuned pre-trained model executes the inference task, and outputs the corresponding inference result; Acquire multiple inference results obtained by the multiple second computing devices, and input the multiple inference results into the MoE gating network, wherein the MoE gating network determines the weights of the multiple fine-tuned pre-training models according to the correlation between the inference task and the multiple fine-tuned pre-training models, and performs weighted fusion on the multiple inference results output by the multiple fine-tuned pre-training models according to the determined weights.

3. The method according to claim 1, characterized in that The step of training the MoE gating network based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices includes: Based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices, the MoE gating network is trained in a federated learning manner.

4. The method according to claim 3, characterized in that The method of training the MoE gating network in a federated learning manner based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices includes: Acquire a training sample set, wherein the training sample set includes a plurality of reasoning tasks of different types, and each reasoning task corresponds to a preset reasoning result; Sending the target reasoning task in the training sample set to the plurality of second computing devices, so that the plurality of second computing devices input the target reasoning task into the fine-tuned pre-trained model, and the fine-tuned pre-trained model executes the target reasoning task and outputs a corresponding reasoning result; Acquire multiple inference results obtained by the multiple second computing devices, and input the multiple inference results into the MoE gating network, and the MoE gating network performs weighted fusion on the multiple inference results, and outputs the weighted fusion inference results; According to the difference between the weighted fused reasoning result output by the MoE gating network and the reasoning result preset by the target reasoning task, the parameters of the MoE gating network are adjusted.

5. The method according to claim 1, characterized in that: The multiple second computing devices adopt the same parameter fine-tuning method to fine-tune the pre-trained model based on the private data they each hold.

6. The method according to claim 5, characterized in that The parameter fine-tuning method is a high-efficiency parameter fine-tuning PEFT method or a full-quantity fine-tuning method.

7. The method according to any one of claims 1 to 6, characterized in that: The pre-trained model includes multiple computing layers; the step of constructing a MoE gating network corresponding to the pre-trained model includes: For at least some of the multiple computing layers included in the pre-trained model, corresponding MoE gating networks are respectively constructed; wherein the at least some of the computing layers include the last computing layer for outputting the inference result.

8. The method according to claim 7, characterized in that The at least part of the computing layers also includes a target computing layer except the last computing layer; the method further includes: Obtaining multiple calculation results output by the target calculation layer contained in the multiple fine-tuned pre-trained models; Inputting the plurality of calculation results into the MoE gating network corresponding to the target calculation layer, and the MoE gating network performs weighted fusion on the plurality of calculation results to obtain a fused calculation result; The fused calculation result is sent to the multiple second computing devices, and the multiple second computing devices input the fused calculation result into the next calculation layer connected to the target calculation layer contained in the respective fine-tuned pre-trained models.

9. A pre-training model fine-tuning method, characterized in that: The method is applied to any second computing device among a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of reasoning tasks; and the method comprises: Obtaining a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider; The pre-trained model is fine-tuned based on the private data held to obtain a fine-tuned pre-trained model, so that the first computing device trains the MoE gating network corresponding to the pre-trained model constructed by the first computing device based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein, The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

10. A pre-training model fine-tuning device, characterized in that: applied to a first computing device corresponding to a model provider, the model provider holding a pre-trained model to be fine-tuned; A plurality of data providers connected to the model provider hold private data corresponding to different types of reasoning tasks; the device comprises: A fine-tuning unit, configured to send the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively perform fine-tuning training on the pre-trained model based on the private data held by them, to obtain a fine-tuned pre-trained model; A gating network training unit, used to construct a MoE gating network corresponding to the pre-training model, and train the MoE gating network based on a plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein, The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

11. A pre-training model fine-tuning device, characterized in that: Applied to any second computing device among a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of reasoning tasks; the apparatus comprises: An acquisition unit, configured to acquire a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider; A gated network training unit, configured to fine-tune the pre-trained model based on the private data held, to obtain a fine-tuned pre-trained model, so that the first computing device trains the MoE gated network corresponding to the pre-trained model constructed by the first computing device based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices; wherein, The trained MoE gating network is used to: determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the reasoning task to be executed and the multiple fine-tuned pre-trained models, and weightedly fuse the multiple reasoning results output by the multiple fine-tuned pre-trained models when executing the reasoning task according to the determined weights.

12. A computing device, characterized in that: include: Memory and processor; The memory has stored thereon computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, the method according to any one of claims 1 to 8 or 9 is performed.

13. A computer-readable storage medium, characterized in that: A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 8 or 9 is implemented.

14. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 8 or 9 is implemented.

Images