Pre-trained model fine-tuning

Abstract

The present disclosure provides a pre-trained model fine-tuning method and a related device. The method is applied to a first computing device corresponding to a model provider, and a plurality of data providers hold private data corresponding to different inference tasks. The method comprises: sending a pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to a plurality of data providers, so that the plurality of second computing devices perform fine-tuning training on the pre-trained model on the basis of private data held by the second computing devices; and constructing an MoE gating network corresponding to the pre-trained model, and training the MoE gating network on the basis of a plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices, wherein the MoE gating network is used for: determining weights of the plurality of fine-tuned pre-trained models on the basis of correlations between an inference task to be executed and the plurality of fine-tuned pre-trained models, and performing, on the basis of the determined weights, weighted fusion on a plurality of inference results outputted by the plurality of fine-tuned pre-trained models executing the inference task.

Description

Fine-tuning of pre-trained model TECHNICAL FIELD

[0001] One or more embodiments of the present disclosure relate to the technical field of pre-trained model, and in particular to fine-tuning of pre-trained model. BACKGROUND

[0002] A pre-trained model is a language model constructed by a deep neural network containing more than hundreds of billions of parameters, which is pre-trained on a large and diverse public dataset using a self-supervised learning method and has mastered many language phenomena.

[0003] In order to apply the pre-trained model to various inference tasks related to different application scenarios, it is necessary to further fine-tune the pre-trained model with datasets related to different application scenarios. However, the datasets used in the fine-tuning stage often contain private data held by each participant in the fine-tuning. If each participant directly shares the datasets used in the fine-tuning stage to fine-tune the pre-trained model with all the data, it is easy to cause leakage of private data and infringe the interests of users. SUMMARY

[0004] Therefore, one or more embodiments of the present disclosure provide a pre-trained model fine-tuning method and related equipment.

[0005] In a first aspect, the present disclosure provides a pre-trained model fine-tuning method applied to a first computing device corresponding to a model provider, wherein the model provider holds a pre-trained model to be fine-tuned; a plurality of data providers interfacing with the model provider hold private data corresponding to different types of inference tasks; the method comprises: sending the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively fine-tune the pre-trained model based on the private data held thereby to obtain fine-tuned pre-trained models; constructing a MoE gating network corresponding to the pre-trained model, and training the MoE gating network based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gating network is configured to: determine weights of the plurality of fine-tuned pre-trained models according to the correlation between an inference task to be executed and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models in executing the inference task according to the determined weights.

[0006] In a second aspect, the present disclosure provides a pre-trained model fine-tuning method, applied to any second computing device corresponding to a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of inference tasks; the method comprises: obtaining a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider; fine-tuning the pre-trained model based on the held private data to obtain a fine-tuned pre-trained model, so that the first computing device trains the MoE gating network corresponding to the pre-trained model constructed by the first computing device based on a plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gating network is used to: determine the weights of the plurality of fine-tuned pre-trained models according to the correlation between the inference task to be executed and the plurality of fine-tuned pre-trained models, and according to the determined weights, the plurality of inference results output by the plurality of fine-tuned pre-trained models executing the inference task are weighted and fused.

[0007] In a third aspect, the present disclosure provides a pre-trained model fine-tuning device, applied to a first computing device corresponding to a model provider, the model provider holding a pre-trained model to be fine-tuned; a plurality of data providers interfacing with the model provider hold private data corresponding to different types of inference tasks; the device comprises: a fine-tuning unit configured to send the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices fine-tune the pre-trained model based on the held private data to obtain a fine-tuned pre-trained model; a gating network training unit configured to construct a MoE gating network corresponding to the pre-trained model, and train the MoE gating network based on a plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gating network is used to: determine the weights of the plurality of fine-tuned pre-trained models according to the correlation between the inference task to be executed and the plurality of fine-tuned pre-trained models, and according to the determined weights, the plurality of inference results output by the plurality of fine-tuned pre-trained models executing the inference task are weighted and fused.

[0008] In a fourth aspect, the present disclosure provides a pre-trained model fine-tuning apparatus, applied to any second computing device of a plurality of second computing devices corresponding to a plurality of data providers; wherein the plurality of data providers hold private data corresponding to different types of inference tasks; the apparatus comprises: an acquisition unit configured to acquire a pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider; a gating network training unit configured to fine-tune the pre-trained model based on the held private data to obtain a fine-tuned pre-trained model, so that the first computing device trains a MoE gating network corresponding to the pre-trained model constructed by the first computing device based on a plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gating network is configured to: determine weights of the plurality of fine-tuned pre-trained models according to a correlation between an inference task to be performed and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models in performing the inference task according to the determined weights.

[0009] Correspondingly, the present disclosure also provides a computing device, comprising: a memory and a processor; the memory stores a computer program / instruction executable by the processor; when the processor executes the computer program / instruction, the pre-trained model fine-tuning method of the first aspect or the second aspect is executed.

[0010] Correspondingly, the present disclosure also provides a computer-readable storage medium, which stores a computer program / instruction, and when the computer program / instruction is executed by a processor, the pre-trained model fine-tuning method of the first aspect or the second aspect is executed.

[0011] Correspondingly, the present disclosure also provides a computer program product, which comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the pre-trained model fine-tuning method of the first aspect or the second aspect is executed.

[0012] In summary, each data provider in the present application can use the private data held by itself to locally fine-tune the pre-trained model to be fine-tuned. Then, the model provider can construct the MoE gating network corresponding to the pre-trained model, and the pre-trained model fine-tuned locally by each data provider can be regarded as each expert in the MoE architecture. Further, the pre-trained model fine-tuned locally by each data provider can be used to train the MoE gating network. The MoE gating network obtained by training can dynamically adjust the weights of the pre-trained models fine-tuned by each data provider for different inference tasks, and based on the weights, the inference results output by the pre-trained models fine-tuned by each party for performing the inference task are weighted and fused to obtain the final global model output. In this way, the global model can play an excellent performance in the inference tasks of various application scenarios while meeting the privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0013] FIG. 1 is a schematic diagram of a system architecture provided by an example embodiment;

[0014] FIG. 2 is a schematic diagram of a pre-trained model fine-tuning method provided by an example embodiment;

[0015] FIG. 3 is a schematic diagram of a structure of a pre-trained model fine-tuning apparatus provided by an example embodiment;

[0016] FIG. 4 is a schematic diagram of another structure of a pre-trained model fine-tuning apparatus provided by an example embodiment;

[0017] FIG. 5 is a schematic diagram of a structure of a computing device provided by an example embodiment. DETAILED DESCRIPTION

[0018] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to designate the same elements, unless otherwise indicated. The embodiments described in the following example embodiments are not representative of all embodiments consistent with one or more embodiments of the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present disclosure, as detailed in the appended claims.

[0019] It should be noted that the steps of the corresponding methods in other embodiments are not necessarily performed in the order shown and described in the present disclosure. In some other embodiments, the steps included in the methods can be more or less than those described in the present disclosure. In addition, a single step described in the present disclosure can be divided into multiple steps for description in other embodiments, and multiple steps described in the present disclosure can be combined into a single step for description in other embodiments.

[0020] It should be noted that the "multiple" in the present application refers to two or more.

[0021] In addition, the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0022] First, some terms in the present disclosure are explained and described to facilitate understanding by those skilled in the art.

[0023] (1) Large Language Models (LLM), also known as pre-training models, are language models constructed by deep neural networks containing more than hundreds of billions of parameters, which are pre-trained on large and diverse public datasets using self-supervised learning methods and have mastered many language phenomena.

[0024] (2) Federated Learning (FL) is a distributed machine learning framework that allows various participants (usually referred to as clients) to collaboratively train a shared global model without the need to centralize private data held by each client, i.e., without the need for each client to share its own private data, thereby allowing learning and improving the ability of the model from multiple data sources while protecting data privacy and security.

[0025] In federated learning, each client can use its own private data to train the global model locally and generate updated model parameters or gradient information, and then send the updated model parameters or gradient information to a central server. Correspondingly, the central server can aggregate the updated model parameters or gradient information sent by multiple clients and update the global model based on the aggregated data. Further, after updating the global model, the new model parameters can be distributed to each client to enable each client to train the updated global model using its own private data, and so on, repeatedly performing the above process until a predetermined training goal is reached.

[0026] (3) Mixture of Experts (MoE), a machine learning architecture, consists of multiple sub-models (called Experts) and a Gating Network. Each Expert can be used to process a certain part or pattern in the data. The Gating Network is used to decide which Experts should be used to process the given input data, and how the outputs of these Experts should be combined.

[0027] For example, the operation flow of MoE is generally as follows: 1. The input data is transmitted to all Experts.

[0028] 2. The Gating Network evaluates the relevance or utility of each Expert for the current input data and assigns a corresponding weight to each Expert based on this.

[0029] 3. Each Expert independently processes the input data and outputs the processing results (such as prediction results or features, etc.).

[0030] 4. According to the weights assigned by the Gating Network, the processing results output by all Experts are weighted and combined (such as simple weighted summation, etc.), thereby obtaining the final model output.

[0031] It should be understood that although the pre-trained model has excellent ability in understanding and generating natural language, in order to apply the pre-trained model to various reasoning tasks related to different application scenarios, it is necessary to further fine-tune the pre-trained model with datasets related to different application scenarios. However, the datasets used in the fine-tuning stage often contain private data held by each participant involved in the fine-tuning. If each participant directly shares the datasets used in the fine-tuning stage to centralize all data to fine-tune the pre-trained model, it is easy to cause the leakage of private data and infringe the interests of users.

[0032] As described above, federated learning, as a new distributed machine learning paradigm, allows multiple participants to collaboratively train a shared global model without sharing data. Based on this, federated learning can be used to fine-tune the pre-trained model jointly with multiple participants. Specifically, each participant can fine-tune the pre-trained model locally using its own private data, then upload the fine-tuning results (such as updated model parameters or gradient information, etc.) to the central server for aggregation, and the central server updates the pre-trained model based on the aggregated data, etc., thereby obtaining the pre-trained model fine-tuned based on the private data of multiple parties, not only protecting the data privacy of each participant, but also utilizing the wide data from different sources and different application scenarios, enhancing the generalization ability of the pre-trained model.

[0033] Although federated learning provides an effective privacy protection mechanism, due to the heterogeneity of the data distribution of the private data held by each participant, the global model obtained by directly aggregating the fine-tuning training results of each party may not meet the individualized needs of all participants, resulting in the final global model not being able to perform well in different application scenarios.

[0034] Based on this, the present disclosure provides a technical solution, which utilizes the MoE architecture, determines the weights of the fine-tuned pre-training models (which can be regarded as individual experts) of each participant for different inference tasks through the MoE gating network, and performs weighted fusion on the inference results output by each party, so as to meet the privacy protection while enabling the global model to perform outstandingly in diversified application scenarios.

[0035] In implementation, the present application can be applied to a first computing device corresponding to a model provider holding a pre-training model to be fine-tuned, and a plurality of data providers interfacing with the model provider and holding private data corresponding to different types of inference tasks. In an illustrated embodiment, the first computing device can first send the pre-training model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers. Further, the plurality of second computing devices can fine-tune the pre-training model based on the private data held by them respectively, and obtain fine-tuned pre-training models respectively. Further, the first computing device can construct a MoE gating network corresponding to the pre-training model, and train the MoE gating network based on the plurality of fine-tuned pre-training models obtained by the plurality of second computing devices. Wherein, the trained MoE gating network can be used to: determine the weights of the plurality of fine-tuned pre-training models according to the correlation between the inference task to be performed and the plurality of fine-tuned pre-training models, and perform weighted fusion on the plurality of inference results output by the plurality of fine-tuned pre-training models in performing the inference task according to the determined weights.

[0036] In the above technical solution, each data provider in the application can use the private data held by itself to locally fine-tune the pre-trained model to be fine-tuned. Then, the model provider can construct the MoE gating network corresponding to the pre-trained model, and the pre-trained model fine-tuned locally by each data provider can be regarded as each expert in the MoE architecture. Further, the pre-trained model fine-tuned locally by each data provider can be used to train the MoE gating network. The MoE gating network obtained by training can dynamically adjust the weights of the pre-trained models fine-tuned by each data provider for different inference tasks, and based on the weights, the inference results output by the pre-trained models fine-tuned by each party for performing the inference task are weighted and fused to obtain the final global model output. In this way, the privacy protection can be met while the global model can play an excellent performance in the inference tasks of various application scenarios.

[0037] Referring to FIG. 1, FIG. 1 is a schematic diagram of a system architecture provided by an exemplary embodiment. As shown in FIG. 1, the system can include a first computing device 100 corresponding to a model provider, and a plurality of computing devices corresponding to a plurality of data providers, for example, including a second computing device 200a, a second computing device 200b, and a second computing device 200c, etc. In an illustrated implementation, the first computing device 100 and the second computing device 200a, the second computing device 200b, and the second computing device 200c can establish a communication connection through any possible way, which is not specifically limited by the present disclosure.

[0038] Among them, the model provider holds a publicly available pre-trained model trained to completion, and expects that the pre-trained model can successfully perform different inference tasks related to various application scenarios through further fine-tuning training.

[0039] Among them, the plurality of data providers hold private data corresponding to different types of inference tasks. For example, the different types of inference tasks can include asset analysis, traffic planning, user interface development, image generation, chart analysis, and question answering, etc., which are not specifically limited by the present disclosure. For example, the private data can include mathematical data (math data) corresponding to asset analysis tasks, code data (code data) corresponding to user interface development tasks, and image data corresponding to image generation tasks, etc., which are not specifically limited by the present disclosure.

[0040] In an illustrated embodiment, the first computing device 100 corresponding to the model provider can send the pre-training model to be fine-tuned held by the first computing device 100 to the second computing device 200a, the second computing device 200b, the second computing device 200c, and the like corresponding to the plurality of data providers. In an illustrated embodiment, the first computing device 100 can broadcast the pre-training model to be fine-tuned to the second computing device 200a, the second computing device 200b, the second computing device 200c, and the like by broadcasting.

[0041] Correspondingly, the second computing device 200a, the second computing device 200b, and the second computing device 200c can receive the pre-training model to be fine-tuned sent by the first computing device 100. Further, the second computing device 200a, the second computing device 200b, and the second computing device 200c can perform fine-tuning training on the pre-training model locally based on the private data held by each of the second computing device 200a, the second computing device 200b, and the second computing device 200c, respectively, to obtain a fine-tuned pre-training model.

[0042] It should be noted that the present disclosure does not particularly limit the specific implementation of fine-tuning training. In an illustrated embodiment, the second computing device 200a, the second computing device 200b, and the second computing device 200c can freeze the parameters of the pre-training model and perform fine-tuning training on the pre-training model using a parameter efficient fine-tuning (PEFT) method or a full fine-tuning method. For example, the PEFT method can include LoRA (Low-Rank Adaptation of LLMs), prefix tuning, prompt tuning, and the like, which are not particularly limited by the present disclosure.

[0043] In addition, in order to ensure that the fine-tuned parameters obtained by each party can act on the same position in the pre-training model, for example, the same dimension of the same model parameter, the second computing device 200a, the second computing device 200b, and the second computing device 200c need to use the same fine-tuning method, for example, LoRA or Prefix Tuning, and the like, which are not particularly limited by the present disclosure.

[0044] Further, the first computing device 100 can construct a MoE gating network corresponding to the pre-training model. The network structure of the gating network can include a linear layer or a self-attention mechanism (Self-Attention) layer, and the like, which are not particularly limited by the present disclosure.

[0045] Further, the first computing device 100 can train the MoE gating network based on the fine-tuned pre-trained models obtained by the second computing device 200a, the second computing device 200b, and the second computing device 200c respectively. In an illustrated embodiment, the multiple fine-tuned pre-trained models can be jointly used to train the MoE gating network in a federated learning manner, which will be described in detail below with reference to the corresponding embodiment of FIG. 2.

[0046] The trained MoE gating network can be used to determine the weights of the multiple fine-tuned pre-trained models according to the correlation between the inference task to be performed and the multiple fine-tuned pre-trained models, and to perform weighted fusion on the multiple inference results output by the multiple fine-tuned pre-trained models according to the determined weights.

[0047] In an illustrated embodiment, the first computing device 100 is a central server in federated learning, and the second computing device 200a, the second computing device 200b, and the second computing device 200c are multiple participants (or clients) in federated learning. In an illustrated embodiment, the first computing device 100, the second computing device 200a, the second computing device 200b, and the second computing device 200c can be smart wearable devices, smartphones, tablet computers, notebook computers, desktop computers, servers, or server clusters or cloud computing centers composed of multiple servers, etc. with the above functions, and the present disclosure does not make specific limitations thereto.

[0048] Please refer to FIG. 2, which is a flowchart of a pre-trained model fine-tuning method according to an example embodiment. The method can be applied to the system architecture shown in FIG. 1, and specifically can be applied to the first computing device corresponding to the model provider shown in FIG. 1. As shown in FIG. 2, the method can specifically include the following steps S201-S202.

[0049] Step S201, sending a pre-trained model to be fine-tuned to multiple second computing devices corresponding to multiple data providers, so that the multiple second computing devices fine-tune and train the pre-trained model based on the private data they hold, to obtain fine-tuned pre-trained models.

[0050] First of all, it should be noted that the complete technical solution provided in the present application can include the fine-tuning phase of the pre-trained model, the training phase of the MoE gating network, and the inference phase in actual application, which will be described in turn as follows. (1) Fine-tuning phase of pre-trained model

[0051] As described above, the model provider holds the pre-trained model to be fine-tuned, and the plurality of data providers hold private data corresponding to different types of inference tasks, which can be specifically referred to the description of the embodiment corresponding to FIG. 1, and will not be described here again.

[0052] In an illustrated embodiment, the first computing device corresponding to the model provider can first send the pre-trained model to be fine-tuned held by it to the plurality of second computing devices corresponding to the plurality of data providers.

[0053] Correspondingly, the plurality of second computing devices can receive the pre-trained model and respectively fine-tune the pre-trained model locally based on the private data held.

[0054] In an illustrated embodiment, the plurality of second computing devices can freeze the parameters of the pre-trained model before fine-tuning. Based on this, after the plurality of second computing devices perform fine-tuning locally, each of the plurality of second computing devices can obtain some newly added parameters (i.e., fine-tuning parameters), but the parameters of the pre-trained model will not be updated.

[0055] In an illustrated embodiment, the pre-trained model described above can be regarded as a backbone large model, and the model parameters thereof can be used to provide basic inference capability. The fine-tuning parameters obtained by the plurality of second computing devices can be used to provide individualized inference capability corresponding to the private data thereof.

[0056] In an illustrated embodiment, the model parameters of the pre-trained model and the fine-tuning parameters obtained by each of the plurality of second computing devices can constitute a fine-tuned pre-trained model. That is, in the fine-tuning stage of the pre-trained model, the plurality of second computing devices obtain a plurality of fine-tuned pre-trained models in total. In an illustrated embodiment, the fine-tuning parameters obtained by each of the plurality of second computing devices can be regarded as an expert in MoE, or the fine-tuned pre-trained model constituted by the model parameters of the pre-trained model and the fine-tuning parameters obtained by each of the plurality of second computing devices can be regarded as an expert, and the like, which are not specifically limited by the present disclosure.

[0057] It should be noted that the present disclosure does not specifically limit the specific implementation of the fine-tuning described above. In an illustrated embodiment, the plurality of second computing devices can use the PEFT method or the full fine-tuning method to fine-tune the pre-trained model, which can be specifically referred to the description of the embodiment corresponding to FIG. 1, and will not be described here again. It should be understood that if the plurality of second computing devices all use the full fine-tuning method to fine-tune the pre-trained model, the newly added parameters obtained by each of the plurality of second computing devices after fine-tuning can be the difference between the parameters of the fine-tuned pre-trained model and the parameters of the original pre-trained model, which is not specifically limited by the present disclosure.

[0058] In step S202, an MoE gating network corresponding to the pre-trained model is constructed, and the MoE gating network is trained based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gating network is configured to: determine weights of the plurality of fine-tuned pre-trained models according to a correlation between a to-be-executed inference task and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models in executing the inference task according to the determined weights.

[0059] Further, after the plurality of second computing devices respectively complete the fine-tuning training locally, the first computing device can construct an MoE gating network corresponding to the pre-trained model.

[0060] In an illustrated implementation, the pre-trained model can include a plurality of computing layers, and accordingly, an MoE gating network corresponding to at least part of the plurality of computing layers can be constructed according to a preset gating strategy.

[0061] In an illustrated implementation, the at least part of the computing layers can include a last computing layer, i.e., an output layer, for outputting inference results. Accordingly, the MoE gating network corresponding to the output layer can be configured to: determine weights of the plurality of fine-tuned pre-trained models according to a correlation between a to-be-executed inference task and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models in executing the inference task according to the determined weights, to obtain a final output result of the model.

[0062] In an illustrated implementation, the at least part of the computing layers can further include a plurality of target computing layers other than the output layer. For example, the pre-trained model can include 32 layers, and the target computing layer can be the 3rd layer, the 10th layer, or the 20th layer, etc., which are not limited in the present disclosure. Accordingly, the MoE gating network corresponding to the target computing layer can be configured to: determine weights of the plurality of fine-tuned pre-trained models according to a correlation between a to-be-executed inference task and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models in executing the inference task according to the determined weights, to obtain a fused computing result. Further, the first computing device can send the fused computing result to the plurality of second computing devices respectively, and accordingly, the plurality of second computing devices can input the fused computing result to a next computing layer connected to the target computing layer in the fine-tuned pre-trained model, for subsequent computation.

[0063] In summary, the disclosure does not particularly limit the number and corresponding positions of the constructed gating networks. In an illustrated embodiment, considering that the inference results output by multiple fine-tuned training models need to be weighted and fused in the end, it is necessary to construct an MoE gating network for the output layer of the pre-trained model, and further, corresponding MoE gating networks can also be adaptively constructed for the remaining several calculation layers according to actual conditions and computing requirements. In addition, it needs to be noted that the calculation layers to which the multiple second computing devices perform fine-tuning training are irrelevant to the calculation layers corresponding to the constructed MoE gating networks. For example, the multiple second computing devices can perform fine-tuning for the 4th layer of the pre-trained model respectively, and correspondingly, the obtained multiple fine-tuning parameters can be located in the 4th layer. Further, the first computing device can construct a corresponding MoE gating network for the 4th layer of the pre-trained model, or the first computing device can also construct a corresponding MoE gating network for the 5th layer or the 6th layer of the pre-trained model, and so on, which are not particularly limited by the disclosure.

[0064] Further, after the MoE gating network is constructed, the MoE gating network can be trained based on the multiple fine-tuned pre-trained models obtained by the multiple second computing devices, so that the MoE gating network can dynamically and accurately determine the weights of the multiple fine-tuned pre-trained models according to different inference tasks.

[0065] In an illustrated embodiment, the first computing device and the multiple second computing devices can train the MoE gating network in a federated learning manner. For example, the MoE gating network can be trained in a horizontal federated learning manner. It needs to be noted that the disclosure does not particularly limit the parameter aggregation manner in horizontal federated learning, which can be any parameter aggregation manner such as federated averaging, adaptive averaging, etc., or enhanced secure aggregation manner such as differential privacy, etc.

[0066] Specifically, the first computing device can first obtain a training sample set, which can include multiple inference tasks of different types, and each inference task corresponds to a preset inference result, which is equivalent to a sample label.

[0067] Further, the first computing device can send a target inference task in the training sample set to the multiple second computing devices. Correspondingly, the multiple second computing devices can freeze the pre-trained model and the local fine-tuning parameters, which is equivalent to freezing the local fine-tuned pre-trained model. Then, each first computing device in the multiple second computing devices can input the target inference task into the local fine-tuned pre-trained model, execute the target inference task by the fine-tuned pre-trained model, and output the corresponding inference result.

[0068] Further, the first computing device can obtain a plurality of inference results obtained by the plurality of second computing devices, and input the plurality of inference results into the pre-constructed MoE gating network, weight and fuse the plurality of inference results by the MoE gating network, and output the inference result after weight and fusion.

[0069] Further, the first computing device can adjust and optimize the parameters of the MoE gating network according to the difference between the inference result after weight and fusion and the preset inference result of the target inference task.

[0070] Further, the first computing device can also send other inference tasks in the training sample set to the plurality of second computing devices to further adjust the parameters of the MoE gating network, repeatedly execute the above steps until the output of the MoE gating network meets the expectation (for example, the difference between the preset inference result is less than a preset value), thereby completing the federated training of the MoE gating network. Wherein, the MoE gating network after training can dynamically and accurately determine the weights of the plurality of fine-tuned pre-training models corresponding to the plurality of second computing devices for different inference tasks input into the pre-training model. (3) Inference phase

[0071] After completing the local fine-tuning of the pre-training model and the federated training of the MoE gating network, the trained global model can be applied to actual inference tasks.

[0072] Next, taking an example of constructing a corresponding MoE gating network for both the target computing layer and the output layer in the pre-training model, the inference process of the trained global model in actual application will be described.

[0073] First, the first computing device can send an inference task corresponding to a user's request to the plurality of second computing devices in response to the request. Correspondingly, the plurality of second computing devices can input the inference task into the respective fine-tuned pre-training model, and execute the inference task by the fine-tuned pre-training model.

[0074] Specifically, in the computing process of executing the inference task, the target computing layer included in each fine-tuned pre-training model can output a corresponding computing result, and then each second computing device can send the computing result output by the target computing layer to the first computing device. Correspondingly, the first computing device can obtain a plurality of computing results sent by the plurality of second computing devices, and input the plurality of computing results into the pre-constructed MoE gating network corresponding to the target computing layer, and weight and fuse the plurality of computing results according to the determined weights by the MoE gating network to obtain a fused computing result. Exemplarily, the plurality of computing results can be weighted and fused, which can specifically include weighted summation of the plurality of computing results.

[0075] Further, the first computing device can return the fusion computing result output by the MoE gating network to the plurality of second computing devices. Correspondingly, each second computing device can input the received fusion computing result into the next computing layer connected with the target computing layer in the fine-tuned pre-trained model for subsequent computing.

[0076] Further, as the computing is performed layer by layer, the plurality of fine-tuned pre-trained models corresponding to the plurality of second computing devices finally respectively output inference results corresponding to the inference task. Correspondingly, the first computing device can obtain the plurality of inference results obtained by the plurality of second computing devices, and input the plurality of inference results into the MoE gating network corresponding to the output layer constructed in advance, so as to fuse the plurality of inference results by weighting according to the determined weights, and obtain the final output result of the model. Further, the first computing device can return the output result to the user, and the like, which are not limited in the present disclosure.

[0077] For example, the inference task can be a user interface development task, and the MoE gating network can determine the weight of the fine-tuned pre-trained model in the second computing device holding code data (equivalent to having programming ability) to be 0.5, the weight of the fine-tuned pre-trained model in the second computing device holding image data (equivalent to having image generation ability) to be 0.4, and the weight of the fine-tuned pre-trained model in the second computing device holding mathematical data (equivalent to having mathematical analysis and calculation) to be 0.1, and the like, which are not limited in the present disclosure.

[0078] For example, the inference task can be a chart analysis task, and the MoE gating network can determine the weight of the fine-tuned pre-trained model in the second computing device holding code data (equivalent to having programming ability) to be 0, the weight of the fine-tuned pre-trained model in the second computing device holding image data (equivalent to having image generation ability) to be 0.3, and the weight of the fine-tuned pre-trained model in the second computing device holding mathematical data (equivalent to having mathematical analysis and calculation) to be 0.7, and the like, which are not limited in the present disclosure.

[0079] Corresponding to the method flow implementation described above, the embodiments of the present disclosure also provide a pre-trained model fine-tuning device. Please refer to FIG. 3, which is a structural schematic diagram of a pre-trained model fine-tuning device provided by an exemplary embodiment. The device 30 can be applied to the system architecture shown in FIG. 1, and specifically can be applied to the first computing device corresponding to the model provider shown in FIG. 1, wherein the model provider holds a pre-trained model to be fine-tuned, and a plurality of data providers interfacing with the model provider hold private data corresponding to different types of inference tasks. As shown in FIG. 3, the device 30 comprises: a fine-tuning unit 301 configured to send the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively fine-tune the pre-trained model based on the private data held thereby to obtain fine-tuned pre-trained models; a gated network training unit 302 configured to construct a MoE gated network corresponding to the pre-trained model, and train the MoE gated network based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein the trained MoE gated network is configured to determine weights of the plurality of fine-tuned pre-trained models according to the correlation between a to-be-executed inference task and the plurality of fine-tuned pre-trained models, and perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-trained models according to the determined weights.

[0080] In an illustrated implementation, the device 30 further comprises a first weighted fusion unit 303 configured to: send a to-be-executed inference task to the plurality of second computing devices, so that the plurality of second computing devices input the inference task into the fine-tuned pre-trained models, execute the inference task by the fine-tuned pre-trained models, and output corresponding inference results; obtain the plurality of inference results obtained by the plurality of second computing devices, and input the plurality of inference results into the MoE gated network, so that the MoE gated network determines weights of the plurality of fine-tuned pre-trained models according to the correlation between the inference task and the plurality of fine-tuned pre-trained models, and performs weighted fusion on the plurality of inference results output by the plurality of fine-tuned pre-trained models according to the determined weights.

[0081] In an illustrated implementation, the gated network training unit 302 is specifically configured to train the MoE gated network in a federated learning manner based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices.

[0082] In an illustrated implementation, the gating network training unit 302 is specifically configured to: obtain a training sample set, the training sample set including a plurality of inference tasks of different types, each inference task corresponding to a preset inference result; send a target inference task in the training sample set to the plurality of second computing devices, so that the plurality of second computing devices input the target inference task into the fine-tuned pre-trained model, execute the target inference task by the fine-tuned pre-trained model, and output a corresponding inference result; obtain a plurality of inference results obtained by the plurality of second computing devices, and input the plurality of inference results into the MoE gating network, perform weighted fusion on the plurality of inference results by the MoE gating network, and output a weighted fusion inference result; and adjust parameters of the MoE gating network according to a difference between the weighted fusion inference result output by the MoE gating network and the preset inference result of the target inference task.

[0083] In an illustrated implementation, the plurality of second computing devices use the same parameter fine-tuning method to fine-tune the pre-trained model based on private data respectively held by the plurality of second computing devices.

[0084] In an illustrated implementation, the parameter fine-tuning method is a high-efficiency parameter fine-tuning (PEFT) method or a full-amount fine-tuning method.

[0085] In an illustrated implementation, the pre-trained model includes a plurality of computing layers; and the gating network training unit 302 is specifically configured to: for at least part of the plurality of computing layers included in the pre-trained model, construct a corresponding MoE gating network respectively; and the at least part of the plurality of computing layers includes a last computing layer for outputting an inference result.

[0086] In an illustrated implementation, the at least part of the plurality of computing layers further includes a target computing layer other than the last computing layer; and the apparatus 30 further includes a second weighted fusion unit 304 configured to: obtain a plurality of computing results output by the target computing layer included in the plurality of fine-tuned pre-trained models; input the plurality of computing results into the MoE gating network corresponding to the target computing layer, perform weighted fusion on the plurality of computing results by the MoE gating network, and obtain a fused computing result; and send the fused computing result to the plurality of second computing devices, so that the plurality of second computing devices input the fused computing result into a next computing layer connected to the target computing layer in the fine-tuned pre-trained model respectively held by the plurality of second computing devices.

[0087] Correspondingly, the embodiment of the disclosure also provides another pre-training model fine-tuning device. Please refer to FIG. 4, which is a structural schematic diagram of another pre-training model fine-tuning device provided by an exemplary embodiment. The device 40 can be applied to the system architecture shown in FIG. 1, and in particular, can be applied to any one of the plurality of second computing devices corresponding to the plurality of data providers shown in FIG. 1, wherein the plurality of data providers hold private data corresponding to different types of inference tasks. As shown in FIG. 4, the device 40 comprises: an acquisition unit 401 configured to acquire a pre-training model to be fine-tuned sent by a first computing device corresponding to a model provider; and a gating network training unit 402 configured to fine-tune the pre-training model based on the held private data to obtain a fine-tuned pre-training model, so that the first computing device trains a MoE gating network corresponding to the pre-training model constructed by the first computing device based on a plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein the trained MoE gating network is configured to determine weights of the plurality of fine-tuned pre-training models according to a correlation between an inference task to be performed and the plurality of fine-tuned pre-training models, and to perform weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-training models in performing the inference task according to the determined weights.

[0088] The implementation process of the functions and roles of each unit in the above device 30 and device 40 is specifically described in the above embodiments, and will not be repeated here. It should be understood that the above device 30 and device 40 can be implemented by software, or by hardware or a combination of software and hardware. For example, as a logical device, it is formed by reading the corresponding computer program instructions into the memory and running by the processor (CPU) of the device. From the hardware level, in addition to the CPU and the memory, the device where the above device is located usually also includes other hardware such as a chip for wireless signal transmission and reception, and / or other hardware such as a board for realizing network communication function.

[0089] The device embodiments described above are only schematic, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical modules, i.e., they can be located in one place, or distributed on multiple network modules. According to actual needs, part or all of the units or modules can be selected to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement without creative labor.

[0090] The apparatuses, units, and modules illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer, and the computer can be specifically a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an e-mail device, a game console, a tablet computer, a wearable device, an in-vehicle computer, or a combination of any of these devices.

[0091] Corresponding to the method embodiments, the embodiments of the present disclosure further provide a computing device. Please refer to FIG. 5, which is a structural schematic diagram of a computing device according to an example embodiment. The computing device can be the first computing device corresponding to the model provider shown in FIG. 1, or any of the second computing devices corresponding to the plurality of data providers. The model provider holds the pre-trained model to be fine-tuned, and the plurality of data providers hold private data corresponding to different types of inference tasks. As shown in FIG. 5, the computing device includes a processor 1001 and a memory 1002, and can further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 can be connected through a bus or other means. As shown in FIG. 5, the memory 1002 includes a computer readable storage medium 1003, which stores computer programs capable of being run by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiments. When running the stored computer programs, the processor 1001 can execute the steps of the pre-trained model fine-tuning method according to the embodiments of the present disclosure.

[0092] The detailed description of the steps of the above pre-trained model fine-tuning method can be found in the previous content, which will not be repeated here.

[0093] Corresponding to the method embodiments, the embodiments of the present disclosure further provide a computer readable storage medium, which stores computer programs. When the computer programs are run by a processor, the steps of the pre-trained model fine-tuning method according to the embodiments of the present disclosure are executed. Please refer to the description of the above embodiments for details, which will not be repeated here.

[0094] The above only describes preferred embodiments of the present disclosure and should not be used to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0095] In one typical arrangement, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0096] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) having a common memory space employable to store the processing module and / or the data module, and / or non-volatile memory, such as flash memory, read-only memory (ROM), etc. The memory is an example of computer readable media.

[0097] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

[0098] Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store information which can be accessed by a computing device. According to the definitions provided herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0099] It is also important to note that the terms "comprises", "comprising", or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0100] Those skilled in the art will understand that embodiments of the present disclosure can be provided as a method, a system or a computer program product. Accordingly, embodiments of the present disclosure can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present disclosure can be embodied in the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied thereon.

Claims

1. A pre-trained model fine-tuning method, applied to a first computing device corresponding to a model provider, the model provider holding a pre-trained model to be fine-tuned; A plurality of data providers interfaced with the model provider hold private data corresponding to different types of inference tasks; the method comprises: sending the pre-training model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices respectively fine-tune the pre-training model based on the private data held by them to obtain fine-tuned pre-training models; constructing an MoE gating network corresponding to the pre-training model, and training the MoE gating network based on the plurality of fine-tuned pre-training models obtained by the plurality of second computing devices; wherein the MoE gating network trained is used to determine the weights of the plurality of fine-tuned pre-training models according to the correlation between the inference task to be performed and the plurality of fine-tuned pre-training models, and to weight and fuse the plurality of inference results output by the plurality of fine-tuned pre-training models in performing the inference task according to the determined weights.

2. The method of claim 1, further comprising: sending an inference task to be performed to the plurality of second computing devices, so that the plurality of second computing devices input the inference task into the fine-tuned pre-training models, execute the inference task by the fine-tuned pre-training models, and output corresponding inference results; obtaining the plurality of inference results obtained by the plurality of second computing devices, and inputting the plurality of inference results into the MoE gating network, so that the MoE gating network determines the weights of the plurality of fine-tuned pre-training models according to the correlation between the inference task and the plurality of fine-tuned pre-training models, and weights and fuses the plurality of inference results output by the plurality of fine-tuned pre-training models according to the determined weights.

3. The method of claim 1, wherein, The training of the MoE gating network based on the plurality of fine-tuned pre-training models obtained by the plurality of second computing devices comprises: training the MoE gating network in a federated learning manner based on the plurality of fine-tuned pre-training models obtained by the plurality of second computing devices.

4. The method of claim 3, wherein, The training of the MoE gating network based on the plurality of fine-tuned pre-training models obtained by the plurality of second computing devices comprises: obtaining a training sample set, the training sample set including a plurality of inference tasks of different types, each inference task corresponding to a preset inference result; sending a target inference task in the training sample set to the plurality of second computing devices, so that the plurality of second computing devices input the target inference task into the fine-tuned pre-training models, execute the target inference task by the fine-tuned pre-training models, and output corresponding inference results; obtaining the plurality of inference results obtained by the plurality of second computing devices, and inputting the plurality of inference results into the MoE gating network, so that the MoE gating network weights and fuses the plurality of inference results, and outputs the inference results after weighting and fusion; According to a difference between a weighted fused inference result output by the MoE gating network and a preset inference result of the target inference task, a parameter of the MoE gating network is adjusted.

5. The method of claim 1, wherein, The plurality of second computing devices adopt the same parameter fine-tuning method to respectively fine-tune the pre-trained model based on the private data held by each of the plurality of second computing devices.

6. The method of claim 5, wherein, The parameter fine-tuning method is a high-efficiency parameter fine-tuning (PEFT) method or a full-amount fine-tuning method.

7. The method of any one of claims 1-6, wherein, The pre-trained model comprises a plurality of computing layers; and the MoE gating network corresponding to the pre-trained model is constructed by: For at least part of the plurality of computing layers included in the pre-trained model, a corresponding MoE gating network is respectively constructed; wherein the at least part of the plurality of computing layers includes a last computing layer for outputting an inference result.

8. The method of claim 7, wherein, The at least part of the plurality of computing layers further includes a target computing layer other than the last computing layer; and the method further comprises: A plurality of computing results output by the target computing layer included in the plurality of fine-tuned pre-trained models are obtained; The plurality of computing results are input to the MoE gating network corresponding to the target computing layer, and the plurality of computing results are weighted and fused by the MoE gating network to obtain a fused computing result; The fused computing result is sent to the plurality of second computing devices, and the plurality of second computing devices input the fused computing result to a next computing layer connected to the target computing layer included in the fine-tuned pre-trained model of each of the plurality of second computing devices. 9.A pre-trained model fine-tuning method, applied to any one of a plurality of second computing devices corresponding to a plurality of data providers; wherein, The plurality of data providers hold private data corresponding to different types of inference tasks; and the method comprises: A pre-trained model to be fine-tuned sent by a first computing device corresponding to a model provider is obtained; The pre-trained model is fine-tuned based on the private data held by the model provider to obtain a fine-tuned pre-trained model, so that the MoE gating network corresponding to the pre-trained model constructed by the first computing device is trained based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein The MoE gating network trained is used to determine weights of the plurality of fine-tuned pre-trained models according to a correlation between a to-be-executed inference task and the plurality of fine-tuned pre-trained models, and to weightedly fuse a plurality of inference results output by the plurality of fine-tuned pre-trained models when the plurality of fine-tuned pre-trained models execute the inference task according to the determined weights.

10. A pre-trained model fine-tuning apparatus applied to a first computing device corresponding to a model provider, the model provider holding a pre-trained model to be fine-tuned; A plurality of data providers interfaced with the model provider hold private data corresponding to different types of inference tasks; and the apparatus comprises: A fine-tuning unit configured to send the pre-trained model to be fine-tuned to a plurality of second computing devices corresponding to the plurality of data providers, so that the plurality of second computing devices fine-tune the pre-trained model based on the private data held by each of the plurality of second computing devices to obtain a fine-tuned pre-trained model; A gating network training unit configured to construct a MoE gating network corresponding to the pre-trained model, and to train the MoE gating network based on the plurality of fine-tuned pre-trained models obtained by the plurality of second computing devices; wherein The MoE gating network after training is used for determining weights of the plurality of fine-tuned pre-training models according to a correlation between an inference task to be performed and the plurality of fine-tuned pre-training models, and performing weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-training models in performing the inference task according to the determined weights.

11. A pre-trained model fine-tuning apparatus applied to any one of a plurality of second computing devices corresponding to a plurality of data providers; wherein, The plurality of data providers hold private data corresponding to different types of inference tasks; the apparatus comprises: An obtaining unit is configured to obtain a pre-training model to be fine-tuned sent by a first computing device corresponding to a model provider; A gating network training unit is configured to fine-tune the pre-training model based on the held private data to obtain a fine-tuned pre-training model, so that the first computing device trains a MoE gating network corresponding to the pre-training model constructed by the first computing device based on a plurality of fine-tuned pre-training models obtained by a plurality of second computing devices; wherein The MoE gating network after training is used for determining weights of the plurality of fine-tuned pre-training models according to a correlation between an inference task to be performed and the plurality of fine-tuned pre-training models, and performing weighted fusion on a plurality of inference results output by the plurality of fine-tuned pre-training models in performing the inference task according to the determined weights.

12. A computing device comprising: A memory and a processor; wherein the memory has computer programs / instructions stored thereon which can be run by the processor; when the processor runs the computer programs / instructions, the method according to any one of claims 1 to 8 or 9 is executed.

13. A computer readable storage medium having stored thereon computer programs / instructions, wherein, The computer programs / instructions are executed by the processor to implement the method according to any one of claims 1 to 8 or 9.

14. A computer program product, wherein, The computer program product comprises computer programs / instructions which are executed by the processor to implement the method according to any one of claims 1 to 8 or 9. The computer program product comprises computer programs / instructions which are executed by the processor to implement the method according to any one of claims 1 to 8 or 9.

Images