Heterogeneous model training method and system based on federated learning

Abstract

The invention provides a heterogeneous model training method and system based on federated learning. The method comprises the following steps: each client initializes a local heterogeneous model, and divides the local heterogeneous model into a feature extractor and a classifier; the server initializes a global adapter; the server issues the global adapter to each client; each client uses the received global adapter as a to-be-updated local adapter, and uses local data to train the to-be-updated local adapter, the feature extractor and the classifier to generate the current optimal local adapter, the feature extractor and the classifier; wherein in the training process, the local adapter to be updated and the feature extractor share a classifier; each client uploads the current optimal local adapter to the server; and the server aggregates all the received local adapters to obtain an updated global adapter, and iteratively updates the global adapter until the local heterogeneous model on each client meets a preset requirement.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a heterogeneous model training method and system based on federated learning. Background Art

[0002] With the booming development of the digital age, artificial intelligence has brought many changes to society. In many fields such as medicine, finance, smart homes, and autonomous driving, artificial intelligence models provide more accurate and efficient solutions to many problems by analyzing and processing massive amounts of data.

[0003] However, with the widespread application of machine learning technology in artificial intelligence, privacy issues are also increasingly concerned. Traditional data-centralized model training methods cannot protect sensitive data, and a model training method that can replace centralized methods is needed. In recent years, federated learning, as an emerging distributed machine learning method, is gradually becoming a key technology for data privacy protection and cross-institutional collaboration. Among them, multiple devices participating in model training (also known as clients) under the control of a central server only need to share local models without sharing sensitive data, so as to collaboratively train a general model with stronger generalization capabilities. This federated learning method of multi-client collaborative training model is equivalent to the traditional centralized data unified training method, while also protecting data privacy. Therefore, federated learning has become a more popular distributed machine learning framework in the big data era.

[0004] Early federated learning aims to train a global model for server collaboration and assumes that the client models are the same, such as the FedAvg algorithm, FedProx, and FedGen. However, in actual application scenarios, federated learning often faces data heterogeneity and model heterogeneity problems. The data heterogeneity problem (also known as the Non-IID problem) refers to the uneven distribution of data in different clients in real scenarios, which does not meet the assumption of independent and identical distribution, and the data distribution of a single client cannot represent all clients. Non-IID makes the model of client cooperative training weak in generalization ability and cannot adapt to the heterogeneous data of all clients. In addition, the model heterogeneity problem is also a major challenge that hinders the further promotion of federated learning. Specifically, some clients want to design a specific model framework for themselves. For example, when different Internet devices (clients) participate in federated training, in addition to being unwilling to share local sensitive data, their model structure information also hopes to be protected. At the same time, considering the differences in computing power and communication efficiency between different clients, the method of using local models with the same structure for different clients is no longer applicable, so model heterogeneity is also a problem that needs to be solved in federated learning. Therefore, there is an increasing number of federated learning methods aimed at training multiple local heterogeneous models, such as FedMD, Fedproto, and FedALA.

[0005] The mainstream methods used in existing heterogeneous federated learning include knowledge distillation, model decoupling, and class average information sharing. However, the knowledge distillation method requires the use of public datasets to calculate soft predictions, but public datasets are usually difficult to obtain and the computational overhead of training the generated model is large; the model decoupling method shares the same part of the model (such as feature extractors or classifiers) to the server aggregation, but the degree of model heterogeneity is limited; the class average information sharing method uploads local class prototypes or class scores to the server aggregation, but the uploaded class average information is prone to privacy leakage. Summary of the invention

[0006] The present invention proposes a heterogeneous model training method and system based on federated learning, which does not require the use of public data sets and is not limited by the degree of heterogeneity of local models. It only requires the addition of an adapter to assist in the training of federated client models, thereby improving the performance of local models while protecting privacy.

[0007] In a first aspect, the present invention provides a heterogeneous model training method based on federated learning, comprising:

[0008] Step 1: Each client initializes a local heterogeneous model and divides the local heterogeneous model into a feature extractor and a classifier;

[0009] Step 2: The server initializes a global adapter;

[0010] Step 3: The server sends the global adapter to each client;

[0011] Step 4: Each client uses the received global adapter as the local adapter to be updated, and uses local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier;

[0012] Step 5: Each client uploads the currently optimal local adapter to the server;

[0013] Step 6: The server aggregates all received local adapters to obtain an updated global adapter, and returns to step 3;

[0014] Step 7: Repeat steps 3 to 6 until the local heterogeneous model on each client meets the preset requirements.

[0015] Furthermore, step 2 specifically includes: the server selects a feature extractor from all clients and initializes it as an initial global adapter.

[0016] Furthermore, step 4 specifically includes:

[0017] The client inputs the local data into the feature extractor and the local adapter to be updated respectively, and obtains the first intermediate feature output by the feature extractor and the second intermediate feature output by the local adapter to be updated;

[0018] Calculate the difference between the first intermediate feature and the second intermediate feature, and use the difference as the distillation loss of the feature extractor and the local adapter to be updated respectively;

[0019] Inputting the first intermediate feature and the second intermediate feature into the classifier to obtain a corresponding first prediction output and a second prediction output;

[0020] Calculate the loss between the first predicted output and the second predicted output and the true value respectively, and record the obtained two loss values ​​as the supervised learning loss of the feature extractor and the local adapter to be updated respectively;

[0021] Calculate the total loss of the feature extractor and the local adapter to be updated based on the distillation loss and supervised learning loss respectively;

[0022] The current optimal local adapter and feature extractor are obtained by minimizing the total loss of the feature extractor and the local adapter to be updated; the current optimal classifier is obtained by minimizing the sum of the supervised learning losses of the feature extractor and the local adapter to be updated.

[0023] Furthermore, a mean square error function is used to calculate the difference between the first intermediate feature and the second intermediate feature.

[0024] Furthermore, a cross entropy loss function is used to calculate the losses between the first prediction output and the second prediction output and the true value, respectively.

[0025] Further, in step 5, the server aggregates all received local adapters to obtain an updated global adapter, which specifically includes:

[0026]

[0027] Among them, δ represents the updated parameters of the global adapter, δ i Indicates the parameters of the local adapter uploaded by client i, D i represents the data of client i, |D i | represents the data volume of client i, and N represents the total data volume of all clients.

[0028] In a second aspect, the present invention provides a heterogeneous model training system based on federated learning, comprising:

[0029] A first initialization module is used to enable each client to initialize a local heterogeneous model and divide the local heterogeneous model into a feature extractor and a classifier;

[0030] A second initialization module is used to enable the server to initialize a global adapter;

[0031] A communication module, used to enable the server to send the global adapter to each client, and enable each client to upload the currently optimal local adapter to the server;

[0032] A training module, used to enable each client to use the received global adapter as the local adapter to be updated, and use local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier;

[0033] The aggregation module is used to enable the server to aggregate all received local adapters to obtain an updated global adapter.

[0034] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0035] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0036] Beneficial effects of the present invention:

[0037] (1) A global adapter is set up on the server. Each client obtains the global adapter from the server and updates it with local data to generate a local adapter. An adapter sharing mechanism is implemented between the client and the server. That is, the client uploads a local adapter to replace the local heterogeneous model to the server for aggregation. The server sends the global adapter to the client to achieve knowledge fusion and transfer across different clients.

[0038] With the help of the adapter sharing mechanism, the learning of global knowledge and local knowledge by client heterogeneous models is effectively promoted, and the problem that it is difficult for the server to aggregate client heterogeneous models is solved. Ultimately, it can effectively alleviate the challenges of data heterogeneity and model heterogeneity in federated learning and improve client model performance.

[0039] (2) The system of the present invention is scalable, that is, it can allow different numbers of clients and different types of clients to participate in federated collaboration.

[0040] (3) During client model training, the local heterogeneous model (i.e., feature extractor and classifier) ​​is trained simultaneously with the local adapter, and a feature bidirectional distillation mechanism is performed to achieve bidirectional knowledge transfer between the local adapter and feature extractor.

[0041] (4) During client model training, a classifier sharing mechanism is executed, that is, the local adapter and the feature extractor share the same classifier, so that the local adapter is trained by supervised learning like the feature extractor. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 One of the flow charts of a heterogeneous model training method based on federated learning provided in an embodiment of the present invention;

[0043] Figure 2 A second flowchart of a heterogeneous model training method based on federated learning provided in an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the structure of a heterogeneous model training system based on federated learning provided in an embodiment of the present invention;

[0045] Figure 4 Comparison of the accuracy of local models obtained by using two different methods of data set partitioning in 10 clients provided by the embodiment of the present invention;

[0046] Figure 5 Comparison of the accuracy of local models obtained by using two different methods of data set partitioning under 20 clients provided by the embodiment of the present invention;

[0047] Figure 6 Comparison of the accuracy of local models obtained by using two different methods of data set partitioning under 50 clients provided by the embodiment of the present invention;

[0048] Figure 7 A structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] like Figure 1As shown, an embodiment of the present invention provides a heterogeneous model training method based on federated learning, comprising the following steps:

[0051] S101: Each client initializes a local heterogeneous model and divides the local heterogeneous model into feature extractors and classifier

[0052] Specifically, and Represent the model architecture of feature extractor and classifier respectively, and θ i They represent the model parameters of the feature extractor and classifier respectively, and i represents the client number.

[0053] S102: The server initializes a global adapter

[0054] Specifically, in this embodiment, the global adapter is initialized in the following manner: the server selects a feature extractor from all clients and initializes it as the initial global adapter. The selection method can be user-defined or randomly selected, which will not be described in detail here.

[0055] S103: The server sends the global adapter Send to each client;

[0056] S104: Each client uses the received global adapter as the local adapter to be updated, and uses the local data x j ∈B k The local adapter, feature extractor and classifier to be updated are trained to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter and feature extractor to be updated share the classifier; wherein, B k Represents the kth batch of data on the current client.

[0057] Specifically, during training, the local adapter and feature extractor to be updated share the classifier of the same local heterogeneous model, so that the local adapter to be updated can be effectively trained by supervised learning like the feature extractor of the local heterogeneous model.

[0058] S105: Each client uploads the currently optimal local adapter to the server;

[0059] S106: The server aggregates all received local adapters to obtain an updated global adapter, and returns to S103;

[0060] Specifically, all clients share local adapters to the server for aggregation to obtain a global adapter, and the server sends the global adapter to the client to promote cross-client knowledge fusion and transfer, thereby achieving effective federated collaboration.

[0061] S107: Repeat steps S103 to S106 until the local heterogeneous model on each client meets the preset requirements.

[0062] Specifically, the preset requirement in this embodiment refers to the convergence of the local heterogeneous model, that is, when the model converges, the process ends.

[0063] The heterogeneous model training method based on federated learning provided by the embodiment of the present invention implements an adapter sharing mechanism between the server and the client, and the local adapter and the global adapter respectively added to the client and the server communicate to realize cross-client knowledge fusion and transfer. During model training, the client uses private data to train its local heterogeneous model and local adapter. Since the feature extractor and the local adapter in the local heterogeneous model share a classifier, the trained local heterogeneous model can learn the knowledge of other clients while maintaining the personalization of its own local heterogeneous model.

[0064] Based on the above embodiments, Figure 2 As shown, an embodiment of the present invention provides a client training method, comprising the following steps:

[0065] The client inputs the local data into the feature extractor and the local adapter to be updated respectively, and obtains the first intermediate feature h output by the feature extractor j and the second intermediate feature h′ output by the local adapter to be updated j ; The process is shown as follows:

[0066]

[0067] It can be understood that since the feature extractor is constructed according to the characteristics of the client, the first intermediate feature h output by the feature extractor is j Similarly, since the local adapter to be updated is the global adapter downloaded from the server, and the global adapter is obtained by aggregating the local adapters of all clients in the previous iteration, the second intermediate feature h′ output by the local adapter to be updated is j With more global information.

[0068] The client calculates the difference between the first intermediate feature and the second intermediate feature, and uses the difference as the distillation loss of the feature extractor and the local adapter to be updated respectively;

[0069] Specifically, in this embodiment, the mean square error function MSE is used to calculate the difference between two intermediate features. The process is as follows:

[0070]

[0071] in, represents the distillation loss of the feature extractor, represents the distillation loss of the local adapter to be updated, B k Indicates the kth batch of data of the current client, |B k | indicates B k The total amount of data in .

[0072] The first intermediate feature and the second intermediate feature are input into the classifier to obtain the corresponding first prediction output p j and the second prediction output p′ j ; The process is as follows:

[0073]

[0074] Calculate the loss between the first predicted output and the second predicted output and the true value respectively, and record the obtained two loss values ​​as the supervised learning loss of the feature extractor and the local adapter to be updated respectively;

[0075] Specifically, in this embodiment, the cross entropy loss function CE is used to calculate the loss between the two predicted outputs and the true value, and the calculation process is expressed as follows:

[0076]

[0077] in, represents the supervised learning loss of the feature extractor, represents the supervised learning loss of the local adapter to be updated, y j Represents data x j The corresponding true label value.

[0078] Calculate the total loss of the feature extractor and the local adapter to be updated based on the distillation loss and supervised learning loss respectively;

[0079] Specifically, the following formula is used in this embodiment to calculate the total loss of the two:

[0080]

[0081]

[0082] in, represents the total loss of the feature extractor, represents the total loss of the local adapter to be updated, and λ represents the weight parameter preset by the user.

[0083] The current optimal local adapter and feature extractor are obtained by minimizing the total loss of the feature extractor and the local adapter to be updated; the current optimal classifier is obtained by minimizing the sum of the supervised learning losses of the feature extractor and the local adapter to be updated.

[0084] Specifically, the embodiment of the present invention adopts the gradient descent method to optimize the model parameters. The process can be expressed as follows:

[0085]

[0086] in, represents the local adapter parameters obtained after the tth iteration on the client, η represents the learning rate, represents the gradient operator, represents the feature extractor parameters obtained after the tth iteration on the client, Represents the classifier parameters obtained after the tth iteration on the client.

[0087] It can be understood that during training, each client can train the local adapter and the local heterogeneous model at the same time, wherein bidirectional knowledge transfer between the feature extractor and the local adapter can be achieved by minimizing the distillation loss of the two. Therefore, in this embodiment, the client can improve the performance of the local heterogeneous model by executing the feature bidirectional distillation mechanism and the classifier sharing mechanism. At the same time, the embodiment of the present invention can allow different numbers of clients and clients of different device types to participate in federated training by executing the adapter sharing mechanism.

[0088] On the basis of the above embodiments, in an embodiment of the present invention, the server updates the global adapter in the following aggregation manner:

[0089]

[0090] Among them, δ represents the updated parameters of the global adapter, δ i Indicates the parameters of the local adapter uploaded by client i, D i represents the data of client i, |D i | represents the data volume of client i, and N represents the total data volume of all clients.

[0091] Corresponding to the above method, such as Figure 3 As shown, an embodiment of the present invention also provides a heterogeneous model training system based on federated learning, including a first initialization module, a second initialization module, a communication module, a training module and an aggregation module.

[0092] Among them, the first initialization module is used to enable each client to initialize the local heterogeneous model and divide the local heterogeneous model into a feature extractor and a classifier; the second initialization module is used to enable the server to initialize a global adapter; the communication module is used to enable the server to send the global adapter to each client, and enable each client to upload the current optimal local adapter to the server; the training module is used to enable each client to use the received global adapter as the local adapter to be updated, and use local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier; the aggregation module is used to enable the server to aggregate all received local adapters to obtain an updated global adapter.

[0093] It should be noted that the heterogeneous model training system based on federated learning provided in the embodiment of the present invention is to implement the above method. Its specific functions can be referred to the above method embodiments and will not be repeated here.

[0094] In order to verify the effectiveness of the solution of the present invention, experiments are conducted using the common CIFAR-100 image classification dataset to test the performance of the local heterogeneous model trained using the solution of the present invention.

[0095] The dataset uses two dataset partitioning methods, namely pathological setting and practical setting. For the pathological setting, we assign 10 of the 100 categories of the dataset to different clients; for the practical setting, we set the Dirichlet concentration to 0.1 to allocate datasets to clients. The lower the Dirichlet concentration, the more unbalanced the data distribution and the stronger the data heterogeneity. The experimental results are shown in Figure 2. Figure 4 , Figure 5 and Figure 6 As shown, under different numbers of clients (10, 20 and 50), the proposed method achieves the highest model accuracy compared with the six baseline methods.

[0096] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 701, a communication interface (Communications Interface) 702, a memory (memory) 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704. The processor 701 can call the logic instructions in the memory 703 to execute the heterogeneous model training method, which includes: step 1: each client initializes a local heterogeneous model and divides the local heterogeneous model into a feature extractor and a classifier; step 2: the server initializes a global adapter; step 3: the server sends the global adapter to each client; step 4: each client uses the received global adapter as the local adapter to be updated, and uses local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter and feature extractor to be updated share the classifier; step 5: each client uploads the current optimal local adapter to the server; step 6: the server aggregates all received local adapters to obtain an updated global adapter, and returns to step 3; step 7: repeat steps 3 to 6 until the local heterogeneous model on each client meets the preset requirements.

[0097] In addition, when the logic instructions in the above-mentioned memory 703 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0098] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: Step 1: Each client initializes a local heterogeneous model and divides the local heterogeneous model into a feature extractor and a classifier; Step 2: The server initializes a global adapter; Step 3: The server sends the global adapter to each client; Step 4: Each client uses the received global adapter as a local adapter to be updated, and uses local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier; Step 5: Each client uploads the current optimal local adapter to the server; Step 6: The server aggregates all received local adapters to obtain an updated global adapter, and returns to Step 3; Step 7: Repeat Steps 3 to 6 until the local heterogeneous model on each client meets the preset requirements.

[0099] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the methods provided by the above-mentioned method embodiments are implemented, for example, including: Step 1: Each client initializes a local heterogeneous model and divides the local heterogeneous model into a feature extractor and a classifier; Step 2: The server initializes a global adapter; Step 3: The server sends the global adapter to each client; Step 4: Each client uses the received global adapter as the local adapter to be updated, and uses local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter and feature extractor to be updated share the classifier; Step 5: Each client uploads the current optimal local adapter to the server; Step 6: The server aggregates all received local adapters to obtain an updated global adapter, and returns to Step 3; Step 7: Repeat Steps 3 to 6 until the local heterogeneous model on each client meets the preset requirements.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A heterogeneous model training method based on federated learning, characterized in that: include: Step 1: Each client initializes a local heterogeneous model and divides the local heterogeneous model into a feature extractor and a classifier; Step 2: The server initializes a global adapter; Step 3: The server sends the global adapter to each client; Step 4: Each client uses the received global adapter as the local adapter to be updated, and uses local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier; Step 5: Each client uploads the currently optimal local adapter to the server; Step 6: The server aggregates all received local adapters to obtain an updated global adapter, and returns to step 3; Step 7: Repeat steps 3 to 6 until the local heterogeneous model on each client meets the preset requirements.

2. According to the heterogeneous model training method based on federated learning according to claim 1, it is characterized in that: Step 2 specifically includes: the server selects a feature extractor from all clients and initializes it as an initial global adapter.

3. The method for training heterogeneous models based on federated learning according to claim 1, characterized in that: Step 4 specifically includes: The client inputs the local data into the feature extractor and the local adapter to be updated respectively, and obtains the first intermediate feature output by the feature extractor and the second intermediate feature output by the local adapter to be updated; Calculate the difference between the first intermediate feature and the second intermediate feature, and use the difference as the distillation loss of the feature extractor and the local adapter to be updated respectively; Inputting the first intermediate feature and the second intermediate feature into the classifier to obtain a corresponding first prediction output and a second prediction output; Calculate the loss between the first predicted output and the second predicted output and the true value respectively, and record the obtained two loss values ​​as the supervised learning loss of the feature extractor and the local adapter to be updated respectively; Calculate the total loss of the feature extractor and the local adapter to be updated based on the distillation loss and supervised learning loss respectively; The current optimal local adapter and feature extractor are obtained by minimizing the total loss of the feature extractor and the local adapter to be updated; the current optimal classifier is obtained by minimizing the sum of the supervised learning losses of the feature extractor and the local adapter to be updated.

4. The method for training heterogeneous models based on federated learning according to claim 3, characterized in that: The difference between the first intermediate feature and the second intermediate feature is calculated using a mean square error function.

5. The method for training heterogeneous models based on federated learning according to claim 3, characterized in that: The cross entropy loss function is used to calculate the loss between the first predicted output and the second predicted output and the true value respectively.

6. A heterogeneous model training method based on federated learning according to any one of claims 1 to 5, characterized in that: In step 5, the server aggregates all received local adapters to obtain an updated global adapter, which specifically includes: Among them, δ represents the updated parameters of the global adapter, δ i Indicates the parameters of the local adapter uploaded by client i, D i represents the data of client i, |D i | represents the data volume of client i, and N represents the total data volume of all clients.

7. A heterogeneous model training system based on federated learning, characterized in that: include: A first initialization module is used to enable each client to initialize a local heterogeneous model and divide the local heterogeneous model into a feature extractor and a classifier; A second initialization module is used to enable the server to initialize a global adapter; A communication module, used to enable the server to send the global adapter to each client, and enable each client to upload the currently optimal local adapter to the server; A training module, used to enable each client to use the received global adapter as the local adapter to be updated, and use local data to train the local adapter, feature extractor and classifier to be updated to generate the current optimal local adapter, feature extractor and classifier; wherein, during the training process, the local adapter to be updated and the feature extractor share the classifier; The aggregation module is used to enable the server to aggregate all received local adapters to obtain an updated global adapter.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images