An edge-cloud multi-level federated learning knowledge transmission method and device

Abstract

The application discloses an end-edge-cloud multi-level federated learning knowledge transmission method and device, the method comprises the following steps: in each round of cloud communication, training the model parameters of the local client model by using the local data to obtain a trained local client model; performing parameter aggregation based on the model parameters of the trained local client model with the same structure to obtain a parameter aggregation result; updating the model parameters of the edge model according to the parameter aggregation result, and performing knowledge transmission by using the updated model parameters of the edge model and a preset public data set. The application overcomes the limitation that the same model is shared between the federated learning client and the server, alleviates the problem of system heterogeneity in the federated learning, and enhances the flexibility of the federated learning.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and in particular to a method and apparatus for knowledge transfer in multi-level federated learning between edge and cloud. Background Technology

[0002] Federated learning, as an emerging distributed machine learning paradigm, allows multiple clients to collaboratively train machine learning models without sharing raw data. This significantly mitigates the risk of data privacy breaches, reduces data transmission overhead, and breaks down data silos. In traditional federated learning, each client trains a shared model using local private data and then transmits the model parameters to the server. The server then uses the FedAvg algorithm or its derivatives to aggregate the client model parameters to update the global and client models. However, this model parameter-based federated learning algorithm requires all client models to have the same structure as the global model, introducing system heterogeneity issues. Specifically, clients participating in federated learning typically have different model training resource capabilities, but the size and complexity of the shared model are limited to accommodate the least resource-rich client for training. Furthermore, allowing clients with heterogeneous resources to train models with the same structure inevitably leads to differences in training performance between clients, lengthening the training time of federated learning. Therefore, it is crucial to invent a more flexible federated learning framework that allows heterogeneous clients to train models with different structures adapted to their resource capabilities. Summary of the Invention

[0003] The present invention aims to at least partially solve one of the technical problems in the related art.

[0004] To address this, this invention proposes a multi-level federated learning knowledge transfer method that supports collaborative training of models with different structures among federated learning clients, while also supporting the training of a larger model on the server side. This overcomes the limitation of sharing the same model between federated learning clients and servers, alleviates the problem of system heterogeneity in federated learning, and enhances the flexibility of federated learning.

[0005] Another objective of this invention is to provide a multi-level federated learning knowledge transmission device for edge-cloud.

[0006] To achieve the above objectives, this invention proposes a multi-level federated learning knowledge transfer method for edge-cloud, comprising:

[0007] The trained local client model;

[0008] The parameter aggregation result is obtained by aggregating the model parameters of the trained local client model with the same structure.

[0009] The model parameters of the edge model are updated based on the parameter aggregation results, and knowledge transfer is performed using the updated model parameters of the edge model and a preset public dataset.

[0010] In addition, the edge-cloud multi-level federated learning knowledge transfer method according to the above embodiments of the present invention may also have the following additional technical features:

[0011] Furthermore, in one embodiment of the present invention, the knowledge transfer using the updated edge model parameters and a preset public dataset includes:

[0012] The model parameters of the server model are trained using the knowledge of the first logits output of each sample and the marginal model in the unlabeled public dataset to obtain the trained server model.

[0013] The model parameters of the edge model are trained using the model parameters of the trained server model and the second logits knowledge of each sample in the output public dataset to obtain the trained edge model;

[0014] The model parameters of the client model are updated based on the model parameters of the trained edge model in order to carry out the next round of model training and knowledge transfer.

[0015] Furthermore, in one embodiment of the present invention, the method further includes using the similarity between the third logits knowledge output by the trained local client model to measure the similarity between client data distributions to obtain a similarity measurement result.

[0016] Furthermore, in one embodiment of the present invention, after obtaining the similarity measurement result, the method further includes:

[0017] Based on the similarity measurement results, the data distribution similarity between any two clients is obtained to obtain the client data distribution similarity matrix;

[0018] Based on the hash mapping of the client data distribution similarity matrix and the local client model structure, local client models with similar data distribution and the same model structure are subjected to parameter aggregation and associated with edge servers to obtain parameter aggregation results and model association results;

[0019] Based on the parameter aggregation results and the local data, a local client model is trained, and parameter aggregation is performed using the model parameters of the trained local client model and the model association results.

[0020] Furthermore, in one embodiment of the present invention, after performing parameter aggregation using the model parameters of the trained local client model and the model association results, the method further includes:

[0021] Update the model parameters of the edge model based on the aggregated parameters;

[0022] The fourth logits knowledge output by the updated edge model in each sample is aggregated based on a weighted ensemble distillation mechanism, and knowledge distillation is performed based on the edge knowledge aggregation result; wherein, the weight of the edge knowledge aggregation result is obtained based on the confidence level of the updated edge model in predicting each sample.

[0023] Furthermore, in one embodiment of the present invention, after performing knowledge distillation based on the edge knowledge aggregation result, the method further includes:

[0024] Calculate the loss value of the server model for each sample in the public dataset to obtain the loss calculation result;

[0025] The sample with the smallest loss value in the loss calculation results is used to train the server model, and the edge model is trained based on the fifth logits knowledge output by the trained server model.

[0026] The model parameters of the client model are updated based on the model parameters of the trained edge model and the model association results, and the next round of model training and knowledge transfer is carried out iteratively until the accuracy of the server model and the client model reaches the preset target value.

[0027] To achieve the above objectives, another aspect of the present invention proposes an edge-cloud multi-level federated learning knowledge transfer device, comprising:

[0028] The model training module is used to train the model parameters of the local client model using local data in each round of cloud communication to obtain a trained local client model.

[0029] The parameter aggregation module is used to aggregate parameters based on the model parameters of the pre-trained local client models with the same structure to obtain parameter aggregation results;

[0030] The knowledge transfer module is used to update the model parameters of the edge model based on the parameter aggregation results, and to transfer knowledge using a preset public dataset.

[0031] The edge-cloud multi-level federated learning knowledge transfer method and apparatus of this invention support collaborative training of models with different structures among federated learning clients, while also supporting the training of a larger model on the server side. This overcomes the limitation of sharing the same model between federated learning clients and servers, alleviates the problem of system heterogeneity in federated learning, and enhances the flexibility of federated learning. Furthermore, the hybrid knowledge transfer method, weighted ensemble distillation method, and server-assisted knowledge selection method in the edge-cloud multi-level federated learning knowledge transfer framework improve the learning performance of federated learning and reduce its resource overhead.

[0032] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0034] Figure 1 This is a flowchart of the edge-cloud multi-level federated learning knowledge transfer method according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of the edge-cloud multi-level federated learning knowledge transmission framework according to an embodiment of the present invention;

[0036] Figure 3 This is a logic diagram of the edge-cloud multi-level federated learning knowledge transmission method according to an embodiment of the present invention;

[0037] Figure 4 This is a performance verification diagram of the edge-cloud multi-level federated learning knowledge transfer framework and method according to an embodiment of the present invention under a model isomorphism setting;

[0038] Figure 5 This is a performance verification diagram of the edge-cloud multi-level federated learning knowledge transfer framework and method according to an embodiment of the present invention in a heterogeneous model setting.

[0039] Figure 6 This is a validity verification diagram for each method in the edge-cloud multi-level federated learning knowledge transfer framework according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram of the structure of an edge-cloud multi-level federated learning knowledge transmission device according to an embodiment of the present invention. Detailed Implementation

[0041] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0043] The following description, with reference to the accompanying drawings, describes the edge-cloud multi-level federated learning knowledge transfer method and apparatus according to embodiments of the present invention.

[0044] Figure 1 This is a flowchart of the edge-cloud multi-level federated learning knowledge transmission method according to an embodiment of the present invention.

[0045] like Figure 1 As shown, the method includes, but is not limited to, the following steps:

[0046] S1, in each round of cloud communication, the model parameters of the local client model are trained using local data to obtain the trained local client model.

[0047] S2, perform parameter aggregation based on the model parameters of a pre-trained local client model with the same structure to obtain the parameter aggregation result;

[0048] S3 updates the model parameters of the edge model based on the parameter aggregation results, and uses the updated model parameters of the edge model and the preset public dataset for knowledge transfer.

[0049] As an example, Figure 2 The edge-cloud multi-level federated learning knowledge transmission framework provided in this embodiment of the invention, such as Figure 2 As shown:

[0050] Specifically, each client trains an arbitrary model based on its own system resource capabilities, while the server trains a larger model due to its relatively abundant resources. Different structures are allowed between client models. The edge-cloud multi-level federated learning knowledge transmission framework provided in the first aspect of the present invention uses edge servers to complete efficient knowledge transmission between heterogeneous client models and larger server models.

[0051] Specifically, in each round of cloud communication, each client trains its local client model using local data. Clients with identical model structures then transmit their model parameters to the edge server. The edge server aggregates the client model parameters and updates its edge model. Because the edge models have different structures, the edge server and the server-side rely on a shared, unlabeled public dataset for knowledge transfer.

[0052] Specifically, each edge model transmits the output logits of each sample in the public dataset to the server. The server uses the public dataset and the logits knowledge from the edge server to train the server model. Then, the server model transmits the logits knowledge of each sample in the public dataset to the edge server. The edge server uses the public dataset and the logits knowledge from the server to train the edge model and then transmits the parameters of the edge model to the client. The client uses this to update the client model and then performs the next round of model training and knowledge transmission.

[0053] As an example, Figure 3 A logical diagram of the edge-cloud multi-level federated learning knowledge transfer method provided in this embodiment of the invention is shown below. Figure 3 As shown:

[0054] Specifically, to improve the quality of knowledge provided to the server, the edge-cloud multi-level federated learning knowledge transfer method of this invention provides a heterogeneous perception hybrid knowledge transfer method that aggregates clients with similar data distributions together and associates them with the same edge server to collaboratively train an edge model. This makes the model an expert in a small domain, proficient in classifying certain categories. This method uses the similarity between the logits output by the client models to measure the similarity between the client data distributions. The similarity between the data distributions of client k and client l is calculated by the following formula:

[0055]

[0056] Among them, S c (·) represents cosine similarity. This represents the model pre-trained using the client k's local private training data. This represents the model's performance on samples in a shared public dataset. The output is obtained from formula (1). Based on this formula, the data distribution similarity between any two clients can be obtained, resulting in a client data distribution similarity matrix for client clustering. Furthermore, this method maintains a hash map recording the client model structure. Based on the calculated data distribution similarity matrix and the client model structure hash map, clients with similar data distributions and identical model structures are clustered into one class and associated with the same edge server. Then, in each cluster, the client uses its local private data to train a local client model, and then transmits the client model parameters to the associated edge server for aggregation.

[0057] Then, each edge server aggregates client model parameters using the following FedAvg algorithm:

[0058]

[0059] in, w represents the local private training dataset of client k. k The model parameters represent client k. The aggregated parameters w will be used to update the edge model. Each edge server updates its edge model using the aggregated model parameters. parameters Then the edge model outputs for each sample in the shared public dataset. The knowledge is transmitted to the server, where it is transferred using the edge-cloud multi-level federated learning knowledge transfer method of this embodiment of the invention. This method provides an integrated distillation mechanism to aggregate the knowledge output by the edge model on each sample and distill the knowledge to the server-side model. The method calculates the weights of the edge knowledge aggregation using the edge model's confidence level in its predictions for each sample. Specifically, for each sample in the shared public dataset... This method calculates the prediction for each edge model. Entropy measures their level of confidence:

[0060]

[0061] Where C represents the number of categories in the classification task. It is calculated using the following formula:

[0062]

[0063] Lower entropy implies higher confidence, and vice versa. Therefore, this method assigns more entropy when aggregating logits from edge models. Smaller marginal models have larger aggregation weights in their logits. For each sample... This method uses the following formula to aggregate logits from the edge server:

[0064]

[0065] Where V represents the number of edge models and the weights Defined as:

[0066]

[0067] Then, for each unlabeled sample Then you can We get a pseudo-label:

[0068]

[0069] Using sample labels The server model can then be trained using the cross-entropy loss function:

[0070]

[0071] The loss function based on knowledge distillation is:

[0072]

[0073] Among them, D KL Represents the KD divergence, and σ(·) represents the softmax function. The aggregated logits are obtained by aggregating edge logits. This represents the logits of the server model's output on the samples. Ultimately, the server model is trained by minimizing the following loss function:

[0074]

[0075] Using the aforementioned weighted ensemble distillation mechanism, knowledge from the edge model can be integrated and distilled into the server model. However, the integrated knowledge, including pseudo-labels and integrated logits, inevitably contains errors, which will affect the training effect of the server model. Therefore, to further improve the quality of knowledge provided for server model training, the edge-cloud multi-level federated learning knowledge transfer method of this invention provides a server-assisted knowledge selection method. This method utilizes information from the server model training process to filter high-quality knowledge. This method leverages a characteristic of machine learning: during training, models often first fit the easier-to-fit correctly labeled samples and then fit the incorrectly labeled samples. Therefore, correctly labeled samples often exhibit smaller loss values ​​compared to incorrectly labeled samples. Based on this, the method iteratively filters samples during the server model training process. Specifically, the method calculates the loss value of the server model for each sample in the public dataset, and then selects the samples with the smallest u% loss values ​​for each category, using only the selected samples for subsequent training rounds of the server model.

[0076] Then, the knowledge from the trained server model is transferred to the edge servers to train edge models. Because the edge models fit the selected samples better, this method only transfers the logits of the server model's output on the selected samples to the edge servers. When each edge server receives the server model's logits on the selected samples... Knowledge of output logits Then, logits are used to assign pseudo-labels to each sample:

[0077]

[0078] The loss function for each edge model, trained using an unlabeled public dataset, is defined as:

[0079]

[0080] Each edge server transmits the parameters of the trained edge model back to its associated client to update the client model, and then iterates through the next round of model training and knowledge transfer until the accuracy of the server model and the client model reaches the target value.

[0081] Furthermore, the effectiveness of the embodiments of the present invention is verified as follows:

[0082] The CIFAR-10 and CIFAR-100 public datasets were used to verify the effectiveness of the edge-cloud multi-level federated learning knowledge transfer framework and method provided in this embodiment of the invention. Specifically, 5000 samples were randomly selected from the datasets, removed from their labels, and used as the public dataset. The remaining samples were allocated to clients as their private training datasets. The edge-cloud multi-level federated learning knowledge transfer framework and method provided in this embodiment of the invention is named FedHKT. Its performance was compared with seven existing federated learning knowledge transfer frameworks and methods, including Local, FedAvg, FedProx, FedMD, FedGKT, FedDF, and FedET. The performance evaluation included three aspects: first, the accuracy of the server model after training; second, the test accuracy of the client model after training on the client's private local data, where each client's local test data and training data have the same data distribution; and third, the test accuracy of the client model after training on the global test dataset, used to evaluate the generalization ability of the client model.

[0083] Figure 4 This is a performance comparison of FedHKT with other methods in the isomorphic model setting, such as... Figure 4 (a) Figure 4 (b) Figure 4 (c) and Figure 4 In (d), both the client and server models are ResNet11. It can be seen that FedHKT improves the accuracy of the server model by an average of 6.6%-32.1%, and the accuracy of the client model also improves in most cases.

[0084] Figure 5 This is a performance comparison of FedHKT and other methods supporting heterogeneous client model structures in a heterogeneous model setting, such as... Figure 5 (a) Figure 5 (b) Figure 5 (c) and Figure 5In (d), the client model is one of ResNet11, ResNet20, or ResNet29, and the server model is ResNet56. It can be seen that FedHKT achieves higher accuracy for both the server and client models in most cases compared to other methods, validating that FedHKT supports more efficient knowledge transfer between heterogeneous client and server models.

[0085] Figure 6 The effectiveness of each method in the edge-cloud multi-level federated learning knowledge transfer framework and method provided in the published embodiments was verified, including the heterogeneous awareness hybrid knowledge transfer method, the weighted ensemble distillation method, and the server-assisted knowledge selection method. This experiment compares the method with the following: 1) a single-level knowledge transfer method, where the logits output by each client model are directly transmitted to the server, and the knowledge selected by the server is also directly transmitted to the client to adjust the client model; 2) FedHKT replacing the ensemble distillation method, replacing the weighted ensemble distillation method of FedHKT with an average ensemble distillation method from the existing FedDF algorithm; 3) FedHKT without a knowledge selection method, omitting the server-assisted knowledge selection method in FedHKT. It can be seen that each method in FedHKT plays a crucial role in improving learning performance.

[0086] The edge-cloud multi-level federated learning knowledge transfer method according to embodiments of the present invention supports collaborative training of models with different structures among federated learning clients, while also supporting the training of a larger model on the server side. This overcomes the limitation of sharing the same model between federated learning clients and servers, alleviates the problem of system heterogeneity in federated learning, and enhances the flexibility of federated learning. The hybrid knowledge transfer method, weighted ensemble distillation method, and server-assisted knowledge selection method in the edge-cloud multi-level federated learning knowledge transfer framework improve the learning performance of federated learning and reduce its resource overhead.

[0087] To achieve the above embodiments, such as Figure 7 As shown, this embodiment also provides an edge-cloud multi-level federated learning knowledge transmission device 10, which includes a model training module 100, a parameter aggregation module 200, and a knowledge transmission module 300.

[0088] The model training module 100 is used to train the model parameters of the local client model using local data in each round of cloud communication to obtain the trained local client model.

[0089] The parameter aggregation module 200 is used to aggregate parameters based on the model parameters of pre-trained local client models with the same structure to obtain parameter aggregation results.

[0090] The knowledge transfer module 300 is used to update the model parameters of the edge model based on the parameter aggregation results and to transfer knowledge using a preset public dataset.

[0091] Furthermore, the knowledge transfer module 300 is also used for:

[0092] The model parameters of the server model are trained using the knowledge of the first logits output of each sample and the marginal model in the unlabeled public dataset to obtain the trained server model.

[0093] The model parameters of the edge model are trained using the model parameters of the pre-trained server model and the second logits knowledge of each sample in the output public dataset to obtain the trained edge model;

[0094] The model parameters of the client model are updated based on the model parameters of the trained edge model in order to carry out the next round of model training and knowledge transfer.

[0095] Furthermore, the aforementioned device 10 also includes an association-cooperative training module, which is used to measure the similarity between client data distributions by utilizing the similarity between the third logits knowledge output by the trained local client model to obtain similarity measurement results.

[0096] Furthermore, the aforementioned collaborative training module is also used for:

[0097] Based on the similarity measurement results, the data distribution similarity between any two clients is obtained to obtain the client data distribution similarity matrix;

[0098] Based on the hash mapping of the client data distribution similarity matrix and the local client model structure, local client models with similar data distribution and the same model structure are subjected to parameter aggregation and associated with edge servers to obtain parameter aggregation results and model association results;

[0099] Based on the parameter aggregation results and the local data, a local client model is trained, and parameter aggregation is performed using the model parameters of the trained local client model and the model association results.

[0100] The edge-cloud multi-level federated learning knowledge transfer device according to embodiments of the present invention supports collaborative training of models with different structures among federated learning clients, while also supporting the training of a larger model on the server side. This overcomes the limitation of sharing the same model between federated learning clients and servers, alleviates the problem of system heterogeneity in federated learning, and enhances the flexibility of federated learning. The hybrid knowledge transfer method, weighted ensemble distillation method, and server-assisted knowledge selection method in the edge-cloud multi-level federated learning knowledge transfer framework improve the learning performance of federated learning and reduce its resource overhead.

[0101] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0102] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for knowledge transfer in edge-cloud multi-level federated learning, characterized in that, Includes the following steps: In each round of cloud communication, the model parameters of the local client model are trained using local data to obtain the trained local client model. The parameter aggregation result is obtained by aggregating the model parameters of the trained local client model with the same structure. The model parameters of the edge model are updated based on the parameter aggregation results, and knowledge transfer is performed using the updated model parameters of the edge model and a preset public dataset. The method further includes using the similarity between the third logits knowledge output by the trained local client model to measure the similarity between client data distributions to obtain similarity measurement results; After obtaining the similarity measurement results, the method further includes: Based on the similarity measurement results, the data distribution similarity between any two clients is obtained to obtain the client data distribution similarity matrix; Based on the hash mapping of the client data distribution similarity matrix and the local client model structure, local client models with similar data distribution and the same model structure are subjected to parameter aggregation and associated with edge servers to obtain parameter aggregation results and model association results; Based on the parameter aggregation results and the local data, a local client model is trained, and parameter aggregation is performed using the model parameters of the trained local client model and the model association results.

2. The method according to claim 1, characterized in that, The knowledge transfer using the updated edge model parameters and a pre-set public dataset includes: The model parameters of the server model are trained using the knowledge of the first logits output of each sample and the marginal model in the unlabeled public dataset to obtain the trained server model. The model parameters of the edge model are trained using the model parameters of the trained server model and the second logits knowledge of each sample in the output public dataset to obtain the trained edge model; The model parameters of the client model are updated based on the model parameters of the trained edge model in order to carry out the next round of model training and knowledge transfer.

3. The method according to claim 2, characterized in that, After performing parameter aggregation using the model parameters of the trained local client model and the model association results, the method further includes: Update the model parameters of the edge model based on the aggregated parameters; The fourth logits knowledge output by the updated edge model in each sample is aggregated based on a weighted ensemble distillation mechanism, and knowledge distillation is performed based on the edge knowledge aggregation result; wherein, the weight of the edge knowledge aggregation result is obtained based on the confidence level of the updated edge model in predicting each sample.

4. The method according to claim 3, characterized in that, After performing knowledge distillation based on the edge knowledge aggregation result, the method further includes: Calculate the loss value of the server model for each sample in the public dataset to obtain the loss calculation result; The sample with the smallest loss value in the loss calculation results is used to train the server model, and the edge model is trained based on the fifth logits knowledge output by the trained server model. The model parameters of the client model are updated based on the model parameters of the trained edge model and the model association results, and the next round of model training and knowledge transfer is carried out iteratively until the accuracy of the server model and the client model reaches the preset target value.

5. A multi-level federated learning knowledge transmission device for edge-cloud computing, characterized in that, include: The model training module is used to train the model parameters of the local client model using local data in each round of cloud communication to obtain a trained local client model. The parameter aggregation module is used to aggregate parameters based on the model parameters of the pre-trained local client models with the same structure to obtain parameter aggregation results; The knowledge transfer module is used to update the model parameters of the edge model based on the parameter aggregation results, and to transfer knowledge using a preset public dataset. The device further includes an association and collaborative training module, which is used to measure the similarity between client data distributions by utilizing the similarity between the third logits knowledge output by the trained local client model to obtain similarity measurement results. The associated collaborative training module is also used for: Based on the similarity measurement results, the data distribution similarity between any two clients is obtained to obtain the client data distribution similarity matrix; Based on the hash mapping of the client data distribution similarity matrix and the local client model structure, local client models with similar data distribution and the same model structure are subjected to parameter aggregation and associated with edge servers to obtain parameter aggregation results and model association results; Based on the parameter aggregation results and the local data, a local client model is trained, and parameter aggregation is performed using the model parameters of the trained local client model and the model association results.

6. The apparatus according to claim 5, characterized in that, The knowledge transmission module is also used for: The model parameters of the server model are trained using the knowledge of the first logits output of each sample and the marginal model in the unlabeled public dataset to obtain the trained server model. The model parameters of the edge model are trained using the model parameters of the trained server model and the second logits knowledge of each sample in the output public dataset to obtain the trained edge model; The model parameters of the client model are updated based on the model parameters of the trained edge model in order to carry out the next round of model training and knowledge transfer.

Images