Training method of personalized model of distillation-based semi-supervised federated learning

Abstract

The invention discloses a training method of a personalized model of distillation-based semi-supervised federated learning, which adopts a knowledge distillation technology, and a client side can select a self-designed model architecture by uploading model prediction rather than model parameters, so that privacy information of the client side about the model is well protected, and the shared data and the local data of the client are used for training together, so that the generalization ability of the model is greatly improved. In addition, the aggregation scheme can perform dynamic aggregation according to the importance degree of knowledge provided by each client, so that the aggregated model prediction better fuses the model knowledge of the client. And after the server finishes aggregation, the model prediction distribution information of the public data is not the pseudo label information, so that the communication transmission efficiency is further improved by utilizing the mode.

Description

technical field [0001] The invention relates to the technical field of federated learning, in particular to a method for training a personalized model based on distillation-based semi-supervised federated learning. Background technique [0002] Federated learning collaboratively trains a global model on the premise that a group of clients do not upload local data sets, and each user can only access their own data, thereby protecting the privacy of users participating in the training. Because of its advantages, federated learning has broad application prospects in industries such as medicine, finance, and artificial intelligence, and has become a research hotspot in recent years. However, federated learning focuses on obtaining a high-quality global model by learning the local data of all participating clients, but since the data of each client is heterogeneous in real scenarios, it cannot train a model when faced with data heterogeneity. Global model for all clients. [00...

AI Technical Summary

Problems solved by technology

[0004] However, to apply knowledge distillation technology to federated learning, it is necessary to ensure that the distillation is performed on the same data set, and the local data of each client in federated learning is different, so how to construct the same data set on the client to achieve distillation is a problem. problem

In federated learning, clients with different data have different importance of the knowledge provided by the model output due to the uneven data quality, so simply averaging it is not an effective aggregation method

Images