Heterogeneous model aggregation method and system based on federated learning

Abstract

The invention relates to the field of federated learning, in particular to a heterogeneous model aggregation method and system based on federated learning, and the method comprises the steps of initializing a neural network model; the method also includes that each client contributes a part of local data and uploads the local data to the server to form a shared data set, and a CGAN model is trained; the client uses a local data set and a data set generated by the CGAN model to train a local model, predicts each data in the shared data set and uploads a prediction score to the server; the server calculates the prediction score deviation degree of each client, takes the reciprocal of a calculation result as a weight, calculates a global prediction score, and uses the global prediction score to perform knowledge distillation on the server model; the client downloads the prediction scores of other client models from the server for cooperative training; and model convergence is performed after multiple iterations. According to the invention, the problem of data heterogeneity of the client side can be solved, the client side model uploads and downloads the prediction score of the shared data set, and the communication traffic between the client side and the server side is reduced.

Description

technical field [0001] The invention relates to the field of federated learning, in particular to a federated learning-based heterogeneous model aggregation method and system. Background technique [0002] Today's deep learning field is developing rapidly, but deep learning has an obvious disadvantage, that is, it requires a large amount of data for training to achieve better performance. In recent years, it has become a worldwide trend to pay attention to data privacy and security. At the same time, most industry data presents the phenomenon of data islands. How to jointly train an excellent model, federated learning is the key technology to solve this problem. [0003] The development of federated learning still faces many challenges. Two of the most important aspects are heterogeneity of client models and differences in local data. Since each client is not necessarily the same and is in a different space, this leads to great differences in the communication volume, com...

AI Technical Summary

Problems solved by technology

[0005] 1. This method still does not solve the problem of client data heterogeneity. Under the condition that the local data sets of each client are not independent and identically distributed, the model performance of each client is quite different, and the fairness is poor.

[0006] 2. Only the simple average method is used to calculate the average prediction score of all client models on the shared data set, without considering the performance differences of each client model, resulting in poor performance of some client models, which will seriously affect the average prediction score quality

[0008] 1. The client still needs to upload and download model parameters to the server, and the traffic problem has not been effectively resolved

Images