Federated learning system based on heterogeneous data

Abstract

The invention relates to a federated learning system based on heterogeneous data, which comprises a central server, K clients, a memory in which a computer program is stored and a processor, and is characterized in that the central server stores a global control variable S and a global model parameter W obtained by each round of federated learning; the global control variable S is used for recording the global model updating direction of the round; the client stores a local control variable Si obtained by each participating client by participating in federated learning every time, the local control variable Si is used for recording the updating direction of a local model of the client participating in federated learning training this time, and the value of i is 1-K.Communication cost of federated learning based on heterogeneous data is reduced, and the convergence speed and the convergence stability of federated learning are improved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a federated learning system based on heterogeneous data. Background technique [0002] Federated learning is a special distributed learning framework. It is mainly used to solve the problem of data islands. In federated learning, data exists on each local client, and all data cannot be gathered in one place for traditional centralized learning. The Federated Averaging Algorithm (FedAvg for short) in the existing federated learning has become the preferred optimization algorithm in the field of federated learning because of its simplicity and low communication overhead. The existing federated averaging algorithm is suitable for independent and identically distributed client data sets. However, on heterogeneous data sets, the federated averaging algorithm will have slow convergence speed and unstable convergence due to differences in client data distribution. A series of techn...

AI Technical Summary

Problems solved by technology

The existing federated averaging algorithm is suitable for independent and identical distribution of client data sets. However, on data sets with strong heterogeneity, the federated averaging algorithm will have slow convergence speed and unstable convergence due to differences in client data distribution. A series of technical problems such as high communication costs and even inability to converge are mainly due to the fact that the federated averaging algorithm has been updated locally many times, and this method will cause the parameter update direction of the global model to deviate from the ideal update direction, which will lead to slow convergence

Images