Federated learning method, device and system

Abstract

The invention discloses a federated learning method, device and system. A federated server transmits an initial model to each client in a unified manner, and the client carries out the training of a model based on the local data of the client after receiving the initial model which is transmitted to each client by the federated server in a unified manner, and obtains an updated gradient; the client sends the updated gradient to the federated server; the federated server carries out aggregation processing on the updated gradients sent by the clients after receiving the updated gradients to obtain a global updated gradient, and carries out singular value decomposition on the global updated gradient; and the client receives the global updated gradient after singular value decomposition sent by the federated server, calculates the global updated gradient according to the global updated gradient after singular value decomposition, and continues to perform model training according to the global updated gradient. The objective is to solve at least one of the problems of high network transmission overhead and unsafe data in an existing federated learning system.

Description

technical field [0001] This application relates to the technical field of machine learning, in particular, to a method, device and system for federated learning. Background technique [0002] Federated learning is a learning method in which data is distributed under different entities. In a federated learning system, the data is distributed on different clients, and the federated server and the client initialize the same model (such as a neural network model) and the same initial parameters of the model. The client first trains on the local data set to obtain the gradient of the model update (the gradient of the model parameters), and then each client entity sends the gradient to the federated server, and the federated server collects the updated gradients of all client entities and aggregates them ( After averaging, summing, etc.), the obtained global gradient is returned to each client entity, so that each client entity can perform model training. [0003] In the process...

AI Technical Summary

Problems solved by technology

[0004] The main purpose of this application is to provide a federated learning method, device and system to solve at least one of the problems of high network transmission overhead and data insecurity in the existing federated learning system

Images