Federated learning method, device and system based on gradient compression

Abstract

The invention discloses a federated learning method, device and system based on gradient compression. A federated server sends initial models to clients in a unified mode; the initial model is trainedby the client to obtain an updated gradient, the updated gradient is quantized to obtain a quantized gradient of the updated gradient, and the quantized gradient of the updated gradient is sent to afederated server; the federated server performs quantity statistics according to different quantized values in the quantized gradients based on all the quantized gradients corresponding to each clientto obtain a statistical result, and returns the statistical result to each client; and the client receives the statistical result sent by the federated server and calculates a global updated gradientaccording to the statistical result so as to continue model training according to the global updated gradient. The objective is to solve the problems of high network transmission overhead and unsafedata in an existing federated learning system.

Description

technical field [0001] The present application relates to the technical field of machine learning, in particular, to a method, device and system for federated learning based on gradient compression. Background technique [0002] Federated learning is a learning method in which data is distributed under different entities. In a federated learning system, the data is distributed on different clients, and the federated server and the client initialize the same model (such as a neural network model) and the same initial parameters of the model. The client first trains on the local data set to obtain the gradient of the model update (the gradient of the model parameters), and then each client entity sends the gradient to the federated server, and the federated server collects the updated gradients of all client entities, and after averaging Return the obtained global gradient to each client entity, so that each client entity can train the model. [0003] In the process of apply...

AI Technical Summary

Problems solved by technology

[0004] The main purpose of this application is to provide a method, device and system for federated learning based on gradient compression to solve the problems of high network transmission overhead and data insecurity in existing federated learning systems

Images