A prediction-based federated learning communication optimization method and system

Abstract

The invention relates to the field of federated machine learning, and discloses a prediction-based federated learning communication optimization method and system. In the present invention, first, the global model and the global variables required in the present invention are initialized, and each user performs local model training according to its local data to obtain local updates. Subsequently, the cloud center predicts its local update according to each user's historical model update trend respectively. Then, by calculating the change of the loss function of the global model when each user adopts its prediction update, the prediction error threshold is set, which includes two steps of initial threshold and dynamic threshold setting. Finally, a global model update strategy is designed according to the set prediction error threshold, and the cloud center uses accurate prediction updates instead of local updates to calculate global model updates. It solves the problem of high communication costs caused by frequent transmission of update parameters between end users and cloud centers in federated learning technology.

Description

technical field [0001] The present invention relates to the field of federated machine learning, and more specifically, to a prediction-based federated learning communication optimization method, which is used to solve the problem of high communication costs caused by frequent transmission of update parameters between end users / devices and cloud centers in federated learning technology. Background technique [0002] As an important branch of artificial intelligence, machine learning has been successfully and widely used in various fields such as pattern recognition, data mining and computer vision. Due to the limited computing resources of terminal devices, the current training of machine learning models usually adopts a cloud-based method. In this method, the data collected by terminal devices, such as pictures, videos, or personal location information, must be uploaded to the cloud center Centralize the training of the model. However, uploading the user's real data will r...

AI Technical Summary

Problems solved by technology

Although these two types of methods can improve the communication efficiency of federated learning to a certain extent, they still have the following shortcomings: the method based on model convergence usually consumes more local computing resources, however, in the federated learning environment , the terminals are usually resource-constrained heterogeneous devices, they do not have enough computing resources to handle the training of complex models, therefore, it is challenging to apply this algorithm to the federated communication optimization of actual scenarios; the importance-based method , the importance or relevance of local updates is judged by an adjustable threshold, and the setting of this threshold is usually based on the goal of maximizing the reduction of communication times. Severe reduction in model accuracy

However, due to the high-dimensionality of model training parameters and the unreliability of the network in the federated learning environment, the communication cost problem has become a basic and important problem in federated learning.

Although the existing research methods have proposed many effective communication optimization methods from the aspects of reducing the amount of communication and the number of communication rounds, they are usually accompanied by other deficiencies, such as the need to consume more local computing resources or severely reduce the training time. Therefore, in order to better solve the high communication cost problem of federated learning, it is necessary to design a method that does not need to consume more local computing resources, but also can greatly reduce the number of communication rounds required while ensuring the accuracy of the training model. rate method

Images