A prediction-based federated learning communication optimization method and system

An optimization method and federated technology, applied in prediction, transmission system, structured data retrieval, etc., can solve the problems of reducing the accuracy of training models, multi-local computing resources, and reducing model accuracy, so as to facilitate efficient implementation and reduce computing Complexity, the effect of reducing accuracy

Active Publication Date: 2022-02-01
CHONGQING UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although these two types of methods can improve the communication efficiency of federated learning to a certain extent, they still have the following shortcomings: the method based on model convergence usually consumes more local computing resources, however, in the federated learning environment , the terminals are usually resource-constrained heterogeneous devices, they do not have enough computing resources to handle the training of complex models, therefore, it is challenging to apply this algorithm to the federated communication optimization of actual scenarios; the importance-based method , the importance or relevance of local updates is judged by an adjustable threshold, and the setting of this threshold is usually based on the goal of maximizing the reduction of communication times. Severe reduction in model accuracy
However, due to the high-dimensionality of model training parameters and the unreliability of the network in the federated learning environment, the communication cost problem has become a basic and important problem in federated learning.
Although the existing research methods have proposed many effective communication optimization methods from the aspects of reducing the amount of communication and the number of communication rounds, they are usually accompanied by other deficiencies, such as the need to consume more local computing resources or severely reduce the training time. Therefore, in order to better solve the high communication cost problem of federated learning, it is necessary to design a method that does not need to consume more local computing resources, but also can greatly reduce the number of communication rounds required while ensuring the accuracy of the training model. rate method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A prediction-based federated learning communication optimization method and system
  • A prediction-based federated learning communication optimization method and system
  • A prediction-based federated learning communication optimization method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0105] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings and embodiments, so as to fully understand the purpose, features and effects of the present invention.

[0106] The specific implementation steps of the present invention will be described below by taking 100 end users jointly training a linear regression model as an example. The expression of the linear regression model is Among them, |k| represents the number of training samples, W represents the training model parameter vector, and X represents the feature vector of the training samples.

[0107] The method provided by the technical solution of the present invention can adopt computer software technology to realize the automatic operation process, figure 1 is the overall method flowchart of the embodiment of the present invention, see figure 1 , combined with figure 2 The specific step flowchart of the emb...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the field of federated machine learning, and discloses a prediction-based federated learning communication optimization method and system. In the present invention, first, the global model and the global variables required in the present invention are initialized, and each user performs local model training according to its local data to obtain local updates. Subsequently, the cloud center predicts its local update according to each user's historical model update trend respectively. Then, by calculating the change of the loss function of the global model when each user adopts its prediction update, the prediction error threshold is set, which includes two steps of initial threshold and dynamic threshold setting. Finally, a global model update strategy is designed according to the set prediction error threshold, and the cloud center uses accurate prediction updates instead of local updates to calculate global model updates. It solves the problem of high communication costs caused by frequent transmission of update parameters between end users and cloud centers in federated learning technology.

Description

technical field [0001] The present invention relates to the field of federated machine learning, and more specifically, to a prediction-based federated learning communication optimization method, which is used to solve the problem of high communication costs caused by frequent transmission of update parameters between end users / devices and cloud centers in federated learning technology. Background technique [0002] As an important branch of artificial intelligence, machine learning has been successfully and widely used in various fields such as pattern recognition, data mining and computer vision. Due to the limited computing resources of terminal devices, the current training of machine learning models usually adopts a cloud-based method. In this method, the data collected by terminal devices, such as pictures, videos, or personal location information, must be uploaded to the cloud center Centralize the training of the model. However, uploading the user's real data will r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06Q10/04G06K9/62G06N3/04G06F16/23
CPCG06Q10/04G06F16/23H04L67/10G06N3/045G06F18/214
Inventor 李开菊梁杰银肖春华
Owner CHONGQING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products