Federated Model Training Using Gradient-Guided Correction Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning in distributed machine learning faces challenges with poor stability and slow model convergence speed due to different data distribution forms across devices, leading to high communication overheads and numerous training rounds.
Innovation Solution
A model training method where a training device sends a gradient to multiple computing devices, which process data to generate correction data based on the same training direction, allowing the training device to update the gradient and train the model efficiently without considering individual data organization forms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning is used to train models on distributed devices with different data distribution forms, then model training can be performed without transmitting private data, but model convergence speed becomes slow and communication overhead increases
Solution Approach 1:
The patent introduces a training direction indicator (gradient) as an intermediary that guides all computing devices to process their local data in a unified direction. This mediator aligns the diverse data distributions across devices without requiring data transmission, thereby maintaining privacy while improving convergence speed by reducing training rounds.
2Reliability
If federated learning is used to train models on distributed devices with different data distribution forms, then model training can be performed without transmitting private data, but communication overhead becomes high due to multiple training rounds
Solution Approach 1:
The gradient as a training direction indicator serves as a mediator that coordinates data processing across distributed devices. By aligning all devices to process data in the same direction guided by this intermediary, the system achieves faster convergence with fewer training rounds, thereby reducing communication overhead while maintaining data privacy.
3Adaptability or versatility
If different data organization forms are processed independently in federated learning, then device autonomy is maintained, but model training stability deteriorates
Solution Approach 1:
The patent applies local quality by allowing each computing device to process its local data in its own organization form (maintaining adaptability) while being guided by a global training direction indicator (ensuring stability). Each device maintains its local data processing characteristics while aligning with the overall training objective, resolving the contradiction between flexibility and stability.
Data Source
AI summary
This application discloses a model training method and apparatus, and a device, and relates to the field of machine learning technologies. Because different computing devices have different data organization forms, a training device obtains correction data determined by each computing device based on a same training direction (a first gradient), so that the training device does not need to consider different data organization forms when training a model based on the correction data. This avoids a problem of poor stability of model training. In addition, all different computing devices run the model and output the correction data based on the same training direction. This helps the training device obtain a more accurate model training direction, thereby reducing a quantity of rounds of model training, and also reducing a quantity of times of communication between the training device and the computing devices.