Semi-Asynchronous Federated Learning Model Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional centralized learning architectures face limitations due to high communication and computing requirements, latency issues, and data privacy and security concerns, which hinder real-time decision-making and efficient data utilization in edge devices, while synchronous and asynchronous federated learning frameworks suffer from low training efficiency and unstable convergence.
Innovation Solution
A semi-asynchronous federated learning method that involves a computing node sending parameters to subnodes, receiving local models, and fusing them based on a model fusion algorithm triggered by a threshold, such as a count or time threshold, to generate a global model, thereby optimizing resource allocation and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous federated learning is used, then model convergence stability is improved, but training efficiency deteriorates due to waiting for all subnodes
Solution Approach 1:
The system dynamically adjusts the synchronization mechanism by introducing a configurable threshold parameter. When the number of received local models reaches the threshold, the computing node performs model fusion immediately without waiting for all subnodes, making the system adaptable between synchronous and asynchronous behaviors based on real-time conditions
Solution Approach 2:
The threshold parameter is adjusted to control the trade-off between convergence stability and training efficiency. By changing this parameter, the system can optimize performance for different scenarios, achieving fast convergence while maintaining stability
2Productivity
If asynchronous federated learning is used, then training efficiency is improved, but model convergence stability deteriorates due to update upon reception
Solution Approach 1:
The system transitions from purely asynchronous to a dynamic semi-asynchronous mode where the decision to fuse models is based on reaching a threshold rather than immediate fusion upon receiving each model, providing dynamic control over convergence behavior
Solution Approach 2:
The computing node monitors the number of received local models and uses this feedback to determine when to perform model fusion. This feedback mechanism ensures that fusion occurs at appropriate moments, maintaining convergence stability while preserving training efficiency
3Adaptability or versatility
If centralized learning architecture is used, then data utilization is improved, but communication and computing requirements increase significantly
Solution Approach 1:
The system segments the learning process into local training phases at subnodes and global fusion phases at the computing node. Each subnode independently trains local models on its own data, segmenting the computational workload and reducing communication requirements while maintaining effective data utilization
Solution Approach 2:
The system extracts only the essential model parameters and gradients needed for global model improvement, rather than transmitting all raw data. This extraction approach enables effective data utilization while significantly reducing communication overhead and computing requirements
Data Source
AI summary
This application provides a method for federated learning. A communication apparatus triggers, by setting a threshold (a time threshold and/or a count threshold), fusion of a local model sent by a terminal device, to generate a global model, and when a fusion weight of the local model is designed, a data feature included in the local model of the terminal device, a lag degree, and a utilization degree of a data feature of a sample set of the corresponding terminal device are comprehensively considered, so that a problem of low training efficiency caused by a synchronization requirement for model uploading versions in a synchronous system can be avoided, and a problem of unstable convergence and a poor generalization capability caused by an “update upon reception” principle of an asynchronous system can be avoided.