Federated Learning Model Training via Prediction Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face challenges in ensuring data privacy, particularly due to concerns about the movement of model updates between user equipment and backend servers, which can compromise local data security.
Innovation Solution
A system that trains a central model using predictions from locally trained models at multiple sites without transferring local data or trained models, utilizing publicly available data samples and knowledge distillation processes to aggregate inference products, ensuring that only relevant portions of the central model are shared, thus maintaining data privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If local data is transferred to central server for model training, then model training effectiveness is improved, but data privacy is compromised
Solution Approach 1:
The patent extracts only the necessary model parameters (gradients, updates) from the local training process and transfers them to the central server, while leaving the actual local data at its source. This extraction approach enables central model training without compromising data privacy, as the raw local data never leaves the local site.
Solution Approach 2:
The patent introduces model parameters (gradients, updates) as an intermediary between local data and the central model. Instead of directly transferring local data to the central server, the system uses these intermediate representations that convey training information while preserving data privacy. The central server receives and processes these intermediaries to update the global model.
2Measurement precision
If complete central model is transmitted to each local server, then model accuracy is improved, but communication cost and time increase
Solution Approach 1:
The patent segments the model transmission process by sending only specific portions of the central model (relevant parameters, task-specific updates) to each local server rather than the complete model. This segmentation reduces communication overhead while maintaining the necessary model accuracy for local tasks.
Solution Approach 2:
The patent applies local quality by tailoring the model parameters transmitted to each local server according to its specific needs and tasks. Each local server receives customized model updates that are relevant to its local data distribution and task requirements, rather than receiving a uniform complete model, thereby optimizing communication efficiency.
3Stability of the object's composition
If model updates are frequently synchronized between local and central servers, then model consistency is improved, but communication overhead increases
Solution Approach 1:
The patent implements periodic action by synchronizing model updates at intervals rather than continuously. Local servers perform multiple local training iterations and synchronize with the central server periodically, which maintains model consistency while significantly reducing communication overhead and energy consumption compared to continuous synchronization.
Solution Approach 2:
The patent applies partial action by performing model synchronization only when necessary (when local models have converged or reached certain milestones) rather than after every training step. This approach maintains sufficient model consistency while avoiding excessive communication overhead associated with overly frequent synchronization.
Data Source
AI summary
Data samples are transmitted from a central server to at least one local server apparatus. The central server receives a set of predictions from the at least one local server apparatus that are based on the transmitted set of data samples. The central server trains a central model based on the received set of predictions. The central model, or a portion of the central model corresponding to a task of interest, can then be sent to the at least one local server apparatus. Neither local data from local sites nor trained models from the local sites are transmitted to the central server. This ensures protection and security of data at the local sites.