Serial Federation for Efficient Non-IID Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional federated learning schemes face inefficiencies in communication cost and low accuracy due to non-IID data structures, and existing methods like local and central learning suffer from biased predictions, data privacy concerns, and catastrophic forgetting.
Innovation Solution
A method called serial federation, where a global model is trained in a cyclic manner by client devices using partitioned local datasets, with a central server managing the process to ensure robustness, security, and computational efficiency through data partitioning and random client sequence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional federated learning schemes based on gradient model update are used, then model training can be performed across multiple devices, but communication cost is high and accuracy performance is low on non-IID data
Solution Approach 1:
The patent segments the global model training process into multiple local training phases, where each client device trains the model independently on its local non-IID data for several epochs before communicating updates. This segmentation reduces the frequency of communication while maintaining model accuracy by allowing local convergence before global aggregation.
Solution Approach 2:
The patent performs preliminary local training actions at each client device before participating in global model aggregation. Clients pre-train the model on their local non-IID data, performing necessary computations and adjustments locally, which reduces the need for frequent communication and improves overall training efficiency.
2Reliability
If local learning is performed separately at each device, then data privacy is maintained, but model generalization capability deteriorates and predictions become biased
Solution Approach 1:
The patent merges multiple local models trained on different non-IID data distributions through federated averaging. By combining the knowledge from diverse local models while maintaining data locality, the global model achieves better generalization capability and reduced prediction bias compared to purely local learning.
3Reliability
If central learning aggregates all data to a single server, then model accuracy improves through sufficient data, but data privacy and security concerns increase
Solution Approach 1:
The patent introduces an intermediary federated learning system that enables collaborative model training without direct data sharing. The server acts as a mediator that coordinates training, aggregates model updates, and maintains a global model, while client devices keep their data locally. This intermediary architecture achieves sufficient data diversity for accurate modeling while preserving data privacy and security.
4Reliability
If federated learning periodically aggregates local models at a central server, then data privacy is preserved, but communication overhead increases and convergence slows
Solution Approach 1:
The patent implements periodic local training phases at each client device before model aggregation. Instead of continuous communication, clients perform training in periodic intervals, updating the global model less frequently but with more substantial local improvements. This periodic action reduces communication overhead while maintaining training effectiveness.
5Loss of energy
If cyclic institutional incremental learning is used for decentralized training, then communication cost reduces, but robustness against client errors and catastrophic forgetting increases
Solution Approach 1:
The patent incorporates feedback mechanisms where the central server monitors model updates from client devices, validates convergence, and provides guidance for handling non-IID data. This feedback loop enables the system to detect and correct potential errors, prevent catastrophic forgetting through proper aggregation strategies, and maintain robustness while reducing communication costs through efficient update protocols.
Data Source
AI summary
The computer-implemented method for computationally efficient machine learning comprises a) generating, by the server computing device, a set of initial model parameters; b) generating, by the server computing device, a set of partitioning parameters including at least the parameters Number of partitions and Minimum number of data points in the partitions; c) communicating, by the server computing device, the set of initial model parameters and the partitioning parameters to each client computing device; d) obtaining, by each client computing device, global values corresponding to the set of initial parameters from the server computing device; e) selecting, by the server computing device, a client computing device based on a global sequence of all client computing devices; f) communicating, by the server computing device, machine learning-based model parameters to the selected client computing device; g) obtaining, by the selected client computing device, global values corresponding to the machine learning-based model parameters; h) incrementally training, by the selected client computing device, the machine-learned model based, at least in part, on a partition of the local dataset to obtain updated machine learning-based model parameters, said partition being selected from the local dataset according to a predefined scheme; i) communicating, by the selected client computing device, the updated machine learning-based model parameters to the server computing device; j) repeating steps d)-i) until all predefined conditions are satisfied.