Federated Learning Single-Round Convergence via Weighted Voting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning methods require multiple rounds of training and communication of model weights, which can be inefficient and compromise data privacy, especially when dealing with large datasets and diverse data sources across different nodes.
Innovation Solution
The implementation of a federated learning system with single-round convergence using a network of federated models that employ knowledge transfer and weighted voting to aggregate models across multiple tiers, allowing for industry-specific models to be created while preserving differential privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple rounds of training and communication are used in federated learning, then model accuracy can be improved, but training time and communication overhead increase significantly
Solution Approach 1:
The patent applies preliminary action by performing local model training at each node before aggregation. Each node trains its model locally using local data in advance, and then the pre-trained models are aggregated at the federation level. This allows the system to leverage local computational resources and data without requiring multiple iterative communication rounds, thereby reducing training time while maintaining model accuracy.
Solution Approach 2:
The patent segments the federated learning process into independent local training phases and centralized aggregation phases. Each node independently trains its model locally, and then these segmented local models are aggregated at the federation level. This segmentation eliminates the need for multiple iterative communication rounds between nodes and the central server, significantly reducing communication overhead and training time.
2Reliability
If multiple rounds of weight communication are performed, then model convergence can be achieved, but data privacy risks increase due to repeated exposure
Solution Approach 1:
The patent extracts only the necessary model parameters (weights and biases) for aggregation while leaving the sensitive training data localized at each node. By taking out only the model parameters rather than the raw data, the system achieves model convergence through aggregation while minimizing privacy risks associated with data exposure during communication rounds.
Solution Approach 2:
The patent performs preliminary local model training at each node before aggregation, so that the models are already converged to a certain extent before being combined. This preliminary action reduces the need for multiple iterative aggregation rounds, thereby limiting the number of times model parameters need to be communicated and reducing cumulative privacy risks.
3Adaptability or versatility
If federated learning is applied across multiple nodes with diverse data, then learning from diverse data sources is enabled, but communication overhead and computational load increase
Solution Approach 1:
The patent segments the federated network into multiple independent nodes that each perform local model training on their own diverse data sources. These segmented nodes then contribute their locally trained models to the federation for aggregation. This segmentation allows the system to leverage diverse data across nodes while avoiding the need for continuous communication during training, thereby reducing communication overhead.
Solution Approach 2:
Each node in the federation performs self-service by independently training its own model using local data and computational resources. This self-service approach allows nodes to contribute diverse data learnings without requiring constant coordination or communication with other nodes during the training process, reducing overall system complexity and communication overhead.
Data Source
AI summary
A method including classifying a cluster training dataset by, for each datapoint of the cluster training dataset: obtaining a student prediction from each of a plurality of respective student machine-learning models for each a plurality of nodes of a cluster; performing a voting of the student predictions from the plurality of respective student machine-learning models of at least a portion of the plurality of nodes of the cluster to determine a respective classification for the datapoint; and labeling the datapoint of the cluster training dataset with the respective classification. The method also can include training a cluster machine-learning model of the cluster using the cluster training dataset, as classified. Other embodiments are described.