Random-Sampling Consensus for Non-IID Semi-Supervised Federated Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing federated learning methods struggle with non-IID data distributions among local clients, leading to performance deterioration and information leakage, particularly in semi-supervised learning scenarios where unlabeled data is utilized.
Innovation Solution
The RSCFed method employs random sampling consensus learning, using distance-reweighted model aggregation (DMA) to dynamically adjust weights based on model distances and incorporate both labeled and unlabeled clients, enhancing model robustness through multiple sub-consensus model aggregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If existing FSSL methods share correlation matrix or equally average model weights among local clients, then model aggregation is simplified, but performance accuracy deteriorates due to Non-IID data distribution and information leakage
Solution Approach 1:
The patent implements dynamic aggregation weights that adapt based on client data characteristics and model performance. Instead of fixed equal weighting, the system calculates weights dynamically considering the Non-IID data distribution across clients, allowing the aggregation process to respond to varying data qualities and client contributions, thereby resolving the contradiction between simplified aggregation and accurate performance
Solution Approach 2:
The patent applies local quality by treating each client's data distribution and model characteristics individually. Rather than assuming uniform data quality across all clients, the system evaluates and weights each client's contribution based on its local data properties, preventing information leakage from clients with problematic Non-IID distributions while preserving valuable local knowledge
2Reliability
If increase weights for labeled clients in aggregation, then model accuracy improves, but the system becomes biased toward clients with labeled data and reduces ability to utilize unlabeled data effectively
Solution Approach 1:
The patent dynamically adjusts aggregation weights based on multiple parameters including data labels, model performance metrics, and data distribution characteristics. This multi-parameter weighting system allows flexible adaptation where labeled clients receive appropriate weight for accuracy while unlabeled clients contribute through consistency regularization and other unsupervised learning signals, maintaining both accuracy and versatility
Solution Approach 2:
The patent applies partial action by having different client types contribute differently to the global model. Labeled clients provide supervised learning signals with higher weight for accuracy-critical updates, while unlabeled clients provide unsupervised consistency signals that complement the supervised learning without dominating the training process, achieving both accuracy improvement and effective unlabeled data utilization
3Productivity
If use standard federated averaging for aggregation, then communication efficiency is maintained, but model robustness decreases under Non-IID settings with mixed labeled and unlabeled data
Solution Approach 1:
The patent maintains communication efficiency through periodic federated averaging cycles while introducing dynamic weight adjustment within each communication round. The system performs multiple local aggregation steps with adaptive weights before a single communication round to the central server, achieving robustness improvement without proportionally increasing communication overhead
Solution Approach 2:
The patent segments the aggregation process into multiple stages: local client-model aggregation with dynamic weights, sub-consensus model formation, and global model aggregation. This segmentation allows robustness-critical local adaptations to occur without requiring constant server communication, maintaining productivity while improving reliability through structured multi-stage aggregation
Data Source
AI summary
A method and systems for random sampling consensus federated (RSCFed) learning in non-IID settings are provided. The method includes randomly sampling local clients, assigning a current global model to the randomly sampled local clients for initialization at beginning of a synchronization round, conducting local training on the randomly sampled local clients, collecting local models from the randomly sampled local clients and executing distance-reweighted model aggregation (DMA) on the collected local models to obtain a sub-consensus model, repeating above steps multiple times to obtain a set of sub-consensus models, and aggregating a new model based on the sub-consensus models to be next global model.