Federated Learning Model Inversion for Fairness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning methods face challenges in achieving fairness due to statistical heterogeneity among client devices, leading to biased models that perform poorly on minority classes.
Innovation Solution
The method involves performing model inversion on a global model at both client devices and a server to generate synthetic data, which is then used to augment datasets and train models, thereby reducing variance in performance across different client devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning focuses on learning a single global model, then the model can be trained on heterogeneous distributed networks in a privacy-preserving manner, but the model suffers from divergence due to statistical heterogeneity of data distributions across client devices
Solution Approach 1:
The patent segments the training process into local client-level training and global server-level aggregation. Each client trains on its own heterogeneous data distribution locally, then updates are aggregated at the server. This segmentation allows each segment to adapt to local data characteristics while maintaining overall system convergence through coordinated aggregation.
Solution Approach 2:
The patent applies local quality by allowing each client device to have its own local model that is optimized for its specific data distribution. Instead of forcing a single global model on all clients, each client maintains a model tailored to its local data characteristics, improving both convergence reliability and adaptability to heterogeneous distributions.
2Productivity
If federated learning uses a weighted average of local client models, then the global model can be updated, but bias is generated towards dominant data distribution, reducing fairness
Solution Approach 1:
The patent applies partial action by using only a subset of client models for aggregation in each communication round rather than requiring all clients. This selective aggregation reduces bias towards dominant distributions and improves fairness while maintaining update efficiency. The server can choose diverse clients with different data distributions to participate in each round.
Solution Approach 2:
The patent changes the aggregation parameters by using adaptive weighting schemes that account for data distribution characteristics. Instead of simple uniform or sample-size-based weighting, the system adjusts aggregation weights to compensate for dominant distributions, thereby improving performance uniformity across minority and majority classes while maintaining efficient model updates.
3Manufacturing precision
If model inversion is performed to generate synthetic data for data augmentation, then fairness and performance uniformity are improved, but computational complexity and time increase
Solution Approach 1:
The patent applies preliminary action by generating synthetic data through model inversion in advance, before the main training process. The synthetic data is pre-generated and stored, then used during training to augment datasets. This preliminary preparation reduces the computational burden during actual training iterations and improves performance uniformity without excessively increasing total training time.
Solution Approach 2:
The patent uses copying by generating synthetic copies of data through model inversion. Instead of collecting and processing additional real data, the system creates synthetic copies that mimic the statistical properties of real data. These copied synthetic samples are then used to augment training datasets, improving fairness while avoiding the time cost of collecting and processing real diverse data.
Data Source
AI summary
Methods and devices are provided for performing federated learning. A global model is distributed from a server to a plurality of client devices. At each of the plurality of client devices: model inversion is performed on the global model to generate synthetic data; the global model is on an augmented dataset of collected data and the synthetic data to generate a respective client model; and the respective client model is transmitted to the server. At the server: client models are received from the plurality of client devices, where each client model is received from a respective client device of the plurality of client devices; model inversion is performed on each client model to generate a synthetic dataset; the client models are averaged to generate an averaged model; and the averaged model is trained using the synthetic dataset to generate an updated model.