Federated Learning Data Rebalancing for Unbiased Local Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning models trained on biased data distributions suffer from overfitting and reduced accuracy due to skewed input and output distributions, leading to inaccurate classification of variations in training data.
Innovation Solution
Adjust local data distributions at user equipment (UEs) to match target data distribution conditions by receiving updates and adjusting the number of data instances to align with a target distribution, ensuring unbiased gradients are used for training the federated learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If federated learning models are trained on local data instances with biased distributions, then the training process can proceed with available data, but the model accuracy deteriorates due to overfitting and reduced ability to classify variations
Solution Approach 1:
The patent changes the distribution parameters of local data instances by receiving update messages from the network node that specify target distribution characteristics. The UE adjusts the number of data instances in its local dataset to match these target distribution parameters, transforming the biased data distribution into an unbiased one that aligns with the global data distribution, thereby improving model accuracy while maintaining training efficiency
Solution Approach 2:
The patent implements a feedback mechanism where the UE transmits information about its current data distribution to the network node, receives update messages with target distribution specifications, and adjusts its local data instances accordingly. This closed-loop feedback process ensures that local data distributions are continuously aligned with global distribution characteristics, resolving the contradiction between training efficiency and model accuracy
2Ease of manufacture
If the number of data instances per class is imbalanced in local datasets, then data collection is simplified, but gradient bias increases leading to poor generalization
Solution Approach 1:
The patent applies parameter changes by adjusting the number of data instances of each class in the local dataset based on update messages from the network node. The UE modifies its data instance counts to match target distribution parameters that reflect the global class balance, thereby correcting gradient bias while maintaining the simplicity of local data collection without requiring complex coordinated sampling
Solution Approach 2:
The patent segments the data distribution adjustment process into independent class-level modifications. Each class's data instance count is adjusted separately based on its specific target parameter, allowing the UE to maintain simple local data collection while achieving overall distribution balance through individual class adjustments that collectively eliminate gradient bias
Data Source
AI summary
A method for wireless communication at a network node includes receiving a first message indicating one or more distributions of a group of local data instances stored at the user equipment (UE), the group of local data instances associated with a local dataset associated with a machine learning model implemented at the UE, each one of the group of local data instances associated with a respective class of a group of classes. The method also includes transmitting, associated with receiving the first message, a second message indicating an update to the group of local data instances, based on the one or more distributions of the group of local data instances failing to satisfy one or more data distribution conditions. The method further includes receiving, associated with the update to the group of local data instances, a third message, indicating one or more first parameters associated with the machine learning model.