Federated Learning Server Class Imbalance Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, the server lacks direct knowledge of class-label distribution in client training data, leading to potential class imbalance issues that affect classification accuracy, and existing methods may invade client privacy by estimating distributions directly.
Innovation Solution
A method where the server estimates class imbalance in training data using connection weights, bias terms, and probability distributions transmitted by the server, allowing for action on clients with significant imbalance without accessing their data, thus maintaining privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the server estimates class imbalance using direct distribution estimation methods, then classification accuracy can be improved, but client privacy is violated
Solution Approach 1:
The patent introduces an intermediary estimation approach where the server infers class imbalance distributions indirectly through federated learning parameter updates rather than directly accessing client data. The server uses the relationship between model parameter changes and class distributions as a mediator to obtain imbalance information without violating privacy.
Solution Approach 2:
The patent replaces the traditional mechanical approach of direct data access with a mathematical inference system. By substituting direct distribution estimation with inference based on federated learning parameter dynamics, the system achieves the same goal (class imbalance detection) without the privacy-invasive mechanism.
2Loss of information
If the server has no knowledge of class imbalance in client data, then client privacy is maintained, but classification accuracy deteriorates due to unaddressed class imbalance
Solution Approach 1:
The patent implements a feedback mechanism where the server continuously monitors federated learning parameter updates to infer class imbalance conditions. This feedback loop allows the server to detect imbalance issues and adjust the training process accordingly, maintaining both privacy and accuracy.
Solution Approach 2:
The system enables the server to self-diagnose class imbalance conditions by analyzing its own received parameter updates from federated learning. Rather than requiring external information about client data distributions, the server uses the learning process itself to generate the necessary diagnostic information.
3Device complexity
If traditional federated learning is used without class imbalance estimation, then system simplicity is maintained, but model training performance deteriorates due to class imbalance
Solution Approach 1:
The patent performs preliminary estimation of class imbalance conditions during the federated learning process itself, before final model deployment. By integrating the estimation into the existing training workflow, the system addresses class imbalance without requiring separate complex preprocessing or postprocessing steps.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for estimating class-imbalance in training data of an iterative learning process comprises transmitting a global parameter vector of the computational task and a vector of the bias terms for a current iteration round of the iterative learning process to the client entities. The method comprises receiving, per client entity and for the current global iteration round, a local model parameter vector with locally computed computational results, connection weights of the connection matrix, and an updated vector of the bias terms. The method comprises estimating the class-imbalance in the local training data per each given client entity as a function of the connection weights, the bias terms, a probability distribution is a function of the bias terms as transmitted by the server entity, the number of local iteration rounds where all locally available training data was used, and the learning rate. The method comprises performing an action for any of the client entities for which the class-imbalance is larger than a threshold.