Machine Learning Label Bias Correction via Training Data Re-weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning approaches struggle to effectively reduce bias in classification models, particularly in resource-constrained environments, as post-processing techniques require additional resources and constrained optimization methods are unstable and complex.
Innovation Solution
A computer-implemented method that re-weights training examples based on fairness constraints, such as demographic parity and equal opportunity, to correct bias in machine-learned classification models by iteratively updating weights and re-training the model, using Lagrange multipliers and exponential weighting to normalize weights, thereby aligning with true, unbiased labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If post-processing techniques are used to enforce fairness on a learned model, then fairness constraints are satisfied, but predictive accuracy deteriorates and additional computational resources are required
Solution Approach 1:
The patent applies preliminary action by incorporating fairness constraints directly into the training algorithm before the model is deployed. Rather than correcting bias after training through post-processing, the method proactively addresses fairness during the training phase by modifying the loss function to include fairness constraints, thereby preventing bias from being learned in the first place.
Solution Approach 2:
The patent employs parameter changes by modifying the training objective function to include fairness constraints as additional parameters. The loss function is transformed from a simple accuracy-based objective to a composite objective that balances predictive accuracy with fairness metrics, allowing the model to learn both goals simultaneously through gradient-based optimization.
2Reliability
If post-processing calibration operations are performed to enforce fairness, then fairness is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent extracts the fairness enforcement mechanism from the post-processing stage and integrates it into the training stage. By removing the need for separate calibration operations after training, the method eliminates the additional complexity and resource requirements that would otherwise be needed for post-processing fairness correction.
3Reliability
If constrained optimization approaches are used to incorporate fairness into training, then fairness constraints are addressed, but training stability deteriorates and convergence becomes difficult
Solution Approach 1:
The patent transforms the constrained optimization problem into an unconstrained optimization problem by incorporating fairness constraints directly into the loss function through Lagrange multipliers. This parameter transformation allows standard gradient-based optimization algorithms to be used, maintaining training stability and convergence properties while still enforcing fairness constraints.
4Reliability
If additional rounds of training are performed to achieve convergence in constrained optimization, then fairness constraints are satisfied, but processing resources and time are increased
Solution Approach 1:
The patent performs preliminary action by integrating fairness constraints into the training objective from the beginning, allowing the model to learn fairness-aware representations in a single training pass. This eliminates the need for multiple alternative rounds of training that would otherwise be required to achieve convergence in constrained optimization approaches.
Data Source
AI summary
The present disclosure is directed to systems and methods for identifying and correcting label bias in machine learning via intelligent re-weighting of training examples. In particular, aspects of the present disclosure leverage a problem formulation which assumes the existence of underlying, unknown, and unbiased labels which are overwritten by an agent who intends to provide accurate labels but may have biases towards certain groups. Despite the fact that a biased training dataset provides only observations of the biased labels, the systems and methods described herein can nevertheless correct the bias by re-weighting the data points without changing the labels.