Training Instance Influence Scoring for Post-Hoc Fairness Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bias mitigation algorithms in machine learning models often require retraining or are computationally expensive, and fail to provide control over the fairness-accuracy trade-off, especially when dealing with pre-trained models.
Innovation Solution
A system that calculates a fairness influence score for training instances using a WoodFisher-based Inverse-Hessian Vector Product (IHVP) scheme, allowing post-hoc unfairness mitigation by removing the effect of biased training instances without refitting the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bias mitigation algorithms are applied to pre-trained models, then fairness is improved, but computational cost increases due to refitting requirements
Solution Approach 1:
The patent computes the influence of training instances on model predictions in advance using the WoodFisher approximation during a preliminary phase. This allows the system to identify and remove biased training instances before final model deployment, avoiding the need for expensive refitting operations while achieving fairness improvements.
Solution Approach 2:
The patent replaces the mechanical process of refitting the entire model (which is computationally expensive) with a substitution approach using the WoodFisher approximation. This mathematical substitution allows the system to estimate the impact of individual training instances without performing full model retraining, significantly reducing computational costs.
2Reliability
If training instances are removed to mitigate bias, then group disparities are reduced, but model accuracy may deteriorate
Solution Approach 1:
The patent applies local quality by selectively removing only the most influential biased training instances rather than removing all training instances uniformly. The WoodFisher influence scores allow the system to identify specific local problems (biased instances) and address them individually, preserving the overall quality and accuracy of the model while improving fairness.
Solution Approach 2:
The patent changes the parameter of training instance selection by using influence scores to determine which instances to remove. Instead of random or uniform removal, the system adjusts the composition of the training set based on calculated influence parameters, optimizing the trade-off between fairness and accuracy by removing only the most harmful instances.
3Reliability
If existing bias mitigation algorithms are used, then fairness is addressed, but control over fairness-accuracy trade-off is lost
Solution Approach 1:
The patent introduces dynamics by allowing the system to adaptively select the number and type of training instances to remove based on the specific fairness-accuracy trade-off requirements. The influence scores provide a dynamic mechanism to adjust the degree of bias mitigation, enabling the system to optimize for fairness when needed while maintaining accuracy when important, giving users control over the trade-off.
Solution Approach 2:
The patent implements feedback by using the WoodFisher influence scores to identify and remove biased instances, then evaluating the impact on model performance. This feedback loop allows the system to adjust the bias mitigation strategy based on observed outcomes, providing continuous control over the fairness-accuracy trade-off and enabling iterative optimization.
Data Source
AI summary
One or more systems, devices, computer program products and/or computer implemented methods of use provided herein relate to a process of mitigating biased training instances associated with a machine learning model without additional refitting of the machine learning model. A system can comprise a memory that stores computer executable components, and a processor that executed the computer executable components stored in the memory, wherein the computer executable components can comprise a training data influence estimation component and an influence mitigation component. The training data influence estimation component can receive a pre-trained machine learning model and calculate a fairness influence score of training instances on group fairness metrics associated with the pre-trained machine learning model. The influence mitigation component can perform post-hoc unfairness mitigation by removing the effect of at least one training instance based on the fairness influence score to mitigate biased training instances without refitting the pre-trained machine learning model.