Classification Model Bias Correction via Record Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing classification models face challenges in accurately determining bias and fairness, as current metrics may not be effective across various scenarios, leading to potential unfair treatment of different groups.
Innovation Solution
A method that involves perturbing original records to generate perturbed records, calculating confidence values for both, determining a final confidence value using the direction of distance traveled, and re-training the model based on biased records to correct bias, utilizing a computer-implemented system to assess and improve model fairness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing fairness metrics (parity metric, equality of opportunity metric, equality of odds metric, bounded error loss metric) are used to measure classification model fairness, then the model behavior can be evaluated, but the metrics are not accurate across different scenarios
Solution Approach 1:
The patent changes the parameters used for fairness measurement from traditional metrics (parity, equality of opportunity, equality of odds, bounded error loss) to a new parameter based on confidence value differences. This parameter change enables accurate fairness assessment across different scenarios by focusing on how confidence values change when protected attributes are modified, rather than relying on scenario-specific metric definitions
Solution Approach 2:
The patent creates a universal fairness measurement approach that works across all classification scenarios by using confidence value differences as a common metric. The method can evaluate fairness for any classification model and any protected attribute (race, gender, age, etc.) without requiring scenario-specific metric selection, making it universally applicable
2Reliability
If traditional fairness metrics are applied to determine model bias, then model behavior can be analyzed, but the determination of bias is not accurate
Solution Approach 1:
The patent creates perturbed copies of original data records by modifying protected attributes (e.g., changing race from 'white' to 'black', or gender from 'male' to 'female'). These copied records with altered protected attributes are then processed through the classification model to measure confidence value differences, providing a reliable and straightforward method for bias determination without complex analytical systems
3Measurement precision
If confidence value differences are measured to determine bias, then accurate fairness assessment is achieved, but the computational process becomes more complex
Solution Approach 1:
The classification model serves itself by processing both original and perturbed records through the same model infrastructure. The system leverages the model's own confidence output mechanism to measure fairness, requiring no external fairness assessment tools or complex additional computational systems - the model's existing confidence calculation capability is repurposed for fairness measurement
Data Source
AI summary
Provided are techniques for correcting a classification model. For each original record of a plurality of original records that are processed by a classification model: the original record is perturbed; for the original record, an original confidence value is obtained for each class of a plurality of classes; for the perturbed record, a perturbed confidence value is obtained for each class of the plurality of classes; a final confidence value is determined using each original confidence value, each perturbed confidence value, and a direction of distance travelled; and a determination is made of whether the original record is biased based on the final confidence value. Then, it is determined whether the classification model is biased based on the original records that are determined to be biased. In response to determining that the classification model is biased, the classification model is corrected, otherwise, the classification model is deployed.