Fair Selective Classification via Mutual Information Regularization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selective classification methods can magnify disparities between groups, leading to unfairness in classification systems, especially when reducing coverage, as existing fairness criteria like demographic parity and equalized opportunities fail to guarantee fairness in scenarios where classifiers abstain from making decisions.
Innovation Solution
The implementation of sufficiency criteria through a novel upper bound of conditional mutual information as a regularizer, ensuring that predictive scores provide equal accuracy across groups and increasing precision for all groups as coverage decreases, thereby mitigating disparities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If selective classification is applied to improve model performance by allowing abstentions, then accuracy is improved, but disparities between groups are magnified
Solution Approach 1:
The patent changes the optimization parameter from standard accuracy maximization to a modified objective that incorporates fairness constraints. By reformulating the selective classification objective to include group fairness metrics, the system simultaneously improves accuracy while preventing disparity magnification through parameter-based optimization control
Solution Approach 2:
The patent implements feedback mechanisms by monitoring group-specific performance metrics during training and adjustment. The system uses observed disparities as feedback signals to dynamically adjust decision thresholds and abstention strategies, ensuring that accuracy improvements do not come at the cost of increased group disparities
2Object-generated harmful factors
If existing fairness criteria like demographic parity are applied, then fairness is improved, but precision rates deteriorate
Solution Approach 1:
The patent segments the fairness objective into group-specific precision rate optimization rather than applying a single demographic parity constraint. By dividing the problem into group-level precision maximization tasks, the system achieves fairness through equalized precision across groups while maintaining overall precision performance
Solution Approach 2:
The patent changes the fairness constraint parameter from demographic parity (which controls false positive rates) to precision rate equalization. This parameter transformation allows the system to achieve fairness goals while preserving and even improving precision rates by directly optimizing the relevant performance metric
3Reliability
If selective classification reduces coverage to improve accuracy, then model performance is improved, but computational efficiency deteriorates
Solution Approach 1:
The patent applies partial action by implementing selective classification with optimized abstention thresholds that prevent excessive coverage reduction. The system performs classification only when necessary (partial action) while using learned confidence thresholds to minimize unnecessary computations, achieving improved performance without excessive computational overhead from universal re-evaluation
Data Source
AI summary
One or more group-specific aggregate losses, one or more group-agnostic aggregate losses, and a joint loss are computed. A regularizer loss is computed based on the one or more group-specific aggregate losses and the one or more group-agnostic aggregate losses. One or more group-specific models are trained based on the one or more group-specific aggregate losses. A feature extractor is updated based on the regularizer loss and a joint classifier is updated based on the joint loss.