Decision Matrix Thresholding for Fairer ML Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning models suffer from bias-related issues such as type-1 and type-2 errors, inaccuracies, inefficiency, limited applicability, and compromise of privacy and security due to rigid parity constraints across data attributes, particularly in high arity attributes and multiple protected attributes.
Innovation Solution
A bias correction system that employs a fairness deviation constraint and decision matrix to iteratively modify decision thresholds, allowing for flexible application across various machine learning architectures, handling multiple attributes, and preserving privacy and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rigid parity constraints are applied across data attributes to ensure fairness, then bias is reduced, but accuracy and efficiency deteriorate due to type-1 and type-2 errors
Solution Approach 1:
The patent transforms the rigid parity constraint into a flexible fairness deviation constraint by changing the parameter from equal treatment to bounded differential treatment. The decision matrix applies different thresholds to different protected attribute values, with each threshold carefully selected to satisfy the fairness deviation constraint while optimizing accuracy. This parameter change allows the system to move from a one-size-fits-all approach to a customized approach for each attribute value.
Solution Approach 2:
The patent introduces dynamic threshold adjustment through the decision matrix, where thresholds are not fixed but are adapted based on the specific protected attribute values and their corresponding fairness requirements. The system dynamically selects thresholds that balance fairness and accuracy for each subgroup, rather than applying a static uniform constraint across all groups.
2Reliability
If conventional fairness constraints are applied, then bias is controlled, but flexibility and applicability across different machine learning architectures are limited
Solution Approach 1:
The decision matrix is designed as a universal post-processing component that can be applied to any machine learning model output, regardless of the underlying architecture. The fairness deviation constraint framework provides a model-agnostic approach that works with different algorithms and prediction types, making the solution broadly applicable across diverse machine learning systems while maintaining bias control.
3Reliability
If decision thresholds are modified to accommodate multiple high arity data attributes, then fairness across sub-populations is improved, but computational complexity increases
Solution Approach 1:
The patent segments the complexity management by handling each protected attribute value separately through individual decision thresholds in the decision matrix. Rather than attempting to optimize all attributes simultaneously in a complex joint optimization, the system divides the problem into manageable per-attribute components, each with its own threshold that satisfies the fairness deviation constraint independently.
4Reliability
If traditional fairness approaches are used, then bias is addressed, but privacy and security of underlying data and learning mechanism are compromised
Solution Approach 1:
The decision matrix serves as an intermediary layer between the machine learning model and the final decision. It processes the model outputs without requiring access to the underlying training data or model parameters, thereby maintaining privacy and security. The fairness deviation constraint is enforced at this intermediate stage, allowing bias mitigation while preserving the confidentiality of the underlying learning mechanism.
Data Source
AI summary
The present disclosure relates to systems, methods, and non-transitory computer readable media that control bias in machine learning models by utilizing a fairness deviation constraint to learn a decision matrix that modifies machine learning model predictions. In one or more embodiments, the disclosed systems generate, utilizing a machine learning model, predicted classification probabilities from a plurality of samples comprising a plurality of values for a data attribute. Moreover, the disclosed systems determine utilizing a decision matrix and the predicted classification probabilities, that the machine learning model fails to satisfy a fairness deviation constraint with respect to a value of the data attribute. In addition, the disclosed systems generate a modified decision matrix for the machine learning model to satisfy the fairness deviation constraint by selecting a modified decision threshold for the value of the data attribute.