Bias-Causing Feature Detection in Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models often operate as 'black boxes,' making decisions differently on various data segments, potentially leading to legally impermissible or questionable outcomes, such as gender bias in loan approvals.
Innovation Solution
A method and system that evaluate machine learning model behavior by computing output differences between data segments, determining reasons for these differences based on feature importance distributions, and generating a remediation engine to adjust features and reduce output disparities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models operate as black boxes to make decisions, then decision-making speed and automation are improved, but transparency and fairness of decisions deteriorate
Solution Approach 1:
The patent introduces an intermediary analysis layer that sits between the black box model and the final decision output. This intermediary computes feature importance distributions and compares them across data segments to identify bias-causing features, thereby providing transparency without slowing down the core decision-making process.
Solution Approach 2:
The patent segments the feature space into different importance distribution groups and compares these segments across protected and non-protected groups. By dividing the analysis into manageable feature importance segments, the system can identify which specific feature segments cause discriminatory outcomes while maintaining overall model efficiency.
2Measurement precision
If machine learning models use complex feature interactions to improve accuracy, then prediction accuracy is improved, but identification of bias-causing features becomes more difficult
Solution Approach 1:
The patent replaces traditional mechanical interpretation methods with a computational approach based on feature importance distributions. Instead of manually analyzing complex feature interactions, the system uses automated computations to measure and compare feature importance across different groups, making it feasible to detect bias even in highly complex models.
Solution Approach 2:
The feature importance distribution analysis acts as an intermediary that bridges the gap between complex model internals and interpretable bias detection. This intermediary layer translates complex feature interactions into measurable importance metrics that can be compared across data segments to identify discriminatory patterns.
3Device complexity
If machine learning models are deployed without bias evaluation to maintain simplicity, then system complexity is reduced, but legal and ethical compliance deteriorates
Solution Approach 1:
The patent performs bias evaluation as a preliminary action before model deployment or during the modeling process. By computing feature importance distributions and identifying bias-causing features in advance, the system ensures legal and ethical compliance is addressed proactively rather than reactively, maintaining simplicity while ensuring compliance.
Solution Approach 2:
The patent implements a feedback mechanism where the analysis of feature importance distributions feeds back into the model development process. This feedback loop identifies bias-causing features and enables corrective actions, ensuring the model meets legal and ethical requirements while maintaining system simplicity through automated analysis.
Data Source
AI summary
A computing machine receives a representation of a machine learning model, a representation of a first data segment, and a representation of a second data segment. The computing machine computes an output difference between an output of the machine learning model applied to the first data segment and an output of the machine learning model applied to the second data segment. The computing machine determines a set of reasons for the computed output difference based on a set of metrics defining distance between feature importance distributions, the set of reasons identifying a set of features from a feature vector of the machine learning model along with a relative contribution of each feature to the computed output difference. The computing machine provides an output representing the set of reasons.