Detecting Indirect Bias in Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional bias detection techniques in machine learning models are limited to direct bias analysis, failing to detect transitive and hidden biases that arise from attributes not present in the training data but correlated with neighboring data, which can lead to biased predictions in applications like loan approvals.
Innovation Solution
A computer-implemented method and system that analyzes data not part of the training dataset to detect transitive and hidden biases by determining correlations between neighboring data and the model's output, using machine-based classification and natural language processing to rank attributes contributing to bias, enabling remediation steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional bias detection techniques are used, then direct bias can be detected, but transitive and hidden biases cannot be detected
Solution Approach 1:
The patent extends bias detection from the traditional single dimension of direct attribute analysis to multiple dimensions by incorporating neighboring data attributes. The system analyzes not only the attributes directly present in training data but also attributes from neighboring data sources, creating a multi-dimensional detection framework that captures transitive and hidden biases that conventional single-dimension approaches miss.
Solution Approach 2:
The patent introduces an intermediary mechanism that connects the machine learning model to external neighboring data sources. This intermediary layer analyzes correlations between model predictions and attributes from neighboring data, enabling the detection of indirect biases that do not appear in the direct training data but influence model behavior through correlated attributes.
2Device complexity
If bias detection is limited to training data attributes, then the detection process is simple, but indirect biases are missed
Solution Approach 1:
The patent segments the bias detection process into distinct modules: (1) direct bias detection using training data attributes, (2) neighboring data attribute analysis, (3) correlation computation between model output and neighboring attributes, and (4) transitive bias identification. This segmentation allows the system to maintain manageable complexity while comprehensively addressing multiple bias types through a structured multi-stage approach.
3Measurement precision
If comprehensive bias detection including neighboring data is performed, then detection accuracy improves, but computational resources increase
Solution Approach 1:
The patent applies partial action by selectively analyzing only the most relevant neighboring data attributes based on their correlation with model predictions. Rather than exhaustively processing all possible neighboring attributes, the system identifies and focuses on those attributes that show significant correlation, reducing computational overhead while maintaining high detection accuracy for the most critical bias sources.
Data Source
AI summary
Systems and methods for detecting indirect bias in machine learning models are provided. A computer-implemented method includes: receiving, by a computer device, a user request to detect transitive bias in a machine learning model; determining, by the computer device, correlations of attributes of neighboring data not included in a dataset of the machine learning model; ranking, by the computer device, the attributes based on the determined correlations; and returning, by the computer device, a list of the ranked attributes to a user that generated the user request.