Discrimination Detection Engine for ML Data Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Businesses face challenges in detecting direct and indirect discrimination in large data sets used for decision-making, as existing methods may inadvertently treat individuals differently based on correlated attributes, leading to discriminatory patterns.
Innovation Solution
A system utilizing machine learning techniques to identify discriminatory patterns by evaluating mutual information metrics in data records, comparing them to thresholds, and generating listings of potentially discriminatory attributes, which can include those related to protected groups or unrelated attributes, to detect both direct and indirect discrimination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical and machine learning techniques are used to support business decisions, then decision-making accuracy is improved, but discriminatory patterns may be introduced through direct and indirect discrimination
Solution Approach 1:
The system performs preliminary detection of discriminatory patterns in training data and feature sets before deploying machine learning models for business decisions. By identifying and flagging potentially discriminatory attributes in advance, the system prevents discrimination from being embedded in decision-making algorithms, thus maintaining both accuracy and fairness.
Solution Approach 2:
The patent introduces an intermediary discrimination detection layer that sits between raw data and machine learning models. This intermediary component evaluates features for discriminatory potential using statistical metrics (such as correlation coefficients and information gain) and can transform or remove problematic features, allowing the ML model to maintain accuracy while avoiding discriminatory outputs.
2Reliability
If existing methods are used to detect discrimination, then some discriminatory patterns can be identified, but the methods are insufficient for detecting indirect discrimination through correlated attributes
Solution Approach 1:
The system extends discrimination detection from direct attribute examination to multi-dimensional analysis by evaluating correlations between features and protected attributes, as well as interactions between multiple features. This dimensional expansion enables detection of indirect discrimination where no single attribute directly indicates protected group membership, but combinations of attributes create discriminatory patterns.
Solution Approach 2:
The detection methodology segments the analysis into multiple independent evaluation stages: (1) direct discrimination detection on individual attributes, (2) indirect discrimination detection through correlation analysis with protected attributes, and (3) interaction effect detection among feature combinations. This segmented approach comprehensively covers various discrimination types that single-method approaches would miss.
Data Source
AI summary
A front end receives a request for data specifying a data type. A query handler retrieves data of the data type comprising a plurality of data records from at least one database. The query handler assigns a classification attribute to each data record using a pre-defined classification policy stored in a policy store. A discrimination detection engine statistically evaluates the classification attributes for the data to identify a mutual information metric. The query handler generates a listing of one or more discriminatory attributes and corresponding mutual information metric contributing to discriminatory data patterns based on the mutual information metric.