Source of Bias Identification Tool for Dataset De-biasing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence computing systems trained on biased datasets can generate incorrect results due to innate biases, leading to unfair outcomes in applications like credit approval, where biases against certain age groups or ethnicities are not readily discernible and propagate to operational parameters, affecting downstream systems.
Innovation Solution
A method and tool for identifying sources of bias in datasets using a Source of Bias Identification (SoBI) computing tool, which executes bias detection operations to generate sub-groups for protected attributes, analyzes these sub-groups based on fairness metrics, and outputs bias notifications to de-bias the dataset, thereby correcting AI systems and retraining them with de-biased data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If bias detection operations are performed on AI system outputs to identify sources of bias in datasets, then the fairness and reliability of AI decision-making is improved, but the complexity of the system increases due to additional analysis layers and processing steps
Solution Approach 1:
The patent segments the dataset into multiple sub-groups based on protected attributes (e.g., age, ethnicity, gender) and analyzes each sub-group separately to identify bias sources. This segmentation allows the system to pinpoint specific biased sub-groups without needing to overhaul the entire AI system, thereby improving fairness while managing complexity through targeted analysis.
Solution Approach 2:
The patent introduces an intermediary bias analysis layer between the AI system and its outputs. This intermediary component performs bias detection operations by analyzing relationships between protected attributes and output characteristics, identifying biased sub-groups without requiring changes to the core AI decision-making engine, thus improving reliability while isolating complexity to a separate modular component.
2Reliability
If the dataset is de-biased by removing or modifying data from identified biased sub-groups, then the fairness of AI outputs is improved, but the quantity and diversity of training data is reduced
Solution Approach 1:
The patent extracts and removes only the specific biased sub-groups from the dataset that were identified through bias detection operations, rather than removing entire protected attribute categories. This selective extraction approach eliminates the harmful biased data while preserving the majority of the training dataset, thereby improving fairness without significantly reducing the overall quantity and diversity of available training data.
Solution Approach 2:
The patent applies local quality changes by modifying only the specific biased portions of the dataset (identified biased sub-groups) while leaving the rest of the data intact. This localized modification ensures that fairness improvements are achieved precisely where needed, without unnecessarily reducing the overall training data volume or diversity of non-biased data.
3Measurement precision
If detailed bias analysis is performed on each sub-group to precisely identify bias sources, then the precision of bias identification is improved, but the time and computational resources required increase
Solution Approach 1:
The patent segments the analysis process into discrete steps: first performing initial bias detection on the overall dataset, then dividing protected attributes into sub-groups, and finally conducting detailed analysis only on sub-groups that show signs of bias. This segmented approach achieves high precision in identifying bias sources while avoiding the need to perform detailed analysis on all data points, thereby reducing overall detection time and computational resource requirements.
Solution Approach 2:
The patent applies partial action by performing comprehensive bias analysis only on sub-groups that are identified as potentially biased through initial screening, rather than analyzing every sub-group in detail. This partial analysis approach achieves sufficient precision for identifying bias sources while significantly reducing the time and computational resources required compared to exhaustive analysis of all data.
Data Source
AI summary
A source of bias identification (SoBI) tool is provided that identifies sources of bias in a dataset. A bias detection operation is performed on results of a computer model, based on an input dataset, to generate groupings of values for a protected attribute corresponding to a detected bias in the operation of the computer model. The SoBI tool generates a plurality of sub-groups for each grouping of values. Each sub-group comprises an individual value, or a sub-range, for the protected attribute. The SoBI tool analyzes each of the sub-groups in the plurality of sub-groups, based on at least one source of bias identification criterion, to identify one or more sources of bias in the input dataset. The SoBI tool outputs a bias notification to an authorized computing device specifying the one or more sources of bias in the input dataset.