AI Model Bias Detection Using Protected Gradient Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models often operate with embedded bias, leading to reduced accuracy and difficulty in detecting and correcting such bias, which is crucial for improving their usefulness and fairness.
Innovation Solution
A method to detect bias in AI models by calculating a protected gradient score using a bias direction and principal component analysis on token embeddings, aggregating scores to determine a fairness indicator, and providing alerts when the indicator falls below a threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional outcome fairness testing is used to detect bias in AI models, then fairness assessment can be performed, but the detection process is difficult and the results are challenging to use for debugging and fixing models
Solution Approach 1:
The patent segments the bias detection process into distinct components: (1) generating counterfactual inputs by modifying protected attributes, (2) calculating fairness metrics for each input-output pair, (3) aggregating results to determine overall model fairness. This segmentation makes the complex detection process more manageable and interpretable for debugging purposes.
Solution Approach 2:
The patent introduces counterfactual inputs as intermediaries between the original input and the fairness assessment. By generating intermediate counterfactual examples that differ only in protected attributes, the system creates a bridge that enables precise measurement of bias while maintaining a structured analysis framework that aids interpretation.
2Reliability
If post processing steps are added to correct bias in AI models, then fairness can be improved, but the model pipeline becomes more complex and less efficient
Solution Approach 1:
The patent performs fairness assessment and identification of biased predictions as a preliminary action before final model deployment or use. By detecting and flagging biased predictions in advance through counterfactual analysis, the system enables corrective actions to be taken beforehand, reducing the need for complex post-processing steps and allowing for more efficient model pipelines.
3Reliability
If fairness requirements are built into the training process, then bias can be reduced, but the training process becomes more complex and time-consuming
Solution Approach 1:
The patent enables the model to essentially self-diagnose fairness issues through automated counterfactual analysis. By implementing fairness detection as a self-service capability that the model performs on its own outputs, the system reduces the need for extended training processes while maintaining the ability to identify and address bias efficiently.
4Loss of information
If traditional explainable AI methods are used to provide fairness explanations, then post-hoc explanations can be generated, but automatic identification of fairness issues in individual explanations and generalization to systematic model bias remains unresolved
Solution Approach 1:
The patent implements a feedback mechanism where counterfactual analysis results directly inform the identification of fairness issues in individual explanations. By comparing predictions across counterfactual inputs and feeding back the differences, the system automatically identifies which specific explanations contain fairness issues and can generalize these findings to detect systematic model bias, thereby increasing automation while maintaining explanation completeness.
Data Source
AI summary
Systems and methods to receive a text sentence and an identification of a class, calculate a bias direction with respect to the class for an embedding model used by an artificial intelligence model to be analyzed, calculate a protected gradient score for each token in the text sentence, aggregate the protected gradient scores of the tokens to form a sentence-level protected gradient score for the sentence, determine a fairness indicator of the AI model based on the sentence-level protected gradient score, and responsive to the fairness indicator being below a threshold, prevent deployment of the AI model. Intelligent detection of bias of an AI model with respect to a protected class is determined via a protected gradient score for the protected class at a token level of a text sentence.