AI Bias Estimation via Unsupervised Deep Neural Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bias detection and estimation methods for machine learning models are limited by inconsistent and insufficient outputs, as each method explores a different ethical aspect of bias and requires different inputs, necessitating domain expert involvement and ensemble methods for comprehensive evaluation.
Innovation Solution
A system and method using a pre-trained unsupervised deep neural network for bias estimation in AI models, which generates bias vectors indicating the degree of bias for each feature, allowing for both targeted and non-targeted evaluations in a single execution without requiring domain experts, and producing scaled and complete bias estimations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple bias detection methods are applied to explore different ethical aspects of bias, then the comprehensiveness of bias detection is improved, but the complexity of the evaluation process increases and contradictory outputs arise
Solution Approach 1:
The patent combines multiple bias detection methods into a single unified framework that processes all ethical aspects simultaneously. The system integrates demographic parity, equalized odds, and other fairness metrics into one coherent evaluation process that produces consistent results across all bias dimensions, eliminating the need to manually coordinate multiple separate methods.
Solution Approach 2:
The patent creates a universal bias detection system that can evaluate multiple ethical aspects and fairness metrics through a single platform. This multi-functional system handles different types of bias detection (protected attribute bias, indirect bias, feature importance bias) and produces standardized outputs that can be compared across all methods without requiring separate evaluation processes.
2Reliability
If ensemble methods are used to ensure comprehensive bias detection, then the reliability of bias detection is improved, but the resource consumption and manual effort increase
Solution Approach 1:
The patent implements an automated bias detection system that performs comprehensive ensemble evaluation without requiring manual expert intervention. The system automatically selects appropriate bias detection methods, processes the data through multiple fairness metrics, and generates unified results, eliminating the need for domain experts to manually coordinate and interpret results from multiple separate methods.
Solution Approach 2:
The patent performs preliminary processing and standardization of bias detection methods before actual evaluation. The system pre-configures multiple bias detection algorithms, establishes unified output formats, and prepares the evaluation framework in advance, so that when comprehensive bias detection is needed, the system can execute the ensemble method efficiently without requiring manual setup or coordination during the actual evaluation process.
3Measurement precision
If different bias detection methods are applied, then various ethical aspects of bias are explored, but the outputs are in different ranges and scales making comparison difficult
Solution Approach 1:
The patent applies parameter transformation to standardize the output of different bias detection methods. The system transforms results from various fairness metrics (demographic parity, equalized odds, etc.) into a unified scale and range, allowing direct comparison of bias levels across different ethical aspects and methods. This parameter standardization enables meaningful aggregation and comparison of results that would otherwise be incomparable.
4Measurement precision
If domain experts are involved to adjust each bias detection method, then the accuracy of bias detection is improved, but the ease of operation deteriorates
Solution Approach 1:
The patent creates a self-configuring bias detection system that automatically adapts to different datasets and evaluation scenarios without requiring domain expert intervention. The system autonomously selects appropriate bias detection methods, configures their parameters, and processes evaluations, making the tool accessible and easy to use while maintaining high detection accuracy through automated expert-level decision-making.
Data Source
AI summary
A system for bias estimation in Artificial Intelligence (AI) models using a pre-trained unsupervised deep neural network, comprising a bias vector generator implemented by at least one processor that executes an unsupervised DNN with a predetermined loss function. The bias vector generator is adapted to store a given ML model to be examined, with predetermined features; store a test-set of one or more test data samples being input data samples; receive a feature vector consisting of one or more input samples; output a bias vector indicating the degree of bias for each feature, according to said one or more input samples. The system also comprises a post-processor which is adapted to receive a set of bias vectors generated by said bias vector generator; process said bias vectors; calculate a bias estimation for every feature of said ML model, based on predictions of said ML model; provide a final bias estimation for each examined feature.