Auto-Encoder Bias Detection and Mitigation With Counterfactual Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models are prone to bias due to reliance on well-curated datasets, leading to erroneous classifications when low-level features like color or texture dominate over shape, and existing bias mitigation strategies require human intervention or prior knowledge, lacking concept-level understanding.
Innovation Solution
A method and system using an auto-encoder to generate latent embeddings, compute attribution scores with a permutation invariant neural network, and create counterfactual images to identify and mitigate bias in deep learning models without explicit bias labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models are trained on large domain-specific natural datasets, then data-driven features are captured effectively, but the models become vulnerable to bias and adversarial perturbation
Solution Approach 1:
The patent introduces an intermediary explanation system that acts as a mediator between the deep learning model and the input data. This explanation system generates concept-level interpretations of model predictions, allowing users to verify whether the model is using appropriate features for decision-making. The intermediary layer provides transparency into the model's reasoning process without requiring changes to the underlying deep learning architecture.
2Difficulty of detecting and measuring
If visual explanation methods like GradCAM are used to understand model decisions, then pixel-level importance is identified, but concept-level understanding of predictions is not provided
Solution Approach 1:
The patent transitions from spatial dimension analysis (pixel-level importance maps) to conceptual dimension analysis (concept-level explanations). Instead of showing which pixels are important, the system identifies which high-level concepts (e.g., 'stripes', 'spots', 'shape') are driving the model's predictions. This dimensional shift from spatial to conceptual space provides meaningful interpretability that aligns with human understanding.
3Reliability
If data augmentation is used to reduce bias effects, then certain types of biases are mitigated, but human intervention is still required to identify biases
Solution Approach 1:
The patent enables the system to automatically identify and analyze biases without requiring human expert intervention. The explanation system autonomously generates concept-level interpretations of model predictions and identifies potential bias sources by analyzing which concepts are over-relied upon. This self-service capability allows automatic bias detection and mitigation strategy formulation.
4Adaptability or versatility
If Concept Activation Vectors are used for humanlike control over model learning, then concept importance is quantified, but the approach assumes the trained network is not biased
Solution Approach 1:
The patent implements a feedback mechanism where the explanation system continuously monitors model predictions and provides information about concept-level reasoning. This feedback loop allows users to verify whether the model is using appropriate concepts and to identify when bias may be present. The system adapts by highlighting concepts that should be important versus those that indicate potential bias, enabling dynamic adjustment of trust in model predictions.
Data Source
AI summary
This disclosure relates generally to identification and mitigation of bias while training deep learning models. Conventional methods do not provide effective methods for bias identification, and they require pre-defined concepts and rules for bias mitigation. The embodiments of the present disclosure train an auto-encoder to produce a generalized representation of an input image by decomposing into a set of latent embedding. The set of latent embedding are used to learn the shape and color concepts of the input image. The feature specialization is done by training an auto-encoder to reconstruct the input image using the shape embedding modulated by color embedding. To identify the bias, permutation invariant neural network is trained for classification task and attribution scores corresponding to each concept embedding are computed. The method also performs de-biasing the classifier by training it with a set of counterfactual images generated by modifying the latent embedding learned by the auto-encoder.