Counterfactual Image Generation for Natural Classifier Explanations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for analyzing neural network behavior, particularly in critical tasks like medical diagnosis and autonomous driving, suffer from limitations such as providing false high-confidence predictions and failing to adequately explain classification boundaries, often producing adversarial or impractical counterfactuals.
Innovation Solution
A method to generate non-adversarial counterfactuals without requiring new neural network training, by optimizing an altered image for a different classification while maintaining similarity to the original image's filter activations, using a Gram matrix to summarize texture and ensure natural-looking counterfactuals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If saliency methods are used to explain model predictions, then attention distribution can be visualized, but they cannot adequately express classification explanations attributed to global illumination changes or missing discriminatory features
Solution Approach 1:
The patent creates counterfactual images that copy the structure and appearance of real images while modifying specific features to change model predictions. These counterfactuals serve as tangible copies that demonstrate what features are necessary for correct classification, overcoming the limitations of abstract saliency maps.
Solution Approach 2:
The patent systematically changes image parameters (such as presence/absence of defects, lighting conditions, object positions) to generate counterfactual examples. By controlling which parameters are modified, the method can specifically test and explain the model's sensitivity to different features while maintaining realistic image appearance.
2Loss of information
If optimization approaches work directly on pixel level to produce counterfactuals, then classification decisions can be explained, but the produced counterfactuals are typically adversarial and fall outside the space of natural images
Solution Approach 1:
Instead of optimizing all pixel values freely, the patent constrains optimization to specific meaningful parameters (such as defect presence, object attributes, scene conditions). This ensures that counterfactuals remain within the space of natural images while still demonstrating classification boundaries.
Solution Approach 2:
The patent introduces an intermediary constraint system that mediates between the optimization goal (changing classification) and the realism requirement. By using learned representations and controlled parameter spaces as intermediaries, the method generates counterfactuals that are both explanatory and realistic.
3Reliability
If generative models are used to produce natural-looking counterfactuals, then realism is improved, but training a generative model is required which is not possible or practical when only few data points are available
Solution Approach 1:
The patent makes the existing trained classifier serve dual purposes: both for making predictions and for generating explanatory counterfactuals. By leveraging the classifier's existing learned representations and decision boundaries, the method eliminates the need for separate generative model training while still producing natural-looking counterfactuals.
Solution Approach 2:
The patent uses controlled parameter modifications within the existing model framework to generate counterfactuals, avoiding the need to train new generative models. This approach maintains naturalness by operating within the distribution learned by the original classifier while requiring no additional training data or model complexity.
4Loss of information
If counterfactuals are generated to understand nonlinear decision boundaries, then classification explanations are provided, but the counterfactuals may be adversarial and produce different classification for reasons not likely to occur in practice
Solution Approach 1:
The patent focuses parameter changes on realistic, semantically meaningful attributes (such as defect characteristics, object properties, environmental conditions) rather than arbitrary pixel modifications. This ensures that the counterfactuals reflect real-world variations that actually occur in practice, making the decision boundary analysis practically relevant.
Solution Approach 2:
The patent uses the classifier's own learned feature representations as an intermediary to guide counterfactual generation. This intermediary ensures that the transformations applied to images follow the same decision logic the classifier uses in practice, preventing adversarial artifacts while maintaining explanatory power for understanding nonlinear decision boundaries.
Data Source
Figure 1a~1b
Figure 1c
Figure 1d
AI summary
Some embodiments are directed to a altering an input image to provide an interpretability of a trained classifier. The altered image (161) may be determined by optimizing at least: a classification score (450) of the altered image to reduce a distance between a classification of the altered image and a target classification, and a similarity score (460) to reduce a distance between an average filter activation difference and a measure for a difference in filter activations between the input image and the altered image.