Contrastive Image Explanations via Pertinent Feature Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) lack transparency in their decision-making processes, making it difficult for humans to trust and understand their predictions, particularly in critical applications like medicine and transportation, where incorrect predictions can have severe consequences.
Innovation Solution
The method generates humanly-interpretable contrastive explanations by creating 'Pertinent Positive' and 'Pertinent Negative' images, which are modified versions of the original image that highlight the features crucial for the DNN's classification, allowing for a clearer understanding of the prediction rationale.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a DNN classifier model is used to achieve high accuracy in image classification, then the classification performance is improved, but the transparency and interpretability of the decision-making process deteriorates
Solution Approach 1:
The patent segments the image classification task into identifying specific high-level features (such as objects, shapes, or patterns) that the DNN relies on for its prediction. By breaking down the complex decision process into identifiable feature components, the system provides transparent explanations about which features were most influential in the classification, thus resolving the contradiction between high accuracy and decision transparency.
Solution Approach 2:
The patent introduces an intermediary explanation layer that bridges the gap between the DNN's internal decision-making process and human understanding. This intermediary component analyzes the DNN's predictions and generates human-interpretable explanations about which high-level features were critical, allowing users to understand the classification rationale without sacrificing the underlying model's accuracy.
2Measurement precision
If deep learning models are used to improve classification performance, then the model's intelligence and accuracy are enhanced, but the ease of understanding and troubleshooting deteriorates
Solution Approach 1:
The patent extracts and highlights the specific high-level features that the DNN relies on for its predictions. By taking out and presenting these key features separately from the complex neural network architecture, the system makes the decision-making process easier to understand and troubleshoot while maintaining the model's high classification performance.
Solution Approach 2:
The patent employs visual highlighting techniques (analogous to color changes) to emphasize the regions or features in the input image that the DNN considers most important for its prediction. This visual transformation makes the abstract neural network decisions concrete and easily interpretable by humans, improving ease of understanding without affecting classification accuracy.
3Productivity
If DNN systems operate autonomously without explanations, then the productivity and efficiency are improved, but the reliability and trust in critical applications deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the system not only provides classifications but also simultaneously generates explanations about the reasoning behind each prediction. This feedback loop of providing both the prediction and its rationale maintains high processing efficiency while building trust and reliability in critical applications by allowing users to verify and understand the decisions.
Data Source
AI summary
In an embodiment, a method for generating contrastive information for a classifier prediction comprises receiving image data representative of an input image, using a deep learning classifier model to predict a first classification for the input image, evaluating the input image using a plurality of classifier functions corresponding to respective high-level features to identify one or more of the high-level features absent from the input image, and identifying, from among the high-level features absent from the input image, a pertinent-negative feature that, if added to the input image, will result in the deep learning classifier model predicting a second classification for the modified input image, the second classification being different from the first classification. In an embodiment, the method includes creating a pertinent-positive image that is a modified version of the input image that has the first classification and fewer than all superpixels of the input image.