CNN Inference Error Localization via Refine Image and Importance Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing analysis techniques fail to specify image sections causing incorrect inference with adequate precision in image recognition processes using convolutional neural networks (CNNs).
Innovation Solution
An analysis apparatus that generates a refine image with a maximized correct label score from an incorrect inference image, using information related to the inference target, and creates a map indicating the degree of importance for each pixel to identify the image section causing incorrect inference by superimposing maps indicating pixel changes and attention degrees.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing analysis techniques (activation maximization, BP, GBP) are used to specify image sections causing incorrect inference, then the analysis can be performed, but the precision in specifying the image section is insufficient
Solution Approach 1:
The patent combines multiple analysis maps (attention map, saliency map, and gradient-based map) into a single integrated importance map. This merging of multiple analysis approaches allows the system to leverage the strengths of each individual method while compensating for their weaknesses, thereby improving both the precision of image section specification and the reliability of incorrect inference cause identification.
Solution Approach 2:
The patent creates a composite analysis approach by integrating different types of information (attention weights, saliency values, and gradient magnitudes) into a unified importance map. This composite methodology combines multiple data sources and analysis techniques to produce a more accurate and reliable specification of image sections causing incorrect inference.
2Measurement precision
If a refine image with maximized correct label score is generated using information related to inference target, then the precision of specifying image section is improved, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-processing the input image to extract relevant features and generate initial maps (attention map, saliency map) before the main analysis. This preliminary preparation organizes the data in advance, making the subsequent generation of the refine image and importance map more efficient and less computationally intensive.
Solution Approach 2:
The patent introduces intermediate processing steps and data structures (such as the attention map and saliency map as intermediaries) that facilitate the generation of the final importance map. These intermediaries break down the complex processing into manageable stages, reducing the overall computational complexity while maintaining high precision in image section specification.
Data Source
AI summary
A method includes: generating a refine image having a maximized correct label score of inference from an incorrect image by which an incorrect label is inferred by a neural network; generating a third map by superimposing a first map and a second map, the first map indicating pixels to each of which a change is made in generating the refine image, of plural pixels of the incorrect image, the second map indicating a degree of attention for each local region in the refine image, the each local region being a region that has drawn attention at the time of inference by the neural network, and the third map indicating a degree of importance for each pixel for inferring a correct label; and specifying an image section based on a pixel value of the third map, the image section corresponding to a region causing incorrect inference in the incorrect image.