Image Classification Visualization Using Shielding Likelihood Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for visualizing deep learning models (DL models) to identify important image areas for classification are time-consuming due to the need to calculate class similarities for multiple shielding images.
Innovation Solution
A method using a first model to calculate the likelihood of a target object in an image, and a second model based on Taylor expansion to approximate the likelihood of shielding images, reducing the need for repeated calculations by using the first likelihood as a basis for specifying important image areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If class similarity is calculated for all shielding images using the DL model, then the important image area for discrimination can be specified, but the processing time becomes excessively long
Solution Approach 1:
The patent pre-calculates and stores the gradient information of the DL model during a training phase or initialization phase. This gradient information, which represents the sensitivity of the model output to input changes, is prepared in advance so that when shielding images are generated, the class similarity can be quickly estimated using the pre-computed gradients rather than re-running the full DL model inference for each shielding image.
Solution Approach 2:
The patent introduces an intermediary computational approach by using the gradient information as a mediator between the DL model and the shielding image analysis. Instead of directly computing class similarity for each shielding image through full model inference, the gradient serves as an intermediate representation that captures the essential sensitivity information, enabling faster approximation of discrimination importance while maintaining accuracy.
2Measurement precision
If multiple shielding images are generated and processed to identify important areas, then the visualization accuracy improves, but the computational complexity increases
Solution Approach 1:
The gradient information required for computing class similarity is pre-computed and stored before the actual shielding image processing begins. This preliminary preparation eliminates the need to perform complex DL model inference operations repeatedly during the shielding image analysis phase, thereby reducing computational complexity while preserving visualization accuracy.
Solution Approach 2:
The patent uses the pre-computed gradient information as a copy or approximation of the full DL model's sensitivity characteristics. This gradient copy allows for rapid estimation of class similarity across multiple shielding images without repeatedly invoking the computationally intensive DL model, thus reducing complexity while maintaining the ability to accurately identify important image areas.
Data Source
AI summary
An information processing apparatus acquires an image; generates a shielding image in which a part of an area included in an area of the acquired image has been shielded; calculates, by inputting the image to a first model that has been trained, first likelihood of the target object included in the image; calculates, by inputting the shielding image to a second model that calculates an approximation value of likelihood of the target object included in the image when the image is input, second likelihood corresponding to the approximation value of the likelihood of the target object included in the shielding image; specifies, based on the first likelihood and the second likelihood, an area that affects discrimination of the class and that is included in the area of the image; and displays the specified area that affects discrimination of the class.