CNN Feature Extraction Using Activation Maps for Similar Image Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image search technologies fail to discriminate between regions useful and not useful for similar image search, leading to masking of relevant features and preventing suitable feature amounts from being acquired.
Innovation Solution
A feature amount acquisition device using a convolutional neural network (CNN) and class activation maps (CAM) to generate activation maps, allowing for selective masking and extraction of high-activation-level regions, generating a CAM-masked feature vector suitable for similar image search.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If masking processing is applied to regions other than the search target, then background noise is removed and search precision is improved, but useful regions for similar image search are also masked and feature amounts suitable for similar image search cannot be acquired
Solution Approach 1:
The patent applies different quality treatments to different regions of the image based on their activation levels. High-activation regions (useful for search) are preserved with full quality, while low-activation regions (background noise) are masked. This local differentiation resolves the contradiction by selectively treating regions rather than uniformly masking entire non-target areas.
Solution Approach 2:
The patent changes the parameter of region importance by using activation levels from CNN to dynamically determine which regions to mask. Instead of fixed masking rules, the system adapts the masking parameter based on the learned activation levels, allowing useful regions to be identified and preserved while still removing background noise.
2Object-affected harmful factors
If faithful masking of non-target regions is performed, then background interference is reduced, but discrimination between useful and non-useful regions is lost and relevant features are prevented from being acquired
Solution Approach 1:
The patent introduces an intermediary mechanism (activation map from CNN) that mediates between the need to remove background interference and the need to preserve useful features. The activation levels serve as an intermediary criterion to intelligently determine which regions to mask, rather than using direct faithful masking that loses useful information.
3Ease of manufacture
If uniform masking is applied to all non-target regions, then processing simplicity is maintained, but useful regions for similar image search are incorrectly masked
Solution Approach 1:
The patent segments the image into multiple regions based on activation levels, dividing the image into high-activation regions (preserved) and low-activation regions (masked). This segmentation approach maintains processing simplicity through automated region division while improving feature discrimination accuracy by treating different regions differently based on their usefulness.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A feature amount acquisition device (100) includes an activation level calculator (12) deriving, as an activation level, a level at which, in a CNN classifier (11) including a plurality of layers and configured to, by processing input data based on image data of an input image capturing a first target and a second target around the first target in the layers, output a classification result of the first target, an unit in a layer among the layers influences a classification result and a feature amount acquirer (14) acquiring, based on the derived activation level and the image data of the input image, a feature amount of the input image so that a feature amount of a low activation level image region that is a region in the input image corresponding to a second unit having a lower activation level than a first unit is smaller than a feature amount of a high activation level image region that is a region in the input image corresponding to the first unit.