Top-Down Heat Map Generation for High Resolution Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing methods using convolutional neural networks struggle to generate heat maps that accurately display the basis for object determination in high resolution, as heat maps generated by Class Activation Mapping (CAM) have lower resolution while those by Integrated Gradients (IG) are difficult to interpret in terms of regional object determination.
Innovation Solution
An image processor method that generates top-down heat maps by extracting regions of interest (ROI) from multiple heat maps, applying corresponding portions to integrate them, and using techniques like hard-mask and soft-mask methods to enhance resolution and accuracy of object determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If heat map is generated by CAM technique, then regional object determination is improved, but resolution deteriorates
Solution Approach 1:
The patent segments the heat map generation process into multiple stages: first generating a low-resolution heat map using CAM to identify regional objects, then generating multiple high-resolution heat maps from different convolutional layers, and finally integrating them. This segmentation allows each stage to optimize for its specific purpose without compromising overall performance.
Solution Approach 2:
The patent merges heat maps from multiple convolutional layers with different resolutions by aligning them through interpolation and averaging. This combining process integrates the regional accuracy of lower layers with the detailed information from higher layers, achieving both high resolution and accurate regional determination.
2Manufacturing precision
If heat map is generated by IG technique, then resolution is improved, but regional object determination interpretability deteriorates
Solution Approach 1:
The patent applies different processing qualities to different parts of the heat map generation process. Lower convolutional layers produce heat maps optimized for regional determination with appropriate resolution, while higher layers produce heat maps optimized for fine detail. Each layer's heat map maintains the quality necessary for its specific function.
Solution Approach 2:
The patent adds a hierarchical dimension to heat map generation by incorporating multiple convolutional layers at different depths. This multi-dimensional approach allows simultaneous preservation of regional information from lower layers and detailed information from higher layers through the integration process.
3Manufacturing precision
If multiple heat maps are integrated to improve resolution, then computational complexity increases
Solution Approach 1:
The patent performs preliminary actions by generating heat maps from multiple convolutional layers in parallel during the forward propagation phase. This allows the integration process to work with pre-computed data rather than requiring complex real-time calculations, reducing overall computational burden.
Solution Approach 2:
The patent creates simplified copies of heat map data from different layers and processes them through standardization and interpolation procedures. This copying approach allows efficient integration without requiring complex custom processing for each layer's unique characteristics.
Data Source
AI summary
Disclosed is an operating method of an image processor communicating with a memory device that includes receiving raw image data from the memory device, receiving first to n-th filter data from the memory device, generating first to n-th heat maps based on a convolution neural network (CNN) operation of the first to n-th filter data and the raw image data, extracting a first region of interest (ROI) from a first heat map among the first to n-th heat maps, extracting a first portion corresponding to the first ROI from a second heat map among the first to n-th heat maps, and generating a first top-down heat map by applying the first portion of the second heat map to the first ROI of the first heat map. ‘n’ is a natural number greater than ‘2’.