Region of Interest Pooling Using Feature Map Pyramid
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Region of Interest (ROI) pooling operations in Convolutional Neural Networks (CNNs) are computationally and memory-intensive, making them difficult to deploy on embedded devices with limited hardware resources and power budgets, hindering efficient object detection.
Innovation Solution
A fast and energy-efficient ROI pooling method is implemented using a feature map pyramid and power-of-two downsampling, which minimizes resampling computation and is compatible with embedded devices, utilizing hardware engines for efficient pooling operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ROI pooling operations are used to achieve accurate object detection, then detection precision is improved, but computational complexity and memory usage increase significantly
Solution Approach 1:
The patent segments the feature map processing by creating a feature map pyramid with multiple scaled versions (original feature map, 1/2 scaled, 1/4 scaled, etc.). Each scaled feature map handles different size ranges of region proposals, dividing the computational workload across multiple specialized structures rather than processing all proposals through a single complex pooling operation.
Solution Approach 2:
The patent performs preliminary downsampling to create scaled feature maps before the actual ROI pooling operation. By pre-computing these scaled versions using power-of-two downsampling, the system prepares the data structure in advance, avoiding expensive resampling operations during the critical ROI pooling stage and reducing real-time computational complexity.
2Measurement precision
If conventional ROI pooling operations are used to maintain detection accuracy, then object detection performance is preserved, but energy consumption increases
Solution Approach 1:
By segmenting proposals into different size categories and assigning them to appropriate scaled feature maps, the patent avoids performing full-resolution pooling on all regions. Small proposals can be processed on downsampled feature maps, significantly reducing the number of computations and energy consumption while maintaining detection accuracy.
Solution Approach 2:
The patent changes the spatial resolution parameter of feature maps by creating multiple scaled versions. This allows the system to adaptively select the appropriate feature map resolution for each proposal size, reducing computational parameters (number of operations) for small proposals while preserving high-resolution processing for large proposals, thereby optimizing energy consumption.
3Measurement precision
If conventional ROI pooling operations are used to achieve accurate feature extraction, then detection accuracy is improved, but operation speed decreases
Solution Approach 1:
The patent performs preliminary downsampling to generate scaled feature maps before ROI pooling. This pre-computation allows the system to quickly select and process the appropriate scaled feature map for each proposal without performing expensive real-time resampling operations, significantly improving operation speed while maintaining feature extraction accuracy through the use of pre-computed high-quality downscaled representations.
Solution Approach 2:
By segmenting the processing into different scaled feature maps, the patent enables parallel processing of multiple proposal sizes simultaneously. Each scaled feature map can be processed independently and in parallel, increasing overall throughput and operation speed while maintaining accurate feature extraction for each proposal size category.
4Measurement precision
If conventional ROI pooling operations are used to process all proposals, then comprehensive detection coverage is achieved, but memory usage increases
Solution Approach 1:
The patent segments the feature representation into multiple scaled feature maps stored in a pyramid structure. Instead of maintaining and processing one large high-resolution feature map for all proposals, the system stores multiple smaller downsampled feature maps. This segmentation reduces peak memory usage while maintaining comprehensive detection coverage by allowing proposals to be processed at the appropriate resolution level.
Solution Approach 2:
The patent adds a scale dimension to the feature map representation by creating a pyramid structure with multiple scaled versions. This dimensional organization allows the system to efficiently manage memory by storing feature maps at different resolutions, reducing the total memory footprint compared to storing only high-resolution feature maps, while still providing comprehensive coverage for proposals of all sizes through appropriate scale selection.
Data Source
AI summary
An apparatus comprises a memory, one or more hardware engines, and a processor. The memory may be configured to store a feature map pyramid comprising an original feature map and a plurality of scaled feature maps. The one or more hardware engines may be configured to perform pooling operations utilizing power of two down sampling. The processor may be configured to generate the feature map pyramid from the original feature map using said one or more hardware engines.