Deep Learning Keypoint Detection via Receptive Field Information Change
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based keypoint detection technologies lag behind handcrafted methods in terms of repeatability and matching score, necessitating an improvement in scale information extraction for better performance.
Innovation Solution
An apparatus and method utilizing a feature extractor, information accumulation pyramid module, information change detection module, keypoint detection module, and continuous scale estimation module to extract features from input images, calculate information changes across receptive fields, and estimate keypoint scales, enhancing deep learning-based keypoint detection robustness to scale changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If deep learning-based keypoint detection is used, then automation and processing speed are improved, but repeatability and matching score deteriorate compared to handcrafted methods
Solution Approach 1:
The patent introduces scale information as an additional dimension by processing features at multiple scales through the information accumulation pyramid module. This multi-scale approach allows the deep learning model to capture keypoints with scale context, improving repeatability and matching score while maintaining automation. The scale dimension compensates for the limitations of single-scale deep learning features.
Solution Approach 2:
The patent changes the parameter representation by explicitly estimating continuous scale parameters for each keypoint through the continuous scale estimation module. Instead of relying solely on spatial coordinates, the system now outputs both location and scale parameters, enabling better keypoint description and matching performance while maintaining automated detection.
2Device complexity
If only keypoint extraction is performed without scale information, then processing simplicity is maintained, but keypoint description and matching performance deteriorate
Solution Approach 1:
The patent segments the keypoint detection process into distinct functional modules: feature extraction, information accumulation pyramid for multi-scale processing, information change detection, and continuous scale estimation. This modular segmentation allows the system to incorporate complex multi-scale analysis while maintaining clear processing flow and manageable complexity through structured organization.
Solution Approach 2:
The information accumulation pyramid module acts as an intermediary between the feature extractor and the keypoint detection module. It processes features at multiple scales and provides enriched multi-scale feature representations to subsequent modules, enabling improved keypoint description and matching without requiring complete redesign of the entire system.
3Measurement precision
If multi-scale filter responses are processed, then scale information extraction is improved, but computational complexity increases
Solution Approach 1:
The information accumulation pyramid module processes features at multiple scales, but selectively focuses computational resources on the most informative scales for keypoint detection. Rather than exhaustively processing all possible scales, the system applies multi-scale filtering strategically to extract critical scale information while limiting unnecessary computational overhead.
Solution Approach 2:
The system performs preliminary multi-scale feature processing through the information accumulation pyramid before keypoint detection. By pre-computing and organizing multi-scale feature representations in advance, the system reduces the computational burden during the actual keypoint detection and scale estimation phases, improving overall efficiency.
Data Source
AI summary
Disclosed herein are an apparatus and method for detecting a keypoint based on deep learning robust to scale changes based on information change across receptive fields. The apparatus for detecting a keypoint based on deep learning robust to scale changes based on information change across receptive fields includes a feature extractor for extracting a feature from an input image based on a pre-trained deep learning neural network, an information accumulation pyramid module for outputting, from the feature, at least two filter responses corresponding to receptive fields having different scales, an information change detection module for calculating an information change between the at least two filter responses, a keypoint detection module for creating a score map having a keypoint probability of each pixel based on the information change, and a continuous scale estimation module for estimating a scale of a receptive field having a biggest information change for each pixel.