Configurable Keypoint Descriptor Generation for Hardware Image Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing algorithms executed on central processing units (CPUs) consume significant bandwidth and increase power consumption, necessitating the development of hardware-based image processing pipelines to optimize resource utilization.
Innovation Solution
Implementing a pyramid image generator circuit and a keypoint descriptor generator circuit within an image signal processor to generate image pyramids at different octaves and scales, determining intensity values and comparison results for keypoint descriptors, and ordering bit values based on importance levels to reduce data size while maintaining feature representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image processing algorithms are executed on CPU, then processing flexibility is maintained, but bandwidth consumption and power consumption increase significantly
Solution Approach 1:
The image processing pipeline is segmented into dedicated hardware components (pyramid image generator circuit, keypoint descriptor generator circuit) that handle specific processing tasks. This segmentation allows critical path operations to be accelerated in hardware while maintaining CPU involvement for higher-level decision making, thus reducing overall power consumption while preserving processing flexibility.
Solution Approach 2:
The patent introduces an intermediary hardware processing stage between the CPU and image data. This intermediary circuit performs preliminary image processing operations (pyramid generation, keypoint descriptor computation) that would otherwise burden the CPU, thereby reducing CPU bandwidth consumption and power usage while maintaining the system's adaptive processing capabilities.
2Loss of information
If all comparison results are retained in keypoint descriptors, then feature representation completeness is maintained, but data size increases
Solution Approach 1:
The patent extracts and retains only the most significant comparison results in the keypoint descriptor, discarding less important bits. This selective extraction maintains the essential feature representation while significantly reducing the data size of the descriptor, achieving a balance between information retention and storage efficiency.
Solution Approach 2:
Different bits in the comparison results are treated with different levels of importance. The patent applies local quality by preserving high-importance comparison results while reducing or eliminating low-importance ones, thereby maintaining feature representation completeness for critical aspects while minimizing overall data size.
Data Source
AI summary
Embodiments relate to generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit and a keypoint descriptor generator circuit. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit determines intensity values of sample points in the pyramid images for a keypoint and determines comparison results of comparisons between the intensity values of pairs of the sample points. The keypoint descriptor generator circuit generate bit values defining the comparison results for the keypoint, each bit value corresponding with one of the comparison results, and generate a sequence of the bit values defining an ordering of the comparison results based on importance levels of the comparisons, where the importance level of each comparison defines how much the comparison is representative of features. Bit values for comparisons having the lowest importance levels may be excluded from the sequence.