Sliding Window Keypoint Descriptor Buffer for Reduced Image Memory

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing algorithms executed on central processing units (CPUs) consume significant bandwidth and increase power consumption, necessitating the implementation of hardware components for image processing that can perform corrections or enhancements without overburdening CPU resources.

Innovation Solution

A circuit comprising a pyramid image generator, keypoint descriptor generator, and pyramid image buffer that utilize a sliding window mechanism to process and store image pyramids and response maps in stages, reducing memory requirements by storing only necessary portions of images at different times.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If entire image pyramids and response maps are stored in memory for keypoint descriptor generation, then complete image processing can be performed, but memory size requirements increase significantly

Engineering Contradiction:

Improvecomplete image processingVSAvoidmemory size

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The image pyramid is divided into multiple octaves, and each octave is processed separately. The buffer stores only the current octave being processed along with a limited number of previous octaves, rather than storing the entire image pyramid. This segmentation allows complete processing while significantly reducing memory requirements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The buffer pre-allocates memory for storing pyramid images at different octaves and scales before processing begins. By preparing the buffer structure in advance with predetermined allocation for different octave levels, the system avoids needing to store all images simultaneously during processing.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If image processing is performed on CPU, then flexibility in processing algorithms is maintained, but power consumption and bandwidth usage increase

Engineering Contradiction:

Improveprocessing flexibilityVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

A dedicated hardware buffer circuit is introduced as an intermediary between the image sensor and the processing units. This buffer stores pyramid images locally, allowing processing to occur with reduced bandwidth requirements and lower power consumption while maintaining algorithmic flexibility through programmable access patterns.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12526381B2Sliding window for image keypoint detection and descriptor generation

Publication Date: 2026.01.13 APPLE INC

AI summary

Embodiments relate to extracting features from images, such as by identifying keypoints and generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit, a keypoint descriptor generator circuit, and a pyramid image buffer. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit processes the pyramid images for keypoint descriptor generation. The pyramid image buffer stores different portions of the pyramid images generated by the pyramid image generator circuit at different times and provides the stored portions of the pyramid images to the keypoint descriptor generator circuit for keypoint descriptor generation. When first portions of the pyramid images are no longer needed for the keypoint descriptor generation, the first portions are removed from the pyramid image buffer to provide space for second portions of the pyramid images that are needed for the keypoint descriptor generation.