FPGA ORB Feature Extraction Accelerator for Low-Latency Stream Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing ORB feature extraction algorithms face significant challenges in speed and efficiency, especially on low-power platforms like small robots or drones, due to the complex and non-parallelizable computation of rotated BRIEF (rBRIEF) descriptors, which results in high latency and resource consumption.
Innovation Solution
A stream processing-based non-blocking ORB feature extraction accelerator is implemented on an FPGA, featuring a downsampling module, Gaussian filtering, corner detection, non-maximum suppression, and an rBRIEF computation module that performs window column selection, computation, and FIFO operations in parallel using a cache management algorithm, along with a hardware sorting module for efficient descriptor sorting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional blocking computation is used for rBRIEF descriptor calculation, then computation can be completed, but data throughput is reduced and latency increases
Solution Approach 1:
The patent implements a stream processing architecture where the rBRIEF computation module continuously processes incoming pixel streams without blocking. The FIFO buffer enables continuous data flow by decoupling the computation stage from the input stage, allowing new pixels to be loaded while previous pixels are being processed, thus eliminating idle time and maintaining continuous useful action throughout the computation pipeline.
Solution Approach 2:
The computation process is segmented into distinct modular stages: input module, FIFO buffer, rBRIEF computation module, and output module. Each stage operates independently and can process data in parallel, with the FIFO buffer acting as a separator that enables simultaneous data loading and computation operations without interference between stages.
2Measurement precision
If rBRIEF descriptor computation is performed with high accuracy, then feature quality is maintained, but computational complexity increases and processing speed decreases
Solution Approach 1:
The system performs preliminary actions by pre-calculating and storing gradient values in the FIFO buffer before the actual rBRIEF computation. This preliminary preparation of data in the buffer allows the computation module to operate at full speed without waiting for data preparation, thus maintaining high processing speed while ensuring accurate feature quality through complete computation.
3Ease of operation
If hardware sorting is implemented for descriptor output, then descriptor organization is improved, but resource occupation increases
Solution Approach 1:
The patent introduces a dedicated sorting module as an intermediary component between the rBRIEF computation module and the output stage. This specialized sorting hardware efficiently organizes descriptors by their response values without requiring complex resource allocation in other parts of the system, thus achieving good descriptor organization while controlling overall resource occupation through functional specialization.
Data Source
AI summary
The provided is a stream processing-based non-blocking oriented FAST and rotated BRIEF (ORB) feature extraction accelerator implemented by a field programmable gate array (FPGA), which mainly includes two innovations: A stream processing-based non-blocking hardware architecture and a cache management algorithm are provided. The accelerator precisely controls and buffers each column of an rBRIEF descriptor computation window by using an algorithm, allowing to receive a new input pixel stream while computing a descriptor, thereby achieving non-blocking processing. An efficient hardware sorting design embedded in an accelerator is provided. Based on a count sorting algorithm, minimal resources are used to implement rBRIEF sorting on hardware, and the rBRIEF sorting is embedded in the accelerator. The accelerator ensures quality of a feature point while achieving high-speed feature point extraction, without significantly reducing accuracy of ORB_SLAM and other algorithms.