Hardware Feature Tracking With Optical Flow for Subpixel Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to track large numbers of feature points in real-time due to resource constraints and inaccuracies in subpixel tracking, particularly in automotive applications, as software-based solutions are inadequate and hardware-accelerated processors are not directly applicable to feature point tracking.
Innovation Solution
A hardware-based approach using optical flow accelerators and programmable vision accelerators to track feature points, enabling subpixel tracking by determining flow vectors for pixel and subpixel locations, and processing image data to identify and manage feature points across images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software-based feature point tracking is used, then flexibility and ease of implementation are improved, but tracking speed and real-time performance deteriorate due to linear dependence on the number of feature points
Solution Approach 1:
The patent replaces software-based feature point tracking with a hardware-accelerated processor that performs optical flow estimation. This substitution transforms the tracking operation from software execution to hardware acceleration, enabling parallel processing and significantly improving tracking speed while maintaining real-time performance for hundreds or thousands of feature points
Solution Approach 2:
The patent introduces an intermediary processing stage between image capture and feature point tracking. The hardware-accelerated processor first performs optical flow estimation to generate flow vectors, which are then used by the feature point tracker. This intermediary step enables efficient hardware acceleration while maintaining tracking accuracy
2Productivity
If hardware-accelerated processors perform optical flow estimation, then processing speed is improved, but subpixel tracking accuracy deteriorates due to constraints on spatial locations of features
Solution Approach 1:
The patent implements a dynamic tracking system where the feature point tracker can operate at different precision levels. The system dynamically adjusts between tracking feature points at pixel locations and subpixel locations based on the specific application requirements, enabling both high-speed processing and high-precision subpixel tracking
Solution Approach 2:
The patent segments the tracking process into two distinct stages: optical flow estimation performed by the hardware-accelerated processor for speed, and feature point tracking that can operate at pixel or subpixel locations for accuracy. This segmentation allows each stage to optimize for its specific function while working together to achieve both speed and precision
3Reliability
If the number of tracked feature points is increased, then tracking coverage and object recognition are improved, but resource requirements and computational complexity worsen
Solution Approach 1:
The patent replaces software-based tracking with hardware-accelerated optical flow estimation, enabling the system to handle hundreds or thousands of feature points simultaneously. This hardware acceleration provides the computational power needed to increase tracking coverage without proportionally increasing resource requirements
Solution Approach 2:
The patent performs preliminary optical flow estimation across the entire image sequence before conducting feature point tracking. This preliminary action pre-computes motion information that is then reused during tracking, significantly reducing the computational complexity of tracking large numbers of feature points
Data Source
AI summary
In various examples, techniques for using hardware feature trackers in autonomous or semi-autonomous systems are described. Systems and methods are disclosed that use a processor(s) to determine flow vectors associated with pixel locations in a first image. The systems also use the processor(s) to determine a location of a feature point in a second image based at least on one or more of the flow vectors and a subpixel location of the feature point in the first image. In some examples, the processor(s) may include an optical flow accelerator (OFA) that includes a hardware unit storing a lookup table that is used to determine the location of the feature point in the second image. In some examples, the processor(s) may include an OFA to determine the flow vectors and a vision processor to determine the location of the feature point in the second image.