Optical Flow Feature Tracking for Real-Time Subpixel Localization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for tracking feature points in autonomous or semi-autonomous systems, such as those used in automotive applications, face challenges with software-based tracking that requires excessive resources and cannot accurately track large numbers of feature points or perform subpixel tracking due to hardware-accelerated processor limitations.

Innovation Solution

A hardware-based approach using optical flow accelerators and programmable vision accelerators to track feature points, enabling the tracking of hundreds or thousands of feature points with subpixel accuracy by processing image data through first and second processors to determine flow vectors and locate feature points in subsequent images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If software-based feature point tracking is used, then flexibility and ease of implementation are improved, but resource consumption increases and tracking speed becomes prohibitive for real-time applications

Engineering Contradiction:

Improveease of implementationVSAvoidtracking speed

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent replaces software-based feature point tracking with a hardware-accelerated processor that performs optical flow estimation. This substitution of computational mechanics with dedicated hardware circuitry enables parallel processing of multiple feature points simultaneously, achieving real-time tracking speeds while reducing resource consumption compared to software implementations.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary hardware-accelerated processor between the image input and the feature point tracking output. This intermediary device performs optical flow estimation to generate displacement information, which is then used by the feature point tracker, thereby bridging the gap between raw image data and tracked feature points with improved efficiency.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If hardware-accelerated processors are used for optical flow estimation, then processing speed is improved, but subpixel location tracking capability is lost

Engineering Contradiction:

Improveprocessing speedVSAvoidsubpixel location accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent segments the processing into two distinct stages: first, a hardware-accelerated processor performs optical flow estimation at pixel level for high-speed processing; second, a feature point tracker processes the displacement information to determine subpixel locations. This segmentation allows each stage to optimize for its specific function, maintaining both speed and precision.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent uses optical flow estimation results as an intermediary between the hardware-accelerated processor and the feature point tracking system. The displacement information generated by the hardware processor serves as intermediate data that enables the tracker to achieve subpixel accuracy without sacrificing the speed benefits of hardware acceleration.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If software-based feature point tracking is used, then adaptability to different applications is improved, but resource requirements become excessive for real-time performance

Engineering Contradiction:

Improveapplication adaptabilityVSAvoidresource consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent replaces general-purpose software processing with dedicated hardware circuitry designed specifically for optical flow estimation and feature point tracking. This substitution reduces resource consumption by implementing the tracking functionality in efficient hardware logic that can operate in parallel, while maintaining adaptability through configurable parameters and interfaces that support various autonomous system applications.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12494038B2Hardware-based feature tracker for autonomous systems and applications

Publication Date: 2025.12.09 NVIDIA CORP

AI summary

In various examples, techniques for using hardware feature trackers in autonomous or semi-autonomous systems are described. Systems and methods are disclosed that use a processor(s) to determine flow vectors associated with pixel locations in a first image. The systems also use the processor(s) to determine a location of a feature point in a second image based at least on one or more of the flow vectors and a subpixel location of the feature point in the first image. In some examples, the processor(s) may include an optical flow accelerator (OFA) that includes a hardware unit storing a lookup table that is used to determine the location of the feature point in the second image. In some examples, the processor(s) may include an OFA to determine the flow vectors and a vision processor to determine the location of the feature point in the second image.