Recurrent State Estimator for Event Sensor Feature Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Feature tracking operations using event camera data are inefficient in terms of power consumption and computational resources, particularly for systems with limited resources, due to the computationally intensive nature of deriving image data from pixel events.
Innovation Solution
The implementation of recurrent state estimation techniques to process pixel events from event-driven sensors, allowing for the tracking of feature movement without the need to derive image data, using methods such as recurrent neural networks or stochastic state estimators to determine characteristics and track gaze or feature movement based on sparse pixel event data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image data is derived from pixel events for feature tracking, then feature tracking functionality is achieved, but computational load and power consumption increase
Solution Approach 1:
The patent extracts only the essential information needed for feature tracking directly from pixel events, rather than converting all pixel events to complete image data. This selective extraction approach maintains feature tracking functionality while significantly reducing computational load and power consumption by processing only the necessary subset of information.
Solution Approach 2:
The patent applies partial action by performing only the minimum necessary processing to achieve feature tracking. Instead of fully reconstructing image data from pixel events, the system processes pixel events partially and directly, extracting feature characteristics without completing the full image derivation process, thereby reducing computational resources and power consumption.
2Reliability
If image data is derived from pixel events for feature tracking, then feature tracking functionality is achieved, but computational resources are consumed
Solution Approach 1:
The patent extracts only the essential information needed for feature tracking directly from pixel events, rather than converting all pixel events to complete image data. This selective extraction approach maintains feature tracking functionality while significantly reducing computational load and power consumption by processing only the necessary subset of information.
Solution Approach 2:
The patent applies partial action by performing only the minimum necessary processing to achieve feature tracking. Instead of fully reconstructing image data from pixel events, the system processes pixel events partially and directly, extracting feature characteristics without completing the full image derivation process, thereby reducing computational resources and power consumption.
3Loss of information
If pixel events are processed to derive image data, then complete image information is obtained, but processing time increases
Solution Approach 1:
The patent extracts only the essential information needed for feature tracking directly from pixel events, rather than converting all pixel events to complete image data. This selective extraction approach maintains feature tracking functionality while significantly reducing computational load and power consumption by processing only the necessary subset of information.
Solution Approach 2:
The patent skips the intermediate step of fully deriving image data from pixel events. Instead, it rushes through the processing by directly extracting feature characteristics from pixel events in a streamlined manner, eliminating unnecessary processing steps and reducing overall processing time while maintaining sufficient information for feature tracking.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces computational load and power consumption by directly processing sparse pixel events, improving the efficiency of feature tracking and gaze tracking functionalities, especially in systems with limited resources.
Implementation Method 1
Each respective pixel event is generated in response to a specific pixel within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold
Data Source
AI summary
In one implementation, a method includes receiving pixel events output by an event sensor that correspond to a feature disposed within a field of view of the event sensor. Each respective pixel event is generated in response to a specific pixel within a pixel array of the event sensor detecting a change in light intensity that exceeds a comparator threshold. A characteristic of the feature is determined at a first time based on the pixel events and a previous characteristic of the feature at a second time that precedes the first time. Movement of the feature relative to the event sensor is tracked over time based on the characteristic and the previous characteristic.