Hybrid Vision Motion Vector Estimation Using Event-Triggered ROI Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing event-based sensors have not been fully utilized in combination with other devices to calculate motion vectors of various objects with high accuracy.
Innovation Solution
An information processing device and method that combines a frame-based vision sensor with an event-based sensor to detect objects, set regions of interest, count event volumes, generate images based on event signals, and calculate motion vectors using a detection, setting, counting, and calculation sections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a frame-based vision sensor is used to capture images, then the system can obtain complete image frames, but the temporal resolution and speed of detecting motion changes are limited by the fixed scanning interval
Solution Approach 1:
The patent combines a frame-based vision sensor and an event-based sensor into a hybrid system. The frame-based sensor provides complete image frames for accurate motion vector calculation, while the event-based sensor detects luminance changes asynchronously to provide high temporal resolution motion information, resolving the contradiction between measurement precision and speed.
Solution Approach 2:
The event-based sensor acts as an intermediary that bridges the temporal resolution gap of frame-based sensors. It generates event signals at high speed based on luminance changes, which are then integrated with frame-based processing to achieve both high temporal resolution and accurate motion vector calculation.
2Speed
If an event-based sensor operates asynchronously to detect luminance changes, then the temporal resolution and speed are improved, but the ability to calculate accurate motion vectors is insufficient when used alone
Solution Approach 1:
The patent merges the asynchronous event-based sensor output with the frame-based vision sensor output. The event signals provide high-speed temporal information while the frame-based sensor provides the complete image context necessary for accurate motion vector calculation, achieving both high speed and high precision.
Solution Approach 2:
The system segments the motion detection task into two parts: the event-based sensor captures high-frequency luminance changes for temporal resolution, while the frame-based sensor captures complete image frames for accurate motion vector calculation. This segmentation allows each sensor to optimize its strength while compensating for the other's limitations.
3Speed
If the system processes event signals from all pixels continuously, then the temporal resolution is maximized, but the power consumption increases compared to frame-based sensors
Solution Approach 1:
The event-based sensor performs partial action by only generating event signals for pixels that detect luminance changes exceeding a threshold, rather than continuously processing all pixels. This reduces power consumption while maintaining high temporal resolution for regions of interest, where full event signal processing is applied.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Accurately calculates motion vectors of various objects by optimizing image generation and motion vector calculation based on the characteristics of each region of interest, improving temporal resolution and tracking accuracy.
Implementation Method 1
an event-based sensor that asynchronously generates an event signal upon detection of a change in intensity of light incident on each pixel
Data Source
AI summary
Provided is an information processing device including a detection section, a setting section, a counting section, an image generation section, and a calculation section. The detection section detects an object according to a first image obtained using a frame-based vision sensor. The setting section sets, in the first image, at least one region of interest including at least a portion of the object. The counting section counts event volume of an event signal in a region of attention corresponding to the region of interest according to an event signal generated by an event-based sensor. The image generation section builds a second image according to the event signal in a case where a predetermined condition is satisfied by the event volume counted by the counting section. The calculation section calculates a motion vector of the region of attention in the second image.