Hybrid Motion Estimation Using Event-Based Video Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Motion vector estimation in video processing is computationally intensive and prone to errors due to limited video frame rates, blurring of rapidly moving objects, and occlusions, forcing trade-offs between accuracy and complexity.
Innovation Solution
A hybrid motion estimation system combining frame-based and event-based video capture, where event-based cameras provide high temporal resolution pixel motion events, which are mapped to tiled regions of frame-based camera images to estimate motion with greater accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional frame-based video capture is used for motion estimation, then the system is simple to implement, but temporal resolution is limited and motion accuracy deteriorates
Solution Approach 1:
The patent combines frame-based video capture and event-based video capture into a hybrid system. The frame-based camera provides spatial information and structural context, while the event-based camera provides high temporal resolution motion information. By merging these two complementary approaches, the system achieves improved motion estimation accuracy without requiring complete replacement of the conventional system, thus managing complexity effectively.
Solution Approach 2:
The patent introduces an intermediary processing layer that receives data from both frame-based and event-based cameras. This intermediary system integrates the two data streams, aligns them in space and time, and produces unified motion estimates. The intermediary acts as a mediator that harmonizes the different characteristics of the two capture methods, enabling their synergistic effect while managing system complexity through modular architecture.
2Measurement precision
If frame rate is increased to improve temporal resolution, then motion estimation accuracy improves, but processing complexity and power consumption increase
Solution Approach 1:
Instead of continuously capturing at extremely high frame rates (excessive action), the system uses event-based cameras to capture only when motion events occur (partial action). This selective capturing approach provides high temporal resolution for motion estimation while avoiding the continuous processing overhead of high-frame-rate video streams, thus improving processing efficiency.
Solution Approach 2:
The patent replaces the mechanical/frame-based temporal sampling system with an event-driven asynchronous capture system. Instead of fixed-time frame sampling, the system uses event-triggered capture based on pixel changes, substituting the mechanical scanning process with a more efficient event-based mechanism that reduces processing requirements while maintaining high temporal resolution.
3Measurement precision
If computational resources are increased to improve motion estimation accuracy, then processing power increases, but available resources are exhausted
Solution Approach 1:
The patent segments the image into multiple tiles or regions, allowing motion estimation to be performed independently on smaller units. This segmentation enables parallel processing and reduces the computational burden on any single processing unit. By dividing the complex motion estimation task into smaller manageable segments, the system achieves accurate motion vectors while distributing the processing load to avoid exhausting available computational resources.
Data Source
AI summary
Techniques are provided for motion estimation using hybrid video imaging based on frame-based capture and event-based capture. A methodology implementing the techniques according to an embodiment includes receiving a sequence of pixel events, generated asynchronously by an event-based video camera, and receiving a sequence of image frames generated by a frame-based video camera at a frame sampling period. The method also includes integrating a subset of the sequence of pixel events, occurring within the frame sampling period between two of the image frames, to generate a pixel motion vector representing motion of the pixel between the two image frames. The method further includes mapping the pixel motion vector to a tile of one of the image frames to generate an estimated motion vector associated with that tile.