Event Camera Motion Estimation via Patch Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion estimation techniques using event-based cameras face challenges in achieving accurate and fast motion detection while conserving computational resources, especially in dynamic lighting conditions and requiring high computational complexity.
Innovation Solution
A method utilizing a dynamic vision sensor to obtain events from pixels, generating images based on these events, and processing them through a first neural network for visual motion estimation and a second neural network for confidence scoring, thereby reducing latency and computational cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional motion estimation techniques are used with event-based cameras, then motion detection can be performed, but computational complexity increases and latency increases
Solution Approach 1:
The patent divides the image into multiple patches and processes each patch independently through separate neural network streams. This segmentation allows parallel processing of different regions, reducing overall computational complexity while maintaining motion estimation accuracy through localized analysis of each patch
Solution Approach 2:
The patent applies partial action by focusing computational resources only on patches that contain motion information rather than processing the entire image uniformly. The confidence map identifies which patches require detailed motion estimation, allowing the system to perform partial processing only where necessary
2Measurement precision
If traditional motion estimation techniques are used with event-based cameras, then motion detection can be performed, but processing time increases
Solution Approach 1:
By segmenting the image into patches and processing them in parallel through multiple neural network streams, the patent reduces processing latency. Each patch can be processed independently and simultaneously, rather than sequentially processing the entire image, thereby speeding up motion estimation
Solution Approach 2:
The patent performs preliminary action by generating confidence maps before detailed motion estimation. The confidence map pre-identifies patches containing motion information, allowing the system to prepare and prioritize processing of relevant patches, reducing overall processing time
3Speed
If event-based cameras are used, then low latency vision is achieved, but computational resources are consumed for processing
Solution Approach 1:
The patent extracts only the essential motion information from event-based camera data by processing localized patches rather than analyzing all pixels in the entire field of view. This extraction approach reduces computational resource consumption while preserving the low latency advantage of event-based cameras
Solution Approach 2:
The patent applies partial action by processing only patches that contain motion information rather than the entire image. The confidence-based filtering ensures computational resources are allocated only to relevant regions, reducing overall resource consumption while maintaining fast response times
4Measurement precision
If comprehensive motion estimation is performed across the entire image, then accuracy is improved, but computational cost increases
Solution Approach 1:
The patent segments the image into multiple patches and processes each through dedicated neural network streams. This segmentation enables accurate motion estimation in each local region while reducing overall computational cost through parallel processing and localized analysis
Solution Approach 2:
The patent performs partial motion estimation only on patches identified as containing motion information through confidence mapping. This selective processing maintains accuracy for motion-containing regions while avoiding unnecessary computational expenditure on static regions
Data Source
AI summary
A method may include obtaining a set of events, of a set of pixels of a dynamic vision sensor, associated with an object; determining a set of voltages of the set of pixels, based on the set of events; generating a set of images, based on the set of voltages of the set of pixels; inputting the set of images into a first neural network configured to output a visual motion estimation of the object; inputting the set of images into a second neural network configured to output a confidence score of the visual motion estimation output by the first neural network; obtaining the visual motion estimation of the object and the confidence score of the visual motion estimation of the object, based on inputting the set of images into the first neural network and the second neural network; and providing the visual motion estimation of the object and the confidence score.