Generalized Event Camera With Intensity Fusion for Scene Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Event cameras capture high-precision timing information but lack sufficient scene information for image reconstruction, requiring post-acquisition algorithms that introduce hardware complexity and spatial/temporal misalignment.
Innovation Solution
A method and system that integrates intensity information with event detection, using high-speed cameras like SPADs to transmit intensity changes and store flux values, allowing for real-time image reconstruction without full frame storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If event cameras capture only changes in per-pixel brightness to achieve high-speed timing information at low readout rates, then timing precision and readout efficiency are improved, but scene information is lost and image reconstruction becomes impossible
Solution Approach 1:
The patent segments the camera output into two distinct streams: event data (change information) and intensity data (scene information). Each stream is processed independently through dedicated pipelines, allowing timing precision from events while preserving scene context from intensity frames for subsequent fusion
Solution Approach 2:
The patent merges event data and intensity data into a unified representation that combines both change detection capabilities and scene information. The fused output enables image reconstruction while maintaining high-speed timing information, resolving the information loss problem
2Loss of information
If post-acquisition algorithms are used to supplement event camera output with conventional intensity frames, then scene information is recovered, but hardware complexity increases and spatial/temporal misalignment occurs
Solution Approach 1:
The patent performs preliminary actions by capturing both event data and intensity data simultaneously during the same acquisition process. This pre-capture of dual data streams eliminates the need for subsequent post-acquisition algorithms to supplement missing information, reducing hardware complexity while maintaining scene information
Solution Approach 2:
The patent introduces a data fusion module as an intermediary that combines event and intensity data streams. This intermediary component handles the integration of different data types, managing spatial and temporal alignment internally rather than requiring complex external processing
3Loss of information
If conventional cameras capture full frames at high frame rates to preserve scene information, then image quality is maintained, but readout rate increases and timing precision is lost
Solution Approach 1:
The patent extracts only the necessary change information from full frames as event data, while separately capturing intensity information. This extraction approach maintains scene information through intensity frames while event-driven readout preserves timing precision by only reading out changed pixels
4Measurement precision
If event cameras operate at high frame rates to capture rapid changes, then timing precision is improved, but data volume increases and processing load increases
Solution Approach 1:
The patent extracts only change information as events rather than transmitting all pixel data. This selective extraction maintains high timing precision for dynamic changes while dramatically reducing data volume by transmitting only necessary change information rather than complete frames
Data Source
AI summary
Methods and systems for detecting changes via an event camera are disclosed. The methods and systems include: monitoring a plurality of pixel measurements from an image sensor, determining an estimate of intensity for a scene using a current frame; detecting changes of the plurality of pixel measurements using the estimate of intensity for the scene; maintaining a stored flux value for each of a first plurality of pixels, wherein the first plurality pixels have not changed intensities; transmitting change information for each of a second plurality of pixels, wherein the second plurality of pixels have changed intensities; determining if a change in the scene has occurred based on the change information for each of the second plurality of pixels; triggering an event in response to the change in the scene; and rendering an event camera image. Other aspects, embodiments, and features are also claimed and described.