CIS-EVS Sensor Fusion for Motion-Blur Image Deblurring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital imaging systems, particularly those using CMOS image sensors (CIS), struggle with motion blur in dynamic scenes, leading to degraded image quality and hindering tasks like object recognition and measurement, while Event Vision Sensors (EVS) excel at capturing motion but lack dense intensity information.
Innovation Solution
A hybrid image sensor combining CIS and EVS pixels, where EVS data is used to deblur CIS data, and fusion masks and weights are generated to combine deblurred CIS data streams, effectively reducing motion blur and enhancing image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If integration time is increased to improve signal-to-noise ratio, then image quality improves, but motion blur worsens
Solution Approach 1:
The patent divides the imaging task into two separate sensor types: CIS pixels for capturing intensity information with long integration times, and EVS pixels for capturing motion events with high temporal resolution. This segmentation allows each sensor to optimize for its specific function without compromise.
Solution Approach 2:
The patent introduces EVS data as an intermediary that bridges the gap between CIS intensity data and motion information. The EVS events are used to deblur the CIS images by providing temporal information about motion, effectively mediating between the conflicting requirements of long exposure for quality and short exposure for sharpness.
2Object-affected harmful factors
If integration time is reduced to mitigate motion blur, then motion blur worsens, but light sensitivity and signal-to-noise ratio deteriorate
Solution Approach 1:
The patent segments the sensor array into CIS pixels optimized for light collection and EVS pixels optimized for motion detection. This allows the system to maintain long integration times for light sensitivity while using EVS data to correct motion blur effects.
Solution Approach 2:
EVS data serves as an intermediary that enables the system to use long integration times for light collection while still achieving motion-free images. The EVS events provide the temporal information needed to deblur the long-exposure CIS images.
3Speed
If EVS is used to capture motion information, then temporal resolution improves, but dense intensity information is lost
Solution Approach 1:
The patent merges CIS and EVS sensor arrays into a hybrid imaging system where both sensor types capture the same scene simultaneously. This combination allows the system to benefit from both high temporal resolution (EVS) and dense intensity information (CIS).
Solution Approach 2:
The sensor array is segmented into functional zones: CIS pixels for intensity capture and EVS pixels for motion event detection. This segmentation allows each sensor type to specialize in its strength while the combined output provides both high temporal resolution and complete intensity information.
4Measurement precision
If computational deblurring algorithms are applied to reconstruct sharp images, then image quality improves, but computational complexity and processing time increase
Solution Approach 1:
The patent performs deblurring in the domain of EVS events rather than performing complex iterative deconvolution algorithms on the final image. By processing motion information at the event level before image reconstruction, the system reduces computational complexity while achieving similar or better deblurring效果.
Data Source
AI summary
The present disclosure describes an image system comprising a hybrid image sensor and control circuitry. The hybrid sensor includes an event-driven sensing array with multiple event vision sensor (EVS) pixels and a pixel array with multiple CMOS image sensor (CIS) pixels. EVS pixels capture contrast data within a first time interval, while CIS pixels capture light intensity data during second and third time intervals. The control circuitry uses the EVS data to deblur the first CIS data, generates fusion masks and weights based on the EVS and CIS data, and fuses the deblurred and subsequent CIS data using these masks and weights. The second time interval occurs before the third time interval.