Dynamic Vision Sensor and Active Pixel Sensor Fusion for High-Speed Image Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High dynamic range imaging technologies face challenges in capturing fast-moving objects due to low sensitivity, low resolution, and noise issues, particularly when dealing with varying brightness levels in a scene, leading to difficulties in accurately reflecting movement information.
Innovation Solution
An image capturing method that combines RGB image frames with pseudo color filter array (CFA) frames, generated based on color event data, to adjust exposure regions and frame rates, allowing for accurate representation of object movement through the use of active pixel sensors and dynamic vision sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional high dynamic range imaging is used to capture fast-moving objects, then brightness information can be captured, but movement information is lost due to low frame rate and excessive exposure
Solution Approach 1:
The patent divides the imaging task into two separate sensor systems: an active pixel sensor for capturing brightness information and a dynamic vision sensor for capturing movement information. This segmentation allows each sensor to optimize for its specific function, with the DVS operating at a much higher frame rate to capture fast movements that would be missed by traditional HDR imaging.
Solution Approach 2:
The patent adds a temporal dimension to the imaging process by using event-based detection in the dynamic vision sensor. Instead of capturing complete frames at fixed intervals, the DVS detects and records only changes in brightness over time, creating a high-speed temporal representation of movement that complements the spatial brightness information from the APS.
2Illumination intensity
If exposure time is increased to capture bright regions, then brightness information is improved, but movement information is lost due to motion blur
Solution Approach 1:
The patent separates the capture of brightness information and movement information into different sensor systems. The active pixel sensor uses longer exposure times to capture adequate brightness in various regions, while the dynamic vision sensor uses extremely short exposure times to freeze motion and capture movement details without blur.
Solution Approach 2:
The dynamic vision sensor acts as an intermediary that captures movement information independently of the exposure time settings of the main imaging sensor. This intermediary system provides high-speed movement data that can be integrated with the brightness information to produce images that accurately represent both illumination and motion.
3Speed
If frame rate is increased to capture movement, then movement information is improved, but noise increases and power consumption rises
Solution Approach 1:
The patent extracts only the essential movement information using event-based detection in the dynamic vision sensor. Instead of capturing and processing complete high-speed video frames that would generate massive amounts of noise, the system extracts only the relevant brightness change events, significantly reducing noise while maintaining high temporal resolution for movement capture.
4Measurement precision
If separate sensors are used for brightness and movement, then movement capture is improved, but device complexity increases
Solution Approach 1:
The dynamic vision sensor performs multiple functions: it captures movement information at high speed, provides temporal contrast enhancement, and enables event-based processing. This multi-functionality reduces the need for additional specialized components, as the DVS serves as both a high-speed motion detector and a temporal reference system for the overall imaging process.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the capture of high dynamic images with improved movement representation, reducing noise and increasing frame rate while maintaining low power consumption, effectively addressing the limitations of traditional high dynamic range imaging.
Implementation Method 1
receiving, by an image sensor, a sequence of images including a plurality of red, green, and blue (RGB) image frames and color event data
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A method and apparatus for capturing images with a high degree of movements in a captured scene are provided. The method includes receiving, by an image sensor, a sequence of images including a plurality of RGB image frames and color event data, generating a plurality of pseudo color filter array (CFA) frames based on the color event data, and generating dynamic images by combining the plurality of RGB image frames and the plurality of pseudo CFA frames.