Multi-Vision Sensor Metadata for Image Pipeline Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image sensors struggle with motion blur and low signal-to-noise ratio in dynamic environments, while event-based vision sensors face challenges in disentangling information from background noise, leading to inefficiencies in image processing tasks.
Innovation Solution
A multi-vision solution combining active pixel sensors (APS) and event-based sensors (EVS) generates metadata to stabilize the image-event data stream relationship, using metadata such as saturated area masks and event pixel locations to enhance the robustness of image processing pipelines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional image sensors are used to capture images in dynamic environments, then image capture is simple, but motion blur and low signal-to-noise ratio occur
Solution Approach 1:
The patent combines conventional APS sensors with event-based EVS sensors into a hybrid vision system. The APS sensor captures full-frame images while the EVS sensor detects brightness changes asynchronously, providing complementary information that resolves motion blur and improves signal-to-noise ratio in dynamic environments
Solution Approach 2:
The patent introduces metadata as an intermediary element that bridges APS and EVS data streams. This metadata includes event pixel locations and saturated area masks, enabling the processing pipeline to selectively combine information from both sensor types to improve image quality without direct integration of the sensor outputs
2Speed
If event-based vision sensors are used to detect dynamic changes, then motion detection capability is improved, but disentangling information from background noise becomes difficult
Solution Approach 1:
The patent applies local quality by generating saturated area masks that identify specific regions where pixels have saturated. This allows the processing pipeline to focus computational resources on non-saturated regions where event data is most useful for disentangling signal from noise, rather than uniformly processing the entire image
3Adaptability or versatility
If hybrid sensor data is processed separately, then data processing is straightforward, but computational effort and processing time increase
Solution Approach 1:
The patent performs preliminary actions by pre-processing both APS and EVS data streams independently before integration. This includes generating metadata such as event pixel locations and saturated area masks in advance, which streamlines the subsequent fusion process and reduces computational complexity during real-time processing
Data Source
AI summary
A system comprising circuitry configured to perform an image processing task, the circuitry comprising a multi-vision solution configured to provide APS data and EVS data, and the circuitry being configured to generate metadata and to provide the metadata to an image processing pipeline comprising an algorithm configured to perform the image processing task.