Weighted Visual Inference Using DVS and CIS Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for determining visual inference using multiple sensors, such as CMOS, DVS, and IMU, often result in incorrect visual inference due to the disadvantages of each sensor, including motion blur, noise, and drift.
Innovation Solution
A method is proposed that calculates a weighted visual inference by combining the confidence of dynamic vision sensor (DVS) and contact image sensor (CIS) measurements. This involves determining DVS feature confidence based on parameters like noise, track length, and number of features tracked, and CIS feature confidence based on feature velocities, then calculating a weighted combination of these confidences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CIS sensor is used for visual measurements, then feature-rich measurements are obtained, but motion blur occurs for fast scene changes and frame rate is low
Solution Approach 1:
The patent combines measurements from both CIS and DVS sensors to create a unified visual inference system. The CIS provides feature-rich measurements while the DVS provides high-speed temporal information, and their measurements are merged through a common inference framework that leverages the strengths of both sensor types.
Solution Approach 2:
The patent introduces an intermediary processing layer that transforms DVS event streams into a format compatible with CIS-based visual inference algorithms. This intermediary representation allows the high-speed DVS data to be integrated into the feature-rich CIS measurement framework without direct conflict between the different measurement paradigms.
2Productivity
If DVS sensor is used for visual measurements, then high data-rate and high dynamic range are achieved, but noise and variable latency occur
Solution Approach 1:
The patent applies local quality by using DVS measurements selectively for specific aspects of visual inference where its high data-rate and dynamic range are most beneficial, such as detecting fast motion and bright objects. The CIS sensor is used for other aspects where its lower noise characteristics are more suitable, creating a spatially and functionally differentiated measurement strategy.
Solution Approach 2:
The patent dynamically adjusts the weighting and contribution of DVS measurements based on scene conditions. When the scene contains fast motion or high-contrast elements that benefit from DVS capabilities, the system increases reliance on DVS data. When noise and latency become problematic, the system reduces DVS contribution and relies more on CIS measurements.
3Reliability
If multiple sensors are used for visual measurements, then sensor deficiencies are overcome, but visual inference becomes incorrect due to improper measurements
Solution Approach 1:
The patent implements feedback mechanisms where the visual inference system continuously monitors the quality and consistency of measurements from both CIS and DVS sensors. Based on this feedback, the system dynamically adjusts the weighting and trust placed in each sensor's measurements, ensuring that unreliable measurements do not degrade the overall visual inference accuracy.
Data Source
AI summary
The embodiments herein provide a method of obtaining a weighted combination of dynamic vision sensor (DVS) measurements and contact image sensor (CIS) measurements for determining visual inference in an electronic device, the method includes receiving, by the electronic device, a DVS image and a CIS image from the image sensor; determining, by the electronic device, a plurality of parameters associated with the DVS image and feature velocities of a plurality of CIS features present in the CIS image; determining, by the electronic device, a determined DVS feature confidence based on the plurality of parameters associated with the DVS image; determining, by the electronic device, a determined CIS feature confidence based on the feature velocities of the plurality of features present in the CIS image; and calculating, by the electronic device, a weighted visual inference based on the determined DVS feature confidence and the determined CIS feature confidence.