DVS Pose Estimation via IMU Fusion and Weighted Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Dynamic Vision Sensors (DVS) face challenges in accurately estimating camera movement due to sparse and highly variant features in frames, leading to difficulties in feature-based image matching and landmark correspondence, which results in vulnerable movement estimation and drift in sensor pose estimation.
Innovation Solution
A DVS pose-estimation system that combines a Dynamic Vision Sensor, a 3D transformation estimator, and an inertial measurement unit for sensor fusion, using confidence-level values and multiscale temporal resolution to estimate camera pose by detecting DVS events and inertial movements, and correcting pose estimates through frame-integration time adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature-based image matching is used for DVS frame comparison, then movement estimation can be performed, but the sparse and highly variant features cause matching difficulty and reduce accuracy
Solution Approach 1:
The patent introduces an inertial measurement unit (IMU) as an intermediary sensor to provide motion information that complements the sparse DVS features. The IMU data serves as a mediator between DVS frames, enabling more reliable movement estimation even when feature matching is difficult due to sparsity and high variability of DVS events
Solution Approach 2:
The patent combines data from multiple sensor types (DVS and IMU) into a composite estimation system. This sensor fusion approach integrates the event-based visual information with inertial measurement data, creating a more robust and accurate movement estimation that overcomes the limitations of using either sensor type alone
2Quantity of substance
If frame integration time is extended to improve feature matching, then more events are available for matching, but temporal drift accumulates and reduces pose estimation accuracy
Solution Approach 1:
The patent implements a feedback mechanism where the IMU provides continuous motion information that corrects and refines the pose estimation over time. This feedback loop prevents temporal drift from accumulating by continuously comparing expected motion (from IMU) with observed motion (from DVS), allowing for shorter frame integration times while maintaining accuracy
Solution Approach 2:
The patent uses periodic sensor fusion updates where IMU data is continuously integrated with DVS frame comparisons at regular intervals. This periodic correction mechanism allows the system to use shorter DVS frame integration times while maintaining pose estimation accuracy through frequent recalibration using inertial measurements
3Ease of operation
If landmark correspondence is not extracted due to sparse features, then feature matching can proceed, but cross-checking and drift correction become difficult
Solution Approach 1:
The IMU serves as an intermediary that provides alternative means for cross-checking motion estimates. When landmark correspondence cannot be reliably extracted from sparse DVS features, the IMU data acts as a mediator to verify and correct pose estimates, maintaining reliability without requiring complex landmark extraction and matching
Data Source
AI summary
A Dynamic Vision Sensor (DVS) pose-estimation system includes a DVS, a transformation estimator, an inertial measurement unit (IMU) and a camera-pose estimator based on sensor fusion. The DVS detects DVS events and shapes frames based on a number of accumulated DVS events. The transformation estimator estimates a 3D transformation of the DVS camera based on an estimated depth and matches confidence-level values within a camera-projection model such that at least one of a plurality of DVS events detected during a first frame corresponds to a DVS event detected during a second subsequent frame. The IMU detects inertial movements of the DVS with respect to world coordinates between the first and second frames. The camera-pose estimator combines information from a change in a pose of the camera-projection model between the first frame and the second frame based on the estimated transformation and the detected inertial movements of the DVS.