Multiscale weighted matching and sensor fusion for dynamic vision sensor tracking

Abstract

The invention relates to a multiscale weighted matching and sensor fusion for dynamic vision sensor (DVS) tracking. A dynamic vision sensor pose-estimation system includes a DVS, a transformation estimator, an inertial measurement unit (IMU) and a camera-pose estimator based on sensor fusion. The DVS detects DVS events and shapes frames based on a number of accumulated DVS events. The transformation estimator estimates a 3D transformation of the DVS camera based on an estimated depth and matches confidence-level values within a camera-projection model such that at least one of a plurality of DVS events detected during a first frame corresponds to a DVS event detected during a second subsequent frame. The IMU detects inertial movements of the DVS with respect to world coordinates between the first and second frames. The camera-pose estimator combines information from a change in a pose of the camera-projection model between the first frame and the second frame based on the estimated transformation and the detected inertial movements of the DVS.

Description

[0001] This patent application claims the benefit of priority to US Provisional Patent Application No. 62 / 437,027, filed December 20, 2016, the disclosure of which is hereby incorporated by reference in its entirety. technical field [0002] The subject matter disclosed herein relates generally to dynamic vision sensors (DVS), and more particularly, to an apparatus and method for estimating the pose of a DVS. Background technique [0003] The output of the DVS is the event-based change in brightness sensed by the camera. Typically, the output of a DVS is a stream of events in which each event is associated with a specific state (ie, the event location within the image sensor array and a binary state indicating a positive or negative change in brightness). A certain number of DVS events are sampled to form an image in which pixel locations containing one or more events are set to non-zero and other pixel locations are all set to zero. The value of each non-zero pixel can be ...

AI Technical Summary

Problems solved by technology

The main difficulties related to DVS camera motion or tracking include: (1) features within each DVS frame are sparse and highly variable, thus feature-based image matching becomes difficult, (if even possible) leading to motion estimation Accuracy suffers; (2) Due to the lack of extraction of key features, the corresponding landmarks are unavailable due to DVS motion

Therefore, it may be difficult to cross-check current estimates of camera motion or pose, and it may be difficult to reference landmarks to reduce sensor motion estimate offsets

Images