Camera Pose Tracking with Dynamic Vision Sensor and IMU
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for determining and tracking camera pose, such as those using photographic cameras and depth sensors, are power-intensive and lack sufficient accuracy and low-latency performance, making them unsuitable for low-power devices like mobile phones and drones.
Innovation Solution
A system comprising a Dynamic Vision Sensor (DVS), an Inertial Measurement Unit (IMU), and a memory storing a 3D map, which determines camera pose by generating DVS images, inertial data, and comparing them with synthesized images based on keyframes and prior poses, allowing for efficient power use and accurate tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If photographic cameras and depth sensors are used for camera pose determination, then measurement precision is improved, but use of energy increases significantly
Solution Approach 1:
The patent changes the operating parameters of the vision sensor from continuous high-power photography to event-driven low-power detection. The DVS operates asynchronously, generating images only when visual events occur, thereby reducing power consumption while maintaining pose estimation accuracy through selective sampling of visually significant moments.
Solution Approach 2:
The patent replaces the mechanical/chemical imaging process of traditional photographic cameras with an electronic event-driven sensing mechanism. The DVS uses electronic pixel-level event generation based on brightness changes, eliminating the need for continuous mechanical shutter operation and associated power consumption while enabling efficient pose determination.
2Use of energy by moving object
If conventional low-cost GPS receivers are used for position determination, then use of energy is reduced, but measurement precision deteriorates to several meters accuracy
Solution Approach 1:
The patent merges multiple sensing modalities (DVS event data, IMU inertial measurements, and 3D map data) into a unified pose estimation system. This fusion approach compensates for the limitations of individual sensors, achieving high-precision positioning without relying on power-intensive GPS while maintaining low energy consumption through efficient sensor integration.
Solution Approach 2:
The patent introduces 3D map data and feature tracking as intermediary elements between the DVS/IMU sensors and the final pose determination. These intermediaries process and correlate sensor data with environmental models, enabling accurate pose estimation without direct reliance on low-precision GPS receivers, thus maintaining both accuracy and energy efficiency.
3Measurement precision
If high frame rate DVS operation is used for accurate tracking, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent implements dynamic frame rate adjustment based on motion detection and tracking requirements. The DVS operates at high frame rates only when visual events indicate significant motion or when tracking accuracy is critical, and reduces operation during stable periods. This dynamic adaptation maintains tracking precision while minimizing unnecessary energy consumption during low-activity intervals.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
A system for determining and tracking camera pose includes a dynamic vision sensor (DVS) configured to generate a current DVS image, an inertial measurement unit (IMU) configured to generate inertial data, and a memory. The memory is configured to store a 3-dimensional (3D) map of a known 3D environment. The system may also include a processor coupled to the memory. The processor is configured to initiate operations including determining a current camera pose for the DVS based on the current DVS image, the inertial data, the 3D map, and a prior camera pose.