Pose Tracking Using Dynamic Vision Sensors and IMU
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 6-DOF pose tracking technologies in VR and AR face challenges with high system delay, inaccurate pose tracking, and robustness issues due to limitations in optical tracking systems and LED marker layouts, which affect user experience and immersion.
Innovation Solution
A pose tracking method and apparatus that utilize a dynamic vision sensor camera to detect periodically changing pixels, classify them into clusters, and establish translation vectors to match 2D and 3D points, while incorporating inertial measurement unit data for rotation information, to estimate the pose of a trackable target with improved accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a CMOS camera is used for optical tracking, then the system can capture images, but the system delay exceeds 16.7ms which does not satisfy low delay requirement for VR
Solution Approach 1:
The patent uses periodically changing LED markers that emit light in specific patterns (e.g., alternating on/off states) to enable the dynamic vision sensor to detect changes over time. This periodic action allows the system to identify markers by their temporal patterns rather than requiring continuous high-frame-rate imaging, thereby reducing system delay while maintaining tracking capability.
2Measurement precision
If LED markers are used for pose tracking, then pose information can be obtained, but the layout of LED lamp and detection status greatly affect accuracy and robustness
Solution Approach 1:
The patent employs dynamic vision sensors that detect temporal changes in pixel values rather than static images. The LED markers are configured to change their emission patterns over time, allowing the system to distinguish markers based on their temporal dynamics. This dynamic approach makes the system robust to variations in marker layout and detection conditions, as the temporal patterns provide unique identification regardless of spatial arrangement.
3Measurement precision
If traditional optical tracking systems are used, then pose can be tracked, but it takes great amount of time to solve translational relationship between 3D LED marker and 2D pixel projection which affects real-time system
Solution Approach 1:
The patent extracts only the essential information needed for pose tracking by using a dynamic vision sensor to detect periodic changes in pixel values. Instead of processing complete images or solving complex translational relationships between 3D markers and 2D projections, the system extracts temporal change patterns from pixels, significantly reducing computational complexity while maintaining tracking accuracy in real-time.
Data Source
AI summary
A pose tracking method and apparatus are disclosed. The pose tracking method includes obtaining an image of a trackable target having a plurality of markers, detecting first points in the obtained image to which the markers are projected, matching the first points and second points corresponding to positions of the markers in a coordinate system set based on the trackable target based on rotation information of the trackable target, and estimating a pose of the trackable target based on matching pairs of the first points and the second points.