AR Feature Detection Using Pixel Vectors and Pose Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) and virtual reality (VR) systems face challenges in accurately detecting and tracking real-world features due to reliance on resource-intensive depth information, which leads to inaccurate and unreliable scene information, degrading user experience and requiring frequent adjustments to maintain accurate mappings.
Innovation Solution
The method involves obtaining pass-through image data from an image sensor, identifying features using pixel characterization vectors, and transforming AR display markers to track objects across changing poses, reducing the need for depth sensors and improving measurement accuracy and user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If depth information is used to detect and track real-world features in AR/VR scenes, then feature detection capability is improved, but resource consumption increases and measurement accuracy decreases
Solution Approach 1:
The patent extracts and removes the depth sensor component from the AR/VR system, relying solely on 2D image data from standard cameras. This extraction eliminates the resource-intensive depth information processing while maintaining feature detection capability through innovative 2D image analysis techniques including pixel characterization vectors and pose estimation algorithms.
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system with a computational approach using 2D image processing. Instead of using physical depth sensors (mechanical/optical system), the system uses algorithmic processing of 2D images with pixel characterization vectors and pose information to achieve feature detection and tracking.
2Difficulty of detecting and measuring
If depth information is used for feature tracking, then feature detection capability is improved, but measurement accuracy and reliability deteriorate
Solution Approach 1:
The patent compensates for the lack of depth information by utilizing the temporal dimension (multiple frames over time) and pose information. By tracking pixel characterization vectors across multiple frames and incorporating device pose data, the system achieves accurate 3D feature tracking using only 2D image data, effectively adding temporal and pose dimensions to compensate for missing spatial depth information.
Solution Approach 2:
The system implements feedback mechanisms by continuously tracking pixel characterization vectors across multiple frames and using pose information to correct and refine feature positions. This feedback loop allows the system to maintain measurement accuracy by constantly adjusting feature positions based on observed changes in the 2D image sequence and known pose transformations.
3Difficulty of detecting and measuring
If depth sensors are used in AR/VR systems, then feature detection capability is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts and removes the depth sensor component from the AR/VR system architecture, demonstrating that feature detection and tracking can be achieved using only standard 2D cameras. This simplification reduces device complexity while maintaining core functionality through advanced 2D image processing techniques.
Solution Approach 2:
The patent makes the standard 2D camera perform multiple functions: it serves both as a general imaging device and as a depth-sensing equivalent through computational techniques. By using pixel characterization vectors and pose estimation on standard camera data, the system enables the universal 2D camera to replace specialized depth sensors, reducing overall system complexity.
4Measurement precision
If AR display markers are frequently recalibrated to maintain accuracy, then measurement precision is improved, but user experience and system reliability deteriorate
Solution Approach 1:
The patent performs preliminary calibration by establishing pixel characterization vectors and feature positions in advance using 2D image data and pose information. This preliminary action creates a robust foundation that remains stable across frame transitions, eliminating the need for frequent recalibration and thereby improving system reliability and user experience.
Solution Approach 2:
The system maintains continuous tracking of AR display markers by using pixel characterization vectors and pose transformations across frames. This continuous action ensures measurement precision is maintained without interruption or frequent recalibration, providing stable and reliable augmented reality experiences throughout extended usage periods.
Data Source
AI summary
A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.