Extended Reality Pose Estimation With Motion-Guided Key-Point Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing key-point detection and matching techniques in SLAM are not specialized for the SLAM context, leading to inaccurate results and increased resource consumption, especially with high-resolution and wide-angle cameras, and specialized training increases running time.
Innovation Solution
A deep learning-based key-point detector that dynamically determines the area of the frame for key-point detection based on estimated motion, using an IMU sensor to ensure efficient and accurate pose estimation by focusing on the overlapping region between frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional key-point detection techniques are used, then the system can operate with simpler algorithms, but the detection accuracy and repeatability are insufficient for SLAM
Solution Approach 1:
The patent replaces traditional mechanical image processing algorithms with a deep learning-based key-point detection system. The neural network model automatically learns to identify SLAM-appropriate key points from image data, substituting manual algorithm design with data-driven learning to achieve superior detection accuracy and repeatability.
Solution Approach 2:
The patent changes the detection parameters by training the key-point detector specifically for SLAM tasks rather than using general-purpose detectors. The model learns optimal detection parameters through training on SLAM-relevant data, enabling it to identify key points that are most useful for pose estimation and mapping.
2Measurement precision
If deep neural networks are used for key-point detection, then detection accuracy improves, but computational cost and running time increase
Solution Approach 1:
The patent applies partial action by detecting key points only in specific regions of interest within the image rather than processing the entire image uniformly. The system identifies and processes only the portions of the image that contain relevant key points for SLAM, reducing unnecessary computational effort while maintaining high accuracy.
Solution Approach 2:
The patent performs preliminary action through pre-training the deep neural network model offline before actual SLAM operation. The model is trained in advance on large datasets to recognize SLAM-appropriate key points, so that during real-time operation, the system only needs to infer using the pre-learned patterns, significantly reducing inference time.
3Quantity of substance
If key-point detection is performed on the entire frame, then all possible key points are detected, but resource consumption increases and irrelevant areas waste computational power
Solution Approach 1:
The patent segments the image into relevant and irrelevant regions based on the deep learning model's understanding of SLAM-appropriate content. The system automatically identifies which portions of the image contain useful key points and directs computational resources only to those segments, avoiding wasted processing on empty or irrelevant areas.
Solution Approach 2:
The patent applies local quality by making the detection process adaptive to different regions of the image. The deep learning model dynamically adjusts its detection sensitivity and key-point identification based on the local content of each image region, concentrating computational effort where key points are likely to exist and reducing effort where they are unlikely.
4Reliability
If specialized training is applied to improve robustness, then key-point detection becomes more reliable, but running time during inference increases
Solution Approach 1:
The patent performs the time-consuming specialized training as a preliminary action during the model development phase. The deep neural network is trained offline on large datasets of images with ground truth key point annotations, learning robust patterns for reliable detection. Once trained, the model can be deployed for real-time SLAM operation without requiring continuous training, thus separating training time from inference time.
Data Source
AI summary
A method for estimating a pose of an extended reality (XR) apparatus is provided. The method includes receiving, by the XR apparatus, at least one first frame received from at least one camera, determining, by the XR apparatus, a first set of key points in the at least one first frame, receiving, by the XR apparatus, at least one second frame received from the at least one camera, tracking, by the XR apparatus, movements of each key point of the first set of key points of the at least one first frame in the at least one second frame, determining, by the XR apparatus, a second set of key points in the at least one second frame based on the tracked movements of each key point of the first set of key points in the at least one second frame, tracking, by the XR apparatus, movements of each key point of the second set of key points of the at least one second frame, and determining, by the XR apparatus, the pose of the XR apparatus based on the tracked movements of each key point of the second set of key points of the at least one second frame.