AR Object Pose Detection via Local Feature Extraction and Server Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality (AR) display devices with limited computational resources face challenges in detecting and tracking objects under varying lighting conditions and occlusions, leading to lag and inaccurate virtual object alignment with real-world objects, especially when CAD models are unavailable or inaccurate.
Innovation Solution
The system uses an AR display device that accesses image and depth data to detect features, forms training clusters, and stores them in a database, allowing for local object recognition and pose determination, with the option to download additional information from a server if local recognition fails, using convolutional neural networks (CNNs) for feature extraction and matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If object detection and tracking are performed on portable AR devices, then augmented reality functionality is enabled, but computational resources are insufficient leading to lag and inaccurate alignment
Solution Approach 1:
The system segments the computational workload by separating feature extraction (performed locally on the portable device) from pose determination (performed on a remote server with greater computational resources). This division allows the portable device to maintain AR functionality while offloading intensive calculations to ensure accurate alignment without exceeding local computational limits.
Solution Approach 2:
The system introduces an intermediary server that receives image data from portable devices, performs computationally intensive pose determination using the extracted features, and returns alignment information to the devices. This intermediary enables accurate virtual object alignment by providing additional computational resources without requiring the portable device itself to have high processing power.
2Speed
If feature extraction is performed locally on portable devices, then processing speed is improved, but accuracy decreases due to limited computational resources
Solution Approach 1:
The system segments the processing pipeline into two stages: rapid feature extraction performed locally on the portable device (prioritizing speed), and precise pose determination performed on a remote server (prioritizing accuracy). This segmentation allows each stage to be optimized for its primary goal while maintaining overall system efficiency.
Solution Approach 2:
The system performs preliminary feature extraction locally before transmitting data to the server. This preliminary action reduces the amount of data that needs to be processed remotely and allows the server to focus computational resources on the most critical pose determination calculations, thereby improving both speed and accuracy.
3Measurement precision
If training data is continuously updated, then object recognition accuracy improves over time, but device memory and processing load increase
Solution Approach 1:
The system extracts and stores only the essential feature representations in local training databases rather than storing complete image datasets. This extraction approach maintains object recognition accuracy by preserving key discriminative features while dramatically reducing the volume of training data that must be stored and processed locally on portable devices.
Data Source
AI summary
A system and method for detecting a pose of an object is described. An augmented reality display device accesses first sensor data from an image sensor and a depth sensor of the augmented reality display device. The first sensor data includes a first plurality of images of an object and corresponding depth data relative to the augmented reality display device and the object. The augmented reality display device detects first features corresponding to the object by applying a convolutional neural network to the first sensor data, forms a plurality of training clusters based on the first features, and stores the plurality of training clusters in a training database.