Adaptive Feature Descriptors for Heterogeneous AR Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial reality systems face challenges in sharing three-dimensional map data across heterogeneous devices without compromising privacy or performance, as high-detail feature descriptors from high-performant devices can strain less-capable devices, while low-detail descriptors may underutilize more capable devices, and traditional methods involve sharing raw image data that compromises user privacy.
Innovation Solution
The method involves generating and sharing feature descriptors using machine-learning models on artificial-reality devices, where high-detail descriptors are adapted to device-specific capabilities, allowing each device to generate and share descriptors optimized for its performance, while maintaining privacy by not sharing raw image data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-detail feature descriptors are shared between devices, then localization and tracking performance is improved, but device performance is compromised on less-capable devices
Solution Approach 1:
The system generates feature descriptors at different levels of detail tailored to each device's capabilities. High-detail descriptors are generated for high-performant devices while low-detail descriptors are generated for lesser-performant devices, ensuring that each device receives appropriately scaled data that optimizes its performance without causing strain or instability
Solution Approach 2:
The system dynamically adapts the detail level of feature descriptors based on the receiving device's computational capabilities. The descriptor generation process adjusts resolution and complexity in real-time according to device performance thresholds, allowing seamless operation across heterogeneous device spectrum while maintaining optimal localization and tracking accuracy for each device type
2Reliability
If low-detail feature descriptors are shared, then device performance is maintained on lesser-performant devices, but localization and tracking performance is underutilized on more capable devices
Solution Approach 1:
The system segments feature descriptors into multiple detail levels (high-detail and low-detail) and distributes them according to device capabilities. This segmentation allows high-performant devices to access and utilize high-detail descriptors for optimal performance while lesser-performant devices receive and process low-detail descriptors that match their computational capacity, preventing resource waste and performance underutilization
Solution Approach 2:
The system changes the parameter of descriptor detail level based on device performance characteristics. By adjusting the resolution and complexity parameters of feature descriptors dynamically according to the receiving device's capabilities, the system optimizes both performance and resource utilization across the entire device spectrum without forcing uniform descriptor quality on all devices
3Loss of information
If raw image data is shared between devices, then complete information is available for processing, but user privacy is compromised
Solution Approach 1:
The system extracts only the necessary feature information from raw images and generates feature descriptors that contain localized spatial and visual characteristics. By taking out only the essential features needed for localization and tracking while discarding unnecessary raw image data, the system maintains information completeness for the specific function while eliminating privacy risks associated with sharing complete images
Solution Approach 2:
The system creates simplified copies of visual information in the form of feature descriptors rather than sharing original raw images. These descriptor copies retain the essential spatial and visual features needed for mapping and localization functions while stripping away privacy-sensitive details, allowing information sharing without compromising user privacy
Data Source
AI summary
A method includes a computing system associated with a first artificial-reality device accessing sensor data of a real environment, generating a first plurality of feature descriptors for the real environment, and generating a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors. The computing system may further cause the first plurality of feature descriptors to be transmitted using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors. The third plurality of feature descriptors are configured to be used by a second artificial-reality device for tracking the real environment. The computing system tracks the real environment using the second plurality of feature descriptors.