Stationary XR Tracking Without Depth Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR and AR systems require depth sensors, which increase cost and complexity, leading to suboptimal tracking and erratic behavior of virtual elements due to environmental variability and user handling issues, particularly in mobile devices.
Innovation Solution
Implementing XR experiences on a stationary device with a secure enclosure that houses a camera, screen, and processors, using machine learning models trained for a specific environment to provide realistic virtual content without depth sensors, and employing predefined segmentation masks to avoid complex in-painting processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth sensors are added to mobile AR devices, then tracking precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent removes the depth sensor component from the AR system entirely. Instead of using active depth sensing hardware, the system extracts depth information passively from monocular video feeds through machine learning models, thereby eliminating the complexity and cost of depth sensors while maintaining tracking precision
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system with a computational approach using machine learning models that process 2D video images to infer 3D spatial relationships, substituting physical sensing hardware with algorithmic processing
2Reliability
If complex machine learning models and in-painting processes are used, then AR realism is improved, but processing time and computational resources increase
Solution Approach 1:
The patent pre-trains machine learning models specifically for each target environment before AR operation. This preliminary action allows the models to be optimized for particular settings (rooms, landscapes, etc.), enabling faster and more accurate processing during actual AR use without requiring complex real-time in-painting operations
Solution Approach 2:
The system changes the parameters of the machine learning models by training them with environment-specific data and adjusting their configuration for particular settings. This allows the models to achieve high realism with optimized processing requirements tailored to each specific AR context
3Adaptability or versatility
If AR systems are not tailored to specific environments, then adaptability is improved, but measurement precision and tracking accuracy deteriorate
Solution Approach 1:
The patent implements a dynamic system where machine learning models are selected and configured based on the specific environment being analyzed. The system adapts its computational approach in real-time by choosing appropriate pre-trained models for different settings (indoor/outdoor, lighting conditions, spatial characteristics), maintaining both versatility and precision
Solution Approach 2:
The patent applies different machine learning models and processing parameters tailored to specific environmental characteristics. Each environment receives customized model selection and configuration optimized for its unique features, ensuring high tracking accuracy for each local context while maintaining overall system adaptability
Data Source
AI summary
Methods and systems are disclosed for generating an extended reality (XR) experience using a statically positioned device. The system receives, from a camera of a stationary device, an image depicting a real-world object, the camera being directed in a stationary manner towards a specified field of view of a real-world environment. The system analyzes the image using a machine learning model to predict tracking information for the real-world object, the machine learning model trained based on a plurality of training images depicting real-world objects in the specified field of view of the real-world environment and corresponding ground-truth tracking information for the real-world objects. The system selects an extended reality (XR) experience from a plurality of XR experiences and overlays one or more XR elements associated with the XR experience on the image based on the predicted tracking information to generate a modified image.