Cloud-Based Pose Tracking Using 3D Environment Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pose-tracking technologies for XR devices are inefficient and unreliable due to error-prone feature tracking, especially in dynamic environments with motion blur and lighting changes, and are unsuitable for low-power XR devices due to computational and storage limitations.
Innovation Solution
A system and method utilizing a server with a 3D environment model to receive images and pose data from a client device, generate reconstructed images, determine spatial transformations, and calculate actual poses, thereby improving pose tracking accuracy without requiring significant computational resources on the client device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pose-tracking means rely on identifying tracking features in images, then pose tracking can be performed, but accuracy deteriorates due to error-prone feature tracking in dynamic environments
Solution Approach 1:
The patent introduces a 3D environment model as an intermediary reference framework. Instead of directly tracking features in 2D images which are error-prone, the system uses the 3D model as a mediator to establish accurate spatial relationships. The 3D model provides stable reference points and geometric constraints that enable precise pose calculation even when image features are unreliable due to motion blur, lighting changes, or dynamic content.
Solution Approach 2:
The patent transitions from 2D image-based pose tracking to 3D space-based pose tracking. By representing the environment as a three-dimensional model with known geometric structure, the system adds a dimensional layer of accuracy. This 3D perspective allows the system to disambiguate ambiguous 2D image features and calculate precise poses using spatial geometry and camera calibration data.
2Measurement precision
If XR devices store and process 3D reconstruction models locally, then pose tracking accuracy improves, but device complexity and power consumption increase
Solution Approach 1:
The patent extracts the computationally intensive 3D environment model and pose calculation functions from the XR device and places them in a cloud-based processing system. The XR device only needs to capture 2D images and transmit minimal pose data, while the cloud server handles the complex 3D reconstruction, model matching, and precise pose calculation. This extraction significantly reduces the computational burden on the device.
Solution Approach 2:
The patent uses a simplified 2D image copy from the XR device as input to the cloud system, which then generates the accurate 3D pose information. Instead of requiring the device to maintain a full 3D model, the system creates a virtual copy of the 3D environment model in the cloud and uses this copy for processing. This allows accurate 3D pose tracking without the device needing to store or process the full 3D data.
Data Source
AI summary
A system includes a server and a data repository storing a three-dimensional environment model, where the server is configured to: receive, from a client device a first image of a real-world environment captured by camera along with information indicative of a first measured pose of the client device; utilise the 3D environment model to generate a first reconstructed image from a perspective of a first measured pose; determine a transformation indicative of a difference in the first measured pose and a first actual pose of the client device; use the transformation to calculate the first actual pose; and send information indicative of at least one of: the first actual pose, or the first spatial transformation, to the client device for calculating subsequent actual poses.