Distributed Pose Estimation for Extended Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational complexity of pose estimation systems imposes significant power and resource demands, limiting their performance and scalability in applications such as tracking and localization, especially in mobile and wearable devices.
Innovation Solution
The approach involves distributing pose estimation, tracking, and mapping functions across different devices to reduce computational, power, and thermal requirements, with communication delays accounted for to prevent drift and tracking errors. This is achieved through a split tracking and mapping system, where a tracking frontend performs real-time localization and a mapping backend handles compute-intensive mapping tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If pose estimation is performed locally on mobile and wearable devices, then real-time tracking and localization can be achieved, but power consumption and thermal requirements increase significantly
Solution Approach 1:
The system divides pose estimation functionality into two segments: a tracking frontend running locally on the wearable device that handles real-time pose tracking, and a mapping backend running on a remote server that handles computationally intensive mapping operations. This segmentation allows the device to maintain real-time tracking performance while offloading power-consuming mapping computations to the server.
Solution Approach 2:
A communication interface acts as an intermediary between the tracking frontend and mapping backend, enabling the device to offload computational tasks to a remote server while maintaining real-time operational capability. The intermediary manages data exchange and task distribution, allowing the device to achieve real-time performance without bearing the full computational burden.
2Temperature
If computational complexity of pose estimation is reduced on wearable devices, then power and thermal requirements decrease, but tracking precision and mapping accuracy may deteriorate
Solution Approach 1:
The mapping backend server acts as an intermediary that performs high-precision mapping computations remotely. The tracking frontend sends raw sensor data and intermediate results to the backend, which returns refined mapping information. This allows the wearable device to maintain simple, low-power local processing while achieving high tracking and mapping accuracy through the remote intermediary's computational power.
Solution Approach 2:
The system performs preliminary pose estimation and tracking at the frontend with reduced computational complexity, then uses preliminary results to guide more accurate backend processing. The backend performs refined mapping and pose correction based on the preliminary frontend estimates, achieving high accuracy without requiring the frontend to perform full-precision computations locally.
3Measurement precision
If mapping computations are performed locally, then mapping accuracy can be maintained, but device weight and complexity increase
Solution Approach 1:
The system extracts the computationally intensive mapping backend functionality from the wearable device and relocates it to a remote server. The device retains only the essential tracking frontend components needed for real-time operation. This extraction eliminates the need for heavy local mapping hardware while maintaining mapping accuracy through remote processing.
Solution Approach 2:
The mapping backend creates and maintains a copy of the environment map on the remote server, which is then transmitted to the device as needed. Instead of requiring the device to generate and store complete high-precision maps locally, the system uses a remote copy of the mapping data, reducing device weight while maintaining access to accurate mapping information.
4Productivity
If distributed architecture is implemented across multiple devices, then computational load is reduced, but communication delays and synchronization errors increase
Solution Approach 1:
The tracking frontend continuously receives feedback from the mapping backend in the form of corrected pose estimates and mapping updates. This feedback loop allows the system to compensate for communication delays and drift by constantly adjusting local tracking based on remote mapping information, maintaining synchronization accuracy despite the distributed architecture's inherent latency.
Solution Approach 2:
The system performs preliminary pose estimation and tracking operations at the frontend before receiving updated mapping data from the backend. These preliminary actions use local sensor data and previous states to maintain continuous tracking, then are refined using backend feedback. This preliminary action approach ensures continuous operation while accommodating communication delays without sacrificing synchronization accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.