XR Overlay Tracking With Edgecloud Motion Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality (XR) systems face challenges in accurately placing visual overlays due to network, processing, and rendering latency, particularly with moving objects, and lack the computational efficiency to handle high-definition data streams efficiently, leading to suboptimal user experiences.
Innovation Solution
A system that utilizes motion probability fields (MPFs) distributed across edgecloud resources, combining local sensor data with edgecloud processing to predict object movement and optimize XR overlay placement, leveraging 5G New Radio (NR) for low latency and high-speed data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If XR systems use distributed computing with edgecloud resources, then computational efficiency and resource utilization improve, but system complexity and network dependency increase
Solution Approach 1:
The system divides computational tasks between local XR devices and edgecloud resources. Local devices handle sensor data capture and preliminary processing, while edgecloud resources perform intensive motion prediction and overlay optimization. This segmentation enables high computational efficiency without requiring every device to have full processing capabilities, thus managing overall system complexity through functional distribution.
Solution Approach 2:
The patent introduces motion probability fields (MPFs) as an intermediary data structure that bridges local sensor data and edgecloud processing. MPFs represent predicted object motion and serve as a standardized interface for communication between XR devices and edgecloud services, simplifying the interaction protocol and reducing the complexity of direct system integration.
2Measurement precision
If the system processes high-definition data streams in real-time, then overlay placement accuracy improves, but processing time and latency increase
Solution Approach 1:
The system performs motion prediction in advance using motion probability fields that are updated continuously based on historical object movement patterns. By pre-computing likely motion trajectories before the actual overlay rendering is needed, the system reduces real-time processing requirements and latency while maintaining high placement accuracy for moving objects.
Solution Approach 2:
The patent changes the parameter representation from raw sensor data to motion probability fields that encode predicted object positions and trajectories. This parameter transformation allows the system to work with compressed, pre-processed motion information rather than processing full high-definition video streams in real-time, thereby reducing processing time while maintaining accuracy.
3Measurement precision
If the system uses motion probability fields for object tracking, then overlay placement accuracy for moving objects improves, but computational resources required increase
Solution Approach 1:
Instead of processing all sensor data at full resolution continuously, the system applies motion probability field techniques selectively to objects that require tracking. By identifying which objects need precise tracking based on their motion characteristics and applying MPF algorithms only to those, the system achieves high tracking accuracy for relevant objects while reducing overall computational resource consumption.
Data Source
AI summary
A method of a user device to track objects in an extended reality (XR) environment includes capturing an image of a physical environment associated with the XR environment, sending the image, a first timestamp for the image, and object information for at least one object in the image to a tracking service, receiving an object identifier, object tracking information, and a second timestamp for the at least one object from the tracking service, performing time and motion compensation on the object tracking information to correlate the object tracking information and object identifier with the at least one object in the image of the physical environment, and generating or updating an XR overlay for the at least one object using time and motion compensated object tracking information.