XR Wearable Remote Assistance With 3D Spatial Annotations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR wearable devices face challenges in secure and efficient remote user assistance due to high bandwidth consumption, increased computational load, and labor-intensive real-time assistance, which strain network resources and reduce device performance.
Innovation Solution
The XR wearable device streams captured images to a computing device, allowing a remote user to add annotations within a 3D world coordinate space, using an insert window that adjusts perspective to appear within the 3D space, reducing the need for continuous high-fidelity data transmission and computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time remote assistance is provided through continuous high-fidelity data transmission, then user assistance quality is improved, but network bandwidth consumption increases and device performance decreases
Solution Approach 1:
The patent creates a visual copy of the remote user's presence and annotations within the XR display, allowing the local user to perceive remote assistance without transmitting continuous high-fidelity video data. The system captures key visual information and reconstructs it as annotations and avatar representations, significantly reducing bandwidth requirements while maintaining assistance quality.
Solution Approach 2:
The patent segments the remote assistance signal into discrete components: user avatar, annotations, and spatial position data. These segmented elements are transmitted independently and rendered at appropriate depths in the XR display, enabling efficient bandwidth utilization while maintaining comprehensive remote presence information.
2Reliability
If real-time remote assistance is provided through continuous data transmission, then user assistance quality is improved, but computational load increases
Solution Approach 1:
Instead of processing and transmitting continuous video streams requiring high computational power, the system creates simplified visual copies (avatars and annotations) that convey remote presence information. This copying approach reduces the computational burden on both transmitting and receiving devices while maintaining effective communication.
Solution Approach 2:
The patent transmits only the essential partial information needed for remote assistance (position, annotation data, avatar visual characteristics) rather than complete continuous video feeds. This partial action approach reduces computational load while providing sufficient information for effective remote collaboration.
3Measurement precision
If continuous high-fidelity data transmission is used for remote assistance, then assistance accuracy is improved, but network resources are strained
Solution Approach 1:
The system creates a simplified visual copy of the remote user and their annotations, transmitting only the essential spatial and visual information rather than complete video data. This copying method maintains assistance accuracy for navigation and collaboration while consuming minimal network resources.
Solution Approach 2:
The patent transitions from two-dimensional video transmission to three-dimensional spatial annotation representation. By placing annotations and avatars at specific depths in the XR display space, the system conveys precise spatial information without transmitting volumetric video data, thus maintaining accuracy while reducing network bandwidth requirements.
Data Source
AI summary
Example computer readable storage, extended reality (XR) wearable devices, and methods for remote presence are disclosed where example methods comprise: capturing, by an image capturing device of the XR wearable device, an image corresponding to a first user view of a real-world scene, sending the image to a computing device, receiving, from the computing device, an indication to pause sending next images to the computing device, and determining a plurality of three-dimensional (3D) coordinates corresponding to a plurality of positions within the image. The method may further include sending, to the computing device, the plurality of 3D coordinates corresponding to the plurality of positions within the image, and receiving, from the computing device, an indication of an augmentation and 3D coordinates associated with the augmentation.