XR Clock Synchronization Using Local Audio-Visual Timing Signals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR devices face challenges in synchronizing clocks accurately for shared experiences due to network congestion, latency, and asymmetric routes, leading to misaligned virtual content and audio-visual lag.
Innovation Solution
Image-based and audio-based user-in-the-loop techniques are employed to estimate time offsets between colocated XR devices, using sensory data such as facial landmarks and audio signals to synchronize clocks and establish a shared coordinate system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network-based clock synchronization is used for XR devices, then time synchronization can be achieved across distributed devices, but network congestion and latency cause time offset errors and misaligned virtual content
Solution Approach 1:
The patent introduces an intermediary synchronization mechanism that uses audio-based timing signals as a mediator between XR devices. Instead of relying solely on network time protocol (NTP) which is vulnerable to network conditions, the system uses locally generated audio signals captured by microphones to establish a shared time reference. This intermediary approach bypasses network congestion and latency issues while maintaining synchronization accuracy across distributed devices.
Solution Approach 2:
The patent replaces the traditional network-based electronic synchronization system with an audio-based synchronization mechanism. By using audio signals that can be captured and timestamped locally, the system substitutes the vulnerable network communication path with a more reliable acoustic reference that is less susceptible to network congestion and latency, thereby improving both measurement precision and reliability of time synchronization.
2Measurement precision
If user-in-the-loop techniques with sensory data processing are employed, then time offset estimation accuracy improves, but computing resources and processing time increase
Solution Approach 1:
The patent extracts only the essential features from sensory data for time offset estimation. Instead of processing complete audio or visual datasets, the system identifies and extracts specific timing-critical features such as audio signal arrival times and visual event timestamps. This extraction approach maintains high measurement precision for time offset estimation while significantly reducing the computational burden and device complexity by focusing only on the most relevant data elements.
Solution Approach 2:
The patent applies partial action by using a subset of available sensory data specifically tailored for time synchronization purposes. Rather than fully processing all sensory inputs from multiple sensors, the system selectively processes only the audio and visual data that directly contribute to time offset estimation. This partial processing approach achieves sufficient accuracy for synchronization while avoiding the excessive computational resources that would be required for complete sensory data analysis.
3Adaptability or versatility
If multiple sensory data types are processed for synchronization, then synchronization accuracy under varying conditions improves, but loss of time and computational overhead increase
Solution Approach 1:
The patent implements periodic synchronization using audio-based timing signals at optimized intervals. Instead of continuously processing multiple sensory data streams, the system uses periodically generated audio signals that serve as synchronization references. This periodic approach maintains adaptability to varying conditions by providing regular synchronization updates while significantly reducing the time loss and computational overhead associated with continuous multi-sensory processing.
Data Source
AI summary
A first extended reality (XR) device and a second XR device are colocated in an environment. The first XR device captures sensory data of a wearer of the second XR device. The sensory data is used to determine a time offset between a first clock of the first XR device and a second clock of the second XR device. The first clock and the second clock are synchronized based on the time offset and a shared coordinate system is established. The shared coordinate system enables alignment of virtual content that is simultaneously presented by the first XR device and the second XR device based on the synchronization of the first clock and the second clock.