Camera Clock Synchronization for Virtual Telepresence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) systems, particularly lightweight augmented reality (AR) head-mounted displays (HMDs), face challenges in synchronizing camera clocks for coherent 3D representation and generating full body models for virtual telepresence, especially when using companion devices and lacking integrated body and pose tracking cameras.
Innovation Solution
The implementation of a method that maps capture times from a local clock to a network time, determines phase delta information between camera captures, and adjusts capture times to synchronize image acquisition across devices, combined with machine learning models for head and body mesh estimation to generate a full body model for avatars.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Weight of moving object
If multiple devices with separate clocks are used for virtual telepresence, then device complexity is reduced and weight is decreased, but time synchronization between devices deteriorates
Solution Approach 1:
A server acts as an intermediary time synchronization coordinator that receives time offset information from multiple devices, calculates phase deltas between their clocks, and sends adjustment instructions back to the devices. This mediator resolves the time synchronization problem without requiring devices to directly coordinate with each other, enabling lightweight devices with separate clocks to work together coherently.
Solution Approach 2:
The system implements a feedback mechanism where devices report their capture times and clock differences to the server, which then calculates phase deltas and sends correction instructions back to the devices. This closed-loop feedback enables continuous time synchronization adjustment, ensuring that images from multiple devices are temporally aligned despite having separate clocks.
2Device complexity
If capture times from different clocks are used directly, then device complexity is reduced, but manufacturing precision of time alignment deteriorates
Solution Approach 1:
The system transforms capture times from different clock references into a common time reference frame by calculating time offsets and phase deltas. This parameter transformation enables precise temporal alignment of images from multiple devices without requiring complex hardware synchronization mechanisms, maintaining simple device architecture while achieving high time alignment precision.
3Measurement precision
If phase delta calculation and capture time adjustment are implemented, then time synchronization precision is improved, but device complexity increases
Solution Approach 1:
The server serves as an external computational intermediary that performs the complex phase delta calculations and generates time adjustment instructions. Devices only need to report their capture times and receive adjustment commands, significantly reducing the computational burden and complexity on the devices themselves while maintaining high time synchronization precision through sophisticated server-side processing.
Data Source
AI summary
Techniques and systems are provided for capturing images by a first device. For instance, a process can include obtaining a first image from a first camera, the first image being associated with a first capture time based on a first clock; mapping the first capture time to a second clock to obtain a second capture time, the second capture time based on a second clock, and wherein the second clock is based on a network time; associating the second capture time with the first image; obtaining a second image from a second camera of another device, the second image including a third capture time based on the second clock; determining phase delta information based on a time difference between the second capture time and the third capture time; and outputting the phase delta information to adjust a next capture time of at least one of the first or second camera.