Latency-Based Display Manager for VR Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for high visual display resolution in VR and AR systems, particularly with head-mounted display devices, leads to issues with image latency, where delayed rendering can cause users to be unaware of their surroundings, potentially resulting in collisions or injuries, as the images displayed do not accurately reflect the user's position in real-time.
Innovation Solution
A Tracking-based Display Manager system that determines and utilizes latency information to predict the user's future position, allowing the image rendering system to generate and display images corresponding to the predicted position, and initiates safe mode operations if latency exceeds a threshold, replacing simulated environment images with actual environment views to alert the user of potential hazards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image rendering resolution is increased to improve visual display quality, then display resolution is improved, but image latency increases causing safety issues
Solution Approach 1:
The system performs preliminary actions by predicting the user's future position based on current tracking data and latency measurements. Image rendering is preemptively adjusted to compensate for anticipated latency delays, ensuring that displayed images correspond to where the user will be rather than where they currently are. This preliminary positioning action resolves the contradiction by proactively addressing the time loss without reducing resolution.
2Measurement precision
If real-time tracking is implemented to improve position accuracy, then position accuracy is improved, but system complexity increases
Solution Approach 1:
The system implements feedback by continuously measuring actual latency values and using this information to adjust image rendering timing and position predictions. The latency measurement feedback loop allows the system to adapt to varying processing speeds and hardware configurations without requiring complex manual calibration, thereby achieving high position accuracy while managing system complexity through automated adaptive control.
3Measurement precision
If latency compensation is applied to improve position correspondence, then position accuracy is improved, but risk of collision increases when latency is high
Solution Approach 1:
The system applies dynamics by making the safety mechanism adaptive rather than static. When latency remains within acceptable thresholds, normal VR/AR rendering continues with latency compensation. When latency exceeds the threshold, the system dynamically switches to safe mode, replacing virtual images with real-world camera feeds. This dynamic response resolves the contradiction by maintaining position correspondence during normal operation while eliminating collision risk during high-latency conditions.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Techniques are described for using information from tracking position of a display device to control display of image data. The display device may, for example, be a head-mounted display ("HMD") device used for virtual reality ("VR") and/or augmented reality ("AR") display of images showing part of a simulated environment around a user wearing the HMD device, and the tracking including determining a position of the HMD device in an actual physical environment (e.g., location and/or orientation in 3D space). Operations of the HMD device or other display device and of an associated image rendering system that provides images for display is improved by determining and using information about a latency or other delay between acquiring tracking data for the device and displaying corresponding images on the device, including to initiate a safe mode operation of the device if the determined delay is too large.