Asynchronous Warp Engine for Wearable Display Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing a high-quality, fast, and asynchronous time warp on consumer hardware for wearable display devices is challenging due to issues with latency, throughput, and graphics hardware utilization in virtual reality systems.
Innovation Solution
A method and system that utilize a separate warp engine to perform just-in-time warping of images, asynchronously with respect to the GPU rendering, to correct for lens distortion and update positions, reducing motion-to-photon delay and memory bandwidth demands by writing only the necessary warped image portions to a cache for immediate display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If asynchronous time warp is implemented on consumer hardware, then motion-to-photon delay is reduced and frame rate is improved, but device complexity and implementation difficulty increase
Solution Approach 1:
The system segments the image processing pipeline by separating the GPU rendering function from the time warp function. The GPU renders images to a back buffer while a dedicated time warp engine independently warps and composites images asynchronously, allowing parallel processing without increasing overall system complexity
Solution Approach 2:
A dedicated time warp engine acts as an intermediary component between the GPU and display. This intermediary handles the complex asynchronous warping operations, memory management, and image compositing, isolating the complexity from both the rendering and display subsystems
2Measurement precision
If high-quality time warp is implemented, then image quality and latency are improved, but processing speed and throughput decrease
Solution Approach 1:
The system performs preliminary actions by pre-calculating warp transforms and preparing image data in advance. The time warp engine pre-processes images to the display resolution and applies warp transforms before they are needed for display, reducing real-time processing requirements
Solution Approach 2:
The asynchronous time warp operation maintains continuous useful action by operating independently of the GPU rendering cycle. The time warp engine continuously processes and composites images without waiting for GPU completion, ensuring steady throughput and consistent frame rates
3Ease of operation
If synchronous rendering with display refresh is used, then hardware utilization is simplified, but frame rate consistency and latency performance worsen
Solution Approach 1:
The system introduces dynamic operation by allowing the time warp engine to operate asynchronously with variable timing independent of the fixed display refresh cycle. This dynamic approach allows the rendering and warping processes to proceed at their own optimal rates while maintaining synchronization with the display through buffer management
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Certain aspects of the present disclosure provide methods and apparatus for operating a wearable display device. Certain aspects of the present disclosure provide a method for operating a wearable display device. The method includes determining a position of the wearable display device based on a motion sensor. The method includes rendering, by a graphics processing unit, an image based on the determined position. The method includes determining a first updated position of the wearable display device based on the motion sensor. The method includes warping, by a warp engine, a first portion of the rendered image based on the first updated position. The method includes displaying the warped first portion of the rendered image on a display of the wearable display device.