Asynchronous Time and Space Warp Region of Interest Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In split-rendered systems, such as those used in gaming virtual reality and augmented reality applications, transmitting per-pixel depth data from the host device to the wearable display device consumes significant bandwidth, leading to latency and positional judder issues due to the delay in updating the user's head pose during image rendering.
Innovation Solution
The system employs time and space warping techniques using metadata from a region of interest (ROI) determined by eye tracking or content information, which reduces the need for transmitting per-pixel depth data, thereby minimizing motion-to-photon latency and positional judder, by generating a single depth plane and motion vector metadata for the ROI.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If per-pixel depth data is transmitted from host device to wearable display device, then time and space warping accuracy is improved, but bandwidth consumption increases significantly
Solution Approach 1:
The patent divides the full depth buffer into multiple depth planes based on depth range segmentation. Instead of transmitting all per-pixel depth data, only selected depth planes are extracted and transmitted to the wearable display device, reducing bandwidth consumption while maintaining sufficient warping accuracy for the region of interest.
Solution Approach 2:
The patent extracts only the essential depth information (selected depth planes) from the complete depth buffer, removing redundant data. This extraction process isolates the critical depth layers needed for accurate time and space warping, transmitting only these planes rather than the full per-pixel depth dataset.
2Reliability
If per-pixel depth data is transmitted, then positional judder is reduced, but latency increases due to data transmission time
Solution Approach 1:
By segmenting the depth buffer into discrete planes, the system reduces the total data volume that needs to be transmitted. This segmentation enables faster transmission of essential depth information, reducing motion-to-photon latency while maintaining positional stability through the preserved depth planes.
Solution Approach 2:
The patent transmits a partial set of depth planes rather than the complete depth buffer. This partial action approach provides sufficient depth information for effective time and space warping to reduce positional judder, while avoiding the excessive data transmission time that would result from sending all per-pixel depth data.
3Manufacturing precision
If full depth buffer is transmitted, then warping quality is improved, but device complexity increases
Solution Approach 1:
The depth buffer is segmented into multiple planes, and only selected planes are transmitted and processed. This segmentation simplifies the data processing complexity at the wearable display device while maintaining warping quality, as the system only needs to handle the essential depth planes rather than the complete depth buffer.
Solution Approach 2:
The essential depth information is extracted from the full depth buffer, removing unnecessary data. This extraction reduces the complexity of data processing at the wearable device while preserving the warping quality needed for accurate time and space correction.
Data Source
AI summary
A method and a system for warping a rendered frame is disclosed. On a host device of a split-rendering system, the method includes generating the rendered frame based on head tracking information of a user. The method also includes identifying a region of interest (ROI) of the rendered frame. The method also includes generating metadata for a warping operation from the ROI. The method further include transmitting the rendered frame and the metadata for a warping operation of the rendered frame. On a client device of the split-rendering system, the method includes transmitting head tracking information of a user by a client device. The method also includes receiving the rendered frame and metadata. The method further includes warping the rendered frame using the metadata and display pose information. The host device and the client device may be combined into an all-in-one head mounted display.