Compositor Layer Extrapolation for AR/VR Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR/VR systems face challenges in rendering high-resolution compositor layers due to power constraints, leading to issues like low pixel density (PPD), double aliasing, and occlusion problems, which affect the crispness and readability of menu or panel layers.
Innovation Solution
The use of extrapolation algorithms to generate compositor layers separately from the eye buffer, allowing for improved PPD and avoiding occlusion issues by decoupling layer frame resolution from eye buffer resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If compositor layers are baked into the scene in the eye buffer, then the rendering process is simplified, but the occluded portions of the scene cannot be recovered when the user's viewpoint changes and the layer resolution is limited by the eye buffer's low PPD
Solution Approach 1:
The patent segments the rendering process by separating compositor layer rendering from scene rendering. Compositor layers are rendered independently at high resolution into a separate buffer, then composited with the scene in the display buffer. This allows layers to maintain high PPD independent of eye buffer resolution and enables occlusion recovery since layers are not permanently baked into the low-resolution eye buffer.
2Device complexity
If the layer frame is baked into the scene at low PPD, then the rendering complexity is reduced, but the sharpness and readability of text in menus or panels deteriorates
Solution Approach 1:
The patent introduces a separate rendering dimension by creating an independent compositor layer buffer distinct from the eye buffer. Layers are rendered at high resolution in this separate dimension, then composited with the scene. This dimensional separation allows high PPD for text without increasing eye buffer complexity, as layers bypass the low-resolution eye buffer path entirely.
3Adaptability or versatility
If content in the eye buffer is sampled twice to account for head movement and lens distortion, then the image can be adjusted for viewpoint changes, but the resolution decreases due to double aliasing
Solution Approach 1:
The patent extracts compositor layers from the eye buffer sampling path entirely. Layers are rendered independently at high resolution and composited directly into the display buffer, bypassing the double sampling process that affects eye buffer content. This extraction eliminates double aliasing for layer content while preserving viewpoint adjustment capabilities through separate warping of the layer buffer.
Data Source
AI summary
In one embodiment, a method may obtain, from an application, (a) an image and (b) a layer frame having a first pose in front of the image. The method may generate, for a first viewpoint associated with a first time, a first display frame by separately rendering the image and the layer frame having the first pose into a display buffer. The method may display the first display frame at the first time. The method may determine an extrapolated pose for the layer frame based on the first pose of the layer frame and a second pose of a previously submitted layer frame. The method may generate, for a second viewpoint associated with a second time, a second display frame by separately rendering the image and the layer frame having the extrapolated pose into the display buffer. The method may display the second display frame at the second time.