Foveated Image Upsampling Using Edge Layers to Cut Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increase in image quality for high-quality content display systems, such as HMDs, leads to increased bandwidth and processing requirements, resulting in implementation issues like latency and inefficiencies in foveated rendering techniques.
Innovation Solution
An image processing system that utilizes gaze tracking to determine the user's point of focus and applies an upsampling process using supplementary information layers, such as edge and alpha blend masks, to enhance image quality in focused regions, reducing latency and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image quality is increased for high-quality content display, then image quality is improved, but bandwidth requirements and processing requirements increase
Solution Approach 1:
The patent applies foveated rendering techniques that render different regions of the display at different quality levels. The foveal region (center of gaze) is rendered at high quality while peripheral regions are rendered at lower quality, reducing overall bandwidth requirements while maintaining perceived image quality. This is achieved through gaze tracking to identify the foveal region and applying resolution scaling to adjust rendering quality in different areas.
2Manufacturing precision
If image quality is increased for high-quality content display, then image quality is improved, but processing time and latency increase
Solution Approach 1:
By rendering only the foveal region at high quality and peripheral regions at lower quality, the patent reduces the total processing workload. This localized approach to quality rendering decreases processing time and latency while maintaining high perceived image quality in the visually critical foveal region.
Solution Approach 2:
The patent renders only the necessary portion of the image at high quality (the foveal region) rather than the entire image. This partial action approach reduces processing requirements and latency while sufficient quality is maintained where the user actually looks.
3Quantity of substance
If foveated rendering is applied to reduce data size, then bandwidth requirements are reduced, but implementation complexity increases
Solution Approach 1:
The system performs gaze tracking and foveal region identification in advance of rendering, allowing the rendering pipeline to be pre-configured with the appropriate quality levels for different regions. This preliminary determination of render regions simplifies the overall implementation by establishing quality requirements before the rendering process begins.
Data Source
AI summary
A content rendering and display system operable to display content to a viewer, the system comprising a rendering unit operable to render an image having a first resolution in a first render pass, and to generate supplementary information in additional render passes, wherein the rendered image and the supplementary information are stored in a multi-layer image structure, an upsampling unit operable to perform an upsampling process on the rendered image so as to generate an image having a portion at a second resolution higher than the first resolution, wherein the upsampling process comprises a morphological antialiasing process that uses the supplementary information as an input, and a display device operable to display the upsampled image, wherein the supplementary information comprises edge information for the rendered image.