Foveated Image Rendering With Temporal Multiplexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing foveated rendering techniques face challenges in achieving high accuracy, fast speed, and low latency in eye gaze tracking, leading to compromised viewer experience and perceptual artifacts due to subsampling assumptions that may not align with actual eye movements.
Innovation Solution
Implementing temporal and binocular multiplexing of spatial profiles, where the foveated zone is dynamically shifted and rendered at native resolution, alternating with peripheral zones at lower resolution across frames or between eyes, reducing perceptual artifacts without requiring precise eye gaze tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If eye gaze tracking is used to dynamically adjust the foveated zone, then rendering quality in the fovea is improved, but system complexity and latency increase
Solution Approach 1:
The patent implements dynamic foveated zone adjustment through temporal multiplexing, where the high-resolution foveated zone is shifted between adjacent frames rather than requiring real-time eye tracking. This dynamic approach maintains rendering quality while reducing system complexity by eliminating the need for complex eye gaze tracking hardware and software.
2Productivity
If the foveated zone is fixed at the center, then rendering computational load is reduced, but viewer experience deteriorates when gaze shifts occur
Solution Approach 1:
The patent employs periodic action through temporal multiplexing, where the high-resolution foveated zone alternates positions between adjacent frames. This periodic shifting creates the perception of a larger effective foveated region without requiring continuous computational adjustment, thereby maintaining rendering efficiency while improving perceptual quality across different gaze positions.
3Loss of energy
If peripheral zones are rendered at lower resolution, then bandwidth and computational resources are reduced, but perceptual artifacts increase
Solution Approach 1:
The patent transitions from spatial resolution adjustment to temporal resolution adjustment by shifting the foveated zone position between frames. This dimensional change from space to time allows peripheral zones to be rendered at lower resolution in each individual frame while maintaining overall perceptual quality through the temporal sequence, thereby reducing computational resources and bandwidth without significantly increasing perceptual artifacts.
Data Source
AI summary
A method includes rendering a foveated image that includes a foveated zone and a peripheral zone. The foveated zone has a first set of image data and is rendered at a first pixel resolution and the peripheral zone has a second set of image data and is rendered at a second, lower, pixel resolution. The method also includes packing the first set of image data into a first image block, packing the second set of image data into a second image block, and generating a control packet that includes rendering information associated with the foveated image. The method further includes concatenating the control packet with the first image block and the second image block to form a frame, transmitting the frame to a display unit, parsing the control packet, decoding the control packet to obtain the rendering information, and projecting a display image rendered according to the rendering information.