Foveated Rendering Display Apparatus for XR Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display apparatuses for extended-reality (XR) environments face challenges in providing immersive experiences due to computationally intensive rendering requirements, leading to excessive processing burdens on rendering servers and sub-optimal visual fidelity, especially when displaying high-frame-rate, high-resolution images with varying resolutions.
Innovation Solution
A display apparatus incorporating foveated rendering, utilizing gaze-tracking data to determine a user's gaze direction and adjust image resolutions accordingly, with a processor sending resolution information to a rendering server to generate high-resolution images only for the gaze region and lower-resolution images for the remaining field of view, thereby reducing computational load and enhancing user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple displays or projectors are employed to display XR environment images, then the visual coverage area is improved, but the optical combination quality deteriorates resulting in a disjointed visual scene
Solution Approach 1:
The display apparatus is divided into multiple independent display units (first display unit and second display unit), each projecting a portion of the XR environment. This segmentation allows each unit to operate independently while contributing to the overall visual coverage, resolving the contradiction by maintaining area expansion without compromising individual projection quality.
2Manufacturing precision
If fixed foveated displaying is employed with central portion rendered at higher resolution, then the central viewport resolution is improved, but the adaptability to user gaze movement deteriorates
Solution Approach 1:
The resolution allocation in the display apparatus is made dynamic rather than fixed. The system continuously adjusts which regions receive high-resolution rendering based on real-time gaze tracking data, allowing the high-resolution focus area to move with the user's gaze. This dynamic adaptation resolves the contradiction by maintaining central resolution quality while enabling flexibility to follow gaze movements.
Solution Approach 2:
The system incorporates gaze tracking feedback to dynamically adjust rendering resolution. The gaze tracking device provides continuous feedback about user gaze direction, which the processing unit uses to reposition the high-resolution rendering focus. This feedback loop enables the system to adapt to gaze movements while maintaining high resolution in the focal area, resolving the contradiction between fixed high resolution and gaze adaptability.
3Manufacturing precision
If high-resolution images are rendered for the entire field of view, then the overall image quality is improved, but the processing burden on the rendering server increases excessively
Solution Approach 1:
The system applies different quality levels to different regions of the XR environment based on user gaze. The focal region (where the user is looking) receives high-resolution rendering, while peripheral regions use lower resolution. This local quality differentiation maintains overall image quality in the important areas while significantly reducing the processing burden, resolving the contradiction between comprehensive high quality and processing complexity.
Solution Approach 2:
Instead of rendering high resolution for the entire field of view (excessive action), the system applies high resolution only to the necessary focal region (partial action). This partial rendering approach provides sufficient image quality where the user is actually looking while avoiding the excessive processing burden of rendering the entire scene at high resolution, resolving the contradiction between overall quality and processing load.
4Device complexity
If single-resolution display is employed for XR environment, then the display complexity is reduced, but the immersive experience quality deteriorates
Solution Approach 1:
The display system is segmented into multiple display units with potentially different resolutions, allowing each unit to contribute to the overall immersive experience. This segmentation enables the system to maintain relatively simple individual display components while achieving high overall quality through their combined output, resolving the contradiction between display simplicity and immersive quality.
Solution Approach 2:
Multiple display units are merged to create a unified XR environment presentation. By combining the output of multiple displays, the system achieves high immersive quality that would be difficult to obtain from a single display, while each individual display unit remains relatively simple in design. This merging approach resolves the contradiction between display complexity and experience quality.
Data Source
AI summary
A display apparatus including first and second light sources, gaze-tracking means, and processor(s) configured to: process gaze-tracking data to determine gaze direction; identify gaze region; determine first and second portions of gaze region; send, to rendering server, resolution information indicative of at least one of: gaze direction, gaze region, first and second portions of gaze region, different required resolutions of at least two input images; receive said input images comprising first input image(s) and second input image(s), from rendering server; process first input image(s) to generate first region of first output image and second region of second output image; process second input image(s) to generate remaining regions of first and second output images; and display first and second output images via first and second light sources.