VR Display Rendering with Foveated Surface Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual reality head-mounted display systems are memory bandwidth and power intensive due to the need for high-resolution frames to be rendered and time-warped at high frame rates, leading to inefficient memory transactions and bandwidth usage.
Innovation Solution
The method and data processing system generate input surfaces with peripheral regions at lower fidelity than central regions or with multiple surfaces at different fidelities, allowing for the selection of parts based on view orientation data to form output surfaces, reducing memory and power consumption by using lower fidelity data for edges or rapid head movements where detail is not noticeable.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution frames are rendered and time-warped at high frame rates, then the virtual reality display quality is improved, but memory bandwidth consumption and power usage increase
Solution Approach 1:
The patent applies local quality by rendering the central region of the input surface at high resolution while rendering peripheral regions at lower resolution. This allows the system to maintain high display quality in the foveal region where users focus their attention, while reducing memory bandwidth and power consumption in peripheral regions where detail is less noticeable. The selective rendering approach directly resolves the contradiction between overall display quality and energy consumption.
Solution Approach 2:
The patent segments the input surface into multiple regions (central region and peripheral regions) with different rendering resolutions. By dividing the rendering task into segments with different quality requirements, the system can optimize resource allocation - using high computational resources for the central region and lower resources for peripheral regions, thereby reducing total power usage while maintaining acceptable display quality.
2Measurement precision
If high-resolution frames are rendered and time-warped at high frame rates, then the virtual reality display quality is improved, but memory bandwidth consumption increases
Solution Approach 1:
The patent reduces memory bandwidth consumption by storing and transferring peripheral regions at lower resolution than central regions. Since memory bandwidth is consumed proportionally to the amount and resolution of data being transferred, rendering and storing peripheral regions at lower fidelity directly reduces the quantity of data that must be moved through the memory subsystem, while central regions maintain high resolution for quality.
Solution Approach 2:
By segmenting the input surface into central and peripheral regions with different resolutions, the patent creates separate data streams with different bandwidth requirements. This allows the memory subsystem to handle high-resolution data only for the central region, while peripheral regions use lower bandwidth, thereby reducing total memory bandwidth consumption.
3Use of energy by moving object
If peripheral regions are rendered at lower fidelity, then memory bandwidth and power usage are reduced, but overall image quality may deteriorate
Solution Approach 1:
The patent applies local quality by matching rendering resolution to human visual perception characteristics. The central region, corresponding to the foveal vision area where users focus attention, is rendered at high resolution to maintain image quality. Peripheral regions, corresponding to peripheral vision where detail discrimination is reduced, are rendered at lower resolution. This selective quality approach ensures that power reduction in peripheral regions does not noticeably impact overall user experience.
4Measurement precision
If the entire input surface is rendered at high fidelity, then maximum image quality is achieved, but processing efficiency and productivity decrease
Solution Approach 1:
The patent segments the rendering task into high-priority central region rendering and lower-priority peripheral region rendering. By processing the central region at high resolution first and peripheral regions at lower resolution, the system improves processing efficiency while maintaining acceptable overall quality. This segmentation allows for better resource allocation and faster frame generation.
Solution Approach 2:
The patent applies partial action by rendering only the necessary portions of the input surface at high fidelity (the central region), while using reduced fidelity for peripheral regions. This partial high-quality rendering approach achieves sufficient image quality for the user's focal area without the excessive processing requirements of rendering the entire surface at maximum resolution, thereby improving processing efficiency.
Data Source
AI summary
A data processing system for providing an output surface for display. The data processing system includes rendering circuitry operable to generate one or more input surfaces to be used for providing an output surface for display. The rendering circuitry is operable to generate a peripheral region of an input surface at a lower fidelity than the fidelity at which a central region of the input surface is generated or is operable to generate one of a plurality of input surfaces at a lower fidelity than the fidelity at which another of the plurality of input surfaces is generated. The data processing system also includes display composition circuitry operable to select part of at least one of the one or more generated input surfaces based on received view orientation data to provide an output surface for display.