Foveated Display Stream Compression via Region Reshaping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current foveated rendering techniques do not effectively reduce the volume of information transmitted to displays in head-mounted devices, as conventional display stream compression methods introduce visible errors when applied to multiplexed high-acuity and low-acuity regions of different dimensions, leading to increased system complexity and power consumption.
Innovation Solution
Reshaping and reorganizing the high-acuity region based on the dimensions of the low-acuity region to form a display stream, which is then encoded using display stream compression (DSC) techniques, allowing for efficient multiplexing and reducing buffer sizes and decoding latencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional DSC compression is applied to multiplexed high-acuity and low-acuity regions of different dimensions, then compression is achieved, but visible errors and artifacts are introduced in the uncompressed data
Solution Approach 1:
The display stream is segmented into separate high-acuity and low-acuity region data streams that are processed independently. The DSC compression is applied separately to each region rather than to the multiplexed stream, preserving visual quality while achieving compression. The segmented streams are then multiplexed for transmission.
2Productivity
If foveated rendering is implemented without reshaping, then rendering efficiency is improved, but preparation time for multiplexing and buffer sizes increase
Solution Approach 1:
The high-acuity region data is reshaped in advance to match the dimensions of the low-acuity region before multiplexing. This preliminary reshaping action prepares the data streams for efficient compression and transmission, reducing preparation time during the actual display pipeline operation and optimizing buffer size requirements.
3Reliability
If high-resolution content is transmitted for the entire field-of-view, then visual quality is maintained, but bandwidth requirements exceed DisplayPort limits and power consumption increases
Solution Approach 1:
Different resolution qualities are applied to different regions of the display based on human visual acuity characteristics. The high-acuity region (central foveal area) receives full-resolution content, while the low-acuity regions (peripheral areas) receive downsampled content. This local quality differentiation maintains perceived visual quality while significantly reducing the total data transmission volume and power consumption.
4Reliability
If uncompressed display stream is transmitted, then visual quality is preserved, but transmission bandwidth exceeds protocol limits and system hardware complexity increases
Solution Approach 1:
The display stream parameters are changed by applying DSC compression with specifically tuned slice heights that are integer fractions of the high-acuity region height. This parameter adjustment enables effective compression of the multiplexed streams while maintaining visual quality, reducing bandwidth requirements to fit within DisplayPort limits, and simplifying hardware requirements by avoiding the need for higher-bandwidth interfaces.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processing unit (120) is configured to render first pixels (420) representative of a high-acuity region (410) in the image (400) and second pixels (415) representative of a low-acuity region (405) in the image. A shaper (155) is configured to reorganize the first pixels based on at least one dimension of the low-acuity region. A multiplexer (160) is configured to multiplex the reorganized first pixels and the second pixels to form a display stream. An encoder (125) is configured to compress the display stream for transmission to a display (130). A decoder (135) is configured to decompress the display stream. A demultiplexer (165) is configured to demultiplex the first pixels and the second pixels. Another processing unit (175) is configured to blend the first pixels and the second pixels to form blended pixel values representative of the image for presentation on a screen (180).