Metamerisation for Near-Eye Display Foveated Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image rendering technologies for Near-Eye Displays (NEDs) face high computational loads due to uniform sampling, with foveated rendering and ray-tracing methods either introducing unnatural blur or being too processing-intensive, and existing methods fail to account for the different perceptual characteristics of the fovea and periphery.
Innovation Solution
A method and system that divide an input image into foveal and peripheral regions, determining statistics for the periphery to create a metamer that matches the original image's perception, reducing computational requirements while maintaining image quality by using metamers to construct peripheral regions, which are perceived as identical to the original.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If uniform sampling is used to render images at high resolution, then image quality is improved, but computational load increases quadratically
Solution Approach 1:
The image is divided into foveal and peripheral regions, allowing different rendering strategies to be applied to each region. The foveal region receives high-resolution rendering while peripheral regions use lower-resolution metamers, thus reducing overall computational load while maintaining perceived image quality.
Solution Approach 2:
Different quality levels are applied to different regions of the image based on human visual system characteristics. The foveal region maintains high resolution and detail, while peripheral regions use band-limited metamers with fewer samples, optimizing the balance between computational efficiency and perceived quality.
2Power
If foveated rendering is used to reduce computational effort in peripheral regions, then computational load is reduced, but the blur appears unnatural and does not match HVS perception
Solution Approach 1:
Band-limited metamers are generated as alternative representations of peripheral image regions. These metamers are constructed from fewer samples and have reduced frequency content, but they are designed to be perceptually indistinguishable from the original high-resolution regions, thus maintaining perceptual accuracy while reducing computational load.
3Measurement precision
If ray-tracing is used to cast more rays to the foveal area, then foveal resolution is improved, but processing time becomes too slow for large and dynamic scenes
Solution Approach 1:
The rendering process is segmented into foveal and peripheral regions with different sampling densities. The foveal region uses appropriate sampling methods while peripheral regions use band-limited metamers, avoiding the need for ray-tracing in low-priority areas and thus reducing overall processing time.
4Speed
If neural networks are used to reconstruct images from dense foveal and sparse peripheral samples, then temporal resolution is improved, but the loss is uniform and does not consider different perceptual characteristics
Solution Approach 1:
The metamerisation process applies different quality levels to different spatial regions. Peripheral regions use band-limited metamers constructed from fewer samples, while the foveal region maintains higher quality. This spatially-varying approach matches human perceptual characteristics better than uniform loss methods.
Data Source
AI summary
A method for creating a metamer for an image for a display, the method comprising receiving a first input image, dividing the input image into a plurality of regions comprising a foveal region and at least one peripheral region, wherein each region of the plurality of regions comprises a plurality of pixels, determining, for each of the at least one peripheral region, the distribution of statistics associated with each of the at least one peripheral region, for each of the at least one peripheral region, identifying a metamer for the peripheral region wherein the metamer has similar and/or identical distribution of statistics to the associated peripheral region, and creating an output image based on the foveal region and the metamer for each of the at least one peripheral region such that the peripheral region of the output image is perceived to be the same as the peripheral region of the input image when perceived by a viewer of the image.