Foveated Image Upscaling Using Variable Kernel Sizes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing quality of images in content display contexts, such as HMDs, leads to higher data requirements and processing demands, resulting in increased bandwidth and latency issues.
Innovation Solution
An image processing method that receives gaze data to determine the user's gaze location, and performs an upscaling process using different kernel sizes for different regions of the image, with a larger kernel size used for the gaze-focused region and a smaller kernel size for the remaining regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If image quality is increased to provide high-quality imagery for HMD, then image quality is improved, but bandwidth requirements and processing demands increase significantly
Solution Approach 1:
The patent applies local quality by rendering different regions of the image at different resolutions based on user gaze data. The foveal region (where the user is looking) is rendered at high resolution, while peripheral regions are rendered at lower resolution. This resolves the contradiction by providing high image quality only where needed, reducing overall data size while maintaining perceived quality.
2Manufacturing precision
If image quality is increased to provide high-quality imagery for HMD, then image quality is improved, but processing time and latency increase
Solution Approach 1:
The patent reduces processing time and latency by applying local quality - rendering only the foveal region at high resolution while using lower resolution for peripheral regions. This selective approach significantly reduces the total processing workload compared to rendering the entire image at high quality, thereby reducing latency while maintaining perceived image quality.
3Quantity of substance
If foveated rendering is used to reduce data size, then data size is reduced, but image quality in non-focused areas deteriorates
Solution Approach 1:
The patent applies local quality by rendering different regions of the image at different resolutions based on user gaze data. The foveal region (where the user is looking) is rendered at high resolution, while peripheral regions are rendered at lower resolution. This resolves the contradiction by providing high image quality only where needed, reducing overall data size while maintaining perceived quality.
4Manufacturing precision
If uniform high resolution is applied to the entire image, then image quality is improved, but computational cost increases
Solution Approach 1:
The patent applies local quality by rendering different regions of the image at different resolutions based on user gaze data. The foveal region (where the user is looking) is rendered at high resolution, while peripheral regions are rendered at lower resolution. This resolves the contradiction by providing high image quality only where needed, reducing overall data size while maintaining perceived quality.
Data Source
AI summary
There is provided an image processing method. The method includes receiving an image, receiving gaze data indicative of a gaze location of a user for the image, performing an upscaling process on at least part of the received image to increase quality of the at least part of the received image, and outputting the upscaled image to a display device. Performing the upscaling process includes upscaling a first region of the received image corresponding to the gaze location of the user for the image using a first kernel size and upscaling a second region of the received image using a second kernel size, where the first kernel size is larger than the second kernel size.