Foveated Image Processing for Head-Mounted Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing techniques for head-mounted displays (HMDs) fail to produce optimal foveated images, resulting in suboptimal resolution, improper pixel brightness, and misalignment issues during optical combination, leading to a poor user experience.
Innovation Solution
A system and method that process images by generating a first image with reduced intensity and resolution, and a second image corresponding to a cropped region of the input image, with adjusted intensity, to be rendered on separate image renderers, ensuring optimal overlap and blending to mimic human vision, accommodating slight misalignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional image processing techniques are used to generate different resolution images, then the processing complexity is reduced, but the image quality and alignment precision deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing transformation parameters (rotation angles, scaling factors, translation vectors) for multiple resolution levels before actual image processing. When generating foveated images, these pre-computed parameters are directly applied to quickly transform the high-resolution reference image into multiple resolution versions, eliminating the need for complex real-time processing while ensuring precise alignment between different resolution images.
Solution Approach 2:
The patent replaces complex mechanical image alignment systems with mathematical transformation operations. Instead of using physical adjustment mechanisms to align images of different resolutions, the system uses calculated transformation matrices and coordinate mapping to precisely position each resolution level, achieving sub-pixel alignment accuracy through computational methods rather than mechanical means.
2Adaptability or versatility
If multiple images with different resolutions are optically combined, then foveated imaging is achieved, but misalignment causes improper pixel intensities along common edges
Solution Approach 1:
The patent applies local quality by implementing different processing strategies for different regions of the image. The foveal region (center) uses high-resolution images with precise transformation, while the peripheral regions use progressively lower resolution images. The transition between regions uses locally adapted blending functions that smooth intensity variations. This regional differentiation maintains pixel intensity accuracy at edges while enabling the multi-resolution foveated imaging structure.
Solution Approach 2:
The patent introduces an intermediary processing layer that includes gradient computation and blending functions. Before optically combining multiple resolution images, the system calculates gradient fields at edges and applies intermediate blending operations to smooth transitions. This intermediary processing prevents improper pixel intensities along common edges by gradually transitioning between different resolution levels rather than abrupt combinations.
3Manufacturing precision
If high-resolution images are displayed across the entire field of view, then image quality is maximized, but the energy consumption and device complexity increase
Solution Approach 1:
The patent segments the visual field into multiple resolution zones: a central foveal region displaying high-resolution images and peripheral regions displaying progressively lower resolution images. This segmentation allows the system to concentrate computational and display resources only where visually necessary, reducing overall energy consumption while maintaining high image quality in the critical central viewing area. The segmented approach enables variable resolution rendering that adapts to human visual acuity characteristics.
Solution Approach 2:
The patent applies local quality by assigning different resolution levels to different spatial locations in the display field. The central foveal region receives high-resolution rendering with full detail, while peripheral regions use lower resolution with reduced detail. This localized quality differentiation matches human visual perception, where acuity is highest at the center of gaze and decreases toward the periphery, thereby reducing energy consumption without perceptibly degrading the overall visual experience.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
Disclosed is system (100, 200) for processing images for display apparatus, system is communicably coupled to display apparatus (102, 202), display apparatus comprising first image Tenderer (104, 204, 406) and second image Tenderer (106, 206), system comprises image source (108, 208, 408) and processor (110, 210), wherein image source produces input image (300), processor of system being configured to: process input image to generate first image (302, 402) such that first region of first image is blurred and its intensity is reduced with respect to intensity of corresponding region of input image; and process input image to generate second image, second image corresponding to cropped region of input image, intensity of second image being adjusted according to intensity of aforesaid first region; wherein processor of system or processor (112, 212) of display apparatus renders first and second images at first and second image renderers, respectively, projections of rendered first and second images being optically combined such that projection of rendered second image overlaps with projection of first region of first image.