Eye-Gaze Spatial Foveation for Lower-Load AR Image Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in providing a comfortable and natural presentation of virtual image elements amidst real-world imagery due to the complexity of the human visual perception system, leading to inefficiencies in memory and processing loads.
Innovation Solution
The method and system utilize eye gaze location to perform spatial foveation and defoveation processes, reducing memory access and power consumption by compressing image quality in regions outside the user's focus and employing a warp reprojection processor for head pose correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional image processing is used in augmented reality systems, then complete image data is processed and displayed, but memory access and processing load increase significantly
Solution Approach 1:
The image is divided into multiple regions based on eye gaze location, with the foveal region (where the user is looking) processed at full resolution and peripheral regions processed at reduced resolution. This segmentation allows the system to maintain high image quality where needed while reducing overall processing load.
Solution Approach 2:
Different quality levels are applied to different spatial regions of the image. The foveal region maintains high quality for natural perception, while peripheral regions use lower quality to reduce bandwidth and processing requirements. This local differentiation resolves the contradiction between complete image processing and reduced load.
2Reliability
If high quality image data is transmitted and processed, then image quality is maintained, but memory consumption and power usage increase
Solution Approach 1:
The image data is segmented into high-quality foveal region and reduced-quality peripheral regions, allowing the system to transmit and process only necessary high-quality data for the foveal region while using lower quality for peripheral areas, thereby reducing overall power consumption.
Solution Approach 2:
The system dynamically changes the quality parameter of image data based on eye gaze location. High quality is applied to the foveal region and reduced quality to peripheral regions, optimizing the balance between image quality and power consumption.
3Productivity
If spatial foveation is applied to reduce processing load, then memory access is reduced, but image quality may be compromised in peripheral regions
Solution Approach 1:
The system applies different quality levels locally: high quality is maintained in the foveal region where the user is looking, while peripheral regions use reduced quality. This ensures that image quality is not compromised where it matters most while still achieving processing efficiency gains.
4Reliability
If complete image data is processed, then all regions are maintained at full resolution, but system resources are consumed unnecessarily
Solution Approach 1:
The system extracts and processes only the essential high-quality image data for the foveal region, while using reduced quality for peripheral regions. This extraction approach removes unnecessary processing from peripheral areas, reducing energy waste while maintaining quality where needed.
Data Source
AI summary
A method includes determining an eye gaze location of a user and generating a spatial foveation map based on the eye gaze location. The method also includes receiving an image, forming a spatially foveated image using the image and the spatial foveation map, and transmitting the spatially foveated image to a wearable device. The method further includes spatially defoveating the spatially foveated image to produce a spatially defoveated image and displaying the spatially defoveated image.