Gaze-Based Exposure for Mixed Reality HDR Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed and augmented reality systems face challenges in providing an optimal exposure experience, as existing auto-exposure methods can result in lost detail and artifacts in peripheral areas, leading to lower image quality.
Innovation Solution
A gaze-based exposure method is implemented in MR systems, where the camera exposure is adjusted based on the user's gaze direction, using selective auto-exposure for a region of interest and ambient lighting information, preserving HDR details and compensating the rest of the scene, thereby providing a foveated HDR experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional auto-exposure methods are used in MR systems, then the exposure is applied uniformly across the entire scene, but this results in lost detail and artifacts in peripheral areas, leading to lower image quality
Solution Approach 1:
The patent applies different exposure qualities to different regions of the image based on the user's gaze. The foveal region (where the user is looking) receives optimized exposure processing, while peripheral regions use standard exposure. This local differentiation ensures high image quality in the important central region without wasting computational resources on peripheral areas, thereby improving overall image quality while minimizing detail loss where it matters most.
2Reliability
If auto-exposure is applied to the entire scene, then all areas are processed equally, but this creates artifacts in peripheral areas where the user is not looking
Solution Approach 1:
The patent segments the image into different regions based on the user's gaze direction. The scene is divided into a foveal region (center of gaze) and peripheral regions. Auto-exposure processing is selectively applied only to the foveal region, while peripheral regions are excluded from this processing. This segmentation eliminates artifacts in peripheral areas caused by inappropriate exposure adjustments, while maintaining exposure consistency in the important central region.
3Loss of information
If multiple exposures are captured for HDR photography, then detailed information is preserved across different lighting conditions, but this increases the complexity and time required for image processing
Solution Approach 1:
The patent performs preliminary action by capturing a single optimized exposure image targeted at the user's gaze direction, rather than capturing multiple exposures as in traditional HDR photography. By using eye tracking to determine the region of interest in advance, the system can capture one high-quality exposure that is specifically optimized for what the user will actually look at, eliminating the need for multiple captures and subsequent merging processing, thus reducing processing time while preserving HDR details in the important central region.
Data Source
AI summary
A processing pipeline and method for mixed reality systems that utilizes selective auto-exposure for a region of interest in a scene based on gaze and that compensates exposure for the rest of the scene based on ambient lighting information for the scene. Images may be generated for display that provide an exposure-compensated, foveated high dynamic range (HDR) experience for the user.