Face-Weighted Auto Exposure for XR User Enrollment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR environments struggle with generating high-quality graphical representations of users due to suboptimal exposure settings during user enrollment processes, leading to grainy and tone-inaccurate avatars.
Innovation Solution
Implementing face-based autoexposure (AE) techniques that adjust exposure settings based on detected face location within sensor data, switching to scene average AE when the face is not detected, and using ROI-weighted AE when the face is present, along with fusion of multiple frames for higher dynamic range images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If scene average AE process is used during user enrollment, then the exposure settings are consistent with general passthrough video generation, but the exposure quality of face regions deteriorates leading to grainy and tone-inaccurate avatars
Solution Approach 1:
The patent applies different AE processes to different regions of the sensor data. When face location data is detected, a first AE process (face-based AE) is applied specifically to the face region, while a second AE process (scene average AE) is applied to the overall scene. This local differentiation ensures optimal exposure quality for the face region without compromising the overall scene capture, directly resolving the contradiction between exposure quality and operational simplicity.
2Manufacturing precision
If face-based AE techniques are implemented during enrollment, then exposure quality of face regions improves, but device complexity increases due to face detection and mode switching mechanisms
Solution Approach 1:
The patent implements dynamic switching between different AE processes based on the detection state. The system automatically transitions between scene average AE and face-based AE depending on whether face location data is detected, and adjusts the weight of face region pixels dynamically. This dynamic adaptation allows the system to achieve high exposure quality when needed while maintaining operational simplicity when face detection is not applicable, thereby managing device complexity effectively.
3Measurement precision
If ROI-weighted AE process is used with face location data, then detail accuracy of user faces improves, but processing time increases due to additional face detection and weighted processing
Solution Approach 1:
The patent performs face detection and determines face location data in advance before applying the AE process. By pre-identifying the face region and preparing the weighted AE parameters beforehand, the system minimizes processing delays during the actual enrollment capture. This preliminary action allows the system to apply the computationally intensive ROI-weighted AE process efficiently, maintaining high detail accuracy while reducing the perceived processing time.
Data Source
AI summary
Facilitating the capture and processing of enrollment data includes: capturing, e.g., by a head-mounted display (HMD) device operating in a first mode (e.g., a passthrough video mode), first sensor data; performing a first autoexposure (AE) process (e.g., a scene average AE algorithm) on the first sensor data; and then determining that the HMD is operating in a second mode (e.g., a user enrollment mode) that is different than the first mode. Once operating in the second mode, the HMD may proceed by: capturing second sensor data; determining face location data for a subject detected in the second sensor data; performing a second AE process (e.g., a face-weighted AE algorithm) on the second sensor data; and generating a graphical representation for the subject (e.g., a so-called “Persona” or other three-dimensional avatar), based, at least in part, on the second sensor data that has had the second AE process performed on it.