Attentional Region Rendering for XR Passthrough Lightness Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) systems, particularly video see-through (VST) XR devices, face challenges such as limited fields of view and inefficient processing of user focus regions, leading to suboptimal user experience.
Innovation Solution
Dynamic attentional region generation and rendering techniques that separate image frames into an attentional region and a de-attentional region, using weighting distributions to adjust lightness and processing, enhancing user focus on the attentional region while reducing processing efforts on the de-attentional region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If dynamic attentional region generation is applied to enhance user focus on attentional region, then user experience is improved, but device complexity increases
Solution Approach 1:
The image frame is segmented into two distinct regions: an attentional region where the user is focusing and a de-attentional region. Different processing techniques are applied to each region, with the attentional region receiving enhanced processing for improved user experience, while the de-attentional region receives reduced processing to manage computational complexity.
Solution Approach 2:
Different quality levels of processing are applied locally to different regions of the image. The attentional region receives high-quality processing including enhanced sharpness and detail, while the de-attentional region receives lower-quality processing. This local differentiation improves user experience in the focus area without proportionally increasing overall computational load.
2Ease of operation
If lightness is attenuated from center point towards edges of transformed image frames, then attentional region is enhanced, but processing time increases
Solution Approach 1:
The image processing is segmented into two parallel processing paths: one for the attentional region and one for the de-attentional region. The lightness attenuation operation is applied independently to both regions, allowing simultaneous processing. This segmentation reduces the effective processing time by avoiding sequential operations on the entire image.
Solution Approach 2:
Instead of applying full lightness attenuation across the entire image frame, the system applies the attenuation partially only to the de-attentional region while maintaining normal lightness in the attentional region. This partial action reduces the total processing time while still achieving the desired attentional enhancement effect.
3Manufacturing precision
If passthrough transformations are applied to image frames, then rendering accuracy is improved, but processing power consumption increases
Solution Approach 1:
The passthrough transformation is applied selectively to different regions of the image frame. The attentional region receives the full passthrough transformation to maintain rendering accuracy for the user's focus area, while the de-attentional region receives reduced or simplified transformations. This segmentation maintains rendering accuracy where needed while reducing overall processing power consumption.
Data Source
AI summary
A method includes obtaining one or more image frames of a scene and data associated with the one or more image frames where the data includes user eye behavior data. The method also includes applying passthrough transformations on the one or more image frames to generate one or more transformed image frames. The method further includes identifying an attentional region in the one or more transformed image frames based on the user eye behavior data. The method also includes adjusting lightness of the one of more transformed image frames using a weighting distribution to generate one or more modified image frames where the lightness is attenuated from a center point of the attentional region towards edges of the one or more transformed image frames. In addition, the method includes rendering one or more images for display based on the one or more modified image frames.