Gaze-Driven AR/XR Display Focus for Salience-Based Attention Redirection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In immersive AR/XR environments, users often struggle to quickly identify critical visual data due to overwhelming amounts of visual information, leading to potential safety issues, particularly in applications like remote commercial aircraft piloting.
Innovation Solution
A system that utilizes environment imaging and user state sensors to track user gaze and environmental elements, determining salience values based on element identities and awareness, and dynamically adjusts display focus to redirect attention to critical elements using focus adjustments and transitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If users are presented with comprehensive visual data in AR/XR environments, then situational awareness is improved, but cognitive overload increases and response time to critical data deteriorates
Solution Approach 1:
The system applies different visual properties to different regions of the display based on their importance. Critical elements receive enhanced visual treatment (sharp focus, higher brightness) while less important elements are rendered with reduced visual prominence (blurred focus, lower brightness), allowing users to quickly identify critical information without being overwhelmed by all visual data simultaneously
Solution Approach 2:
The display is segmented into multiple focal regions with different levels of sharpness and visual prominence. The system divides the visual field into a focused region (sharp), intermediate region (moderately blurred), and peripheral region (heavily blurred), enabling users to process critical information in the focused region while maintaining contextual awareness of surrounding elements
2Loss of information
If comprehensive visual data is displayed in AR/XR environments, then information completeness is improved, but user attention management deteriorates
Solution Approach 1:
The system dynamically adjusts the focal region and visual prominence of elements based on real-time analysis of user gaze direction, head orientation, and the criticality of environmental elements. The focused region moves and resizes automatically to highlight currently critical elements, while the system continuously reevaluates which elements require attention based on changing situational context
Solution Approach 2:
The system incorporates user state sensors (eye tracking, head tracking) to monitor user attention and behavior in real-time. This feedback is used to dynamically adjust which elements receive enhanced visual treatment, creating a closed-loop system that adapts to individual user needs and attention patterns while managing the complexity of comprehensive visual data presentation
3Loss of time
If focus adjustment is applied to highlight critical elements, then response time to critical data is improved, but immersion in the environment may deteriorate
Solution Approach 1:
Instead of applying uniform visual effects across the entire display, the system applies focus adjustments and visual enhancements only to specific local regions containing critical elements. This localized approach allows critical information to stand out while preserving the natural appearance and immersion of the surrounding environmental context
Solution Approach 2:
The system applies visual enhancement selectively and partially - only to the extent necessary to draw attention to critical elements. The enhancement is calibrated to provide just enough visual prominence to ensure rapid detection of critical data without over-enhancing to the point of creating an artificial or non-immersive appearance
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A system and method for attention redirection is disclosed. The system may include a display (112), an environment imaging sensor (204) configured to receive environment images comprising elements of an environment, a user state sensor (202) configured to receive user images comprising a user gaze, and a controller (102). The controller (102) may be configured to execute program instructions to receive environment images, identify elements and their identities and locations, receive user images comprising the user gaze, track the user gaze direction, determine salience values associated with the elements, and direct a focus adjustment of at least one region of the display (112) based on the salience values.