HMD Video Processing with Gaze-to-Head Tracking Fallback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing head-mounted displays (HMDs) fail to replicate the full range of human physiological and psychovisual responses to viewed images, leading to suboptimal user experiences.
Innovation Solution
Implement gaze and head tracking systems within HMDs, utilizing cameras and processing units to adjust displayed content based on user eye and head movements, incorporating features like foveal rendering and stereoscopic imaging to enhance realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gaze direction detection is used to control video images, then user immersion and interaction naturalness are improved, but system reliability deteriorates when gaze detection fails or provides inaccurate data
Solution Approach 1:
The system dynamically switches between different control parameters (gaze direction vs head orientation) based on detection quality. When gaze detection reliability drops below a threshold, the system changes to using head orientation data instead, maintaining reliable operation while preserving natural interaction where possible
Solution Approach 2:
The controller acts as an intermediary that mediates between the gaze detector/head tracker and the image processor. It monitors the quality of gaze detection data and selectively switches control sources, ensuring reliable operation while maintaining interaction naturalness through intelligent data selection
2Reliability
If multiple detection systems (gaze detector and head tracker) are integrated, then user experience realism is improved, but device complexity increases
Solution Approach 1:
The controller performs multiple functions: it processes gaze detection data, evaluates its quality, switches between different control sources (gaze and head orientation), and coordinates with the image processor. This multi-functionality consolidates complexity into a single control unit rather than requiring separate systems for each function
Solution Approach 2:
The system merges gaze detection and head tracking into a unified control framework where both sensors contribute to a single control objective. The controller combines data from both sources and selectively uses the most reliable one, reducing overall system complexity compared to maintaining entirely separate control pathways
3Reliability
If the system switches control functions from gaze direction to head orientation, then system reliability is improved during detection failures, but interaction naturalness deteriorates
Solution Approach 1:
The system dynamically adjusts its control mechanism based on real-time detection quality. When gaze detection is reliable, it uses gaze-directional control for natural interaction. When detection fails or becomes inaccurate, it dynamically switches to head orientation control to maintain reliability, creating a flexible system that adapts to changing conditions
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
Apparatus comprises a video display to display video images to a user; a gaze detector configured to detect a gaze direction for one or both eyes of the user while the user views the display; a head tracker configured to detect a head orientation of the user; an image processor configured to generate the video images for display by the video display; the image processor being responsive to one or more control functions dependent upon the gaze direction detected by the gaze detector; and a controller configured to detect a predetermined condition and, in response to detection of the predetermined condition, to control the image processor to be responsive to one or more control functions dependent upon the head orientation detected by the head tracker in place of the one or more control functions dependent upon the gaze direction detected by the gaze detector.