HMD Eye Tracking via Internal Cameras and Illumination Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual reality systems provide limited graphical representations of users, leading to a non-immersive experience due to reliance on dedicated peripherals and impractical eye-tracking systems, which are not suited for portable, lightweight head-mounted displays.
Innovation Solution
A head-mounted display with multiple image capture devices positioned to capture various parts of the user's face, including a central image capture device between the user's eyes, coupled with a trained convolutional neural network to determine gaze direction and adjust illumination sources for improved image capture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional facial tracking systems use dedicated peripherals and markers, then tracking accuracy is improved, but device complexity and user burden increase
Solution Approach 1:
The patent combines facial tracking functionality with the existing display optics of the HMD. The same optical elements (lenses, mirrors, waveguides) that deliver visual content to the user are also used to capture images of the user's face. This eliminates the need for separate dedicated tracking cameras and markers, reducing device complexity while maintaining tracking capability through multiple images captured from different angles within the optical path.
2Measurement precision
If dedicated eye tracking devices are added to HMD, then gaze tracking precision is improved, but weight and device complexity increase
Solution Approach 1:
The patent makes the existing image capture devices serve multiple functions: they capture both the visual content for the user and simultaneously capture images of the user's face and eyes for tracking. The same optical path and sensors are used for both display and tracking purposes, eliminating the need for additional dedicated eye tracking hardware that would increase weight and complexity.
3Measurement precision
If multiple image capture devices are positioned within HMD to capture face portions, then facial tracking accuracy is improved, but device complexity increases
Solution Approach 1:
The patent divides the face tracking function across multiple image capture devices positioned at different locations within the HMD (e.g., above, below, left, right of the display). Each device captures a specific portion of the user's face, and the system integrates these segmented views to achieve complete facial tracking. This segmentation allows using existing display optics for tracking while maintaining accuracy through multiple perspectives.
Data Source
AI summary
A virtual reality (VR) or augmented reality (AR) head mounted display (HMD) includes multiple image capture devices positioned within the HMD to capture portions of a face of a user wearing the HMD. Images from an image capture device include a user's eye, while additional images from another image capture device include the user's other eye. The images and the additional images are provided to a controller, which applies a trained model to the images and the additional images to generate a vector identifying a position of the user's head and positions of the user's eye and fixation of each of the user's eyes. Additionally, illumination sources illuminating portions of the user's face include in the images and in the additional images are configured when the user wears the HMD to prevent over-saturation or under-saturation of the images and the additional images.