Monocular Gaze Tracking Across Multiple Screens with 3D Head Pose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze position estimation systems are unreliable and computationally inefficient, particularly when determining gaze positions on multiple screens, due to variations in face-camera-screen alignment, user differences, and lack of depth information, leading to inaccurate and computationally costly solutions.
Innovation Solution
A monocular gaze tracking system using a deep neural network (DNN) for real-time gaze position determination on consumer-grade devices, incorporating head pose tracking, geometrical alignment, and calibration techniques, with a method for predicting reliability and maintaining low computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional gaze position estimation systems are used, then gaze position can be determined, but the systems are unreliable and inaccurate particularly on multiple screens
Solution Approach 1:
The patent transforms the gaze estimation problem from direct 2D image space measurement to 3D normalized face coordinate system. By changing the parameter space from pixel coordinates to normalized 3D facial landmarks, the system achieves screen-independent gaze estimation that works reliably across multiple screens with different geometries and positions.
Solution Approach 2:
The system transitions from 2D monocular image analysis to inferred 3D facial geometry by incorporating head pose estimation and 3D facial landmark models. This dimensional transformation allows the system to recover depth information implicitly, enabling accurate gaze projection onto screens at various distances and angles.
2Measurement precision
If conventional methods are used to determine gaze position, then gaze information can be obtained, but computational complexity increases and real-time performance is compromised
Solution Approach 1:
The system performs preliminary normalization of facial landmarks to a standard 3D coordinate system before gaze calculation. By pre-establishing the normalized face geometry and head pose orientation, the actual gaze projection computation becomes a simple mathematical operation rather than a complex optimization problem, enabling real-time performance.
Solution Approach 2:
The patent replaces complex mechanical multi-camera depth sensing systems with a computational approach using monocular vision combined with head pose estimation. Instead of using multiple physical cameras to capture depth, the system computationally infers 3D geometry from 2D images through normalized landmark transformation, significantly reducing hardware complexity and computational load.
3Device complexity
If monocular camera is used for gaze tracking, then device complexity is reduced, but depth information is lost making gaze determination difficult
Solution Approach 1:
The system introduces head pose estimation as an intermediary component that bridges the gap between monocular 2D images and 3D gaze determination. By first estimating head orientation and then normalizing facial landmarks into a 3D coordinate system aligned with the head, the system recovers implicit depth information without requiring depth-sensing hardware.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Systems and methods for real-time, efficient, monocular gaze position determination that can be performed in real-time on a consumer-grade laptop. Gaze tracking can be used for human-computer interactions, such as window selection, user attention on screen information, gaming, augmented reality, and virtual reality. Gaze position estimation from a monocular camera involves estimating the line-of-sight of a user and intersecting the line-of-sight with a two-dimensional (2D) screen. The system uses a neural network to determine gaze position within about four degrees of accuracy while maintaining very low computational complexity. The system can be used to determine gaze position across multiple screens, determining which screen a user is viewing as well as a gaze target area on the screen. There are many different scenarios in which a gaze position estimation system can be used, including different head poses, different facial expressions, different cameras, different screens, and various illumination scenarios.