Gaze Tracking via Convolutional Neural Network and Head Pose
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gaze tracking systems are limited in determining gaze direction from arbitrary viewpoints, especially when the eyes become self-occluded, as they require the subject's position to remain within a specific field of view, restricting movement and accuracy.
Innovation Solution
A system utilizing a convolutional neural network trained on a 360-degree image dataset from multiple cameras positioned around the subject, which captures head and eye appearances to predict gaze direction vectors, transitioning from eye to head position reliance when eyes are occluded, allowing gaze tracking from various viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dedicated eye-tracking devices with fixed field of view are used, then measurement precision of gaze direction is improved, but device complexity and restriction of subject movement increase
Solution Approach 1:
The patent transitions from 2D eye-tracking to 3D gaze estimation by incorporating head pose information (pitch, yaw, roll angles) alongside eye position data. This dimensional expansion allows the system to accurately track gaze direction even when eyes are occluded, as the head orientation provides additional spatial context for determining where the subject is looking.
Solution Approach 2:
The patent introduces head pose estimation as an intermediary component that bridges the gap between eye position and gaze direction. By first determining head orientation and then combining it with eye position (when visible) or using it alone (when eyes are occluded), the system achieves continuous gaze tracking without requiring direct eye visibility, thus reducing system complexity while maintaining accuracy.
2Measurement precision
If dedicated eye-tracking devices with fixed field of view are used, then measurement precision of gaze direction is improved, but adaptability to arbitrary viewpoints is reduced
Solution Approach 1:
The patent creates a universal gaze tracking system that functions across multiple viewpoints and scenarios. By training the neural network on diverse data including various head poses, distances, and lighting conditions, the system achieves multi-functionality that allows it to accurately estimate gaze direction whether the subject is facing forward, turned away, or whether eyes are visible or occluded, thus greatly enhancing adaptability while maintaining precision.
Solution Approach 2:
The patent employs parameter changes by dynamically adjusting the input parameters to the neural network based on visibility conditions. When eyes are visible, the system uses both head pose and eye position parameters; when eyes are occluded, it relies primarily on head pose parameters. This adaptive parameter selection allows the system to maintain accuracy across varying viewpoints and occlusion conditions.
3Adaptability or versatility
If wearable eye-tracking devices are used, then adaptability to arbitrary viewpoints is improved, but measurement precision and intrusiveness worsen
Solution Approach 1:
The patent uses a non-wearable camera system that captures images of the subject's head and eyes, creating a visual copy of the subject's appearance and eye position. This external observation approach achieves viewpoint flexibility similar to wearable devices while avoiding the intrusiveness and potential accuracy degradation associated with wearing equipment on the subject's face or head.
Data Source
AI summary
A system for determining the gaze direction of a subject includes a camera, a computing device and a machine-readable instruction set. The camera is positioned in an environment to capture image data of head of a subject. The computing device is communicatively coupled to the camera and the computing device includes a processor and a non-transitory computer-readable memory. The machine-readable instruction set is stored in the non-transitory computer-readable memory and causes the computing device to: receive image data from the camera, analyze the image data using a convolutional neural network trained on an image dataset comprising images of a head of a subject captured from viewpoints distributed around up to 360-degrees of head yaw, and predict a gaze direction vector of the subject based upon a combination of head appearance and eye appearance image data from the image dataset.