Featureless Gaze Tracking Under Masks, Low Light, and Distance Changes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking methods relying on facial feature extraction fail in the presence of complex artifacts such as face masks, low lighting, and varying distances, leading to incomplete or inaccurate gaze tracking.
Innovation Solution
A method that uses a convolutional neural network to analyze raw video frames directly, extracting spatial hierarchy features without facial feature extraction, trained on a diverse dataset including various environmental conditions and user demographics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If facial feature extraction methods are used for gaze tracking, then the system can achieve gaze tracking under normal conditions, but the system fails when complex artifacts such as face masks, low lighting, or varying distances are present
Solution Approach 1:
The patent removes the facial feature extraction step from the traditional gaze tracking pipeline. Instead of extracting facial features as an intermediate step, the system directly processes raw video frames through a deep convolutional neural network to predict gaze location, thereby eliminating the vulnerability to facial artifacts while maintaining gaze tracking capability
Solution Approach 2:
The patent introduces a deep convolutional neural network as an intermediary between raw video input and gaze prediction output. This neural network acts as a mediator that automatically learns robust feature representations from raw pixels, bypassing the need for manual facial feature extraction and enabling reliable gaze tracking under varied conditions
2Measurement precision
If a custom ecologically valid dataset including face masks and varying conditions is used for training, then predictive capabilities in harsh conditions are enhanced, but the complexity of data collection and processing increases
Solution Approach 1:
The patent employs preliminary action by collecting and preparing a diverse training dataset in advance that includes various ecological conditions such as face masks, different lighting, and varying distances. This pre-prepared dataset enables the neural network to learn robust patterns before deployment, improving prediction accuracy without adding complexity during actual gaze tracking operations
Solution Approach 2:
The patent applies parameter changes by varying multiple parameters in the training dataset including lighting conditions, subject distance, presence of face masks, and camera angles. This diverse parameter variation in training data enables the model to generalize better to real-world conditions while the system itself remains simple during operation
Data Source
AI summary
Systems and methods are disclosed for gaze tracking. A method includes receiving a video of a user taken by a front-facing camera of a device having a screen, receiving dimensions of the screen, parsing the video into a series of uniform-dimension video frame images, inputting the series of uniform-dimension video frame images to a pretrained artificial neural network, thereby extracting a plurality of features from the series of uniform-dimension video frame images to determine a set of internal spatial hierarchy features on each of the uniform-dimension video frame images, inputting each set of hierarchy features to a fully connected layer, the fully connected layer producing an intermediate physical estimate of the user's gaze location on the screen relative to the device camera in centimeters, determining a series of screen locations based on the intermediate physical estimate and the dimensions of the screen, and labeling the gaze location on the screen.