Gaze Tracking Neural Network for Head-wearable Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current head-mounted eye trackers face challenges such as bulkiness, limited integration into glasses frames, high power consumption, and reduced accuracy due to environmental stray light, making them uncomfortable and less accurate for everyday use.
Innovation Solution
A head-wearable spectacles device with strategically placed cameras and a convolutional neural network system that captures images of both eyes, allowing for improved gaze direction prediction and reduced computational demands, while being designed for ergonomic integration and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution cameras are used for eye tracking, then measurement precision is improved, but device complexity and weight increase
Solution Approach 1:
The patent replaces traditional mechanical feature extraction algorithms with a deep learning-based neural network system. Instead of using complex computer vision algorithms to extract pupil center, infrared glint position, and pupil contours from high-resolution images, the system uses a convolutional neural network that processes image data directly to predict gaze direction, thereby reducing computational complexity while maintaining accuracy
Solution Approach 2:
The patent changes the processing approach by transforming the output from raw image pixels to feature representations through neural network layers. The system processes images at a lower resolution (e.g., 320x240 pixels) rather than requiring high-resolution images, reducing the computational burden while the neural network learns to extract necessary gaze information from these compressed representations
2Measurement precision
If high-resolution cameras and processing units are used, then measurement precision is improved, but power consumption increases
Solution Approach 1:
The patent applies partial action by processing only the necessary portions of the image data through the neural network. Instead of processing entire high-resolution images, the system uses lower-resolution images (320x240 pixels) and extracts only the essential gaze-related features through the neural network's convolutional layers, significantly reducing computational energy requirements while maintaining sufficient accuracy for gaze estimation
Solution Approach 2:
The system replaces energy-intensive traditional computer vision algorithms with a neural network model that has been pre-trained on large datasets. This substitution allows the system to achieve accurate gaze estimation with much lower computational power, enabling real-time processing on mobile devices without requiring high-power processors or continuous high-resolution image capture
3Ease of operation
If cameras are integrated into glasses frames, then ease of operation is improved, but manufacturing precision requirements increase
Solution Approach 1:
The patent incorporates a calibration mechanism that dynamically adjusts the system's understanding of camera positions and eye geometry. Instead of requiring fixed, precision-manufactured camera placements, the system uses an interactive calibration process where the user performs specific eye movements and looks at calibration targets, allowing the neural network to learn the actual geometric relationships between cameras, eyes, and display in real-time, thereby accommodating manufacturing variations
Solution Approach 2:
The calibration process enables the system to self-adjust and self-optimize its gaze estimation parameters for each user. The neural network automatically learns the specific geometric relationships and calibration data during the calibration process, eliminating the need for precise factory calibration and allowing the system to adapt to individual users' facial geometries and camera positions without requiring high manufacturing precision
4Measurement precision
If manual camera adjustment is required for each user, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The calibration process is designed to be user-friendly and automated. Instead of requiring manual camera adjustment or complex setup procedures, the system performs automatic calibration by having the user complete simple instructions such as looking at specific calibration targets or performing eye movements. The neural network automatically captures these actions, extracts the necessary calibration data, and adjusts the gaze estimation parameters without requiring technical expertise or time-consuming manual configuration
Solution Approach 2:
The system performs preliminary calibration actions during an initial setup process, where the user completes calibration tasks before normal usage begins. This preliminary calibration establishes the neural network's understanding of the user's eye geometry and camera positions, allowing for accurate gaze estimation throughout subsequent use without requiring repeated manual adjustment or recalibration, thereby reducing time loss during actual application
Data Source
Figure 1A~1C
Figure 2A~2C
Figure 3A~3D
AI summary
A method for detecting one or more gaze-related parameters of a user includes creating a left image of at least a portion of a left eye of the user using a first camera of a head-wearable device worn by the user, creating a right image of at least a portion of a right eye of the user using a second camera of the head-wearable device, feeding the left and right images together as an input into a convolutional neural network, and, obtaining the one or more gaze-related parameters from the convolutional neural network as a result of the left and right images input.