Coarse-to-Fine Neural Networks for Real-Time Gaze Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking systems are resource-intensive or suffer from high latency, making them unsuitable for applications requiring high accuracy and real-time performance.
Innovation Solution
A two-phase approach using coarse and fine convolutional neural networks (CNNs) for pupil center estimation, optimized for GPU processing, which includes a coarse CNN for initial estimation and a fine CNN for refined localization, with a threshold-based mechanism to reduce redundant computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing gaze tracking systems use high-accuracy algorithms, then measurement precision is improved, but use of energy and computational resources increases
Solution Approach 1:
The patent divides the gaze estimation task into two separate neural networks: a coarse CNN for initial pupil center estimation and a fine CNN for refined localization. This segmentation allows the system to process images in stages, using the coarse network to identify candidate regions and the fine network to provide precise measurements only where needed, thereby reducing overall computational resource consumption while maintaining high accuracy.
Solution Approach 2:
The system applies the fine CNN only to specific regions identified by the coarse CNN rather than processing entire images with the high-accuracy network. This partial action approach ensures that computationally intensive operations are performed only where necessary, reducing energy consumption while preserving measurement precision in critical areas.
2Measurement precision
If existing gaze tracking systems use high-accuracy algorithms, then measurement precision is improved, but productivity decreases
Solution Approach 1:
By segmenting the processing into coarse and fine stages, the system achieves high accuracy through the fine CNN while the coarse CNN quickly filters and prepares data, improving overall processing throughput and productivity without sacrificing measurement precision.
Solution Approach 2:
The fine CNN is applied only to selected regions rather than entire images, reducing the total number of computations required and thereby increasing processing speed and productivity while maintaining high accuracy in the regions that matter most.
3Measurement precision
If existing gaze tracking systems use high-accuracy algorithms, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The coarse CNN performs preliminary processing to identify and prepare candidate regions before the fine CNN applies refined localization. This preliminary action reduces the amount of data the fine network must process, thereby reducing latency and meeting real-time performance requirements while maintaining high accuracy.
Solution Approach 2:
By applying the computationally intensive fine CNN only to specific regions rather than entire images, the system reduces processing time and latency while preserving measurement precision where it is most needed.
Data Source
AI summary
Apparatuses, systems, and techniques are presented to estimate user gaze. In at least one embodiment, one or more neural networks are used to determine coarse and fine gaze estimates for one or more users.