RGB Camera Gaze Tracking With Head Pose for Real-Time Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Consumer-grade computing devices lack the specialized hardware and resources to accurately process eye-gaze data in real-time, necessitating additional hardware purchases and complex configurations, which impedes widespread adoption of gaze-tracking technology.
Innovation Solution
Generate an image grid based on facial images using RGB cameras, process it through a sequence of neural networks, including convolutional and fully connected networks, and integrate head pose information to enhance accuracy and reduce resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If specialized hardware (IR light emitter and sensor) is used for gaze tracking, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts the gaze tracking functionality from specialized hardware and implements it using standard consumer-grade computing device components (RGB camera, processor, memory). This removes the need for separate IR emitter and sensor hardware, thereby reducing device complexity while maintaining gaze tracking capability through software-based image processing of standard video feeds
Solution Approach 2:
The patent makes the standard RGB camera serve multiple functions: it acts as both a general-purpose imaging device and a gaze tracking sensor. By processing standard video feed from the existing camera through specialized algorithms, the system achieves gaze tracking without requiring dedicated hardware, thus reducing overall device complexity while preserving measurement precision
2Measurement precision
If specialized hardware is used for gaze tracking, then measurement precision is improved, but ease of operation deteriorates
Solution Approach 1:
The patent removes the requirement for separate specialized hardware components (IR emitter, IR sensor, calibration tools) and implements gaze tracking entirely through software processing of standard camera feeds. This extraction simplifies the user experience by eliminating complex hardware setup, connection, and calibration steps, making the system as easy to operate as any standard camera application
Solution Approach 2:
The system performs automatic calibration and gaze point determination through software algorithms that process standard video feeds. The computing device itself provides all necessary processing capabilities through its existing processor and memory, eliminating the need for external calibration equipment or specialized operational procedures, thereby greatly improving ease of operation
3Adaptability or versatility
If consumer-grade devices process eye-gaze data, then device versatility is improved, but measurement precision deteriorates
Solution Approach 1:
The patent applies advanced image processing parameters and algorithms to compensate for the lower quality of consumer-grade camera sensors. By using sophisticated techniques such as feature detection, pattern recognition, and statistical analysis on standard video feeds, the system achieves measurement precision comparable to specialized hardware while maintaining broad versatility across consumer devices
Solution Approach 2:
The patent combines multiple processing techniques and data sources (video feed analysis, head pose estimation, eye region detection) to create a composite processing system. This multi-faceted approach compensates for the limitations of consumer-grade hardware by integrating several lower-precision measurements into a higher-precision overall gaze determination, thereby achieving both versatility and accuracy
4Productivity
If real-time gaze processing is implemented, then productivity is improved, but use of energy increases
Solution Approach 1:
The patent segments the gaze tracking processing into distinct functional modules: video feed acquisition, eye region detection, feature extraction, gaze point calculation, and output generation. This segmentation allows the system to process only relevant portions of the video feed (focusing computational resources on eye regions rather than entire frames) and enables selective activation of processing stages, thereby reducing overall energy consumption while maintaining real-time productivity
Solution Approach 2:
The patent implements partial processing by focusing computational effort only on regions of interest (eye areas) within the video feed rather than analyzing entire frames. By applying advanced processing selectively to small portions of the image data and using simplified models for non-critical aspects, the system achieves real-time performance with reduced computational load and lower energy consumption
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Systems and methods are provided for predicting an eye gaze location of an operator of a computing device. In particular, the method generates an image grid that includes regions of interest based on a facial image. The facial image is based on a received image frame of a video stream that captures the operator using the computing device. The image grid further includes a region that indicate rotation information of the face. The method further uses a combination of trained neural networks to extract features of the regions of interest in the image grid and predict the eye gaze location on the screen of the computing device. The trained set of neural networks includes a convolutional neural network. The method optionally generate head pose pitch, roll, and yaw information to improve accuracy of predicting the location of an eye gaze.