3D Gaze Tracking on Mobile Devices Without Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze tracking methods for computing devices are invasive, require additional equipment, have high energy consumption, and lack scalability, especially on mobile devices, while current attention measurement techniques are inaccurate and unreliable.
Innovation Solution
A method using a camera and LiDAR sensor on a computing device to capture 3D eye and face images, combined with machine learning algorithms, including convolutional and fully connected neural networks, to estimate visual attention without calibration, leveraging historical gaze data and depth information for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If invasive eye tracking methods or specialized equipment are used, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses the device's existing camera to capture images of the user's eyes and creates a computational model that copies the functionality of specialized eye tracking hardware. By processing images of the eye region through machine learning algorithms, the system replicates gaze tracking capabilities without requiring dedicated eye tracking sensors or invasive equipment.
Solution Approach 2:
The patent replaces physical/mechanical eye tracking systems (such as infrared sensors, head-mounted devices, or contact lenses) with a software-based computational approach. Instead of using mechanical or optical hardware to directly measure eye position, the system uses image processing and machine learning to estimate gaze from standard camera images, substituting a mechanical measurement system with an information-processing system.
2Measurement precision
If specialized eye tracking hardware is used, then measurement precision is improved, but ease of manufacture and scalability worsen
Solution Approach 1:
The patent makes the device's camera serve multiple functions: it is used both for standard device operation (capturing photos or video) and for gaze tracking by analyzing the eye region in the captured images. This multi-functionality allows the same hardware component to provide both conventional imaging and attention measurement capabilities, eliminating the need for separate specialized equipment and improving scalability.
Solution Approach 2:
The patent enables the computing device to perform its own gaze tracking using its existing camera and processor resources. The device processes its own captured images through machine learning algorithms to determine user attention, making the system self-sufficient without requiring external specialized equipment or continuous calibration procedures.
3Measurement precision
If continuous calibration is required, then measurement precision is improved, but loss of time and ease of operation worsen
Solution Approach 1:
The patent performs preliminary training of the machine learning model during the device setup or initial use, so that the system is pre-calibrated and ready for immediate gaze tracking without requiring continuous calibration sessions. The model learns user-specific characteristics in advance, enabling accurate attention measurement from the start of normal device operation.
Solution Approach 2:
The system uses feedback from the captured images and detected eye features to continuously refine gaze estimates without requiring explicit user calibration actions. The machine learning model processes the visual data and adjusts its predictions based on the detected patterns, enabling the system to adapt and maintain accuracy automatically during normal operation.
4Measurement precision
If expensive sensors are used, then measurement precision is improved, but loss of substance (cost) worsens
Solution Approach 1:
The patent uses inexpensive, widely available camera sensors that are already present in most modern devices instead of expensive specialized eye tracking hardware. By leveraging the existing camera infrastructure and applying software-based processing, the system achieves gaze tracking functionality at minimal additional cost, making the technology accessible and scalable across diverse device platforms.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Accurately and reliably tracks and quantifies visual attention on mobile devices without sensitive data exposure, achieving high performance and scalability.
Implementation Method 1
acquiring, using a camera of the computing device, photo frames of a user when the user is looking at the computing device
Implementation Method 2
acquiring depth information related to the photo frames
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A method and a system for tracking and quantifying visual attention on a computing device are disclosed. The method comprises acquiring photo frames of a user when the user is looking at a computing device; detecting a user's face within each frame and both eyes of the user within the detected user's face, providing a plurality of 3D eye and face images; feeding the plurality of 3D eye and face images to a first machine learning algorithm comprising a set of sub-algorithms, each one of said set of sub-algorithms transforming the right eye into a left eye by performing a specular reflection through a vertical line that intercepts the center of a frontal lens of the camera, providing a new set of transformed images; and estimating a user's attention by feeding the new set of transformed images to a second machine learning algorithm, the latter reverting said specular reflection.