Camera-Based Eye Tracking for Persistent Gaze-Control HMIs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing eye-tracking systems for human-machine interfaces are susceptible to sensor quality issues, user inconvenience, and environmental lighting conditions, leading to inefficiencies and eye fatigue.

Innovation Solution

A camera-based eye-tracking system employing a deep learning algorithm for real-time classification of gaze and eye directions, utilizing a single or multiple cameras, and a supervisory control architecture, capable of continuous operation and high accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Difficulty of detecting and measuring

If electrooculogram (EOG) sensors are used for eye tracking, then eye movement detection capability is provided, but sensor skin-contact quality issues and user inconvenience occur

Engineering Contradiction:

Improveeye movement detectionVSAvoiduser convenience

Core Design Contradiction:

Difficulty of detecting and measuringVSEase of operation

Solution Approach 1:

The patent replaces the mechanical EOG sensor system with an optical camera-based eye tracking system. The camera captures images of the user's eyes, and deep learning algorithms process these images to detect eye movements and gaze directions, eliminating the need for physical skin contact sensors and improving user convenience while maintaining detection capability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary deep learning processing system between the camera and the eye movement detection. Instead of directly measuring electrical signals from the eye, the system uses camera images as an intermediary and applies trained neural networks to extract eye movement information, resolving the contradiction between non-contact operation and accurate detection

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If commercial eye-tracking software control functions are used, then HMI control capability is provided, but complicated eye movements are required causing eye fatigue

Engineering Contradiction:

ImproveHMI control capabilityVSAvoideye fatigue

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The patent changes the parameters of eye movements required for HMI control by training deep learning models to recognize subtle and natural eye movements. Instead of requiring complex deliberate eye patterns, the system learns to interpret natural gaze directions and eye movements for controlling multiple functions, reducing eye fatigue while maintaining control capability

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent implements dynamic adaptation by using continuously trained deep learning models that can adapt to different users and usage scenarios. The system dynamically adjusts its interpretation of eye movements based on learned patterns, enabling natural control without requiring users to perform standardized complicated eye movements that cause fatigue

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If camera systems are used for eye tracking, then non-contact operation is achieved, but environmental lighting conditions influence performance

Engineering Contradiction:

Improvenon-contact operationVSAvoidperformance consistency

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent applies beforehand cushioning by preprocessing camera images to compensate for lighting conditions before eye movement detection. The deep learning system is trained to be robust against varying lighting environments, and image processing techniques are applied to normalize lighting effects, cushioning the impact of environmental factors on detection reliability

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

Solution Approach 2:

The patent implements feedback mechanisms where the eye tracking system continuously monitors detection quality and adjusts its processing parameters based on current lighting conditions. The deep learning model receives feedback about image quality and environmental factors, dynamically adjusting its detection thresholds and parameters to maintain reliable performance across different lighting scenarios

Inventive Principle:

Principle #23Feedback

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables exceptional accuracy (99.99%) in controlling robotic systems with minimal user effort, overcoming limitations of conventional systems and providing precise control in various applications.

Implementation Method 1

acquire first image data of a subject including of a subject's eyes via a first camera system

Methodology Applied

Scientific EffectLight reflection: Reflection

Data Source

PatentUS12578788B2Persistent human-machine interfaces via gaze and eye direction tracking

Publication Date: 2026.03.17 GEORGIA TECH RES CORP

AI summary

A camera-based eye-tracking system and a data classification method for persistent human-machine interfaces (HMI). An exemplary system may employ a deep learning algorithm (e.g., convolutional neural network) for classifying eye directions. In some implementations, the pupil center corneal reflection method of an eye tracker is employed for gaze tracking. The example system can use a supervisory control and data acquisition architecture that can be universally applied to any screen-based HMI task.