Event-Based Eye Tracking With Neural Inference Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional eye tracking systems relying on frame-based cameras are slow and produce large volumes of data, while event-based systems with hand-crafted accumulation regimes result in noisy and artifact-prone intensity images, complicating neural network processing.
Innovation Solution
An eye tracking device using an event-based optical sensor and a controller that generates a signal stream of events, processed by a first neural network to create an inference frame, which is then analyzed by a machine learning module without relying on conventional image frames, utilizing a recurrent neural network and a convolutional neural network for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frame-based cameras are used for eye tracking, then complete image frames can be obtained for analysis, but the system becomes slow and produces large volumes of data
Solution Approach 1:
The patent extracts only the essential information needed for eye tracking from the event stream, rather than capturing complete image frames. By processing individual events and extracting relevant features directly from the event data, the system achieves accurate eye tracking measurements while avoiding the time and data overhead of frame-based capture.
Solution Approach 2:
The patent segments the continuous event stream into meaningful groups or batches for processing, rather than treating it as a complete frame. This segmentation allows the system to process events incrementally and efficiently, reducing latency while maintaining measurement precision through structured analysis of event patterns.
2Measurement precision
If hand-crafted accumulation regimes are used to create intensity images from event data, then a visual representation can be generated, but the images become noisy and artifact-prone
Solution Approach 1:
The patent replaces the mechanical hand-crafted accumulation process with a learned model-based approach. Instead of using fixed algorithms to accumulate events into intensity images, the system employs trained neural networks that learn optimal accumulation strategies from data, thereby generating cleaner representations with reduced noise and artifacts while maintaining measurement precision.
3Device complexity
If conventional computer vision approaches are used with event-based data, then processing can be simplified, but the system still relies on frame-based data access and interpolation
Solution Approach 1:
The patent introduces learned models as an intermediary between raw event data and final eye tracking results. These models serve as a bridge that directly processes event streams without requiring conversion to frame-based representations, eliminating the need for interpolation and frame acquisition while maintaining processing efficiency.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach achieves better reconstruction performance and estimation of eye gaze parameters by generating high-quality inference frames, reducing processing latency and improving accuracy without the need for frame-based data access.
Implementation Method 1
an event-based optical sensor (1), configured to produce a signal stream of events in response to radiation reflected off an eye (2) of a user
Data Source
AI summary
An eye tracking device including an event-based optical sensor, configured to receive radiation reflected off an eye of a user and produce a signal stream of events, each event corresponding to detection of a temporal change in the received radiation at one or more pixels of said optical sensor, and a controller, which is connected to said optical sensor and configured to: a) receive the signal stream of events from the optical sensor, b) generate an inference frame based on the stream of events, c) utilize the inference frame as input to a machine learning module and operate the machine learning module to obtain output data, and e) extract from the output data information related to the eye of the user, wherein the controller is configured to generate the inference frame utilizing a first artificial neural network.