Event-Camera Gaze Direction Detection With Corneal Reflection Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze estimation methods in extended reality (XR) face challenges with high complexity, increased hardware cost, and power consumption due to the use of multiple cameras, particularly infrared cameras, and limited scalability with camera serial interface (CSI)-based dynamic vision sensor (DVS) cameras.
Innovation Solution
A method using a camera parallel interface (CPI)-based DVS camera to determine gaze direction by decoding data from an event camera, analyzing reflected light point information, and employing a regression model to estimate gaze direction, reducing complexity and power consumption while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple cameras (particularly infrared cameras) are used for gaze estimation, then measurement precision is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent extracts and utilizes only the necessary components for gaze estimation - specifically using a single DVS camera to capture reflected light points from the cornea, rather than deploying multiple cameras. This extraction approach maintains measurement precision by focusing on the essential light reflection phenomenon while eliminating redundant hardware complexity
Solution Approach 2:
The patent replaces the mechanical/optical system of multiple cameras with an event-based imaging system (DVS camera) that processes visual information through computational algorithms. This substitution uses the unique event-driven architecture to detect corneal reflections and calculate gaze direction, reducing hardware complexity while maintaining or improving measurement precision
2Measurement precision
If multiple cameras (particularly infrared cameras) are used for gaze estimation, then measurement precision is improved, but power consumption increases
Solution Approach 1:
The patent extracts only the essential functionality needed for gaze estimation by using a single DVS camera instead of multiple infrared cameras. This extraction eliminates the power consumption associated with running multiple camera systems while maintaining the core capability to detect corneal reflections and estimate gaze direction accurately
Solution Approach 2:
The patent changes the operational parameters of the imaging system by using a DVS camera with event-driven architecture instead of traditional continuous-frame infrared cameras. This parameter change allows the system to activate processing only when corneal reflection events occur, significantly reducing overall power consumption while maintaining measurement precision during active gaze estimation
3Adaptability or versatility
If CPI-based DVS camera is used instead of CSI-based DVS camera, then adaptability is improved, but device complexity may increase
Solution Approach 1:
The patent adopts the CPI (Camera Parallel Interface) standard which provides universal compatibility across different DVS camera devices and platforms. The CPI interface enables the gaze estimation system to work with various DVS camera implementations without requiring device-specific adaptations, thereby improving scalability and adaptability while maintaining manageable interface complexity through standardized protocols
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method effectively determines gaze direction with reduced complexity and power consumption, enhancing scalability and efficiency in XR applications.
Implementation Method 1
the reflected light point signal being light that is emitted from a light source and is reflected by a corneal surface
Data Source
AI summary
Provided is a method of determining a gaze direction including obtaining a plurality of image frames by performing decoding based on data acquired by an event camera, determining reflected light point information for each image frame among the plurality of image frames, and determining the gaze direction for each image frame among the plurality of image frames based on the reflected light point information, wherein each image frame among the plurality of image frames includes event data obtained based on a reflected light point signal captured by the event camera, the reflected light point signal being light that is emitted from a light source and is reflected by a corneal surface, and wherein the reflected light point information includes at least one of a reflected light point position and numbers of reflected light points corresponding to a pair of reflected light points obtained based on the event data.