Neural Gaze Difference Estimation From Video Eye Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for measuring gaze difference, such as traditional prism and flashlight methods and image pixel calculations, face challenges in accuracy and operability, especially for children, and are costly and difficult to operate.
Innovation Solution
A method utilizing neural networks for key frame extraction, facial feature point extraction, and gaze difference estimation, combined with a trained gaze difference estimation network, to accurately measure gaze differences from video data without requiring professional devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional prism and flashlight method is used for measuring gaze difference, then measurement can be performed with simple equipment, but measurement precision deteriorates due to difficulty in cooperation and large error
Solution Approach 1:
The patent uses image processing to create a digital copy of the corneal light reflex position, replacing direct physical measurement. By capturing images of the corneal reflex and calculating pixel coordinates, the system achieves precise measurement without requiring complex physical alignment equipment or patient cooperation during the measurement process
Solution Approach 2:
The patent replaces the mechanical prism and flashlight measurement system with an optical-imaging-computation system. Instead of using physical prisms to measure gaze deviation, the system uses cameras to capture corneal reflex images and computational algorithms to calculate gaze difference from pixel coordinates, eliminating the need for mechanical alignment and improving both precision and ease of use
2Measurement precision
If new methods such as image pixel calculation of corneal light reflexes are used, then measurement precision improves, but device complexity and operational difficulty increase
Solution Approach 1:
The patent employs a universal image processing framework that can measure gaze difference for both eyes simultaneously using standard camera equipment. The same computational algorithm processes images from either the right or left eye, making the system versatile and eliminating the need for specialized single-eye measurement devices
Solution Approach 2:
The system uses the patient's own corneal reflex as the measurement marker, requiring no external markers, special coatings, or additional equipment attached to the patient. The corneal reflex naturally occurs when light hits the cornea, and the system simply captures and processes this self-generated optical phenomenon
3Measurement precision
If new measurement methods are adopted, then measurement precision improves, but cost increases due to specialized devices such as VR devices and eye trackers
Solution Approach 1:
The patent uses inexpensive, readily available equipment (standard cameras, computers with image processing software) instead of expensive specialized devices like VR headsets or clinical eye trackers. The system treats the measurement as a temporary, single-use process where standard consumer electronics suffice, eliminating the need for costly proprietary hardware
Data Source
AI summary
Disclosed are a method and apparatus for measuring a gaze difference based on gaze estimation. Video data of a testee gazing at a visual target is obtained, and a face image sequence is obtained from the video data. The face image sequence is input into a first neural network for key frame extraction to obtain a first and second gaze position face image. The face images are input into a second neural network for facial feature point extraction. Cropping is performed based on facial feature point coordinates to obtain a first and second eye region image. A gaze difference estimation network is trained to obtain a trained gaze difference estimation network. The first and second eye region images are input into the trained gaze difference estimation network to obtain a predicted gaze difference between the first and second eye region images.