3D Gaze Deviation Measurement via Neural Network Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for measuring the deviation angle of a gaze position in squint patients are subjective and inconsistent, relying on patient cooperation and examiner interpretation, leading to inaccurate and variable results.
Innovation Solution
A method and apparatus using 3D reconstruction techniques, involving video streaming, neural networks, and 3D face modeling to objectively determine the deviation angle by analyzing facial feature points and eyeball rotation, reducing subjective factors and improving consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional measurement methods (triangular prism, corneal reflection, visual field arc) are used, then the measurement process can be completed, but the results show large deviations and lack objective consistency due to subjective factors of examiners and patient cooperation requirements
Solution Approach 1:
The patent replaces traditional mechanical and manual measurement methods (triangular prism, corneal reflection, visual field arc) with an automated image processing system using neural networks and 3D reconstruction algorithms. The system captures facial images and automatically calculates deviation angles through computer vision, eliminating examiner subjectivity and improving measurement consistency.
Solution Approach 2:
The system enables automatic self-measurement by capturing images of the patient's face and eyes, then using automated algorithms to calculate the deviation angle without requiring continuous examiner intervention or patient-active participation beyond maintaining the prescribed gaze position.
2Productivity
If traditional measurement methods are used, then the measurement can be performed, but the process requires significant patient cooperation and is time-consuming
Solution Approach 1:
The system performs preliminary actions by automatically capturing multiple facial images and pre-processing them through neural networks to extract feature points and reconstruct 3D geometry before final deviation angle calculation, streamlining the examination process and reducing the need for repeated manual measurements.
3Measurement precision
If automated image processing with neural networks is used, then measurement objectivity and consistency are improved, but the device complexity and processing requirements increase
Solution Approach 1:
The patent segments the complex measurement task into distinct processing stages: image capture, facial feature detection through first neural network, key frame selection, eye region analysis through second neural network, 3D reconstruction, and deviation angle calculation. This modular approach manages system complexity while maintaining measurement accuracy.
Data Source
AI summary
Disclosed is a method and apparatus for measuring a deviation angle of a gaze position based on 3D reconstruction. A face image sequence of a testee is acquired. Covering conditions of the face image sequence are obtained. Key frame images are determined and input into a second neural network to obtain feature point heat maps that are converted into facial feature point coordinates. An objective function is constructed. A head pose corresponding to the key frame images is obtained. An eyeball position is initialized. An eyeball rotation angle of a reference gaze position is set as a preset angle. 3D coordinates of an eyeball in a reference gaze position image in a head coordinate system is solved. 3D coordinates of an eyeball in a to-be-measured image in the head coordinate system are fixed. An eyeball rotation angle is solved. A deviation angle of a gaze position is obtained.