3D Gaze Prediction Neural Network Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze detection systems struggle with accurate 3D gaze tracking, particularly when camera and screen configurations change, leading to reduced accuracy and limited support for stereoscopic displays and 3D applications.
Innovation Solution
A deep learning system that trains a neural network using diverse training images to predict 3D gaze information independently of camera and screen configurations, by normalizing 2D images and updating calibration parameters without altering network parameters, allowing for consistent gaze detection across different devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is trained for a specific camera and screen configuration, then prediction accuracy for that configuration is improved, but the system cannot adapt to different configurations without re-training
Solution Approach 1:
The patent introduces calibration parameters that can be adjusted for different camera and screen configurations without re-training the neural network. These parameters include camera intrinsic parameters (focal length, principal point), screen parameters (resolution, size, distance), and gaze mapping parameters. By changing these parameters according to the specific configuration, the system maintains high prediction accuracy across diverse devices.
Solution Approach 2:
The patent separates the configuration-specific aspects from the neural network model itself. The neural network is trained on normalized eye images and learns general gaze patterns, while configuration-specific parameters are handled separately through calibration procedures. This segmentation allows the core model to remain universal while adapting to different configurations through parameter adjustment rather than re-training.
2Adaptability or versatility
If 3D gaze tracking is implemented, then support for stereoscopic displays and 3D applications is improved, but prediction accuracy compared to 2D gaze detection deteriorates
Solution Approach 1:
The patent extends 2D gaze detection to 3D gaze tracking by incorporating depth information through calibration parameters. The system uses camera intrinsic parameters and screen geometry to map 2D eye image coordinates to 3D gaze points in virtual space. This dimensional extension enables support for stereoscopic displays and 3D applications while maintaining accuracy through proper calibration of the mapping parameters.
3Measurement precision
If the neural network is re-trained for new configurations, then prediction accuracy is maintained, but time consumption and computational resources increase
Solution Approach 1:
The patent performs preliminary calibration to determine configuration-specific parameters before actual gaze detection begins. This calibration process, which involves capturing images of the user looking at known reference points and computing transformation parameters, is done once per configuration change rather than continuously. This preliminary action eliminates the need for time-consuming re-training while maintaining accuracy.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Techniques for generating 3D gaze predictions based on a deep learning system are described. In an example, the deep learning system includes a neural network. The neural network is trained with training images. During the training, calibration parameters are initialized and input to the neural network, and are updated through the training. Accordingly, the network parameters of the neural network are updated based in part on the calibration parameters. Upon completion of the training, the neural network is calibrated for a user. This calibration includes initializing and inputting the calibration parameters along with calibration images showing an eye of the user to the neural network. The calibration includes updating the calibration parameters without changing the network parameters by minimizing the loss function of the neural network based on the calibration images. Upon completion of the calibration, the neural network is used to generate 3D gaze information for the user.