Neural Network Gaze Prediction Using Segmented Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze prediction systems are costly, resource-intensive, and require large amounts of impractical training data, leading to overfitting and decreased accuracy due to noisy data, which affects computational efficiency and prediction accuracy.
Innovation Solution
A gaze prediction system that uses a neural network to process input facial images, including query and calibration images, to predict gaze positions with fewer computational resources, employing online calibration and diverse training data from multiple users, reducing the need for extensive personalized training and retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing gaze prediction systems use traditional training methods with large amounts of personalized training data, then they may achieve adequate prediction accuracy, but they require excessive computational resources, time, and storage capacity
Solution Approach 1:
The training data is segmented into two distinct components: (1) generic training data from multiple users that captures general gaze patterns, and (2) minimal personalized calibration data that captures individual-specific characteristics. This segmentation allows the system to achieve accurate predictions without requiring large amounts of personalized training data, thereby reducing computational resources while maintaining prediction accuracy.
Solution Approach 2:
The system changes the parameter of training data quantity from 'large amounts of personalized data' to 'minimal personalized calibration data combined with generic multi-user data'. This parameter change fundamentally alters the training approach, enabling the model to achieve comparable or superior accuracy with significantly reduced computational requirements for data collection and processing.
2Measurement precision
If gaze prediction systems collect large amounts of personalized training data to improve accuracy, then they may achieve better prediction performance, but they suffer from overfitting and noisy data that decrease accuracy
Solution Approach 1:
The system uses generic training data collected from multiple users that serves a universal function of capturing general gaze patterns applicable to all users. This universal training component prevents overfitting to individual noise while the minimal personalized calibration data fine-tunes the model for each user, achieving both accuracy and reliability without requiring large amounts of personalized data.
Solution Approach 2:
Instead of collecting excessive personalized training data that leads to overfitting, the system applies partial action by using only minimal personalized calibration data (e.g., a few images per user) combined with the generic multi-user training data. This partial approach to personalized data collection avoids the harmful effects of overfitting while maintaining individualized prediction accuracy.
3Measurement precision
If gaze prediction systems require extensive personalized training and retraining for each user, then they may achieve accurate predictions, but they become costly and impractical for deployment
Solution Approach 1:
The system performs preliminary action by pre-training the model on generic multi-user training data before deployment. This preliminary training establishes a strong baseline model that works well for all users. When a new user is introduced, only minimal calibration data is needed to adapt the model, eliminating the need for extensive personalized training and making the system practical for deployment across many users at low cost.
Solution Approach 2:
The system uses copying by leveraging training data from multiple users as a template or copy that captures general gaze patterns. Instead of creating entirely new training data for each user, the system copies and adapts the general patterns learned from the multi-user training set, combining them with minimal user-specific calibration data to achieve accurate predictions without extensive personalized training.
4Ease of operation
If gaze prediction systems are implemented on user devices with limited resources, then they provide better accessibility, but they have constrained computational power for running complex models
Solution Approach 1:
The model architecture is segmented into components that can efficiently process the two-stage training approach (generic multi-user training followed by minimal personalized calibration). This segmentation enables the model to achieve high accuracy with reduced computational complexity, making it suitable for deployment on user devices with limited processing power while maintaining accessibility.
Solution Approach 2:
The system changes the parameter of training data requirements from 'large amounts of personalized data' to 'minimal personalized calibration data'. This parameter change reduces the computational burden during both training and inference phases, enabling the gaze prediction system to run on user devices with constrained computational resources while maintaining high prediction accuracy and broad accessibility.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for gaze position prediction. One of the systems includes a neural network configured to obtain a collection of input facial images of a user, wherein the collection of input facial images of the user comprises (i) a query image of the user, (ii) one or more calibration images of the user, and (iii) a respective calibration label that labels a known gaze position of the user for each of the one or more calibration images of the user; and a subsystem configured to process the received collection of input facial images of the user using the neural network to generate an output that characterizes a gaze position of the user. The specification describes examples of gaze prediction systems which require fewer computational resources whilst achieving equal or higher levels of accuracy compared to known gaze predictions systems.