Personalized Eye-Tracking Neural Network Using AR UI Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks for eye tracking struggle with generalization to individual users, requiring large training datasets and lacking personalization for improved accuracy and efficiency.
Innovation Solution
A neural network is retrained using eye images captured during user interactions with virtual user interface devices on a head-mounted augmented reality device, utilizing a retraining set that includes eye images and corresponding virtual UI device locations to adapt the network to a specific user, enhancing its performance for eye tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single global neural network is trained on data from multiple users, then the system can serve multiple users with one model, but the measurement precision and tracking accuracy deteriorate due to individual differences in eye movement patterns
Solution Approach 1:
The patent segments the user population into multiple clusters based on eye movement characteristics, and trains a separate neural network for each cluster. This segmentation approach allows the system to maintain high tracking accuracy for each user group while still supporting multiple users, resolving the contradiction between multi-user support and measurement precision.
2Quantity of substance
If eye tracking data from multiple users is combined for training, then the training data volume increases, but the manufacturing precision and model accuracy worsen due to heterogeneity in eye movement patterns
Solution Approach 1:
The patent segments the training data into homogeneous clusters based on eye movement characteristics before training separate neural networks for each cluster. This segmentation allows the system to utilize large volumes of training data from multiple users while maintaining high model accuracy by ensuring each neural network is trained on homogeneous data with consistent eye movement patterns.
3Measurement precision
If individual neural networks are trained for each user, then the tracking accuracy is maximized, but the device complexity and computational resources increase significantly
Solution Approach 1:
The patent segments users into a limited number of clusters (e.g., 5-10 clusters) based on eye movement characteristics, and trains one neural network per cluster. This segmentation strategy balances tracking accuracy with device complexity by grouping users with similar eye movement patterns together, reducing the number of required neural networks from potentially thousands (one per user) to a manageable number of clusters.
Solution Approach 2:
The patent creates neural networks that serve multiple users within the same cluster, making each neural network multi-functional. This universality approach reduces device complexity by having a single neural network handle tracking for multiple users with similar eye movement characteristics, rather than requiring individual networks for each user.
4Adaptability or versatility
If clustering is performed to group similar users, then the adaptability to individual patterns improves, but the device complexity and processing requirements increase
Solution Approach 1:
The patent performs clustering analysis as a preliminary action during the offline training phase, grouping users by eye movement characteristics before training the neural networks. This preliminary clustering enables the system to adapt to individual patterns effectively while keeping online processing complexity low, as the clustering structure is pre-established and only requires simple classification during actual eye tracking operations.
Data Source
Figure 1
Figure 2~2A
Figure 3
AI summary
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.