Personalized Eye-Tracking Neural Network Retraining From UI Events
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks for eye tracking are not optimized for individual users, leading to suboptimal performance in personal devices like augmented reality headsets, as they are typically trained on large populations and lack personalized data.
Innovation Solution
A method for retraining a neural network for eye tracking using personalized data collected from individual users' interactions with virtual user interface devices, enhancing the network's performance for gaze direction determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a neural network is trained on large population data, then the network can be applied to multiple users, but the performance for individual users is suboptimal
Solution Approach 1:
The patent segments the training process into two distinct phases: population-level training using large datasets to establish baseline performance, followed by user-specific retraining using personalized eye movement data. This segmentation allows the system to maintain broad applicability while achieving individualized precision through separate training stages.
Solution Approach 2:
The system performs preliminary training on population data before deploying to individual users. This preliminary action establishes a robust baseline model that can be quickly adapted to individual users through subsequent retraining with minimal personalized data, resolving the contradiction between broad applicability and individual precision.
2Measurement precision
If a neural network is trained on personalized data, then the eye tracking accuracy improves, but the data collection process becomes more complex
Solution Approach 1:
The system implements self-service data collection where users naturally interact with virtual user interface devices during normal device operation. The system automatically captures eye movement data during these interactions without requiring separate calibration sessions or complex user instructions, thereby simplifying the data collection process while maintaining high accuracy.
Solution Approach 2:
The system uses feedback from user interactions with virtual UI devices to automatically identify and collect relevant eye movement data. This feedback-driven approach streamlines data collection by focusing only on meaningful interactions, reducing complexity while improving tracking accuracy through targeted personalized data.
3Measurement precision
If retraining data is collected during user interactions with virtual UI devices, then personalized performance improves, but the time required for data collection increases
Solution Approach 1:
The system collects personalized training data continuously during normal device usage and user interactions with virtual UI devices. This continuous collection approach eliminates separate calibration phases, allowing the system to accumulate personalized data over time without adding dedicated data collection steps, thus maintaining high performance while minimizing time loss.
Data Source
AI summary
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.