Wearable Eye-Tracking Retraining Using UI Interaction Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks for eye tracking are not optimized for individual users, leading to suboptimal performance in personal devices such as augmented reality headsets, as they are typically trained on large populations and lack personalized data.
Innovation Solution
A method for retraining a neural network for eye tracking using personalized data collected from individual users' interactions with virtual user interface devices, adjusting the network weights to improve accuracy and adapt to specific user behaviors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is trained on large population data, then the network can be applied to general users, but the performance is suboptimal for individual users due to lack of personalized adaptation
Solution Approach 1:
The system performs preliminary collection of user-specific eye tracking data during normal device usage, storing this data for later retraining. This preliminary action accumulates personalized training data without disrupting the user experience, enabling future customization of the neural network for individual users.
Solution Approach 2:
The neural network is designed to be dynamically retrainable, transitioning from a static population-trained model to an adaptive personalized model. The system allows the network weights to be updated based on accumulated user-specific data, making the eye tracking system adaptable to individual user characteristics and behaviors.
2Measurement precision
If personalized data collection is implemented, then individual user accuracy improves, but the system complexity and data processing requirements increase
Solution Approach 1:
The same neural network architecture and training framework used for population-level training is reused for personalized retraining. This universal approach allows the system to handle both general and personalized scenarios with a single unified mechanism, reducing the need for separate complex systems while maintaining individual user accuracy.
Solution Approach 2:
The system creates copies of the original population-trained neural network weights and uses these copies for personalized retraining. This copying approach allows the personalized model to inherit the general capabilities of the population model while adapting to individual users, simplifying the implementation of personalization without requiring entirely new network architectures.
3Reliability
If retraining data is collected during UI interactions, then the data is highly relevant to actual usage scenarios, but the data collection process requires coordination with UI events
Solution Approach 1:
The system uses UI interaction events as feedback triggers to initiate eye tracking data collection. When a user interacts with the device interface, this feedback signal activates the capture of corresponding eye images, ensuring that retraining data is collected during actual usage scenarios and is highly relevant to real-world performance requirements.
Solution Approach 2:
The system pre-establishes the association between UI events and eye tracking data collection protocols. By having this coordination mechanism prepared in advance, the system can efficiently capture relevant data during interactions without requiring complex real-time decision-making, reducing the operational complexity while maintaining data relevance.
Data Source
AI summary
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.