Eye Tracking Neural Network HDR Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing eye-tracking technologies in augmented and virtual reality systems face challenges in accurately determining eye pose due to optical artifacts caused by low dynamic range images of the user's eye, which result in over-exposed or under-exposed portions, particularly in reflections from light emitters.
Innovation Solution
The implementation of a method using machine learning models, specifically convolutional neural networks, to generate high dynamic range images from low dynamic range images of the user's eye, thereby reducing optical artifacts and improving the accuracy of eye pose determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If low dynamic range imaging is used to capture eye images, then the imaging process is simple and fast, but optical artifacts such as over-exposed or under-exposed portions occur, reducing measurement precision
Solution Approach 1:
A machine learning model acts as an intermediary between the LDR image and the final eye pose determination. The model processes the LDR image to generate an HDR image, effectively mediating the transformation from low-quality input to high-quality output for accurate glint detection and eye pose calculation.
Solution Approach 2:
The patent replaces the traditional mechanical/optical HDR imaging system (which would require multiple cameras or complex optical paths) with a computational approach using machine learning. The neural network model substitutes physical HDR imaging mechanisms with algorithmic processing to achieve artifact reduction.
2Object-affected harmful factors
If high dynamic range imaging is implemented through multiple LDR images, then optical artifacts are reduced, but the complexity of the imaging system and processing time increase
Solution Approach 1:
The machine learning model is pre-trained on a large dataset of LDR-HDR image pairs before deployment. This preliminary training phase allows the model to learn the complex mapping between LDR and HDR representations, so that during actual eye tracking, the transformation can be performed rapidly without requiring multiple physical images.
Solution Approach 2:
The patent changes the fundamental parameter from capturing multiple physical images (temporal dimension) to processing a single image through a trained neural network (computational dimension). This parameter change transforms the system from a multi-image acquisition system to a single-image processing system with learned enhancement capabilities.
3Object-affected harmful factors
If high dynamic range imaging is implemented through multiple LDR images, then optical artifacts are reduced, but the processing time and computational resources increase
Solution Approach 1:
The machine learning model is pre-trained offline on extensive datasets, performing the computationally intensive learning phase beforehand. Once trained, the model can rapidly process LDR images to generate HDR images during real-time eye tracking, significantly reducing the processing time during actual operation compared to traditional multi-image HDR methods.
Data Source
AI summary
Enhanced eye-tracking techniques for augmented or virtual reality display systems. An example method includes obtaining an image of an eye of a user of a wearable system, the image depicting glints on the eye caused by respective light emitters, wherein the image is a low dynamic range (LDR) image; generating a high dynamic range (HDR) image via computation of a forward pass of a machine learning model using the image; determining location information associated with the glints as depicted in the HDR image, wherein the location information is usable to inform an eye pose of the eye.