Gaze Tracking with 360-Degree Panoramic Camera
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for gaze tracking lack sufficient precision and variability in data collection, especially outside controlled environments, due to the difficulty in acquiring high-resolution imagery around the eye and the lack of large and diverse training data, which hinders the development of accurate gaze estimation systems for applications like autonomous driving and social robotics.
Innovation Solution
A system utilizing a 360-degree panoramic camera to capture image data of subjects in uncontrolled environments, combined with a neural network that predicts gaze direction vectors and error bounds using a pinball loss function, allowing for the generation of labeled training datasets for neural networks to accurately estimate gaze directions and uncertainty in natural settings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution cameras are used to capture eye imagery, then measurement precision of gaze direction is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses a 360-degree panoramic camera to capture images of the subject's eyes from multiple angles simultaneously, creating multiple copies of the eye imagery. This allows the system to select or combine the best quality images for gaze estimation without requiring multiple high-resolution cameras positioned optimally, thus reducing device complexity while maintaining measurement precision.
Solution Approach 2:
The 360-degree panoramic camera serves multiple functions: it captures eye imagery for gaze tracking, provides comprehensive environmental context, and enables gaze estimation from various viewing angles. This multi-functionality reduces the need for specialized high-resolution eye cameras, thereby reducing device complexity while maintaining or improving gaze measurement precision through redundant angular coverage.
2Measurement precision
If controlled lab environments are used for data collection, then measurement precision is improved, but adaptability to real-world scenarios deteriorates
Solution Approach 1:
The system transitions from static controlled lab environments to dynamic real-world settings by using a 360-degree panoramic camera that can capture gaze data in natural, uncontrolled environments. The camera's ability to capture images from all directions allows it to adapt to varying lighting conditions, subject positions, and environmental factors, maintaining measurement precision while significantly improving adaptability to real-world scenarios.
Solution Approach 2:
The patent adds the dimension of environmental context by using a 360-degree panoramic camera that captures not only eye imagery but also the surrounding environment. This allows the system to collect gaze data in diverse real-world settings while maintaining precision through the ability to process images from multiple angles and compensate for environmental variations that would be controlled away in lab settings.
3Quantity of substance
If 360-degree panoramic camera is used to collect diverse training data, then quantity and variety of training data is improved, but device complexity increases
Solution Approach 1:
The 360-degree panoramic camera performs multiple functions simultaneously: it captures eye imagery for gaze training data, records environmental context for context-aware gaze estimation, and provides multi-angle views that eliminate the need for multiple separate cameras. This multi-functionality generates large volumes of diverse training data while avoiding the device complexity of deploying multiple specialized cameras.
Solution Approach 2:
The system transitions from 2D single-angle camera views to 360-degree omnidirectional coverage, adding angular dimensionality to the training data. This single device generates vastly more diverse training examples by capturing eye imagery from all directions simultaneously, providing both increased data quantity and variety without the complexity of multiple camera systems.
4Measurement precision
If multiple high-resolution cameras are positioned around the eye, then measurement precision is improved, but device complexity and ease of operation worsen
Solution Approach 1:
The patent merges the functionality of multiple high-resolution cameras positioned around the eye into a single 360-degree panoramic camera. This single device captures images from all directions simultaneously, providing the same multi-angle eye imagery that would require multiple separate cameras, thereby maintaining measurement precision while dramatically reducing device complexity and ease of operation.
Solution Approach 2:
The 360-degree panoramic camera serves as a universal imaging solution that replaces multiple specialized high-resolution cameras. It captures eye imagery from all angles in a single device, eliminating the need for complex camera arrangements and simplifying both the hardware setup and operational procedures while maintaining the ability to obtain high-resolution eye images from multiple perspectives.
Data Source
AI summary
A system includes a camera positioned in an environment to capture image data of a subject; a computing device communicatively coupled to the camera, the computing device comprising a processor and a non-transitory computer-readable memory; and a machine-readable instruction set stored in the non-transitory computer-readable memory. The machine-readable instruction set causes the computing device to perform at least the following when executed by the processor: receive the image data from the camera; analyze the image data captured by the camera using a neural network trained on training data generated from a 360-degree panoramic camera configured to collect image data of a subject and a visual target that is moved about an environment; and predict a gaze direction vector of the subject with the neural network.