Gaze Prediction Model Using Camera Calibration Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for tracking driver attention, such as eye-tracking glasses, are expensive, intrusive, and do not effectively predict points of gaze, leading to redundant warnings and a diminished driving experience, as they only track what the driver looks at and not what they might miss outside their field of view.
Innovation Solution
A machine learning model is trained using a camera calibration module and a point of gaze regression module, processing pairs of driver face and scene images from multiple cameras to predict the driver's point of gaze, minimizing Euclidean distance loss and triplet loss to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If eye-tracking glasses are used to track driver's points of gaze, then measurement precision of driver attention is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent uses standard cameras to capture images of the driver's face and eyes, creating a visual copy of the driver's appearance. This copied visual information is then processed through machine learning models to infer gaze direction, replacing the need for specialized eye-tracking glasses while achieving comparable measurement precision
Solution Approach 2:
The patent replaces the mechanical/optical system of eye-tracking glasses with a computational system consisting of standard cameras and machine learning algorithms. The mechanical eye-tracking device is substituted by an information processing system that analyzes facial images to determine gaze direction
2Measurement precision
If eye-tracking glasses are used to track driver's points of gaze, then measurement precision is improved, but ease of operation deteriorates due to intrusive nature
Solution Approach 1:
Instead of requiring drivers to wear eye-tracking glasses, the system uses standard cameras to capture images of the driver's face and processes these images through machine learning to infer gaze direction. This copying approach eliminates the need for intrusive wearable devices while maintaining measurement precision
Solution Approach 2:
The patent replaces expensive, intrusive eye-tracking glasses with inexpensive standard cameras that are already present in modern vehicles. This substitution uses readily available, low-cost components to achieve the same functional goal without compromising driver comfort or convenience
3Measurement precision
If eye-tracking glasses are used, then what the driver looks at is tracked, but reliability of danger detection deteriorates due to limited field of view
Solution Approach 1:
The patent extends the tracking capability from just the gaze point (2D coordinate) to include the broader context of the driver's field of view and head orientation (3D spatial information). By analyzing the relationship between face images and scene images along with embedding vectors, the system infers not only where the driver is looking but also what areas are within their potential field of view, thereby improving danger detection reliability
Data Source
AI summary
A method and system for training a machine learning model for point of gaze prediction simultaneously analyzes the image of the scene and the image of the driver's face using the machine learning model to generate a point of gaze of a driver in traffic environments. The machine learning model comprises a camera calibration module that can compute an embedding vector that represents the spatial configuration between the driver and the camera system. The camera calibration module improves the overall model's performance, which is jointly trained end to end.