Personalized Calibration Functions for Gaze Detection in Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for autonomous vehicles face challenges in accurately predicting user gaze due to variations in individual eye measurements and facial features, leading to skewed training data and inaccurate predictions, especially for diverse populations and those with eyewear, requiring cumbersome retraining and straining on-board computing resources.
Innovation Solution
The implementation of personalized calibration functions that use explicit and implicit calibration methods to refine gaze predictions for individual users, generating calibration functions through instruction or analysis of historical viewing habits, allowing for accurate gaze detection without extensive retraining, thus improving accuracy and reducing computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional DNN training uses high volume of training image data from diverse individuals, then generalization capability is improved, but measurement precision for individual users deteriorates
Solution Approach 1:
The system segments the gaze prediction process into two stages: (1) a pre-trained DNN provides initial gaze predictions using training data from diverse individuals, and (2) a user-specific calibration function refines these predictions for individual accuracy. This segmentation allows the system to benefit from both generalization capability and individual precision without requiring complete retraining.
Solution Approach 2:
The system performs preliminary calibration by collecting ground truth gaze data from the specific user and generating a calibration function before actual autonomous driving operation. This preliminary action creates a personalized adjustment mechanism that refines the general DNN predictions for the specific user's eye geometry and facial features, resolving the contradiction between generalization and individual accuracy.
2Measurement precision
If DNN is retrained with images from specific driver to improve accuracy, then gaze prediction accuracy is improved, but loss of time and computing resources worsens
Solution Approach 1:
The system performs calibration data collection and function generation as a preliminary step during initial vehicle setup or idle periods, rather than during active driving. This allows the calibration to be completed beforehand, so that during autonomous operation, only lightweight calibration function application is needed, not time-consuming retraining.
Solution Approach 2:
Instead of copying the entire DNN model for retraining, the system creates a lightweight calibration function that copies only the necessary adjustment parameters. This calibration function is applied to the pre-trained DNN outputs, achieving user-specific accuracy without the computational burden of full model retraining.
3Reliability
If complete retraining is performed to ensure safety and accuracy, then reliability is improved, but productivity and ease of operation worsen
Solution Approach 1:
The system segments the safety-critical calibration process from routine operation. The calibration function is generated during setup or maintenance periods when productivity impact is minimal, ensuring reliability. During actual autonomous driving, the pre-calibrated system operates efficiently without interruption, maintaining both safety and productivity.
Solution Approach 2:
The system performs self-calibration by automatically collecting ground truth gaze data from the user's natural viewing behavior and autonomously generating the calibration function. This eliminates the need for manual intervention or cumbersome retraining procedures, improving ease of operation while ensuring reliability through accurate user-specific calibration.
Data Source
AI summary
In various examples, systems and methods are disclosed that provide highly accurate gaze predictions that are specific to a particular user by generating and applying, in deployment, personalized calibration functions to outputs and/or layers of a machine learning model. The calibration functions corresponding to a specific user may operate on outputs (e.g., gaze predictions from a machine learning model) to provide updated values and gaze predictions. The calibration functions may also be applied one or more last layers of the machine learning model to operate on features identified by the model and provide values that are more accurate. The calibration functions may be generated using explicit calibration methods by instructing users to gaze at a number of identified ground truth locations within the interior of the vehicle. Once generated, the calibration functions may be modified or refined through implicit gaze calibration points and/or regions based on gaze saliency maps.