Driver Gaze Estimation Using Object-Based Correction at Large Face Angles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing line of sight estimating devices struggle to precisely determine a driver's gaze direction when their face is oriented to the right or left relative to the vehicle's front, leading to inaccurate facial feature point detection and subsequent estimation errors.
Innovation Solution
A line of sight estimating device that includes a feature point detector to identify facial features, a first line of sight estimator to determine gaze direction, a determining unit to assess reliability, an object detector to identify gazed objects, and a correcting unit to adjust the gaze direction based on facial orientation and reliability thresholds, ensuring accurate gaze estimation even when the face is angled.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the monitoring camera captures the driver's face from the front when the driver is facing forward, then the facial feature points can be precisely detected, but the system cannot accurately estimate the line of sight when the driver turns their face to the right or left direction
Solution Approach 1:
The system dynamically adapts its line of sight estimation approach based on the driver's face orientation. When the face is oriented within a predetermined angle from the front, the system uses direct facial feature point-based estimation. When the face orientation exceeds this angle, the system switches to an indirect estimation method using gazed object detection and position, thereby maintaining accuracy across varying face orientations
Solution Approach 2:
The system introduces gazed object detection as an intermediary when direct facial feature point estimation becomes unreliable due to large face orientations. By detecting objects in the driver's field of view and determining their positions, the system uses these objects as mediators to infer the line of sight direction, bypassing the limitation of angled face capture
2Ease of operation
If the driver directs their face to the right or left to confirm surrounding conditions, then the driver can monitor the environment, but the monitoring camera captures the face at an angle making facial feature point detection impossible
Solution Approach 1:
The system dynamically switches between two line of sight estimation methods based on face orientation. For small angles (face nearly forward), it uses precise facial feature point detection. For large angles (face turned right or left), it transitions to gazed object-based estimation, ensuring continuous accurate monitoring regardless of driver's head position
Solution Approach 2:
The system preliminarily determines face orientation before selecting the estimation method. By assessing the angle between the driver's face and the camera's optical axis in advance, the system proactively chooses the appropriate estimation approach, preventing detection failures before they occur
3Device complexity
If the system uses only facial feature points for line of sight estimation, then the estimation is simple, but the reliability is low when the driver's face is angled
Solution Approach 1:
The system dynamically adjusts its complexity based on operating conditions. When face orientation is within acceptable limits, it uses the simpler facial feature point method. When face orientation exceeds thresholds, it activates the more reliable but complex gazed object detection method, optimizing the balance between simplicity and reliability
Solution Approach 2:
The system uses gazed object detection as a mediator to enhance reliability when facial feature points become unreliable due to large face angles. By detecting objects the driver is looking at and using their positions to infer line of sight, the system compensates for the unreliability of direct facial feature point estimation in angled configurations
Data Source
AI summary
The line of sight estimating device has a processor configured to estimate a line of sight direction of a driver using the facial feature points and determine reliability of the line of sight direction based on the reliability of the facial feature points, determine whether or not the driver is in a gazing state based on the line of sight direction and multiple other line of sight directions, when the reliability of the line of sight direction is below a threshold, detect a gazed object estimated to be gazed at by the driver appearing in a front image based on the line of sight direction, when the driver is in the gazing state, and correct the line of sight direction so as to be oriented toward the gazed object, when the reliability of the line of sight direction is below the threshold and the driver is in the gazing state.