Autonomous Vehicle Handover Verification Using Operator Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for transitioning autonomous vehicle control to manual mode do not adequately ensure that the operator has acknowledged primary targets before handing over control, potentially leading to unsafe situations due to lack of awareness of critical objects in the environment.
Innovation Solution
A system that includes an object-detector and operator-monitor to classify targets, verify the operator's gaze direction, and use a controller to ensure the operator has looked at primary targets within a defined time before allowing manual mode takeover, with an alert device to notify the operator of missed targets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system performs verification that the operator has gazed at each primary target more recently than a primary time before executing handover, then operator awareness and safety are improved, but the transition time and system complexity increase
Solution Approach 1:
The system performs preliminary verification of operator gaze direction and target awareness before executing the handover from autonomous to manual mode. The controller monitors whether the operator has gazed at each primary target more recently than a primary time threshold, ensuring the operator is aware of critical objects before control transition occurs.
Solution Approach 2:
The system implements feedback mechanisms where the controller continuously monitors operator gaze direction using an operator monitor, compares it with target locations identified by an object detector, and provides feedback signals to indicate whether the operator has achieved sufficient awareness of primary targets before allowing handover.
2Reliability
If the system verifies operator gaze at primary targets before handover, then safety is improved, but the device complexity increases due to additional monitoring and verification components
Solution Approach 1:
The controller performs multiple functions including monitoring operator gaze direction, identifying primary targets, verifying operator awareness, and controlling the handover process. The operator monitor and object detector are integrated into the existing autonomous vehicle control system, reducing the need for separate dedicated components.
Solution Approach 2:
The system uses the operator's own gaze direction data, captured by the operator monitor, to verify their awareness of targets. The controller self-verify whether the operator has gazed at primary targets by comparing gaze data with target locations, eliminating the need for external verification systems.
3Reliability
If the system requires operator gaze verification before manual mode takeover, then operator awareness is improved, but the productivity and response time deteriorate
Solution Approach 1:
The system verifies operator awareness of primary targets rather than requiring verification for all detected objects. The controller identifies a subset of critical objects as primary targets and only requires the operator to have gazed at these specific targets, reducing the verification burden while maintaining safety.
Solution Approach 2:
The system adjusts the primary time threshold parameter dynamically or optimizes it to balance verification thoroughness with response time requirements. By tuning this temporal parameter, the system achieves adequate operator awareness verification without excessively prolonging the handover process.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system (10) for operating a vehicle includes an object-detector (20), and operator-monitor (24), and a controller (30). The object-detector (20) is used to detect one or more targets (22) proximate to a host-vehicle (12), said host-vehicle (12) operable in an autonomous-mode (14) and a manual-mode (16). The operator-monitor (24) is used to detect a gaze-direction (26) of an operator (18) of the host-vehicle (12). The controller-circuit (30) is in communication with the object-detector (20) and the operator-monitor (24). The controller-circuit (30) is configured to determine a classification (42) of each target (22) detected by the object-detector (20). The classification (42) includes a primary-target (44) and an ignored-target (46). The controller-circuit (30) is further configured to determine that a hand-over (40) of operation of the host-vehicle (12) from the autonomous-mode (14) to the manual-mode (16) is recommended, perform a verification (60) that the operator (18) has gazed at each primary-target (44) more recently than a primary-time (62), and in response to the verification (60), execute the hand-over (40).