Autonomous Vehicle Disengagement Evaluation Using Trajectory Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles often disengage from autonomous-driving mode unnecessarily due to supervisory drivers being overly cautious, making it difficult to distinguish between appropriate and inappropriate disengagements, which hampers training and data consistency in real-world performance testing.
Innovation Solution
A method is introduced to simulate and evaluate disengagements by comparing pre-disengagement planned trajectories with actual or predicted agent behaviors using perception and prediction data, generating evaluation scores to determine the appropriateness of disengagements and providing actionable feedback to supervisory drivers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If supervisory drivers disengage the autonomous operation frequently to handle difficult driving scenarios, then the safety and reliability of the vehicle is improved, but the productivity and efficiency of autonomous driving testing is reduced
Solution Approach 1:
The system implements feedback by providing automated evaluation results to supervisory drivers, showing them when their disengagements were appropriate or inappropriate based on objective analysis of driving scenarios, vehicle performance, and safety metrics. This feedback loop enables drivers to learn and adjust their disengagement behavior over time.
Solution Approach 2:
The system enables self-service by allowing the autonomous vehicle system to automatically evaluate and assess disengagement appropriateness without requiring external reviewer intervention. The vehicle independently analyzes its own performance data, sensor information, and driving scenarios to determine whether disengagements were justified.
2Adaptability or versatility
If supervisory drivers exercise subjective discretion to disengage from autonomous mode, then the adaptability to handle complex scenarios is improved, but the measurement precision and objectivity of performance evaluation deteriorates
Solution Approach 1:
The system replaces subjective driver discretion with objective feedback by automatically analyzing driving scenarios, sensor data, and vehicle performance to evaluate whether disengagements were appropriate. This objective measurement system provides precise, consistent evaluation criteria that eliminate human bias while still adapting to complex scenarios through comprehensive data analysis.
Solution Approach 2:
The system substitutes the mechanical system of human subjective judgment with an automated computational evaluation system. Instead of relying on supervisory drivers' personal discretion and experience, the system uses algorithms to objectively assess disengagement appropriateness based on measurable criteria from sensors, vehicle data, and scenario analysis.
3Reliability
If real-world performance testing is conducted with supervisory driver oversight, then the reliability and safety of autonomous operation are improved, but the loss of time and resources for manual supervision increases
Solution Approach 1:
The system implements self-service by enabling the autonomous vehicle to independently monitor its own performance, evaluate disengagement events, and generate assessment reports without requiring continuous supervisory driver intervention. The vehicle autonomously processes sensor data, analyzes driving scenarios, and determines disengagement appropriateness, freeing supervisory drivers from manual evaluation tasks.
Solution Approach 2:
The system provides automated feedback to supervisory drivers summarizing disengagement evaluations, allowing them to focus on high-level oversight rather than manually analyzing each disengagement event. This feedback mechanism reduces the time supervisors spend on routine evaluation while maintaining comprehensive safety monitoring.
Data Source
AI summary
A method includes generating, while a vehicle is operating in an autonomous-driving mode, a planned trajectory associated with a computing system of the vehicle based on first sensor data capturing an environment of the vehicle. The method further includes, while the vehicle is operating according to the planned trajectory, receiving a disengagement instruction associated that causes the vehicle to disengage from operating in the autonomous-driving mode and switch to operating in a disengagement mode. Subsequent to the vehicle operating in the disengagement mode, the method further includes capturing second sensor data and generating a simulation of the environment. The simulation is based on sensor data associated with the environment and the planned trajectory. Additionally, subsequent to the vehicle operating in the disengagement mode, the method concludes with evaluating a performance of an autonomy system based on the simulation, and providing feedback based on the evaluation.