Autonomous Driving Decision Evaluation Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles lack a framework to objectively evaluate their driving decisions against subjective human decisions, making it difficult to compare and grade their performance in the same driving scenarios.
Innovation Solution
A system that logs sensor outputs and human driving control inputs while the autonomous vehicle is driven manually, allowing for the extraction and grading of human driving decisions and comparison with autonomous vehicle decisions using a data analytics system, which includes a machine learning engine, decision extraction module, and grading module.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If autonomous vehicle driving decisions are made objectively using algorithms, then consistency and reliability are improved, but the ability to account for subjective human decision-making differences deteriorates
Solution Approach 1:
The system creates a virtual copy of human driving decisions by logging sensor outputs and human control inputs during manual driving. This copy is then processed through a machine learning engine that extracts decision patterns, allowing the autonomous vehicle to learn and adapt to subjective human decision-making while maintaining its own objective decision framework
Solution Approach 2:
The system implements a feedback loop where human driving decisions are captured, analyzed, and used to evaluate and potentially adjust autonomous vehicle decisions. The grading module compares autonomous decisions against human decision patterns, providing feedback that enables the system to account for human preferences while maintaining reliability
2Adaptability or versatility
If a framework is created to evaluate autonomous vehicle decisions against human decisions, then adaptability to human standards is improved, but system complexity increases
Solution Approach 1:
The evaluation framework is segmented into distinct functional modules: a logging module that captures sensor and control data, a machine learning engine that extracts decision patterns, and a grading module that evaluates decisions. This segmentation allows each component to perform its specific function independently, managing complexity through modular design
Solution Approach 2:
The machine learning engine acts as an intermediary between raw driving data and decision evaluation. It processes logged sensor outputs and human control inputs to extract meaningful decision patterns, serving as a mediator that translates complex raw data into evaluable decision frameworks without requiring direct complex interactions between all system components
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In one embodiment, systems and methods are disclosed for evaluating autonomous driving vehicle (ADV) driving decisions. A driving scenario is selected, such as a route or destination or type of driving condition. ADV planning and control modules are turned off and do not control the ADV. As a user drives the ADV, sensors detect and periodically log a plurality of objects external to the ADV. Driving control inputs of the human driver are also logged periodically. An ADV driving decision module generates driving decisions with respect to each object detected by the sensors. The ADV driving decisions are logged, but are not used to control the ADV. An ADV driving decision is identified in the logs, and a corresponding human driving decision is extracted, graded, and compared to the ADV driving decision. The ADV driving decision can be graded using the logs and graded human driving decision.