Autonomous Driving Software Validation With Human Driver Benchmarking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for validating autonomous control software for autonomous vehicles lack a comprehensive and efficient means to ensure safety, as they often rely on manual testing, which is time-consuming and risky, and do not provide a reliable benchmark for evaluating the software's performance in various scenarios.
Innovation Solution
A system that uses a validation model simulating an idealized human driver to evaluate autonomous control software through simulated scenarios, allowing for the comparison of the software's performance against this model, with scenarios generated from real events and human responses, and a 'handover time' selection process to assess the software's ability to avoid collisions and ensure safety.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual testing methods are used to validate autonomous control software, then comprehensive safety evaluation can be achieved, but the validation process becomes time-consuming and risky
Solution Approach 1:
The patent creates a validation model that copies and simulates human driver behavior through software, allowing comprehensive safety evaluation to be performed in a virtual environment. This simulation approach enables thousands of test scenarios to be executed rapidly without requiring actual manual testing, thus maintaining reliability while dramatically reducing validation time and eliminating risks to real persons and property
Solution Approach 2:
The validation model is prepared in advance with pre-defined human driver characteristics, response times, and behavioral patterns. By establishing this benchmark model before actual validation begins, the system can immediately conduct comprehensive safety evaluations across multiple scenarios without the time-consuming setup required by manual testing methods
2Measurement precision
If manual testing methods are used to validate autonomous control software, then detailed performance evaluation can be achieved, but the process becomes risky to actual persons and property
Solution Approach 1:
The patent introduces a validation model as an intermediary between the autonomous control software and the test scenarios. This virtual model acts as a safe mediator that captures detailed performance data and human driver behavior metrics without exposing actual persons or property to risk. The intermediary enables precise measurement of software performance while eliminating harmful factors associated with real-world testing
3Reliability
If traditional validation approaches are used, then thorough testing can be performed, but the evaluation process lacks efficiency and scalability
Solution Approach 1:
The validation model is designed to dynamically adapt to different test scenarios and adjust its behavior accordingly. This dynamic capability allows the system to maintain thorough testing across diverse situations while improving efficiency through automated scenario generation and execution. The model can rapidly transition between different driving conditions, vehicle types, and environmental factors without requiring manual reconfiguration
Solution Approach 2:
The validation model serves multiple functions simultaneously: it acts as a benchmark for comparison, generates test scenarios, evaluates software performance, and provides safety certification. This multi-functionality enables thorough testing across all these aspects without requiring separate validation processes, thereby significantly improving evaluation efficiency and scalability while maintaining comprehensive reliability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the disclosure relate to validating autonomous control software for operating a vehicle autonomously. For instance, the autonomous control software is run through a driving scenario to observe an outcome for the autonomous control software. A validation model is run through the driving scenario a plurality of times to observe an outcome for the model for each of the plurality of times. Whether the software passed the driving scenario is determined based on whether the outcome for the software indicates that a virtual vehicle under control of the software collided with another object during the single time. Whether the validation model passed the driving scenario is determined based on whether the outcome for the model indicates that a virtual vehicle under control of the model collided with another object in any one of the plurality of times. The software is validated based on the determinations.