Log-Based Simulation Divergence for Autonomous Vehicle Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicle software testing faces challenges in accurately simulating real-world interactions, as changes in vehicle behavior during simulations can lead to unintended consequences, such as collisions, making it difficult to maintain realistic scenarios and identify critical divergence points for handover times.
Innovation Solution
A method involving log-based simulations where divergence points are identified by comparing simulated vehicle characteristics with actual vehicle data, allowing for adjusted handover times and the use of timers to flag simulations for review, ensuring safe and realistic testing without physical risks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If simulations are run using log data to test autonomous vehicle software, then testing capability is improved, but simulation accuracy deteriorates when vehicle behavior changes during simulation
Solution Approach 1:
The simulation is divided into two distinct phases: a replay phase where historical log data is executed to establish baseline behavior, and a test phase where the software under test is evaluated. This segmentation allows the system to maintain accuracy during replay while enabling flexible testing during the second phase, resolving the contradiction between testing capability and simulation accuracy.
Solution Approach 2:
The system performs preliminary replay of log data before the actual software testing. By first executing the historical trajectory and identifying divergence points, the system establishes a baseline for comparison. This preliminary action ensures that subsequent testing phases can accurately measure deviations caused by the software under test, maintaining simulation accuracy while enabling comprehensive testing.
2Reliability
If handover time is adjusted based on divergence points, then simulation realism is improved, but system complexity increases
Solution Approach 1:
The system continuously monitors the simulated vehicle's behavior against the historical log data and provides feedback when divergence points are detected. When the simulated vehicle deviates from the historical trajectory beyond a threshold, the system identifies this as a divergence point and uses it to adjust the handover time. This feedback mechanism maintains simulation realism by adapting to actual behavior variations while providing a clear, rule-based approach that manages system complexity.
3Adaptability or versatility
If multiple simulations are run to cover various scenarios, then testing coverage is improved, but resource consumption increases
Solution Approach 1:
The system uses a single log data file that contains multiple historical trajectories and scenarios. By replaying this comprehensive log data and identifying divergence points across different scenarios, the system can generate multiple test cases from one source. This multi-functional approach allows extensive testing coverage without proportionally increasing resource consumption, as the same log data serves multiple testing purposes.
Solution Approach 2:
The system performs preliminary processing of log data to extract and organize divergence points before actual testing begins. By pre-identifying critical divergence points and structuring the test scenarios in advance, the system can efficiently execute multiple simulations without redundant processing. This preliminary organization of test cases from log data significantly reduces the computational resources required during the actual testing phase while maintaining comprehensive coverage.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure relate to testing software for operating an autonomous vehicle. For instance, a first simulation may be run using log data and the software to control a first simulated vehicle 670. During this, one or more characteristics of the simulated vehicle may be compared with one or more characteristics of a vehicle from the log data. The comparison may be used to determine a divergence point for starting a timer. In addition, a second simulation may be run using the log data and the software to control a second simulated vehicle. The divergence point may be used to determine a handover time to allow the software to take control of the second simulated vehicle. Whether the software is able to continue through the first simulation before the timer expires without a particular type of event occurring and/or the second simulation without the particular type of event occurring is determined.