Simulations with modified agents for testing autonomous vehicle software
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing autonomous vehicle software struggle to validate safety and effectiveness, particularly in rare or outlier situations, such as interactions with less common road users like motorcyclists, due to limited real-world data examples, leading to inadequate testing of behaviors like lane changes or merging.
Innovation Solution
The method involves running simulations using log data collected from autonomous vehicles, where common road users are modified to simulate less common users, and interactive agents are introduced to respond to the vehicle's behavior, allowing for the analysis of interactions like collisions or near collisions, and flagging simulations for further review.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-world testing is used to validate autonomous vehicle software, then safety and effectiveness can be tested, but testing coverage is limited due to scarcity of rare situation data
Solution Approach 1:
The patent creates virtual copies of real-world driving scenarios by modifying log data to generate synthetic test cases. Rare situations are replicated through data transformation, allowing comprehensive testing without requiring actual physical occurrences of these rare events.
Solution Approach 2:
The system performs preliminary modification of log data to create diverse test scenarios before actual software validation. By pre-processing real driving data to include rare and edge cases, the system prepares comprehensive test suites that cover situations unlikely to occur during normal testing.
2Reliability
If physical real-world testing is conducted for all scenarios, then comprehensive validation is achieved, but testing time and cost increase significantly
Solution Approach 1:
The patent replaces physical mechanical testing with computational simulation. By substituting real-world vehicle testing with software-based simulation of modified log data, the system achieves comprehensive validation without the time and resource constraints of physical testing.
Solution Approach 2:
Comprehensive test scenarios are prepared in advance by modifying log data to include all possible edge cases and rare situations. This preliminary preparation allows parallel processing of multiple test scenarios, dramatically reducing overall validation time compared to sequential physical testing.
3Productivity
If log data is used for simulation testing, then testing speed increases, but testing accuracy decreases due to lack of interactive agent responses
Solution Approach 1:
The patent introduces interactive agents as intermediaries between the autonomous vehicle software and the environment. These agents simulate responses from other road users, providing realistic interaction validation while maintaining the efficiency of simulation-based testing with modified log data.
4Productivity
If common road users are tested extensively, then typical driving scenarios are validated, but rare road user interactions remain untested
Solution Approach 1:
The patent modifies parameters in log data to transform common road user scenarios into rare scenario representations. By changing characteristics such as road user type, behavior patterns, and interaction frequencies in the synthetic data, the system efficiently generates test cases for rare situations while maintaining testing productivity.
Data Source
AI summary
The disclosure relates to testing software for operating an autonomous vehicle. In one instance, a simulation may be run using log data collected by a vehicle operating in an autonomous driving mode. The simulation may be run using the software to control a simulated vehicle and by modifying a characteristic of an agent identified in the log data. During the running of the simulation, that a first type of interaction between the first simulated vehicle and the modified agent will occur may be determined. In response to determining that the particular type of interaction will occur, the modified agent may be replaced by a interactive agent that simulates a road user corresponding to the modified agent that is capable of responding to actions performed by simulated vehicles. That the particular type of interaction between the simulated vehicle and the interactive agent has occurred in the simulation may be determined.