Vehicle Scenario Difficulty Metrics for Selective AV Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Testing autonomous vehicle components requires a large number of simulations, consuming significant computing resources and time, which can decrease safety and prevent other components from being tested.
Innovation Solution
A machine-learned model is used to determine a difficulty metric for scenarios, allowing a subset of scenarios to be simulated, reducing the number of simulations needed while maintaining confidence intervals, and ensuring critical scenarios are included.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large number of simulations are performed to test autonomous vehicle components, then testing coverage and reliability are improved, but computing resource consumption and time increase significantly
Solution Approach 1:
The system uses self-collected log data from autonomous vehicle operations to train a machine-learned model that predicts scenario difficulty. This self-service approach eliminates the need for external test data sources and enables the system to automatically identify and prioritize critical scenarios for simulation, reducing testing time while maintaining validation reliability
Solution Approach 2:
The patent transforms the testing approach by changing the parameter of scenario selection from random or exhaustive sampling to difficulty-based prioritization. The machine-learned model assigns difficulty scores to scenarios based on historical log data, and simulations are selectively performed on high-difficulty scenarios, thereby reducing the total number of simulations needed while improving validation efficiency
2Reliability
If thousands or millions of simulations are run to ensure safety, then component safety is improved, but computing bandwidth and time consumption increase enormously
Solution Approach 1:
The system extracts and isolates the most critical scenarios from the full dataset by using a machine-learned difficulty metric. Instead of simulating all scenarios equally, the system extracts only those scenarios with high difficulty scores that most impact safety validation, thereby dramatically reducing computing resource consumption while maintaining safety assurance
Solution Approach 2:
The patent applies partial action by performing simulations on a selective subset of scenarios rather than the complete dataset. The machine-learned model identifies scenarios that exceed a difficulty threshold, and simulations are performed only on these partial cases, which are sufficient to validate safety without requiring exhaustive testing of all possible scenarios
3Measurement precision
If more simulations are performed to validate components, then validation accuracy is improved, but the time to verify updates increases, decreasing safety response time
Solution Approach 1:
The system performs preliminary action by pre-training a machine-learned model on historical log data to establish difficulty metrics before actual validation occurs. This pre-computed knowledge enables rapid identification of critical scenarios during update verification, allowing accurate validation to be performed quickly without requiring extensive new simulations
Solution Approach 2:
The patent introduces dynamics by making the scenario selection process adaptive rather than static. The machine-learned model dynamically adjusts which scenarios require simulation based on the specific component being validated and the difficulty metrics learned from historical data, enabling the system to optimize validation accuracy for each specific update while minimizing verification time
4Reliability
If comprehensive scenario testing is performed, then component validation thoroughness is improved, but computing bandwidth is consumed, preventing other components from being tested
Solution Approach 1:
The system applies segmentation by dividing the validation process into two distinct phases: (1) offline training of the machine-learned difficulty metric model using historical log data, and (2) online selective simulation of high-difficulty scenarios. This segmentation enables thorough validation of individual components while maintaining high overall productivity by reducing the simulation burden on computing resources
Data Source
AI summary
A pre-trained machine-learned model, pre-generated clusters determined from embeddings generated by the machine-learned model, and/or difficulty metric(s) determined from simulation and associated with the clusters may be transmitted to and used on a vehicle. The machine-learned model may use sensor data to generate an embedding or a difficulty metric characterizing a current scenario encountered by the vehicle and the vehicle may alter operation of the vehicle based on the difficulty metric or difficulty metric(s) for the cluster associated with the embedding.