Autonomous Vehicle Stack Testing for ODD Transition Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for evaluating autonomous vehicle (AV) stacks do not adequately address the safety and performance requirements of level 3 and level 4 autonomy, particularly in scenarios outside the defined operational design domain (ODD), which are crucial for ensuring guaranteed safety and timely responses.
Innovation Solution
A computer system and method for testing AV stacks using ODD-based and specification-based response rules, which evaluate the stack's performance by processing scenario ground truth and internal state data to ensure compliance with driving rules, including transition demands and minimum risk maneuvers within defined time intervals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based safety models are used to evaluate autonomous vehicle trajectories, then safety evaluation capability is improved, but the approach does not adequately address level 3 and level 4 autonomy requirements including transition demands and minimum risk maneuvers
Solution Approach 1:
The testing framework is segmented into multiple independent rule modules: ODD compliance rules, transition demand rules, and minimum risk maneuver rules. Each module independently evaluates a specific aspect of autonomous vehicle behavior, allowing comprehensive coverage of different autonomy levels while maintaining clear separation of concerns and enabling targeted testing of specific functional requirements
Solution Approach 2:
The testing framework achieves universality by designing a multi-functional evaluation system that can assess both basic safety compliance (through ODD rules) and advanced autonomy requirements (through transition demand and MRM rules). This single framework adapts to evaluate different levels of autonomous driving systems, making it versatile across multiple application scenarios and autonomy levels
2Reliability
If extensive real-world data collection is performed to ensure safety, then safety level is improved, but time consumption and testing efficiency deteriorate
Solution Approach 1:
The framework performs preliminary evaluation by checking ODD compliance, transition demand responses, and minimum risk maneuver execution before conducting full safety assessments. This preliminary filtering identifies critical safety issues early in the testing process, reducing the need for extensive real-world data collection and enabling faster iteration cycles
Solution Approach 2:
The framework substitutes physical real-world testing with a virtual rule-based evaluation system. By replacing mechanical data collection processes with computational rule checking, the system achieves high-speed safety assessment without the time constraints of physical testing, enabling rapid validation of autonomous vehicle systems
3Ease of operation
If basic safety rules are applied for evaluation, then evaluation simplicity is improved, but the system fails to detect violations of transition demands and minimum risk maneuvers
Solution Approach 1:
The evaluation system is segmented into distinct rule layers: basic ODD compliance rules provide simple evaluation for fundamental safety, while separate transition demand and minimum risk maneuver rules add specialized detection capabilities. This segmentation maintains simplicity at each layer while collectively achieving comprehensive violation detection through the combination of all rule modules
Solution Approach 2:
The testing framework introduces intermediary rule modules that bridge basic safety evaluation and advanced autonomy requirements. These intermediary transition demand and MRM rules act as mediators between simple ODD compliance checking and complex safety assessment, enabling detection of nuanced violations without overwhelming complexity
Data Source
AI summary
A method for testing performance of a stack for planning ego vehicle trajectories in real or simulated driving scenarios applying driving rules to the scenario ground truth for evaluating the performance of the stack in the scenario, and providing output indicating whether each driving rule has been complied with; wherein the driving rules include at least one ODD-based response rule, wherein applying the ODD-based response rule includes processing the scenario ground truth over multiple time steps, to determine whether the scenario is within the defined ODD at each time step, and thereby detecting a change in the scenario that takes the scenario outside the defined ODD, and processing the internal state data, to determine whether a state change occurred within the stack, within a time interval, the output for the at least one ODD-based response rule indicating whether the state change occurred within the time interval.