Trajectory Planner Rule Activation for Efficient Robot Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating the performance of trajectory planners in autonomous vehicles require significant computational resources, especially when evaluating complex rules over multiple time steps and scenarios, which can be inefficient and non-informative when rules are not applicable.
Innovation Solution
A method that selectively evaluates performance evaluation rules only when their activation conditions are satisfied, using a test oracle to process scenario ground truth and render results on a graphical user interface, thereby reducing computational resources and improving result quality by distinguishing between applicable and inactive rules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all performance evaluation rules are evaluated over multiple time steps for every scenario element, then comprehensive performance assessment is achieved, but computational resource usage increases significantly
Solution Approach 1:
The patent applies partial action by selectively evaluating performance rules only when their activation conditions are satisfied. Instead of evaluating all rules for all scenario elements across multiple time steps, the system evaluates only the subset of rules that are relevant to the current scenario context, thereby reducing computational resource usage while maintaining assessment comprehensiveness for applicable rules.
Solution Approach 2:
The patent segments the performance evaluation process by dividing rules into different categories based on their activation conditions. The test oracle processes scenario ground truth and selectively applies rules based on whether their activation conditions are met, creating distinct evaluation paths for different rule types (e.g., safety rules, comfort rules, efficiency rules) depending on the scenario context.
2Quantity of substance
If performance evaluation rules are evaluated continuously over multiple time steps, then complete performance data is collected, but result quality decreases when rules are not applicable
Solution Approach 1:
The system performs partial evaluation by activating only those rules whose conditions are satisfied in the current scenario. This prevents the generation of meaningless evaluation results for inapplicable rules while still collecting comprehensive data for all applicable rules, thereby improving the overall informativeness of the performance assessment results.
Solution Approach 2:
The patent implements dynamic rule activation where the set of evaluated rules changes based on the scenario context and time step. The test oracle continuously monitors scenario ground truth and dynamically determines which rules should be active at each time step, allowing the evaluation process to adapt to changing scenario conditions and maintain result quality.
3Reliability
If comprehensive rule evaluation is performed for all scenario elements, then thorough safety assessment is achieved, but processing time increases
Solution Approach 1:
The patent applies partial action in the context of safety assessment by evaluating only those rules that are relevant to the current scenario's safety concerns. The test oracle identifies applicable rules based on activation conditions related to scenario elements (e.g., presence of obstacles, traffic signals, road conditions) and evaluates only those rules, maintaining safety assessment thoroughness while reducing processing time.
Solution Approach 2:
The system performs preliminary filtering of rules based on activation conditions before conducting the actual performance evaluation. The test oracle pre-processes scenario ground truth to identify which rules are applicable, and only then proceeds to evaluate those specific rules, thereby avoiding unnecessary processing time while ensuring all relevant safety rules are assessed.
Data Source
AI summary
A computer-implemented method of evaluating the performance of a trajectory planner for a mobile robot in a real or simulated scenario, comprises receiving scenario ground truth of the scenario, the scenario ground truth generated using the trajectory planner to control an ego agent of the scenario responsive to at least one scenario element of the scenario. One or more performance evaluation rules for the scenario and at least one activation condition for each performance evaluation rule are received. A test oracle processes the scenario ground truth to determine whether the activation condition of each performance evaluation rule is satisfied over multiple time steps of the scenario. Each performance evaluation rule is evaluated by the test oracle, to provide at least one test result, only when its activation condition is satisfied.