Meta-Test Case Aggregation for Autonomous Driving Test Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current test methods for autonomous vehicle systems are inefficient in assessing test results and identifying error causes, making it difficult for users to evaluate and improve test executions.
Innovation Solution
A computer-implemented method and system that calculates and aggregates characteristic values from pre-recorded sensor data to evaluate test cases, allowing for meta-test case evaluation and identifying errors, thereby simplifying data evaluation and future test execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If test results are presented with all attached test result artifacts and execution log data, then complete test information is provided, but it becomes difficult and inefficient for users to evaluate test results
Solution Approach 1:
The patent segments the large volume of test data into hierarchical groups (test suites, test cases, individual test results) and introduces meta-test cases that aggregate results at multiple levels. This segmentation allows users to evaluate tests at different granularities without being overwhelmed by complete raw data, resolving the contradiction between information completeness and evaluation efficiency.
Solution Approach 2:
The patent introduces meta-test cases as intermediary elements between raw test data and user evaluation. These meta-test cases aggregate and summarize test results, serving as a mediator that preserves complete test information while presenting it in an efficiently evaluable format, thus resolving the contradiction.
2Measurement precision
If all test cases are evaluated in detail, then comprehensive error identification is achieved, but test execution and evaluation time increases significantly
Solution Approach 1:
The patent segments test evaluation into multiple hierarchical levels (individual test cases, test suites, and meta-test cases). This allows selective evaluation at different levels, enabling comprehensive error identification through aggregated meta-test case analysis without requiring detailed examination of every single test case, thus reducing evaluation time while maintaining accuracy.
Solution Approach 2:
The patent enables partial evaluation by allowing users to focus on meta-test cases that aggregate specific types of errors or test suites of interest. This partial action approach identifies errors comprehensively in relevant areas without performing excessive detailed evaluation across all test cases, resolving the time-accuracy contradiction.
3Reliability
If test data is not aggregated and analyzed systematically, then original test data is preserved, but identified errors cannot be efficiently reused for future test execution
Solution Approach 1:
The patent performs preliminary aggregation and analysis of test data during the current test execution, creating meta-test cases that capture error patterns and characteristics. This preliminary action preserves original test data integrity while preparing processed, analyzed results that can be efficiently reused for planning and executing future tests, thus resolving the contradiction between data integrity and future productivity.
Solution Approach 2:
The patent establishes a feedback mechanism where test results are aggregated into meta-test cases that inform future test execution strategies. The analyzed error patterns and aggregated results provide feedback for improving future test campaigns, enabling more efficient test execution while maintaining reliance on preserved original test data, thus resolving the contradiction.
Data Source
Figure 1
Figure 2
AI summary
The invention relates to a method for analyzing test procedures of a device (10a) and/or function (10b) for at least partial autonomous driving of a motor vehicle, comprising aggregating (S4) the first characteristic value (12) calculated for each of the plurality of test cases (T1) into a second characteristic value (16) representing a meta-test case (T2); and evaluating (S5) the second characteristic value (16) using a predetermined second criterion (18) and by logically combining the first evaluation results (14a) of the plurality of test cases (T1) into a second evaluation result (18a) of the meta-test case (T2). Furthermore, the invention relates to a system (1) for analyzing test procedures of a device (10a) and/or function (10b) for at least partial autonomous driving of a motor vehicle.