Virtual Test Classification for Reliable Simulation-Based Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The reliability of simulation models in model-based testing is limited, leading to a lack of trust in their results, which hinders the incorporation of simulation findings into release decisions for technical systems, particularly in fields like automotive engineering and robotics.
Innovation Solution
A virtual test classifier is introduced to assess the quality of simulation models by combining validation metrics with test requirements, using signal metrics and uncertainty quantification methods to predict simulation errors and ensure trustworthy test results, thereby unifying validation and verification approaches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simulation models are used for testing technical systems, then testing efficiency and early detection of errors are improved, but the reliability and trustworthiness of test results deteriorate
Solution Approach 1:
The patent introduces a validation model as an intermediary between the simulation model and the test results. This validation model compares simulation outputs with real measurement data to assess the quality and reliability of the simulation model, thereby mediating the trustworthiness of test results while maintaining the efficiency benefits of simulation-based testing
Solution Approach 2:
The patent implements a feedback mechanism where validation metrics are calculated by comparing simulation results with real measurement data, and this feedback is used to adjust and improve the simulation model. This continuous feedback loop enhances the reliability of simulation-based testing while maintaining high testing efficiency
2Reliability
If validation metrics are used to assess simulation quality, then the trustworthiness of test results is improved, but the complexity of the testing process increases
Solution Approach 1:
The patent develops a universal validation framework that can be applied across different simulation models and technical systems. The validation model serves multiple functions including quality assessment, error metric calculation, and confidence determination, thereby improving trustworthiness without proportionally increasing process complexity
Solution Approach 2:
The patent transforms complex validation assessments into simplified parameter representations through error metrics and confidence levels. By changing the parameters from detailed validation data to summarized quality indicators, the system maintains high trustworthiness while reducing the perceived complexity of the testing process
Data Source
AI summary
A method for testing a technical system. Tests are carried out with the aid of a simulation of the system. The tests are evaluated with respect to a fulfillment measure of a quantitative requirement on the system and different error measures of the simulation. On the basis of the fulfillment measure and each of the error measures, a classification of the tests is carried out as either reliable or unreliable case by case. A selection among the error measures is made on the basis of a number of the tests classified as reliable.