Virtual Test Classifier for Simulation Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The reliability of simulation models in testing technical systems, such as embedded systems and electronic control units, is limited, leading to mistrust in their results and limited incorporation into release decisions, especially in fields like automotive engineering and robotics.
Innovation Solution
A method that uses a virtual test classifier to combine model validation and product testing by evaluating simulation model error metrics and quantitative requirements, allowing for the selection and refinement of test cases to enhance the reliability of simulation results, thereby expanding the range of reliable test input parameters and improving the trustworthiness of simulation-based testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simulation models are used for testing technical systems, then productivity is improved through automated testing, but reliability deteriorates due to lack of trust in simulation results
Solution Approach 1:
The patent introduces signal metrics as an intermediary mechanism that objectively compares simulation results with real measurements. This mediator quantifies the agreement between virtual and physical system behavior, providing a basis for trusting simulation results. The signal metrics act as a bridge that transforms subjective trust into objective, measurable validation criteria.
Solution Approach 2:
The patent implements feedback loops where simulation results are continuously validated against real measurements using signal metrics. The validation results feed back into the testing process, allowing the system to learn from discrepancies and improve the reliability of simulation models over time. This creates a closed-loop system that progressively enhances trust in simulation-based testing.
2Reliability
If simulation models are validated against real measurements, then reliability is improved through trust in results, but device complexity increases due to additional validation infrastructure
Solution Approach 1:
The patent creates a universal validation framework using signal metrics that can be applied across multiple testing scenarios, system types, and measurement domains. This multi-functional approach consolidates what would otherwise require separate validation mechanisms for each testing case, reducing overall infrastructure complexity while maintaining comprehensive reliability validation.
Solution Approach 2:
The patent transforms the validation process from a qualitative, subjective assessment into a quantitative parameter-based evaluation. By defining specific signal metrics with measurable parameters, the system converts complex validation judgments into straightforward numerical comparisons, simplifying the validation infrastructure while improving reliability assessment.
3Adaptability or versatility
If the range of test input parameters is expanded, then adaptability is improved through comprehensive testing, but measurement precision deteriorates due to limited validation data
Solution Approach 1:
The patent applies partial validation strategically to the most critical test parameters and operating conditions. Rather than requiring exhaustive validation across all possible parameters, the system focuses validation efforts on key parameters that have the greatest impact on system safety and performance. This selective approach maintains measurement precision for critical parameters while enabling expanded testing ranges for less critical parameters.
Data Source
AI summary
A method for testing a technical system. The method includes: tests are carried out with the aid of a simulation of the system, the tests are evaluated with respect to a fulfillment measure of a quantitative requirement on the system and an error measure of the simulation, on the basis of the fulfillment measure and error measure, a classification of the tests as either reliable or unreliable is carried out, and a test database is improved on the basis of the classification.