Simulation Model Validation via Error Distribution Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current simulation methods for technical systems, such as embedded systems in automotive engineering, lack reliability due to limited incorporation of simulation results in release decisions, primarily because of trust issues related to their accuracy and handling of uncertainties.
Innovation Solution
A method that compares simulation model outputs with measured values using error measures like mean squared error, allowing for the validation of time series and scalar comparisons, and incorporates aleatoric and epistemic parameters to estimate model calibration and identify outliers, thereby enhancing the reliability of simulation models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simulation models are used for testing technical systems, then productivity is improved by automating test activities, but reliability deteriorates due to lack of trust in simulation accuracy
Solution Approach 1:
The patent implements a feedback mechanism by comparing simulation model outputs with measured values from physical experiments. The validation module calculates error measures between simulated and measured time signals, and uses this feedback to assess model quality and guide model improvement, thereby increasing trust in simulation results while maintaining automated testing productivity
Solution Approach 2:
The patent replaces pure physical experimentation with a hybrid approach that substitutes some mechanical testing with validated simulation models. By establishing quantitative validation criteria, the simulation model can substitute for physical tests in scenarios where validation thresholds are met, improving productivity while maintaining reliability through the validation framework
2Reliability
If multiple repeated experiments are compared to reduce statistical uncertainties, then reliability is improved, but device complexity increases due to multiple comparisons
Solution Approach 1:
The patent transforms the complex multi-dimensional comparison of multiple time series into a simplified statistical framework by changing parameters to probability distributions and error measures. Instead of manually comparing multiple experiments, the system calculates statistical metrics (mean squared error, confidence intervals) that automatically synthesize multiple comparisons into actionable validation results, improving reliability while managing complexity through statistical aggregation
3Reliability
If validation compares simulation outputs with measured values, then reliability is improved through empirical verification, but measurement precision requirements increase the difficulty of detecting and measuring
Solution Approach 1:
The patent introduces an intermediary validation module that acts as a mediator between simulation outputs and measured values. This module standardizes the comparison process by implementing defined error measures and validation criteria, transforming the complex task of direct comparison into a systematic process that handles data alignment, error calculation, and statistical analysis, thereby improving validation reliability while reducing measurement complexity
Data Source
AI summary
A method for simulating a technical system. Time series are obtained with the aid of a simulation model of the system, variable values being assigned to at least one epistemic parameter of the simulation model, at least one measurement series is obtained by corresponding measurements on the system, for each value of the epistemic parameter, a real-value error measure of the time series obtained for this value with respect to the measurement series and a distribution function of the error measure are calculated, and for the simulation, that value is used, for which the distribution function has the smallest distance to a Heaviside function.