Component Model Discrepancy Analysis for Complex System Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex technical systems with intricately linked components are difficult to validate or verify due to their stochastic and complex nature, making it hard to predict and guarantee desired behavior, especially in systems of systems where complexity grows rapidly.
Innovation Solution
A method involving obtaining models for system components, training machine learning models using validation measurements, propagating test inputs through these models to determine discrepancies, and verifying if the system meets a desired criterion by maximizing the probability of not exceeding a predefined discrepancy threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional validation methods are used on complex technical systems, then validation thoroughness should be high, but the complexity of the validation process and data requirements increase rapidly
Solution Approach 1:
The patent creates virtual copies (digital twins) of system components and their validation measurements. Instead of validating the actual complex system directly, virtual representations are used to simulate and analyze system behavior. This allows thorough validation of complex systems through computationally efficient virtual models, resolving the contradiction between validation thoroughness and process complexity.
Solution Approach 2:
The patent segments the complex technical system into individual components, each with its own validation measurements and machine learning models. By dividing the system into manageable parts and validating them separately, the overall validation complexity is reduced while maintaining comprehensive coverage of system behavior.
2Measurement precision
If extensive real-world data collection is performed to validate complex systems, then validation accuracy improves, but the time and resources required increase significantly
Solution Approach 1:
The patent performs preliminary validation by training machine learning models on validation measurements during the design phase, before actual system deployment. This preliminary action captures system behavior characteristics in advance, eliminating the need for extensive real-world data collection later and significantly reducing validation time while maintaining accuracy.
Solution Approach 2:
Virtual copies of validation measurements are used to represent real system behavior. Instead of collecting extensive real-world data, the patent uses computationally generated virtual measurements that replicate system characteristics, achieving high validation accuracy without the time and resource costs of extensive physical testing.
3Productivity
If machine learning models are trained on validation measurements to predict system behavior, then the need for extensive real-world testing is reduced, but the accuracy of predictions may compromise validation reliability
Solution Approach 1:
The patent incorporates feedback mechanisms where machine learning model predictions are continuously refined using discrepancy information. The difference between predicted and actual validation measurements is used to update and improve model accuracy, ensuring that predictions remain reliable while maintaining high validation efficiency through reduced physical testing.
Solution Approach 2:
The patent replaces traditional mechanical validation approaches (extensive physical testing) with computational methods using machine learning models. This substitution maintains validation reliability by using statistically rigorous methods to compare virtual and real system behavior, while dramatically improving validation efficiency through reduced physical testing requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Method for verifying and/or validating whether a technical system (40) fulfills a desired criterion, wherein the technical system (40) emits output signals based on input signals supplied to the technical system (40), wherein the method comprises the steps of: a. Obtaining models (M1,M2,MC) for a plurality of components (S1,S2,SC) comprised by the technical system (40), wherein a connection between the obtained models characterizes which component passes which signal to which other component; b. Obtaining a plurality of validation measurements, wherein a validation measurement comprises a measurement input and a measurement output (p1,p2,pC-1,pC), wherein the measurement output is obtained from a component (S1,S2,SC) of the technical system (40) for the measurement input if the measurement input is provided to the component (S1,S2,SC); c. For each component (S1,S2,SC), training a machine learning model (V1,V2,VC) to predict measurement outputs (p1,p2,pC-1,pC) of the respective component (S1,S2,SC) based on inputs of the respective component, wherein at least parts of the validation measurements are used as training dataset and wherein the machine learning model (V1,V2,VC) corresponds to the model (M1,M2,MC) obtained for the component; d. Obtaining first test outputs (qM,C) from a last model (MC) based on test inputs (q°), wherein the first test outputs (qM,C) are obtained by propagating the test inputs (q°) through the connection of models; e. Determining, second test outputs (qV,C) from the machine learning model (VC) corresponding to the last model and based on the test inputs (q0) of the models (M1,M2,MC), wherein the second test outputs (qV,C) are obtained by propagating the test inputs (q0) through a connection of the machine learning models (V1,V2,VC), wherein the connection of the machine learning models (V1,V2,VC) is according to the connection of the models (M1,M2,MC) the respective machine learning models (V1,V2,VC) correspond to; f. Determining a discrepancy (d), wherein the discrepancy (d) characterizes a difference between a distribution of first test outputs (qM,C) determined from the last model (MC) and a distribution of second test outputs (qV,C) determined by the machine learning model (VC) corresponding to the last model (MC); g. Verifying and/or validating whether the technical system (40) fulfills the criterion, wherein verifying and/or validating is characterized by maximizing a probability of a distribution of measurement outputs (pC) of a last component (SC) of the technical system (40) to not fulfill the criterion with respect to the distribution of measurement outputs (pC) and under a constraint stipulating that a discrepancy of the distribution of measurement outputs (pC) and the distribution of first test outputs (qM,C) may not exceed the discrepancy (d) determined in step f.