Estimation Model Monitoring with Digital Twin Robustness Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems that utilize artificial intelligence for estimation in critical applications face challenges in verifying the robustness and stability of these models, particularly when dealing with data outside their predefined operational design domain.
Innovation Solution
A predictive monitoring system that generates test data by applying transformations such as noise introduction or abnormal data generation, and then uses a digital twin to simulate the behavior of the estimation model under these conditions, comparing the results to predefined criteria to assess robustness and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data-driven machine learning models are used for estimation in critical systems, then productivity and adaptability are improved, but reliability and robustness become difficult to verify
Solution Approach 1:
The system performs preliminary actions by generating test data with introduced disturbances and abnormalities before actual operation. The digital twin simulates model behavior under various conditions in advance, allowing robustness verification to be performed proactively rather than reactively when failures occur.
Solution Approach 2:
A digital twin (copy) of the estimation model is created to simulate its behavior under test conditions. This copy allows verification of robustness without affecting the actual operational model, enabling safe testing of extreme and abnormal conditions that would be difficult to verify with the original model alone.
2Reliability
If the number of training data is increased to improve model robustness, then reliability is improved, but loss of time and industrial cost increase
Solution Approach 1:
Instead of changing the quantity of training data, the system changes parameters by introducing artificial disturbances and abnormalities to the test data. This allows verification of model robustness under varied conditions without requiring additional training data collection and processing time.
Solution Approach 2:
A digital twin acts as an intermediary between the actual model and the verification process. The digital twin handles the computationally intensive simulation of various conditions, allowing rapid verification without directly burdening the operational model or requiring extensive data collection.
3Adaptability or versatility
If the estimation model operates outside its predefined usage domain, then adaptability is improved, but stability and robustness deteriorate
Solution Approach 1:
The system applies preliminary anti-action by introducing disturbances and abnormalities that counteract potential failures before they occur in actual operation. Test data is deliberately modified to include conditions outside the normal usage domain, allowing the model to be stress-tested and its stability boundaries to be identified.
Solution Approach 2:
The system uses feedback by comparing the digital twin's simulation results with expected behavior under various conditions. When the model shows instability or incorrect behavior under certain conditions, this feedback identifies the boundaries of the usage domain and triggers alerts for retraining or model adjustment.
Data Source
Figure 1
Figure 2
AI summary
The present invention relates to a predictive monitoring device (16) for the operation of an estimation unit (14) of a physical quantity receiving data acquired by a detection unit (12) and applying an estimation model to the acquired data, the predictive monitoring device (16) comprising: - a test data generation unit (20) applying a transformation to the acquired data, - a calculation unit (22) estimating a value of the physical quantity by applying the model to the test data, - an analysis unit (24) of the test value performing a comparison between the test value and an expected value according to a criterion for evaluating the performance of the model, and - a warning unit (26) issuing an alert based on the result of the comparison.