Abnormality Cause Estimation in Reinforcement Learning Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In control systems utilizing multiple machine learning models, it is challenging to determine the main cause of abnormalities, as existing methods lack clarity in distinguishing between evaluation and operation models, leading to difficulties in identifying the root cause of issues.
Innovation Solution
An estimation apparatus and method that acquires an abnormality index from state data when an abnormality occurs, estimating whether the evaluation model or the operation model is the main cause based on this index, and outputs instructions to retrain either model accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple machine learning models (evaluation model and operation model) are used in a control system, then the system's functionality and performance are improved, but it becomes difficult to determine the main cause of abnormalities
Solution Approach 1:
The patent segments the abnormality analysis by creating distinct abnormality indices for the evaluation model and operation model. The estimation unit divides the diagnostic task into separate evaluation paths, allowing independent analysis of each model's contribution to the abnormality through dedicated indices and threshold comparisons.
Solution Approach 2:
The patent introduces abnormality indices as intermediary metrics that mediate between the complex multi-model system and the cause identification task. These indices serve as intermediate representations that translate model outputs into interpretable abnormality measurements, facilitating clearer cause determination.
2Measurement precision
If the system provides detailed abnormality analysis, then the accuracy of cause determination is improved, but the system complexity increases
Solution Approach 1:
The patent segments the diagnostic system into distinct functional components: abnormality index acquisition units for each model type, an estimation unit for synthesis, and an output unit for results. This modular segmentation achieves precise cause determination while managing complexity through organized functional separation.
Solution Approach 2:
The patent uses parameter changes in the form of abnormality indices and threshold values to achieve precise cause determination. By adjusting and comparing index values against predetermined thresholds, the system accurately identifies abnormality causes without requiring complex analytical structures.
3Ease of operation
If the system automatically identifies the main cause of abnormalities, then the ease of operation is improved, but the reliability of the identification may be compromised
Solution Approach 1:
The patent implements feedback mechanisms where abnormality indices are continuously acquired and compared against thresholds, with results fed back to determine the main cause. This feedback loop enables automatic identification while maintaining reliability through systematic validation against predetermined criteria for each model type.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
[Abstract][Means for solving the problem] There is provided an estimation apparatus including an abnormality index acquisition unit configured to acquire, as an abnormality index, an evaluation index which is output by an evaluation model according to state data being input when an abnormality occurs in a facility, among pieces of the state data indicating a state of the facility when an operation model is used to control a control target provided in the facility, the operation model being generated by reinforcement learning in which an output of the evaluation model trained by machine learning to output the evaluation index in accordance with the state of the facility, is set as at least a part of a reward, and outputting an action in accordance with the state of the facility; an estimation unit configured to estimate which of the evaluation model or the operation model is a main cause of the abnormality, based on the abnormality index; and an output unit configured to execute an output in accordance with a result of the estimate.