Reinforcement Learning Process Monitoring for Adaptive Fault Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional process control systems face challenges in accurately detecting anomalies and faults, particularly in changing plant conditions, due to the reliance on static thresholds and PCA-based methods that often result in false positives and alarm fatigue.

Innovation Solution

The implementation of a reinforcement learning model that learns to associate metrics with normal or anomalous states through a metric-reward mapping, dynamically updating a state-action mapping using historical and live plant data to improve anomaly/fault detection and remedial actions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If static thresholds and PCA-based methods are used for anomaly detection, then the system structure is simple, but the detection accuracy deteriorates and false positives increase

Engineering Contradiction:

Improvesystem structureVSAvoidanomaly detection accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent transforms static anomaly detection thresholds into dynamic, adaptive thresholds using reinforcement learning. The system continuously learns from historical and live plant data, adjusting detection thresholds based on changing plant conditions. This dynamic approach resolves the contradiction by maintaining simple system structure while significantly improving detection accuracy through adaptive learning rather than complex static rules.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The reinforcement learning model enables the anomaly detection system to self-improve automatically by learning from its own performance and plant data. The system autonomously adjusts detection parameters and identifies patterns without requiring complex external configuration or manual threshold setting, thereby improving accuracy while keeping the overall system structure relatively simple.

Inventive Principle:

Principle #25Self-service

2Ease of operation

If static thresholds are used for anomaly detection, then the system is easy to operate, but adaptability to changing plant conditions deteriorates

Engineering Contradiction:

Improvesystem operation simplicityVSAvoidadaptability to changing conditions

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent implements a feedback mechanism where the reinforcement learning model continuously receives performance feedback from anomaly detection outcomes and plant condition data. This feedback loop enables the system to automatically adapt to changing plant conditions by learning from past detections and adjusting future detection behavior, resolving the contradiction between ease of operation and adaptability.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs self-adjustment through reinforcement learning, automatically adapting detection parameters to changing plant conditions without requiring operator intervention. This self-service capability maintains ease of operation while dramatically improving adaptability to varying plant states and conditions.

Inventive Principle:

Principle #25Self-service

3Use of energy by moving object

If PCA-based methods with static thresholds are used, then computational resources are conserved, but false positives increase causing alarm fatigue

Engineering Contradiction:

Improvecomputational resource usageVSAvoidfalse positive rate

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

The patent introduces dynamic threshold adjustment through reinforcement learning that adapts detection sensitivity based on learned patterns and current plant conditions. This dynamic approach reduces false positives by distinguishing between normal variations and actual anomalies, improving reliability while maintaining reasonable computational resource usage through efficient learning algorithms.

Inventive Principle:

Principle #15Dynamics

4Measurement precision

If reinforcement learning model is implemented for anomaly detection, then detection accuracy and adaptability improve, but device complexity increases

Engineering Contradiction:

Improveanomaly detection accuracyVSAvoidsystem structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent changes the fundamental parameter of threshold values from static to dynamic through reinforcement learning. Instead of implementing a completely complex new system, it transforms the behavior of existing detection parameters, achieving improved accuracy while limiting complexity growth to the learning component rather than the entire system architecture.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20240119342A1General reinforcement learning framework for process monitoring and anomaly/ fault detection

Publication Date: 2024.04.11 FISHER ROSEMOUNT SYST INC

AI summary

A method includes receiving a metric-reward mapping; and using reinforcement machine learning to train a state-action mapping. A method includes receiving a set of metrics corresponding to an ongoing industrial control process; determining anomaly/fault and normal action values by reference to a reinforcement learning-determined state-action mapping; and causing a remedial action to occur. A process control system includes an anomaly/fault detection device, that receives metrics, determines anomaly/fault and normal action values; and causes a remedial action to occur.