Self-Optimizing Process Control Using Deep Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional control methods in air separation plants and other process plants struggle to ensure optimal operation, particularly during load changes, with ALC controllers providing rapid stability but lacking multi-variable control advantages, while MPC controllers offer stability but are slower and unpredictable.
Innovation Solution
Implement a control system using model-based deep reinforcement learning with a neural network that self-optimizes by continuously improving control strategies through retraining, incorporating a cost function that considers energy consumption and product purity, and utilizing a neural network to predict optimal control inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If ALC controller is used for rapid load changes, then adjustment speed is improved, but multi-variable control capability is lost
Solution Approach 1:
The patent merges ALC controller and MPC controller into a hybrid control system where ALC provides rapid setpoint adjustment for load changes while MPC handles multi-variable optimization. The ALC controller quickly adjusts setpoints in response to load changes, and the MPC controller simultaneously optimizes multiple process variables to achieve both fast response and comprehensive control capability.
Solution Approach 2:
The control system achieves multi-functionality by integrating two control strategies: ALC for rapid load change response and MPC for multi-variable optimization. This universal controller can handle both fast adjustment requirements and complex multi-variable control tasks, making the system adaptable to various operating conditions.
2Stability of the object's composition
If MPC controller is used for multi-variable control, then control stability is improved, but adjustment speed becomes slow and unpredictable
Solution Approach 1:
The patent combines MPC controller with ALC controller to resolve the speed-stability tradeoff. MPC maintains control stability through its predictive algorithm and multi-variable optimization, while ALC provides rapid setpoint adjustment. The hybrid system leverages the strengths of both controllers to achieve both stability and fast response.
Solution Approach 2:
The ALC controller performs preliminary action by quickly adjusting setpoints in response to load changes before the MPC controller fine-tunes the process variables. This preliminary rapid adjustment followed by stabilized optimization enables both fast response and controlled stability.
3Device complexity
If conventional control methods are used, then system complexity is reduced, but control quality during load changes deteriorates
Solution Approach 1:
The patent segments the control system into distinct functional modules: ALC controller for load change response, MPC controller for multi-variable optimization, and neural network for prediction. This modular segmentation allows each component to perform its specialized function effectively while maintaining manageable overall system complexity.
Solution Approach 2:
The neural network acts as an intermediary that predicts process behavior and provides input to both ALC and MPC controllers. This intermediary component enhances control quality by providing predictive information without requiring complete redesign of the control architecture, thus improving reliability while controlling complexity.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The invention relates to a method for operating a process system (100), in which method one or more actuators in the process system (100) are set by means of one or more control values specified by means of a control process, whereby one or more operating parameters of the process system (100) are influenced. The control process is a self-optimizing control process which comprises the use of model-based deep reinforcement learning and the consideration of a cost function. One or more components of the process system (100) are represented in a model by means of neural network, which model is used in the model-based deep reinforcement learning. The present invention also relates to a corresponding process system (100) and to a method for converting a process system (100).