Process Control Using Deep Reinforcement Learning for Load Changes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional control methods in air fractionation plants and other processing systems often fail to ensure optimal operation, particularly during load changes, as they lack predictive mechanisms for setpoint values and stability in multi-variable control environments.
Innovation Solution
A method utilizing model-based deep reinforcement learning with a neural network to self-optimize control strategies, influenced by a cost function that considers energy consumption and product purity, allowing for continuous improvement and better energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional control methods (PID, cascade controllers, ALC control, MPC) are used, then the system can maintain basic operation and stability, but the system fails to ensure optimal operation during load changes and lacks predictive mechanisms for setpoint values
Solution Approach 1:
The neural network is trained in advance with historical operating data and simulation data to learn optimal control strategies before actual load changes occur. This preliminary training enables the system to predict and adapt to load changes proactively rather than reactively, resolving the contradiction by preparing the controller with pre-acquired knowledge of optimal operations under various loading conditions.
Solution Approach 2:
The system implements continuous feedback loops where actual process data is fed back to the neural network controller, which adjusts setpoint values and control parameters in real-time based on deviations from optimal operation. This feedback mechanism enables dynamic optimization during load changes while maintaining stability, addressing both reliability and productivity requirements simultaneously.
2Productivity
If model-based deep reinforcement learning with neural network is implemented, then controller adaptation and energy efficiency improve significantly, but the device complexity and computational requirements increase
Solution Approach 1:
The patent creates a virtual copy of the air separation plant through neural network modeling and digital twins. This virtual model replicates the complex physical system's behavior, allowing the control algorithm to learn and optimize in the virtual environment without affecting the actual plant. The copied model handles the computational complexity while the physical system benefits from optimized control with minimal additional hardware complexity.
Solution Approach 2:
The patent replaces traditional mechanical control systems (PID controllers, cascade controllers, ALC control systems) with an intelligent software-based neural network controller. This substitution eliminates the need for complex hardware modifications and interconnections of multiple controllers, reducing device complexity while achieving superior energy efficiency through data-driven optimization algorithms.
3Adaptability or versatility
If multiple controllers (base controllers, ALC control, trim controllers) are interconnected, then the system can handle multi-variable control, but the system lacks predictive mechanisms and requires intensive higher-level process management
Solution Approach 1:
The patent merges the functions of multiple separate controllers (base controllers, ALC control, trim controllers) into a single unified neural network controller. This consolidation integrates multi-variable control capabilities, predictive mechanisms, and adaptive optimization into one intelligent system, reducing process management complexity while maintaining full adaptability for handling complex multi-variable air separation operations.
Solution Approach 2:
The neural network controller is designed as a universal multi-functional control system that can perform all control tasks previously requiring separate controllers. It simultaneously handles base control, load change adaptation, trim adjustments, and predictive optimization, eliminating the need for intensive higher-level process management while providing comprehensive multi-variable control capabilities through a single intelligent controller.
Data Source
AI summary
The invention relates to a method for operating a process system, in which method one or more actuators in the process system are set by means of one or more manipulated variable values specified by means of a control process, whereby one or more operating parameters of the process system are influenced. The control process is a self-optimizing control process which comprises the use of model-based deep reinforcement learning and the consideration of a cost function. One or more components of the process system are represented in a model by means of neural network, which model is used in the model-based deep reinforcement learning. The present invention also relates to a corresponding process system and to a method for converting a process system.