Industrial Process Control Using Inverse Neural Networks and RL
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing model-based control systems, such as Model Predictive Control (MPC), face challenges in accurately representing complex industrial processes due to limited system information and difficulties in determining actuator limitations, leading to inefficient computational efforts and suboptimal performance.
Innovation Solution
A method and system utilizing a process control device that integrates a trained inverse neural network and Reinforcement Learning (RL) agent to predict process control signals, optimizing convergence rates and constraints, reducing computational effort by leveraging a low-fidelity model and incorporating RL for dynamic adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If model-based control (MPC) is used to control industrial processes, then control performance is improved, but computational effort increases and model accuracy decreases due to simplification
Solution Approach 1:
The patent pre-calculates and stores optimal control actions in lookup tables during an offline phase, so that during online operation, only simple table lookups are needed instead of running complex MPC computations in real-time
Solution Approach 2:
The patent creates simplified representations of the complex MPC controller by storing pre-computed control policies in lookup tables, which approximate the behavior of the full MPC system without requiring its computational resources
2Measurement precision
If accurate nonlinear models are used in MPC, then control accuracy is improved, but model design becomes challenging and computationally expensive
Solution Approach 1:
The patent uses simplified, low-fidelity models instead of accurate nonlinear models, accepting that the model itself is approximate but compensating through the pre-computed lookup tables that capture the optimal control behavior
Solution Approach 2:
The patent creates a simplified model representation that is computationally inexpensive, storing the essential control knowledge in lookup tables rather than maintaining complex nonlinear model structures
3Adaptability or versatility
If Reinforcement Learning is used to tune MPC parameters, then adaptability is improved, but computational cost increases due to intensive exploration
Solution Approach 1:
The patent performs the computationally intensive Reinforcement Learning training offline to pre-compute lookup tables, so that online adaptation can be achieved through simple table lookups without real-time computational burden
Solution Approach 2:
The patent creates a simplified copy of the RL-MPC system by storing pre-trained policies in lookup tables, which provide adaptive behavior without requiring the computational resources of the full RL training process
4Reliability
If exhaustive exploration of control environments is performed, then optimal policy is improved, but computational efficiency decreases
Solution Approach 1:
The patent performs exhaustive exploration and policy optimization in an offline training phase, storing the results in lookup tables that can be quickly queried during online operation without repeating the computational exploration
Solution Approach 2:
The patent creates a simplified representation of the optimally explored policy by storing it in lookup tables, which approximate the exhaustive exploration results without requiring the computational resources of repeated exploration
Data Source
Figure 1~3
Figure 4~6
Figure 7
AI summary
The invention is concerned with a method, process control device, process control system excitation arrangement, control device, process control system, computer program and computer program product for controlling an industrial process. The process control device obtains a prediction of a range of process control signals u(k: k + N) for the industrial process (32), which range of process control signals u(k: k + N) have been predicted using a prediction of a range of process response signals y(k: k + N) in a model of the industrial process (32) and which range of process response signals y(k: k + N) have been predicted based on a range of reference signals r(k: k + N) and applies the predicted process control signals u(k: k + N) in the control of the process (32).