Industrial furnace temperature control optimization method and system based on model predictive control

Abstract

The invention provides an industrial furnace temperature control optimization method and system based on model predictive control. The method comprises the following steps: defining a state space and an action space of a temperature control system; an MPC prediction model is established based on the state space and the action space, a future temperature trajectory is predicted, and a prediction error compensation mechanism and a self-adaptive prediction domain are introduced; constructing a deep reinforcement learning model which comprises a state value network and a strategy network and is used for learning an optimal temperature control strategy; based on the MPC prediction model and the deep reinforcement learning model, constructing a multi-objective optimization model for controlling performance, energy consumption and temperature uniformity objectives, and solving to obtain an optimal control sequence; and taking the optimal control sequence as a reference trajectory, and alternately updating parameters of the deep reinforcement learning model by adopting an Actor-Critic architecture. Through multi-level collaborative optimization, the adaptability, the control precision and the energy utilization efficiency of the industrial furnace temperature control system are improved.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation control technology, and in particular relates to an industrial furnace temperature control optimization method and system based on model predictive control. Background Technology

[0002] In modern industrial production, temperature is a key process parameter, and its control quality directly affects product performance and energy consumption. Traditional industrial furnace temperature control mainly employs methods such as PID and fuzzy control, which have limitations such as slow response speed, poor robustness, and difficulty in adapting to complex operating conditions. In recent years, Model Predictive Control (MPC) has been widely used in industrial temperature control due to its consideration of constraints and future behavior. However, MPC methods rely on accurate mathematical models and struggle to cope with the strong nonlinearity, time-varying characteristics, and modeling uncertainties of temperature systems.

[0003] On the other hand, with the development of artificial intelligence technology, Deep Reinforcement Learning (DRL) has demonstrated powerful adaptive control capabilities. Through continuous interaction and learning between the agent and the environment, DRL can achieve self-optimization of policies in complex dynamic systems. However, current DRL methods mostly focus on the algorithm level, with few engineering application cases, and cannot yet directly guide industrial control practices.

[0004] Current industrial furnace temperature control solutions have the following shortcomings:

[0005] (1) Traditional industrial furnace temperature control methods rely on a fixed prediction domain, which is difficult to adapt to the time-varying nature and disturbance changes of the system, resulting in a decrease in temperature control accuracy;

[0006] (2) The MPC model has a rigid control strategy in complex dynamic environments, making it difficult to flexibly cope with multi-objective optimization requirements and nonlinear state changes;

[0007] (3) The use of single-objective optimization methods can easily lead to local optimal performance, and cannot take into account multiple objectives such as control performance, energy consumption level and temperature field uniformity.

[0008] Therefore, we need to develop an industrial furnace temperature control optimization method and system based on model predictive control, which can improve prediction accuracy, enhance strategy self-optimization, and achieve multi-objective collaborative optimality of industrial furnace temperature control. Summary of the Invention

[0010] The purpose of this invention is to provide an industrial furnace temperature control optimization method and system based on model predictive control, so as to solve the problems mentioned in the background art, such as non-adaptive prediction, rigid strategy, difficulty in taking into account multiple objectives, and limited control accuracy and overall system performance of existing industrial furnace temperature control schemes.

[0011] To achieve the above objectives, on the one hand, the present invention provides an industrial furnace temperature control optimization method based on model predictive control, the method being as follows:

[0012] Step S1: Define the state space and action space to characterize the dynamic characteristics of the industrial furnace temperature optimization control system;

[0013] Step S2: Based on the state space and action space, establish an MPC prediction model to predict the future temperature trajectory of the industrial furnace temperature optimization control system, and introduce a prediction error compensation mechanism and an adaptive prediction domain to optimize the performance of the MPC prediction model.

[0014] Step S3: Construct a deep reinforcement learning model, including designing a state-value network and a policy network, to learn the optimal control strategy of the industrial furnace temperature optimization control system and achieve adaptive optimization control;

[0015] Step S4: Based on the MPC prediction model and the deep reinforcement learning model, the overall objective function is constructed by weighted summing the control performance objective function, energy consumption objective function and temperature uniformity objective function, which serves as a multi-objective optimization model. A constrained optimization problem is constructed based on the multi-objective optimization model, and the optimal temperature control sequence is obtained by solving the constrained optimization problem.

[0016] The multi-objective optimization model is as follows: ,in These are weighting coefficients, representing the relative importance of different objectives. , , These are the objective functions for control performance, energy consumption, and temperature uniformity, respectively.

[0017] Step S5: Using the optimal temperature control sequence as a reference trajectory, the Actor-Critic architecture is adopted to alternately update the policy network and state value network parameters of the deep reinforcement learning model based on the temporal difference error, thereby optimizing the optimal temperature control strategy.

[0018] Based on the aforementioned scheme, the state space is a high-dimensional vector that contains the temperature system at time [time]. All status information: ,in, , indicating time The actual temperature matrix; Represents the temperature deviation matrix; As an energy consumption indicator, it quantifies the energy consumption level of furnace temperature control. This is a vector of process parameters, containing various process conditions that affect furnace temperature. The predicted temperature sequence for the next N steps is an important basis for the control of the MPC prediction model, and ;

[0019] Action space A defines all possible operations of the industrial furnace temperature optimization control system, consisting of control vectors from m independent actuators, as follows: For the first An actuator, whose control vector This represents the sequence of control increments to be applied in the next N steps: , ,in and For the control constraints of the i-th actuator.

[0020] Based on the aforementioned scheme, the establishment of the MPC prediction model includes establishing a single-step state transition function for single-step prediction, using an autoregressive structure for multi-step prediction, and using an adaptive prediction domain and prediction error compensation mechanism for optimization and adaptive improvement.

[0021] The single-step prediction includes:

[0022] Define a single-step state transition function ,Will System state at time 1 Mapped to state of time As a single-step prediction model, its mathematical expression is:

[0023]

[0024] in, , is the system state transition matrix. , is the control gain matrix. , is the bias vector. , which is the time-varying set of model parameters. This represents the control action at time k, corresponding to the control value at time k in the control increment sequence defined in the action space, i.e. It consists of the control increments of m independent actuators at time k.

[0025] Based on the aforementioned scheme, and based on the single-step state transition function A multi-step prediction function is constructed using autoregressive extrapolation. ,Right now:

[0026]

[0027] in, ,for Predict the state sequence step by step. ,for Step control sequence.

[0028] Based on the aforementioned scheme, the MPC prediction model is characterized as follows: , , ,in For based on Future predicted by time information The state of the temperature system, i.e., the future temperature trajectory. For the corresponding output predicted value, Output dimension; output matrix The state space is mapped to the output space, and key indicators to be optimized, including the average and extreme values ​​of furnace temperature, are extracted.

[0029] Based on the aforementioned scheme, the state value network uses a state space. For the estimation of input and output state values, the network structure adopts a multilayer perceptron, and the expression is: ,in, This is the input layer representation in a state-value network, directly corresponding to the input state space. , For the first The hidden state of the layer and For the first Layer weight matrix and bias vector, For activation function, For network parameters, This represents the state-value function, used to estimate the state value in the state space. The expected value of long-term cumulative rewards that can be obtained by following the current strategy.

[0030] Based on the aforementioned scheme, the policy network adopts a Gaussian policy, with a state space. Given the input, output the probability distribution of the action space A, expressed as follows: ,in In the state space The probability distribution of choosing action A. and Let be the mean and standard deviation of state-related actions, respectively, which can be approximated as: ,in and For output of the hidden layer, For policy network parameters, The standard deviation is denoted as .

[0031] Based on the aforementioned scheme, the control performance objective function, energy consumption objective function, and temperature uniformity objective function are constructed using the output equation of the single-step prediction model of the MPC prediction model. The control performance objective function is as follows: ,in To control performance targets, a penalty is imposed on the weighted L2 norm between the predicted output trajectory and the reference trajectory. The output equation is... for The reference trajectory for step prediction express Weighted L2 norm, It is a positive semi-definite matrix.

[0032] Based on the aforementioned scheme, the constrained optimization problem is formulated as follows: The constraints are: , , , , , That is, minimizing the multi-objective function under the constraints of the MPC prediction model. At the same time, control the amount ,state and output Physical constraints must be met, where u corresponds to the control increment sequence defined in action space A, and x corresponds to the state space. The system state variable described in the text, y corresponds to the system output. This is the output matrix, used to map the state space to the output space.

[0033] On the other hand, the present invention provides an industrial furnace temperature control optimization system based on model predictive control, used to execute the industrial furnace temperature control optimization method based on model predictive control described in the aforementioned scheme, comprising: a data acquisition unit, a predictive control unit, a reinforcement learning unit, an industrial computer, an execution control unit, and an industrial furnace object;

[0034] The data acquisition unit is used to realize the real-time acquisition and monitoring of the temperature field inside the industrial furnace, and to realize real-time data exchange with the host computer system.

[0035] The industrial control computer, as the hardware platform of the host computer system, receives the industrial furnace temperature data collected by the data acquisition unit and inputs it into the prediction control unit and the reinforcement learning unit; and the software algorithms of the prediction control unit and the reinforcement learning unit run on the industrial control computer.

[0036] The prediction control unit predicts the future temperature trajectory using the MPC prediction model;

[0037] The reinforcement learning unit is used to learn and optimize policies based on a deep reinforcement learning model and through the actor-critic algorithm.

[0038] The execution control unit receives the optimal temperature control strategy output by the industrial control computer, performs weighted combination of multiple targets, and generates control commands to regulate the temperature of the industrial furnace.

[0039] The present invention has the following advantages and effects compared with the prior art:

[0040] (1) An adaptive prediction domain adjustment mechanism is introduced to dynamically adjust the MPC prediction step size and scale. Combined with online prediction error compensation, the deviation of the MPC model is continuously corrected to achieve adaptive modeling of the time-varying characteristics of the industrial furnace temperature system, which significantly improves the accuracy of future temperature trajectory prediction and the robustness of the control system.

[0041] (2) Combining deep reinforcement learning models, state value network and policy network are designed, and the Actor-Critic architecture is adopted to continuously and alternately update the control policy, thereby improving the policy flexibility and self-optimization capability and realizing the continuous adaptive optimization of the control policy of the industrial furnace temperature control system under complex working conditions.

[0042] (3) Construct a weighted multi-objective optimization model that includes three sub-objective functions: control error, energy consumption, and temperature uniformity. Introduce an adaptive weight adjustment mechanism to dynamically balance each objective, thereby achieving synergistic optimization of multiple performance indicators of the industrial furnace temperature control system and improving overall energy efficiency and product quality consistency. Attached Figure Description

[0043] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0044] Figure 1 This is a flowchart of an industrial furnace temperature control optimization method based on model predictive control provided in an embodiment of the present invention;

[0045] Figure 2 This is a structural diagram of an industrial furnace temperature control optimization system based on model predictive control provided in an embodiment of the present invention. Detailed Implementation

[0046] To more clearly explain the purpose, technical solutions, and advantages of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein. On the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0047] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0050] The present invention will now be described in detail with reference to specific embodiments:

[0051] Example 1

[0052] As attached Figure 1 As shown, Embodiment 1 of the present invention provides an industrial furnace temperature control optimization method based on model predictive control. The specific steps of the method are as follows:

[0053] Step S1: Define the state space and action space to characterize the dynamic characteristics of the industrial furnace temperature optimization control system.

[0054] Preferably, the core of the embodiments of the present invention is to accurately characterize the dynamic characteristics of the industrial furnace temperature optimization control system (hereinafter referred to as the temperature system), which requires the establishment of mathematical representations of the state space and action space;

[0055] Specifically, the state space is a high-dimensional vector that contains the temperature system at time t. All status information:

[0056]

[0057] in, , indicating time The actual temperature matrix, for a given... OK Two-dimensional furnace temperature field at measurement points It can be represented as:

[0058]

[0059] in, , representing the temperature deviation matrix, whose elements are the differences between the actual temperature and the target temperature, i.e. ,and Same shape; This is a temperature change rate matrix, reflecting the dynamic trend of furnace temperature changes, where the elements... Defined as: , The sampling period; As an energy consumption indicator, the energy consumption level of furnace temperature control can be quantified and calculated based on process parameters such as fuel flow rate and heating power. This is a vector of process parameters, which includes various process conditions that affect furnace temperature, such as raw material composition, air-oxygen ratio, and pressure. The predicted temperature sequence for the next N steps is an important basis for the control of the MPC prediction model, where: ;

[0060] Specifically, the action space A defines all possible operations of the industrial furnace temperature optimization control system, consisting of control vectors from m independent actuators, as follows:

[0061]

[0062] For the An actuator, whose control vector This represents the sequence of control increments to be applied in the next N steps:

[0063]

[0064]

[0065] in and For the control constraints of the i-th actuator.

[0066] In summary, state space and action space The definition encompasses the static attributes and dynamic variation patterns of the temperature system in an industrial furnace, laying a mathematical foundation for subsequent modeling, prediction, optimization, and control. In practical applications, state variables and action variables can be appropriately selected and combined according to the specific characteristics of the industrial furnace.

[0067] Furthermore, based on the state space Given the action space A, the state transition process of the temperature system is modeled as a controlled Markov decision process as follows:

[0068]

[0069] in, For an unknown temperature system, the state transition function needs to be approximated using an MPC prediction model and a deep reinforcement learning model. express The system state at a given time corresponds to the state space defined above. The state vector in; express The control action at a given moment corresponds to the action space = in The control increment sequence at time step, i.e. ,Depend on Each independent actuator The control increment at any given time.

[0070] Step S2: Based on the state space and action space An MPC prediction model is established to predict the future temperature trajectory of the temperature system, and a prediction error compensation mechanism and an adaptive prediction domain are introduced to optimize the performance of the MPC prediction model.

[0071] Preferably, based on the state space and action space We will construct an MPC prediction model that can accurately predict future temperature trajectories.

[0072] Specifically, a hierarchical modeling approach is adopted, and the MPC prediction model, including single-step prediction, multi-step prediction, and adaptive improvement, is implemented from the bottom up.

[0073] For example, the single-step prediction includes:

[0074] Define a single-step state transition function ,Will System state at time 1 Mapped to state of time As a single-step prediction model, its mathematical expression is:

[0075]

[0076] in, , is the system state transition matrix. , is the control gain matrix. , is the bias vector. , where is the time-varying set of model parameters. The mathematical expression uses a linear state transition model, which has good computational efficiency and closure; nonlinear systems can also be linearized near their equilibrium points. This represents the control action at time k, corresponding to the control value at time k in the control increment sequence defined in the action space, i.e. It consists of the control increments of m independent actuators at time k.

[0077] Furthermore, based on the single-step state transition function A multi-step prediction function is constructed using autoregressive extrapolation. ,Right now:

[0078]

[0079] in, ,for Predict the state sequence step by step. ,for Multi-step control sequence. By utilizing multi-step prediction, MPC can anticipate future temperature trends and formulate optimal temperature control strategies.

[0080] Furthermore, to adapt to the time-varying characteristics of the temperature system, this embodiment introduces an adaptive prediction domain and a prediction error compensation mechanism to optimize the performance of the MPC prediction model, namely the adaptive improvement.

[0081] Preferably, the adaptive prediction domain is determined by adjusting the prediction step size. This enables dynamic changes in the prediction scale: ,in and These are the minimum and maximum prediction step sizes, respectively. For growth rate, The time exponent is used. The prediction step size varies with the running time from... Dynamic growth to This reduces computational complexity while satisfying terminal constraints.

[0082] For example, the prediction error compensation mechanism is based on the deviation between the actual observed values ​​and the predicted values, and uses optimization algorithms such as stochastic gradient descent to correct the MPC prediction model parameters online. ,in Let be the prediction error at time t. Let F be the norm of the matrix. and These are matrices representing the actual values ​​and the predicted values, respectively. The learning rate is used to iteratively optimize the prediction model, making it continuously approximate the real system.

[0083] Furthermore, combining the above three levels of modeling mechanisms—single-step prediction, multi-step prediction, and adaptive improvement—the MPC prediction model can be described as follows: , , ,in For based on Future predicted by time information The state of the temperature system, i.e., the future temperature trajectory. For the corresponding output predicted value, Output dimension. Output matrix. The state space is mapped to the output space, and key indicators to be optimized, including the average and extreme values ​​of furnace temperature, are extracted.

[0084] Step S3: Construct a deep reinforcement learning model, including a state value network and a policy network, to learn the optimal control strategy of the industrial furnace temperature optimization control system and achieve adaptive optimization control of the temperature system.

[0085] Specifically, this embodiment introduces a deep reinforcement learning model (DRL model) based on the MPC prediction model. Through continuous interaction between the agent and the environment, the optimal temperature control strategy is learned, realizing adaptive optimization control of the industrial furnace temperature optimization control system. The core of the DRL model is the design and training of the state-value network and the policy network.

[0086] For example, define the basic elements required for a DRL model, where the state space... Action space and state transition function All are obtained based on step S1; reward function Where S is the state space at all times, used to quantify the quality of each action, taking into account factors such as control performance, energy consumption level, and temperature uniformity, as follows:

[0087]

[0088] in, Indicates the action performed by the system at time t. The immediate reward value obtained afterward is used to quantify the effectiveness of the control. These are the weighting coefficients. The squared L2 norm of the control action vector at time t is used to quantify control energy consumption. Let t represent the maximum and minimum values ​​of the temperature distribution in the industrial furnace at time t.

[0089] Furthermore, the agent in the DRL model aims to maximize the expected long-term cumulative reward, i.e., the optimal value function: ,in, ∈ S represents the system state at time t, which contains all the state information of the temperature system at time t, such as the actual temperature matrix, temperature deviation matrix, and temperature change rate matrix, corresponding to the state space defined in step S1. As a discount factor, As a control strategy, the optimal value function satisfies the Bellman optimality equation: , where r(S, a) represents the immediate reward obtained by performing action a in state S, P(S'|S, a) represents the probability of transitioning to state S' after performing action a in state S, and γ is the discount factor.

[0090] Furthermore, this embodiment of the invention employs a function approximation method, using the deep reinforcement learning model to approximate the state value network and the policy network. The state value network is used to evaluate the long-term value of a state, and the policy network is used to generate control decisions.

[0091] Specifically, the state value network is based on a state space. The input is used to estimate the value of the output state. The network structure can employ a multilayer perceptron (MLP):

[0092]

[0093] in, This is the input layer representation in a state-value network, directly corresponding to the input state space. , For the first The hidden state of the layer and For the first Layer weight matrix and bias vector, For activation function, For network parameters, This represents the state-value function, used to estimate the state value in the state space. The expected value of long-term cumulative rewards that can be obtained by following the current strategy.

[0094] Furthermore, to improve generalization ability, the input to the state-value network includes not only the basic state variables, i.e., the state space. In addition, information predicted by the MPC prediction model can be introduced to form an extended state space. :

[0095]

[0096] The temperature system predicted by the MPC prediction model in the future The temperature trajectory at step t is spliced ​​with the state space at the current time t to form a spatiotemporal joint feature representation, which enhances the state value network's ability to perceive the future trend of the temperature system.

[0097] Furthermore, the policy network employs a Gaussian policy, with a state space... Given the input, output the probability distribution of the action space A. To satisfy the requirement of a continuous action space, it is as follows:

[0098]

[0099] in, In the state space The probability distribution of choosing action A. and These are the mean and standard deviation of state-related actions, which can be approximated using MLP:

[0100]

[0101] and For output of the hidden layer, For policy network parameters, introduce the standard deviation. It allows for flexible adjustment of the action output.

[0102] Furthermore, the deep reinforcement learning model is trained using an Actor-Critic architecture. The policy gradient is calculated based on Monte Carlo sampling to update the policy network parameters, and the state value network parameters are updated based on temporal difference error. The stability and convergence efficiency of policy training are improved by combining an experience replay mechanism and Gaussian exploration noise.

[0103] Specifically, the state value network is used as the Critic's advantage function:

[0104]

[0105] Furthermore, the policy network of the Actor is updated using the policy gradient theorem:

[0106]

[0107]

[0108] in The learning rate of the policy network. The policy gradient term is approximated using Monte Carlo sampling. Simultaneously, it is based on temporal difference error. Critic network parameters Iterative updates are performed to optimize the evaluation accuracy of the state value function, and its basic form is as follows:

[0109]

[0110]

[0111] in, The learning rate of the Critic network is used to control the step size of parameter updates. This is known as the TD error. The alternating updates of the Actor-Critic algorithm ensure the stability of the policy search and the efficiency of the sample.

[0112] Step S4: Based on the MPC prediction model and the deep reinforcement learning model, the overall objective function is constructed by weighted summing the control performance objective function, energy consumption objective function and temperature uniformity objective function, which serves as a multi-objective optimization model. A constrained optimization problem is constructed based on the multi-objective optimization model, and the optimal temperature control sequence is obtained by solving the constrained optimization problem.

[0113] Specifically, trade-offs are made between multiple dimensions such as control performance, energy efficiency, and product quality. This embodiment, based on MPC optimized control, introduces multi-objective optimization to construct a multi-objective optimization model that comprehensively considers control error, energy consumption level, and temperature uniformity.

[0114] For example, the output equation of the single-step prediction model of the MPC prediction model is defined as follows:

[0115]

[0116] in For k-step prediction output, The output matrix maps the state space to the output space. As an output dimension, it can include multiple indicators such as furnace temperature mean, extreme values, and gradient.

[0117] Based on the output equation, three sub-objective functions are constructed: a control performance objective function, an energy consumption objective function, and a temperature uniformity objective function, as follows:

[0118]

[0119]

[0120]

[0121] in, To control performance targets, a penalty is imposed on the weighted L2 norm between the predicted output trajectory and the reference trajectory. for The reference trajectory for step prediction express Weighted L2 norm, It is a positive semi-definite matrix; To achieve energy consumption targets, penalize control actions. and its change The magnitude of the amplitude, where u represents the control sequence in action space A, and Δu represents the increment of the control value between adjacent time steps, i.e. , and It is a weighted matrix; To achieve uniformity, the uniformity of the temperature field is measured by the difference between the predicted maximum and minimum temperature values. , , These represent the maximum, minimum, and mean values ​​of the predicted output, respectively.

[0122] Furthermore, by weighted summing of the three sub-objectives, a multi-objective function is obtained. As the multi-objective optimization model:

[0123]

[0124] in The weighting coefficient represents the relative importance of different objectives, and the weights can be adjusted online based on factors such as process requirements and operating conditions.

[0125] Furthermore, based on the multi-objective optimization model and the state space... With the action space A, the industrial furnace temperature optimization control problem can be further formalized into a constrained optimization problem, the objective of which is to minimize a multi-objective function while satisfying state, control and output constraints.

[0126] Specifically, the constrained optimization problem can be expressed as:

[0127]

[0128] Constraints:

[0129]

[0130]

[0131]

[0132]

[0133]

[0134] That is, minimizing the multi-objective function under the constraints of the MPC prediction model. At the same time, control the amount ,state and output The system must satisfy physical constraints, where u corresponds to the control increment sequence defined in action space A, x corresponds to the system state variables described in state space S, and y corresponds to the system output, including relevant features of the temperature matrix. Solving this constrained optimization problem yields the optimal temperature control sequence. .

[0135] For example, the constrained optimization problem can be solved using optimization algorithms such as Sequential Quadratic Programming (SQP), Interior Point Method (IP), and Genetic Algorithm (GA) to find the global optimum while meeting real-time requirements. It is worth noting that since there may be conflicts between the three sub-objectives, the selection of weights needs to balance the objectives to avoid suboptimal solutions. An adaptive weight adjustment mechanism can be introduced:

[0136]

[0137]

[0138]

[0139]

[0140] in, As the initial weights, The weights are adjusted for intelligent adaptive adjustment. For the first One goal is Timing deviation, To adjust the rate parameter, the weights will decrease exponentially with the deviation; the larger the deviation, the smaller the weight of the target, thus achieving dynamic priority allocation.

[0141] Furthermore, the optimal temperature control sequence The reference trajectory of the DRL agent in the deep reinforcement learning model constitutes a multi-level collaborative optimization architecture:

[0142]

[0143] in The output trajectory is optimized for MPC and used as the reward function for DRL. The objective value is determined through this hierarchical design. Guided by the MPC optimization results, the DRL will improve its policy, achieving adaptive equilibrium across multiple objectives. Simultaneously, the policy learned by the DRL can also serve as initial values ​​for the MPC, accelerating the convergence of the optimization solution.

[0144] Step S5: Using the optimal temperature control sequence as a reference trajectory, the Actor-Critic architecture is adopted to alternately update the policy network and state value network parameters of the deep reinforcement learning model based on the temporal difference error, thereby optimizing the optimal temperature control strategy.

[0145] Preferably, this embodiment employs an Actor-Critic architecture to implement the policy guidance process based on the reference trajectory, wherein the Actor network... The Critic network is responsible for generating control strategies. Responsible for assessing the value of the state;

[0146] Specifically, the strategy guidance process utilizes temporal differential error (TD error) through an Actor-Critic architecture. The policy network and state-value network of the deep reinforcement learning model are updated alternately. Actor is the decision-making network, and Critic is the evaluation network;

[0147] Furthermore, an experience replay mechanism is introduced to transfer samples generated from the interaction between the deep learning model agent and the environment. The data is stored in the replay cache D, and learning is performed by randomly sampling small batches of samples to break temporal correlations and improve training stability. Simultaneously, the Critic network employs a dual-network structure to create the target network. As The delayed copy, with target network parameters Calculate the target value of TD error : This alleviates the "moving target" problem. Accordingly, the temporal difference error of the Critic network is updated to the TD error target value. Value estimate of the current state The difference is calculated as follows: .

[0148] Furthermore, exponentially decaying Gaussian noise is superimposed on the output of the Actor network. Noise standard deviation As training progresses from the initial value Gradually decrease to Balance exploration and utilization.

[0149] The policy guidance process is as follows: Randomly initialize the Actor network parameters. Critic network parameters Critic target network parameters Then, the replay cache D is initialized, and a training round is initiated; specifically, the environment state is reset in each round. Actions are generated based on the Actor network and Gaussian noise. Perform the action and observe the next state. and rewards The transferred samples are stored in D; small batches of data are randomly sampled from D, based on the TD error. Update the Critic network parameters φ based on policy gradient estimation. Update Actor network parameters Delay updating Critic target network parameters After multiple rounds of training, output the optimized policy network parameters. and state value network parameters Ultimately, this enables precise control and optimization of industrial furnace temperature.

[0150] In summary, the optimal temperature control sequence obtained through multi-objective optimization based on MPC prediction information, combined with the adaptability and generalization of deep reinforcement learning models, enables the furnace temperature control strategy to be continuously optimized in complex and ever-changing industrial environments, significantly improving control performance and resource utilization efficiency.

[0151] In this embodiment, firstly, a state space including a temperature matrix, energy consumption indicators, and process parameters is defined, and a corresponding action space is established. Based on the state and action spaces, an MPC prediction model is constructed. Through single-step prediction and autoregressive multi-step prediction mechanisms, combined with prediction error compensation and an adaptive prediction domain, the accuracy of temperature trajectory prediction is improved. Furthermore, a deep reinforcement learning model is constructed, and a state value network and a policy network are designed to learn the optimal temperature control strategy, compensating for the shortcomings of traditional MPC in dealing with system nonlinearity and modeling uncertainty. Subsequently, a multi-objective optimization problem is constructed based on control performance, energy consumption level, and temperature uniformity, and the optimal temperature control sequence is obtained by solving it. Finally, by using the optimal control sequence as the reference trajectory of the deep reinforcement learning model, an Actor-Critic architecture and a temporal differential error training strategy are adopted to continuously optimize the agent's control strategy, achieving adaptive regulation of the industrial furnace temperature system under complex operating conditions. This invention achieves a comprehensive improvement in temperature control system in terms of control accuracy, response speed, and energy consumption optimization, and has good engineering application value.

[0152] Example 2

[0153] As attached Figure 2 As shown, Embodiment 2 of the present invention provides an industrial furnace temperature control optimization system 200 based on model predictive control. This system comprises a data acquisition unit, a predictive control unit, a reinforcement learning unit, an industrial control computer, an execution control unit, and an industrial furnace object, as detailed below:

[0154] The data acquisition unit includes a temperature sensor array arranged inside the industrial furnace and a data acquisition module installed in the field control cabinet. The temperature sensor array uses common industrial sensors such as thermocouples and resistance temperature detectors (RTDs) to achieve real-time acquisition and monitoring of the temperature field inside the furnace. The temperature signals output by the sensors are processed through isolation amplification, filtering, and A / D conversion before being transmitted to the data acquisition module in digital form. The data acquisition module aggregates and preprocesses the raw data from the temperature sensor array and other process parameter sensors (such as pressure, flow rate, and composition) via communication methods such as bus or industrial Ethernet, and achieves real-time data exchange with the host computer system.

[0155] The software algorithms of the prediction control unit and the reinforcement learning unit run on an industrial control computer. The industrial control computer serves as the hardware platform of the host computer system. It interacts with the data acquisition module through OPC or a custom communication protocol, and interacts with other software and hardware such as the human-machine interface and data storage server through Ethernet.

[0156] The industrial control computer receives industrial furnace temperature data collected by the data acquisition unit. On one hand, it inputs this data into the predictive control unit, where the MPC predictive model predicts the future temperature trajectory and solves the optimal control problem based on constraints. On the other hand, it inputs state information into the reinforcement learning unit, where the DRL model learns and optimizes the policy using an actor-critic algorithm. The outputs of both units are then passed to the execution control unit for comprehensive optimization.

[0157] The execution control unit uses a PLC or PAC as the controller, receives the optimal temperature control strategy output by the industrial computer, performs weighted combination of multiple targets, and generates control commands to drive field actuators such as burners, feed valves, and discharge valves. Through closed-loop control, it achieves precise temperature regulation of the industrial furnace. Simultaneously, the execution control unit feeds back the control effect to the predictive control unit and reinforcement learning unit, realizing dynamic optimization of the industrial furnace temperature.

[0158] Furthermore, the human-machine interface adopts an integrated industrial tablet PC with a touch screen, which communicates with the industrial control computer to realize functions such as visual monitoring, parameter setting, curve tracking, and alarm management of the industrial furnace temperature control optimization system. It also supports remote access and mobile APP, making it convenient for operators to use.

[0159] The data storage and communication components utilize industrial-grade servers or cloud platforms, employing standard database interfaces or industrial communication protocols to achieve data storage, management, analysis, and sharing, and to provide data support for upper-level systems.

[0160] The entire system's power supply, distribution, protection, and other auxiliary components are professionally designed. It uses a fully enclosed control cabinet with built-in air conditioning for cooling and dustproof and dehumidification measures to adapt to various harsh working conditions.

[0161] The aforementioned hardware and software units are seamlessly integrated through standardized interfaces to form a complete industrial furnace temperature control optimization system. This system can be flexibly deployed in new or existing industrial furnace production lines, adapting to different process requirements through parameter tuning. Combined with the MPC prediction and deep reinforcement learning methods proposed in Example 1, this system can significantly improve the temperature control level of industrial furnaces, achieving energy consumption optimization, product quality improvement, and a doubling of production efficiency while ensuring safety and reliability. This provides new ideas and solutions for intelligent and green manufacturing in energy-intensive industries.

[0162] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims. It should be understood that the invention is not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for optimizing temperature control in an industrial furnace based on model predictive control, characterized in that, include: Step S1: Define the state space and action space to characterize the dynamic characteristics of the industrial furnace temperature optimization control system; Step S2: Based on the state space and action space, establish an MPC prediction model to predict the future temperature trajectory of the industrial furnace temperature optimization control system, and introduce a prediction error compensation mechanism and an adaptive prediction domain to optimize the performance of the MPC prediction model. Step S3: Construct a deep reinforcement learning model, including designing a state-value network and a policy network, to learn the optimal control strategy of the industrial furnace temperature optimization control system and achieve adaptive optimization control; Step S4: Based on the MPC prediction model and the deep reinforcement learning model, the overall objective function is constructed by weighted summing the control performance objective function, energy consumption objective function and temperature uniformity objective function, which serves as a multi-objective optimization model. A constrained optimization problem is constructed based on the multi-objective optimization model, and the optimal temperature control sequence is obtained by solving the constrained optimization problem. The multi-objective optimization model is as follows: ,in These are weighting coefficients, representing the relative importance of different objectives. , , These are the objective functions for control performance, energy consumption, and temperature uniformity, respectively. Step S5: Using the optimal temperature control sequence as a reference trajectory, the Actor-Critic architecture is adopted to alternately update the policy network and state value network parameters of the deep reinforcement learning model based on the temporal difference error, thereby optimizing the optimal temperature control strategy.

2. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, The state space is a high-dimensional vector that contains the temperature system at time t. All status information: ,in, , indicating time The actual temperature matrix; Represents the temperature deviation matrix; As an energy consumption indicator, it quantifies the energy consumption level of furnace temperature control. This is a vector of process parameters, containing various process conditions that affect furnace temperature. The predicted temperature sequence for the next N steps is an important basis for the control of the MPC prediction model, and ; Action space A defines all possible operations of the industrial furnace temperature optimization control system, consisting of control vectors from m independent actuators, as follows: For the first An actuator, whose control vector This represents the sequence of control increments to be applied in the next N steps: , ,in and For the control constraints of the i-th actuator.

3. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, The establishment of the MPC prediction model includes establishing a single-step state transition function for single-step prediction, using an autoregressive structure for multi-step prediction, and using an adaptive prediction domain and prediction error compensation mechanism for optimization and adaptive improvement. The single-step prediction includes: Define a single-step state transition function ,Will System state at time 1 Mapped to state of time As a single-step prediction model, its mathematical expression is: ,in, , is the system state transition matrix. , is the control gain matrix. , is the bias vector. , which is the time-varying set of model parameters. This represents the control action at time k, corresponding to the control value at time k in the control increment sequence defined in the action space, i.e. It consists of the control increments of m independent actuators at time k.

4. The industrial furnace temperature control optimization method based on model predictive control according to claim 3, characterized in that, Based on the single-step state transition function A multi-step prediction function is constructed using autoregressive extrapolation. ,Right now: ,in, ,for Predict the state sequence step by step. ,for Step control sequence.

5. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, The MPC prediction model is characterized as follows: , , ,in Based on Future predicted by time information The state of the temperature system, i.e., the future temperature trajectory. For the corresponding output predicted value, Output dimension; output matrix The state space is mapped to the output space, and key indicators to be optimized, including the average and extreme values ​​of furnace temperature, are extracted.

6. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, The state value network, in state space For the estimation of input and output state values, the network structure adopts a multilayer perceptron, and the expression is: ,in, This is the input layer representation in a state-value network, directly corresponding to the input state space. , For the first The hidden state of the layer and For the first The weight matrix and bias vector of the layer, For activation function, For network parameters, This represents the state-value function, used to estimate the state value in the state space. The expected value of long-term cumulative rewards that can be obtained by following the current strategy.

7. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, The policy network employs a Gaussian policy, with a state space. Given the input, output the probability distribution of the action space A, expressed as follows: ,in In the state space The probability distribution of choosing action A. and Let be the mean and standard deviation of state-related actions, respectively, which can be approximated as: ,in and For output of the hidden layer, For policy network parameters, The standard deviation is denoted as .

8. The industrial furnace temperature control optimization method based on model predictive control according to claim 1, characterized in that, Based on the output equation of the single-step prediction model of the MPC prediction model, the control performance objective function, energy consumption objective function, and temperature uniformity objective function are constructed. The control performance objective function is as follows: ,in To control performance targets, a penalty is imposed on the weighted L2 norm between the predicted output trajectory and the reference trajectory. The output equation is... for The reference trajectory for step prediction, express Weighted L2 norm, It is a positive semi-definite matrix.

9. The industrial furnace temperature control optimization method based on model predictive control according to claim 8, characterized in that, The constrained optimization problem is formulated as follows: The constraints are: , , , , , That is, minimizing the multi-objective function under the constraints of the MPC prediction model. At the same time, control the amount ,state and output Physical constraints must be met, where u corresponds to the control increment sequence defined in action space A, and x corresponds to the state space. The system state variable described in the text, y corresponds to the system output. This is the output matrix, used to map the state space to the output space.

10. An industrial furnace temperature control optimization system based on model predictive control, used to execute the industrial furnace temperature control optimization method based on model predictive control as described in any one of claims 1-9, characterized in that, It includes a data acquisition unit, a prediction and control unit, a reinforcement learning unit, an industrial control computer, an execution control unit, and an industrial furnace object; The data acquisition unit is used to realize the real-time acquisition and monitoring of the temperature field inside the industrial furnace, and to realize real-time data exchange with the host computer system. The industrial control computer, as the hardware platform of the host computer system, receives the industrial furnace temperature data collected by the data acquisition unit and inputs it into the prediction control unit and the reinforcement learning unit; and the software algorithms of the prediction control unit and the reinforcement learning unit run on the industrial control computer. The prediction control unit predicts the future temperature trajectory using the MPC prediction model; The reinforcement learning unit is used to learn and optimize policies based on a deep reinforcement learning model and through the actor-critic algorithm. The execution control unit receives the optimal temperature control strategy output by the industrial control computer, performs weighted combination of multiple targets, and generates control commands to regulate the temperature of the industrial furnace.

Images