Multi-model-based intermittent reaction kettle intelligent control method and system

Abstract

The invention discloses a multi-model-based intermittent reaction kettle intelligent control method and system. The method comprises the steps of collecting operation data such as temperature, pressure and reactant concentration of a reaction kettle in real time through a sensor, constructing a soft measurement model by utilizing a machine learning algorithm, and performing training based on historical data so as to predict key operation parameters; when operation conditions change, the system updates model parameters in real time through an online learning model by using a recursive least square method or an online gradient descent method; besides, based on a reinforcement learning algorithm, the system can generate and optimize a control strategy, adjust operation parameters such as the opening degree of a cooling water valve and the power of a heater, and finally execute an optimal control instruction through an actual controller to realize high-precision control of the reaction kettle; according to the method, the control precision, the system response speed and the adaptive capacity are improved, the energy utilization is optimized, and the production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial control technology, and in particular to a multi-model-based batch reactor intelligent control method and system. Background Art

[0002] In industrial production processes, intermittent reactors are widely used in a variety of chemical reaction processes due to their flexible operation and strong adaptability. Traditional PID control methods are difficult to cope with the complex dynamic characteristics of intermittent reactors, such as nonlinearity, multivariable coupling, and frequent changes in operating conditions. They can easily lead to inaccurate control, slow response, or system instability, which are difficult to solve even with fuzzy control combined with PID control. At the same time, when conditions change, it takes a lot of time to determine the fuzzy rules. Therefore, there is an urgent need for a new intelligent control algorithm that can adjust the control strategy in real time to improve the operational stability and production efficiency of the reactor. Summary of the invention

[0003] Technical purpose: In view of the shortcomings of the existing intermittent reactor control technology, the present invention discloses an intelligent control method and system for an intermittent reactor based on multiple models, which can improve the control accuracy and response speed of the reactor.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0005] A multi-model based intermittent reactor intelligent control method specifically comprises the following steps:

[0006] Multiple sensors are used to collect real-time operating data inside the reactor, including temperature, pressure and reactant concentration;

[0007] Utilize machine learning algorithms to build soft-sensing models, and train them based on historical operating data to predict key operating parameters in the reactor.

[0008] Construct an online learning model to update the parameters of the soft sensor model in real time through recursive least squares method or online gradient descent method when the operating conditions of the reactor change;

[0009] Based on the reinforcement learning algorithm, the control strategy is constructed and optimized according to the current state of the reactor and historical operation data;

[0010] The optimal control strategy generated by the reinforcement learning model is used to output the given control instructions and adjust the reactor operating parameters, including the cooling water valve opening and the heater power;

[0011] The generated control instructions are executed by the actual controller to ensure that the reactor remains in the desired operating state and optimize the response speed and stability of the control reactor.

[0012] Preferably, the method further includes cleaning and normalizing the collected operation data after the operation data is collected, so as to convert the data of different physical quantities into standardized data.

[0013] Preferably, the soft sensor model is constructed using a machine learning algorithm, and training the soft sensor model based on historical operation data specifically includes the following steps:

[0014] Extract key features from historical operation data, including the changing trend, maximum value, minimum value and average value of time series data, as well as derived statistical features, for training soft sensor models;

[0015] Select a machine learning model suitable for time series prediction and use historical operation data to train a soft sensor model with the goal of predicting the key operating parameters of the reactor;

[0016] Use the validation set to evaluate the prediction accuracy of the soft sensor model, calculate the error index of the soft sensor model, adjust the hyperparameters of the soft sensor model, and complete the training of the soft sensor model.

[0017] Preferably, constructing an online learning model, when the operating conditions of the reactor change, updating the parameters of the soft sensor model in real time by recursive least squares method or online gradient descent method specifically includes the following steps:

[0018] Use the trained soft sensor model as the initial model. When new operation data arrives, update the parameters of the soft sensor model in real time. The recursive least squares update formula is as follows:

[0019]

[0020] Among them, K t represents the Kalman gain vector, θ t represents the parameter estimate at the current moment, P t represents the covariance matrix of the previous moment, x t Represents the input vector at the current moment, y t Represents the output at the current moment;

[0021] The online learning model continuously acquires new operation data and updates the soft sensor model with the new operation data.

[0022] Preferably, based on the reinforcement learning algorithm, according to the current state of the reactor and historical operation data, constructing and optimizing the control strategy specifically includes the following steps:

[0023] Define the state space, action space, and reward function of reinforcement learning. The state space includes temperature, pressure, and reactant concentration. The action space is the control operations that the reactor can take, including adjusting the opening of the cooling water valve and the power of the heater.

[0024] The control strategy is trained using a deep Q network algorithm, and the strategy is optimized through the experience replay mechanism and the target network mechanism. The Q value update formula is:

[0025]

[0026] Among them, Q(s,a) represents the Q value of the current state s and action a, α represents the learning rate, r represents the immediate reward, s' represents the next state, and γ represents the discount factor;

[0027] During the operation of the reactor, the reinforcement learning model is continuously optimized through feedback from actual operation data.

[0028] Preferably, the formula of the reward function is:

[0029] R(s,a)=-(N target -N current )

[0030] Among them, R(s,a) represents the reward function, N target represents the target operation parameter, N current Indicates the current operation parameters.

[0031] Preferably, the optimal control strategy generated by the reinforcement learning model is used to output a given control instruction, and the adjustment of the reactor operating parameters specifically includes the following steps:

[0032] The optimal control strategy generated by the reinforcement learning model outputs control instructions, which include the opening of the cooling water valve and the power adjustment of the heater;

[0033] Send the generated control instructions to the actual hardware device through the controller interface;

[0034] The adjusted reactor operating status is monitored in real time and continuously fed back to the system for status updates for the next round of control decisions.

[0035] The present invention also provides an intermittent reactor intelligent control system based on multiple models, which is used to implement the above-mentioned intermittent reactor intelligent control method based on multiple models, comprising:

[0036] A data processing module, configured to collect the reactor operating parameters in real time through sensors, and to clean and normalize the data;

[0037] Soft measurement model module, which builds soft measurement models based on machine learning algorithms to predict key operating parameters in the reactor;

[0038] An online learning model module configured to update the parameters of the soft sensor model in real time based on a recursive least squares method or an online gradient descent method to adapt to the dynamic changes in the operating conditions of the reactor;

[0039] Reinforcement learning control model module, a control model built based on reinforcement learning algorithm, configured to dynamically adjust the control strategy to optimize the control performance of the reactor;

[0040] A control output module is used to output corresponding control instructions and adjust the controller operation according to the control strategy generated by the reinforcement learning control model;

[0041] The control execution module is used to execute the control instructions of the control output module through the actual controller to dynamically adjust the operating parameters of the reactor.

[0042] Preferably, the sensors for collecting operation data include temperature sensors, pressure sensors and chemical sensors, which are installed at different locations of the reactor to monitor various operation parameters in real time.

[0043] Beneficial effects: The multi-model-based intermittent reactor intelligent control method and system provided by the present invention have the following beneficial effects:

[0044] 1. The present invention combines the soft measurement model and the reinforcement learning algorithm, monitors the key operating parameters of the reactor in real time, and dynamically adjusts the control strategy, which significantly improves the control accuracy and response speed of the system; by optimizing the control strategy through the reinforcement learning model, it can effectively reduce overshoot and system oscillation, ensure that the reactor reaches a stable state in the shortest time, and improve the stability of the system operation.

[0045] 2. The online learning model of the present invention enables the reactor to adapt to changes in operating conditions in real time, ensuring the reactor's ability to adapt to complex working conditions. In addition, the optimization control strategy based on the reinforcement learning model can dynamically adjust operating parameters such as the cold water valve opening and heater power to avoid excessive consumption and reduce energy consumption, thereby optimizing resource utilization, improving production efficiency and reducing operating costs.

[0046] 3. The multi-model design of the present invention can handle the coupling relationship between multiple variables and the nonlinear characteristics of the system, so that the system can operate effectively in a complex industrial control environment, improve the overall control quality, have a wide range of adaptability, and have strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0048] Figure 1The figure is a flow chart of the intelligent control method of intermittent reactor based on multiple models of the present invention;

[0049] Figure 2 It is a structural diagram of the intelligent control system of the intermittent reactor based on multiple models of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0051] like Figure 1 As shown, the present invention provides an intelligent control method for an intermittent reactor based on a multi-model, which specifically includes the following steps:

[0052] S1. Multiple sensors are used to collect real-time operating data inside the reactor, including temperature, pressure and reactant concentration.

[0053] In one embodiment, various sensors, including temperature sensors, pressure sensors, and chemical sensors, are installed at key locations of the reactor, such as inside and outside the reactor body, to monitor the concentration of reactants.

[0054] The sensor continuously collects the internal operation data of the reactor at a fixed sampling frequency, and transmits this data to the central control system via wireless or wired networks. The collected data stream includes key parameters such as temperature, pressure and reactant concentration at each moment.

[0055] Since the sampling frequencies of sensors may be different, different sensor data need to be timestamped through a data synchronization mechanism to ensure consistency and synchronization in subsequent data processing.

[0056] In one embodiment, after the operation data is collected, the collected operation data is cleaned and normalized to convert the data of different physical quantities into standardized data.

[0057] The collected data is cleaned by removing noise and outliers to ensure the accuracy of subsequent modeling data. Cleaning methods include moving average filtering, Kalman filtering or median filtering. The cleaned data is normalized to standardize data of different dimensions to facilitate the construction of soft sensor models.

[0058] S2. Use machine learning algorithms to build a soft measurement model and train the soft measurement model based on historical operation data to predict key operating parameters in the reactor.

[0059] In one embodiment, a soft sensor model is constructed using a machine learning algorithm, and training the soft sensor model based on historical operation data specifically includes the following steps:

[0060] S21. Extract key features from historical operation data, including the change trend, maximum value, minimum value and average value of time series data, as well as derived statistical features, for training soft sensor models;

[0061] S22. Select a machine learning model suitable for time series prediction, including but not limited to a neural network, a support vector machine or a random forest, and use historical operation data to train a soft sensor model. The goal is to predict key operating parameters of the reactor, such as reactant concentration or product quality. The loss function usually selects the mean square error, and the model parameters are optimized by the gradient descent method.

[0062] S23. Use the validation set to evaluate the prediction accuracy of the soft sensor model, calculate the error indicators of the soft sensor model, such as mean square error, absolute error, etc., adjust the hyperparameters of the soft sensor model such as learning rate, regularization parameter, etc., and complete the training of the soft sensor model.

[0063] S3. Build an online learning model. When the operating conditions of the reactor change, the parameters of the soft sensor model are updated in real time through recursive least squares method or online gradient descent method.

[0064] In one embodiment, an online learning model is constructed. When the operating conditions of the reactor change, the parameters of the soft sensor model are updated in real time by using a recursive least squares method or an online gradient descent method. Specifically, the following steps are included:

[0065] S31, using the trained soft sensor model as the initial model, when new operation data arrives, update the parameters of the soft sensor model in real time, and the recursive least squares update formula is as follows:

[0066]

[0067] Among them, K t represents the Kalman gain vector, θ t represents the parameter estimate at the current moment, P t represents the covariance matrix of the previous moment, which measures the uncertainty of the current parameter estimate, x t Represents the input vector at the current moment, y t Represents the output at the current moment; Represents the product of the current covariance matrix and the input vector, reflecting the relationship between the input data and the uncertainty of parameter estimation; is a normalization factor that ensures the update step size is appropriate and avoids excessive parameter updates.

[0068] S32. The online learning model continuously acquires new operating data and updates the soft sensor model with the new operating data, so that the model can adapt itself when the operating conditions change.

[0069] S4. Based on the reinforcement learning algorithm, the control strategy is constructed and optimized according to the current state of the reactor and historical operation data.

[0070] In one embodiment, based on the reinforcement learning algorithm, according to the current state of the reactor and historical operation data, constructing and optimizing the control strategy specifically includes the following steps:

[0071] S41. Define the state space, action space and reward function of reinforcement learning. The state space includes temperature, pressure and reactant concentration. The action space is the control operations that the reactor can take, such as maintaining the temperature near the target value, including adjusting the opening of the cooling water valve and the power of the heater.

[0072] In one embodiment, the reward function is formulated as:

[0073] R(s,a)=-|N target -N current |

[0074] Among them, R(s,a) represents the reward function, N target represents the target operation parameter, N current Indicates the current operation parameters.

[0075] S42. Use the deep Q network algorithm to train the control strategy, optimize the strategy through the experience replay mechanism and the target network mechanism, and the Q value update formula is:

[0076]

[0077] Among them, Q(s,a) represents the Q value of the current state s and action a, α represents the learning rate, r represents the immediate reward, s' represents the next state, and γ represents the discount factor;

[0078] S43. During the operation of the reactor, the reinforcement learning model is continuously optimized through feedback from actual operation data.

[0079] In a specific embodiment, assuming that the temperature of a reactor needs to be controlled, the state space, action space, reward function, and dynamic models of the reactor and temperature control system are first defined. The specific contents are as follows:

[0080] State space: First, define the state space, which can describe the variables in the reactor temperature control process; obtain the current reactor temperature T current , target temperature T target , the historical state of the control system, that is, the temperature error e at the previous moment t-1 =T target -T current , and two interfering factors: cooling water flow rate L cold , Cooling water temperature Tcold , so the state vector can be defined as S t =[T current , T target , e t-1 , L cold , T cold ].

[0081] Action space: For the temperature control system, the action space does not include the control amount of the cooling water valve. In order to adapt to the characteristics of the discrete action of the deep Q network, the action can be discretized into different cooling gears, and the fine gear is multiple gears A t ={-10,-9,-8,-7,-6,-5,-4,-3,-2,-1,0,1,2,3,4,5,6,7,8,9,10}, each action represents the adjustment of the cooling water valve, such as increasing or decreasing the opening by 10%.

[0082] Reward function: The reward function is used to guide the reinforcement learning agent to optimize the temperature control strategy. The goal of reactor temperature control is to maintain the temperature near the target value, so the reward function is designed based on the temperature error:

[0083] R t =-|T target -T current |

[0084] This negative error reward function can drive the system to minimize the temperature difference.

[0085] The temperature change model of the reactor is simulated in a data-driven way. If the specific parameters of the reactor are available, the simulation environment can be established based on the heat transfer equation and the energy balance equation. If not, historical data can be used to establish a black box model, and the temperature change can be predicted using a data-driven approach.

[0086] In order to better apply the deep Q network algorithm to the temperature control system, it is necessary to introduce experience replay and target network.

[0087] Experience replay: At each time step, the agent stores the state, action, reward, and next state into the experience pool, and then randomly extracts a small batch of experience for training, breaking the temporal correlation of the data and improving the stability of training.

[0088] Target network: Use a dual network architecture of the main network and the target network. The target network is updated at regular intervals to reduce shock during training. The specific training process is as follows:

[0089] 1. Initialize the network: define the main network Q(s,a|θ) and the target network Q'(s,a|θ'), and initialize the experience pool, where s represents the current state, a represents the action, θ represents the main network parameters, and θ' represents the target network parameters;

[0090] 2. Collect experience: Based on the current state vector S t Select action A through the ∈ greedy strategy t , control the cooling water valve, and obtain the next temperature state S according to the physical model t+1 and instant reward R t , will (S t ,A t ,R t ,S t+1 ) is stored in the experience pool;

[0091] 3. Update strategy: randomly extract batches of experience from the experience pool and calculate the target value y = R t +γmaxQ'(S t+1 ,a'|θ'), using gradient descent to minimize the loss L(θ)=(yQ(S t ,A t |θ)) 2 , update the main network parameters.

[0092] 4. Target network update: At each fixed number of steps, copy the parameters of the main network to the target network;

[0093] 5. Iterative training: Continue to train the agent until the model converges and achieves the desired temperature control effect.

[0094] Before actually deploying the system, historical data from the reactor is used for simulation training. The trained deep Q network controller is deployed in the actual system, and online learning or adjustment strategies are used to cope with real-world uncertainties, such as environmental changes or equipment aging.

[0095] S5. Use the optimal control strategy generated by the reinforcement learning model to output the given control instructions and adjust the reactor operating parameters, including the cooling water valve opening and the heater power.

[0096] In one embodiment, the optimal control strategy generated by the reinforcement learning model is used to output a given control instruction, and the adjustment of the reactor operating parameters specifically includes the following steps:

[0097] S51, outputting control instructions based on the optimal control strategy generated by the reinforcement learning model, where the control instructions include the opening of the cooling water valve and the power adjustment of the heater;

[0098] S52, sending the generated control instruction to the actual hardware device, such as the electric valve and the heater control unit, through the controller interface. The controller must have real-time response capability and be able to immediately adjust the operating parameters according to the received control instruction;

[0099] S53, real-time monitoring of the adjusted reactor operating status, and continuous feedback to the system for status update for the next round of control decision-making.

[0100] S6. Execute the generated control instructions through the actual controller to ensure that the reactor is kept in the desired operating state and optimize the response speed and stability of the control reactor.

[0101] The present invention also provides an intermittent reactor intelligent control system based on multiple models, which is used to implement the above-mentioned intermittent reactor intelligent control method based on multiple models, comprising:

[0102] A data processing module, configured to collect the reactor operating parameters in real time through sensors, and to clean and normalize the data;

[0103] Soft measurement model module, which builds soft measurement models based on machine learning algorithms to predict key operating parameters in the reactor;

[0104] An online learning model module configured to update the parameters of the soft sensor model in real time based on a recursive least squares method or an online gradient descent method to adapt to the dynamic changes in the operating conditions of the reactor;

[0105] Reinforcement learning control model module, a control model built based on reinforcement learning algorithm, configured to dynamically adjust the control strategy to optimize the control performance of the reactor;

[0106] A control output module is used to output corresponding control instructions and adjust the controller operation according to the control strategy generated by the reinforcement learning control model;

[0107] The control execution module is used to execute the control instructions of the control output module through the actual controller to dynamically adjust the operating parameters of the reactor.

[0108] like Figure 2 The figure shows a system architecture diagram in one embodiment of the present invention. The system is divided into three layers, namely, a control layer, a model layer and a data layer. The data layer includes a data processing module, the model layer includes a reinforcement learning model, an online learning model and a soft measurement model, and the control layer includes several controllers.

[0109] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-model based batch reactor intelligent control method, characterized in that: The specific steps include: Multiple sensors are used to collect real-time operating data inside the reactor, including temperature, pressure and reactant concentration; Utilize machine learning algorithms to build soft-sensing models, and train them based on historical operating data to predict key operating parameters in the reactor. Construct an online learning model to update the parameters of the soft sensor model in real time through recursive least squares method or online gradient descent method when the operating conditions of the reactor change; Based on the reinforcement learning algorithm, the control strategy is constructed and optimized according to the current state of the reactor and historical operation data; The optimal control strategy generated by the reinforcement learning model is used to output the given control instructions and adjust the reactor operating parameters, including the cooling water valve opening and the heater power; The generated control instructions are executed by the actual controller to ensure that the reactor remains in the desired operating state and optimize the response speed and stability of the control reactor.

2. The intelligent control method for batch reactor based on multiple models according to claim 1 is characterized in that: It also includes cleaning and normalizing the collected operation data after the operation data is collected, and converting data of different physical quantities into standardized data.

3. The intelligent control method for batch reactor based on multiple models according to claim 1 is characterized in that: The soft sensor model is constructed using machine learning algorithms. Training the soft sensor model based on historical operation data includes the following steps: Extract key features from historical operation data, including the changing trend, maximum value, minimum value and average value of time series data, as well as derived statistical features, for training soft sensor models; Select a machine learning model suitable for time series prediction and use historical operation data to train a soft sensor model with the goal of predicting the key operating parameters of the reactor; Use the validation set to evaluate the prediction accuracy of the soft sensor model, calculate the error index of the soft sensor model, adjust the hyperparameters of the soft sensor model, and complete the training of the soft sensor model.

4. The intelligent control method for batch reactor based on multiple models according to claim 1 is characterized in that: Constructing an online learning model, when the operating conditions of the reactor change, the parameters of the soft sensor model are updated in real time by recursive least squares method or online gradient descent method. The specific steps include: Use the trained soft sensor model as the initial model. When new operation data arrives, update the parameters of the soft sensor model in real time. The recursive least squares update formula is as follows: Among them, K t represents the Kalman gain vector, θ t represents the parameter estimate at the current moment, P t represents the covariance matrix of the previous moment, x t Represents the input vector at the current moment, y t Represents the output at the current moment; The online learning model continuously acquires new operation data and updates the soft sensor model with the new operation data.

5. The intelligent control method for batch reactor based on multiple models according to claim 1 is characterized in that: Based on the reinforcement learning algorithm, the control strategy is constructed and optimized according to the current state of the reactor and historical operation data, which specifically includes the following steps: Define the state space, action space, and reward function of reinforcement learning. The state space includes temperature, pressure, and reactant concentration. The action space is the control operations that the reactor can take, including adjusting the opening of the cooling water valve and the power of the heater. The control strategy is trained using a deep Q network algorithm, and the strategy is optimized through the experience replay mechanism and the target network mechanism. The Q value update formula is: Among them, Q(s,a) represents the Q value of the current state s and action a, α represents the learning rate, r represents the immediate reward, s' represents the next state, and γ represents the discount factor; During the operation of the reactor, the reinforcement learning model is continuously optimized through feedback from actual operation data.

6. The intelligent control method for batch reactor based on multiple models according to claim 5 is characterized in that: The formula for the reward function is: R(s,a)=-(N target -N current ) Among them, R(s,a) represents the reward function, N target represents the target operation parameter, N current Indicates the current operation parameters.

7. The intelligent control method for batch reactor based on multiple models according to claim 1 is characterized in that: Using the optimal control strategy generated by the reinforcement learning model, outputting the given control instructions, and adjusting the reactor operating parameters specifically include the following steps: The optimal control strategy generated by the reinforcement learning model outputs control instructions, which include the opening of the cooling water valve and the power adjustment of the heater; Send the generated control instructions to the actual hardware device through the controller interface; The adjusted reactor operating status is monitored in real time and continuously fed back to the system for status updates for the next round of control decisions.

8. An intelligent control system for an intermittent reactor based on multiple models, used to implement an intelligent control method for an intermittent reactor based on multiple models as claimed in any one of claims 1 to 7, characterized in that: include: A data processing module, configured to collect the reactor operating parameters in real time through sensors, and to clean and normalize the data; Soft measurement model module, which builds soft measurement models based on machine learning algorithms to predict key operating parameters in the reactor; An online learning model module configured to update the parameters of the soft sensor model in real time based on a recursive least squares method or an online gradient descent method to adapt to the dynamic changes in the operating conditions of the reactor; Reinforcement learning control model module, a control model built based on reinforcement learning algorithm, configured to dynamically adjust the control strategy to optimize the control performance of the reactor; A control output module is used to output corresponding control instructions and adjust the controller operation according to the control strategy generated by the reinforcement learning control model; The control execution module is used to execute the control instructions of the control output module through the actual controller to dynamically adjust the operating parameters of the reactor.

9. The intelligent control system for batch reactors based on multiple models according to claim 8, characterized in that: Sensors for operating data collection include temperature sensors, pressure sensors and chemical sensors, which are installed in different parts of the reactor to monitor various operating parameters in real time.

Images