A double-target control method for a waste incineration deacidification system based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's garbage incineration deacidification system double target control method, by collecting garbage incineration power plant operation and emission data, and carry out data preprocessing;Then the data after pre-processing are carried out variable screening and construct input variable time series set;Based on time series sample, semi-dry method-wet method deacidification cascade prediction model is established, for predicting final emission concentration.Semi-dry method slurry flow and wet method caustic soda flow control increment are used as control action, and double target reward function is constructed, which considers acid gas emission control and deacidification agent consumption optimization, and deep deterministic policy gradient algorithm is used to construct reinforcement learning control model, to realize the training and optimization of control strategy.Through the continuous interaction of environmental prediction model and reinforcement learning intelligent agent, the optimal deacidification control strategy is obtained, which significantly reduces the deacidification agent consumption while ensuring the stable and standard acid gas emission, further improves the stability, economy and environmental protection of system operation.

Description

Technical Field

[0001] This invention relates to the field of waste incineration flue gas pollution control technology, and mainly to a dual-objective control method for a waste incineration desulfurization system based on reinforcement learning. Background Technology

[0002] With the continuous advancement of urbanization in my country, the amount of urban domestic waste generated continues to increase. Waste-to-energy incineration technology, due to its advantages of volume reduction, harmlessness, and resource recovery, has become one of the main methods for urban domestic waste treatment. However, the incineration process generates a large amount of flue gas pollutants, including acidic gases such as hydrogen chloride (HCl) and sulfur dioxide (SO2). If not properly controlled, these pollutants can not only pollute the environment but also corrode equipment, affecting the safe and stable operation of the incineration system. To reduce the concentration of acidic gas emissions, waste-to-energy incineration plants typically employ flue gas desulfurization systems. Common desulfurization processes include dry, semi-dry, and wet desulfurization, with the combined semi-dry and wet process being widely used in practical engineering. This type of system removes pollutants by injecting alkaline absorbents (such as quicklime slurry or caustic soda solution) into the flue gas to neutralize the acidic gases. Because the flue gas flows sequentially between treatment units, the desulfurization process exhibits significant series characteristics and dynamic coupling.

[0003] Currently, the control methods for acid removal systems in waste incineration power plants mainly include the following two categories: (1) Manual experience-based adjustment. Operators adjust the amount of acid removal agent added based on chimney emission monitoring data and furnace operation conditions, which is highly subjective and difficult to adapt to changes in operating conditions. (2) PID-based automatic control. Usually, the chimney emission concentration is used as the feedback variable for adjustment. However, due to the large fluctuations in waste composition, the acid gas concentration changes significantly during incineration, and the acid removal reaction process has obvious time delay and nonlinear characteristics, making it difficult for traditional control methods to achieve effective adjustment, and problems such as response lag or control fluctuations are likely to occur. At the same time, traditional methods rely on fixed parameters or static models and lack the ability to adaptively adjust to the dynamic characteristics and disturbances of the system. In addition, existing control strategies are mostly aimed at achieving emission standards, lacking optimization constraints on the amount of acid removal agent added, and making it difficult to balance emission control and operational economy. Therefore, there is an urgent need to introduce intelligent control methods that can continuously interact with the system operating environment and dynamically optimize control strategies based on feedback information, so as to achieve synergistic optimization of flue gas emissions and deacidifying agent consumption under complex operating conditions and improve the stability and economy of system operation.

[0004] An existing multi-objective reinforcement learning-based collaborative control method for incineration pollution (CN120650717B) constructs a high-dimensional feature matrix driven by multi-modal sensor data and employs the Deep Deterministic Policy Gradient (DDPG) algorithm to achieve multi-objective optimization control of pollutant emissions, resource utilization, and costs. However, this method mainly focuses on the comprehensive optimization of the entire incineration process and lacks targeted modeling and control for the acid removal system. A multi-pollutant collaborative prediction and intelligent control method for waste incineration flue gas (CN119289371B) predicts the concentrations of pollutants such as HCl, SO2, and NOx based on a Long Short-Term Memory (LSTM) model and combines multi-objective optimization calculations to obtain the optimal absorbent dosage for valve regulation control. However, this method belongs to an indirect control framework of "prediction model + optimization calculation" and does not introduce adaptive strategies such as reinforcement learning for dynamic decision-making and strategy updates. Therefore, the online adaptability of the control strategy remains limited when facing the strong nonlinearity, time-varying nature, and complex disturbances in the waste incineration process.

[0005] There are two main ways to control the desulfurization of waste incineration flue gas: (1) Operators adjust the flow rate of semi-dry slurry and the amount of wet caustic soda added based on the HCl and SO2 emission concentrations of the chimney and the operating conditions of the furnace, taking into account the economic efficiency of operation while ensuring that the emission meets the standards; (2) Automatic control based on PID, using the chimney emission concentration as the feedback variable, performing PID calculation based on the deviation between the actual value and the set value, outputting the control signal of the desulfurization agent addition amount, and adjusting the slurry or caustic soda flow rate through the regulating valve to stabilize the emission concentration near the set range.

[0006] Due to the complex and fluctuating composition of waste, desulfurization systems exhibit significant time-delay and nonlinear characteristics. Existing desulfurization control methods, primarily based on manual experience and traditional PID control, rely heavily on fixed control parameters or static models. These methods lack the ability to adaptively adjust to the dynamic characteristics and complex disturbances of the system, making them ill-suited for complex time-varying conditions. This often leads to problems such as insufficient accuracy in flue gas pollutant emission control, response lag, and control fluctuations, while also resulting in high desulfurization agent consumption. Furthermore, existing control strategies typically prioritize emission compliance, lacking optimization constraints on desulfurization agent dosage, leading to low agent utilization efficiency and making it difficult to balance emission control with operational economics. Summary of the Invention

[0007] This invention proposes a dual-objective control method for a waste incineration desulfurization system based on reinforcement learning. The aim is to overcome the characteristics of large fluctuations in waste composition, complex changes in operating conditions, and significant time delays and inertia in the waste incineration desulfurization system. It also addresses the problems of poor desulfurization control effect, low accuracy of acid gas emission control, large consumption of desulfurization agent, and poor control stability in existing desulfurization methods that rely mainly on manual experience and traditional control methods.

[0008] The present invention is achieved by at least one of the following technical solutions.

[0009] A dual-objective control method for a waste incineration acid removal system based on reinforcement learning includes the following steps: Step 1: Collect operating and emission parameters of the waste incineration power plant and preprocess the collected data; Step 2: Based on the preprocessed data, perform variable selection and construct a set of time series input variables; Step 3: Construct a semi-dry-wet desulfurization cascade prediction model based on time series samples as an environmental prediction model. Input the time series of input variables into the environmental prediction model to predict the intermediate flue gas state after semi-dry desulfurization treatment and predict the final emissions. Step 4: Using the control increments of semi-dry slurry flow rate and wet caustic soda flow rate as control actions, construct a dual-objective reward function that takes into account both acid gas emission control and deacidifying agent consumption optimization, and use a deep deterministic policy gradient algorithm to construct a reinforcement learning control model. Step 5: Train the constructed reinforcement learning control model; Step 6: Deploy the trained reinforcement learning control model in the actual deacidification system for online control.

[0010] Furthermore, the environmental prediction model in step 3 is a semi-dry-wet deacidification tandem prediction model, including a semi-dry deacidification sub-model and a wet deacidification sub-model; The semi-dry deacidification sub-model is used to predict the intermediate flue gas state after semi-dry deacidification treatment, and the output is the intermediate flue gas state at the next moment; the wet deacidification sub-model is used to predict the final emission, and the output is the acid gas concentration in the chimney at the next moment.

[0011] Furthermore, the reinforcement learning control model described in step 4 includes a state space, an action space, a state transition, and a reward function.

[0012] Furthermore, step 4 specifically includes: Step 4.1, State space definition, the state space includes waste feed rate, furnace temperature, main steam flow rate, economizer outlet flue gas oxygen content, chimney HCl equivalent concentration, and chimney SO2 equivalent concentration; Step 4.2, Action Space Definition: The action space includes the semi-dry slurry flow rate control increment and the wet caustic soda flow rate control increment. The semi-dry slurry flow rate control increment represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle; the wet caustic soda flow rate control increment represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle. Step 4.3, State Transition: The state transition refers to the transition of the system state according to the state transition function after the control action is applied to the training environment. The state transition function is composed of a semi-dry deacidification sub-model and a wet deacidification sub-model, and updates the non-deacidification related state variables in combination with historical operating data. Step 4.4: Reward function construction. The reward function is a dual-objective optimization function with flue gas pollutant emissions and deacidifying agent consumption as optimization objectives. At the same time, a control smoothing penalty term is introduced to constrain action changes and improve system stability. The reward function is constructed based on the constructed dual-objective optimization function and the control smoothing penalty term. The dual-objective optimization function includes an HCl emission penalty function, an SO2 emission penalty function, and a deacidifying agent consumption objective function.

[0013] Furthermore, the reinforcement learning control model in step 5 is trained using a deep deterministic policy gradient algorithm. The deep deterministic policy gradient algorithm includes an Actor network, a Critic network, a target network, and an experience replay pool. The Actor network outputs continuous actions, the Critic network evaluates state-action value, the target network is used for stable training, and the experience replay pool is used to store interaction data. All of these networks use multi-layer fully connected feedforward neural networks to achieve nonlinear mapping.

[0014] Furthermore, the actual control quantities corresponding to the action space in step 4.2 are the slurry flow rate and caustic soda flow rate at the next moment, which are respectively generated by the semi-dry slurry flow rate control increment and wet caustic soda flow rate control increment by the agent at time t, plus the slurry flow rate and caustic soda flow rate at the current moment.

[0015] Furthermore, the steps for constructing the reward function in step 4.4 include: Step 4.4.1: Define the bi-objective optimization function as follows:

[0016] in, It is a bi-objective optimization function. For the emission objective function, Let the objective function be the consumption of deacidifying agent. and These are the weighting coefficients.

[0017] Step 4.4.2: To avoid drastic fluctuations in the control quantity, a penalty term for control change is introduced:

[0018] in, The semi-dry slurry flow rate control increment generated by the agent at time t represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle. The increment of wet caustic soda flow control generated by the agent at time t represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle.

[0019] Step 4.4.3: Based on the dual-objective optimization function and control smoothing penalty items The reward function is defined as follows:

[0020] in, Execute the action at time t The immediate reward value is calculated afterward. It is a bi-objective optimization function. To control the smoothing penalty term, To control the weighting coefficients of the smoothing term.

[0021] Furthermore, the emission objective function in step 4.4.1 consists of two items: HCl and SO2.

[0022] in, As a penalty item for emissions, As a penalty item for HCl emissions, SO2 emission penalties and These are the corresponding weights. The penalty functions for HCl emissions and SO2 emissions are as follows:

[0023]

[0024] in, and These are the calculated concentrations of HCl and SO2 in the chimney at the next moment after the control action is executed, obtained from the environmental prediction model. and The emission limits for HCl and SO2 are set according to the local laws and regulations of the power plant. and These are the safety boundary concentrations (below the emission limits) for HCl and SO2, respectively, set according to power plant regulations considering both environmental and economic factors. and These are the penalty coefficients for exceeding emission standards for HCl and SO2, respectively.

[0025] A computer device according to the present invention includes a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, which, when executed by the processor, causes the processor to implement the method described herein.

[0026] The present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements the method described herein.

[0027] Compared with existing technologies, the beneficial effects of the present invention are as follows: This invention constructs a prediction model for the two-stage deacidification process of semi-dry and wet methods, and combines it with reinforcement learning algorithms to achieve intelligent optimization and adjustment of slurry flow rate and caustic soda flow rate. Through continuous interaction with the environment, it improves the accuracy of emission control, significantly reduces the consumption of deacidifying agent while ensuring stable compliance of acid gas emissions, and further enhances the stability, economy and environmental friendliness of the system operation. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be introduced below. Obviously, the drawings described below are only one embodiment of this invention. For those skilled in the art, other similar types of drawings can be obtained based on the drawings without creative effort.

[0029] Figure 1 The following is an overall flowchart of the dual-objective control method for a waste incineration deacidification system based on reinforcement learning, as an example.

[0030] Figure 2 The diagram shows the framework of a dual-objective control method for a waste incineration desulfurization system based on reinforcement learning, as illustrated in this embodiment.

[0031] Figure 3 The flowchart of the DDPG reinforcement learning control method based on the environment prediction model is shown in the example. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described herein are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0033] This embodiment presents a dual-objective control method for a waste incineration desulfurization system based on reinforcement learning, comprising the following steps: Step 1: Data Acquisition and Preprocessing. Collect operating and emission parameters of the waste-to-energy plant, and preprocess the collected data, including handling outliers and data standardization.

[0034] The operating and emission parameters of the waste-to-energy incineration plant in this embodiment include furnace combustion and operating parameters, semi-dry reaction tower operating parameters, flue gas state parameters at the dust collector outlet (intermediate flue gas state parameters), wet scrubbing system operating parameters, and flue gas pollutant emission parameters. The main flue gas pollutants emitted are HCl and SO2. Preprocessing includes outlier handling and data standardization. Outlier handling uses the 3σ criterion to screen outliers, and then replaces them using linear interpolation. Data standardization uses the Z-Score standardization method.

[0035] Step 2: Variable Screening and Time Series Construction. A candidate variable set is constructed based on the process mechanism of the deacidification system. The input variable set for the semi-dry deacidification sub-model is determined by combining mutual information (MI) methods and expert experience. and the set of input variables for the wet deacidification model The time shift values ​​at which the correlation between each variable is maximized are obtained using the MI method. The maximum value L is then used to construct the time series sets of the input variables for the semi-dry deacidification model. and the set of time series input variables for the wet deacidification model .

[0036] Specifically, candidate variables include combustion condition variables, semi-dry acid removal system operation variables, flue gas state variables at the dust collector outlet, and wet acid removal system operation variables. The semi-dry acid removal sub-model targets HCl and SO2 at the dust collector outlet, while the wet acid removal sub-model targets HCl and SO2 at the chimney. The set of input variables for the semi-dry acid removal sub-model is as follows. and the set of input variables for the wet deacidification model The corresponding control variables are already included in each.

[0037] In the time delay analysis in step 2, the process time from fuel input to final combustion and flue gas discharge is first calculated based on actual waste incinerator operating experience. m For each variable after filtering x Pick t time, t-1 time, t- 2 Time to tm Each moment constitutes a new variable x Time series matrix The MI value of this matrix and the HCl concentration is calculated using the MI method, and the value at the moment with the largest MI value is taken. As the time delay value of this variable, Take the largest value among all variables. The length of the constructed time series L . useL Let's construct the time series sets of input variables for the two models respectively:

[0038]

[0039] in, This is the set of time series input variables for the semi-dry deacidification model. This is the set of time series input variables for the wet deacidification model. Let be the vector of relevant input variables for the semi-dry deacidification model at time t. Let be the vector of relevant input variables for the wet deacidification model at time t. L This represents the length of the time series window.

[0040] Step 3: Construct an environmental prediction model. The environmental prediction model is a semi-dry-wet deacidification tandem prediction model, including a semi-dry deacidification sub-model and a wet deacidification sub-model.

[0041] (1) Establish a semi-dry deacidification sub-model to predict the state of intermediate flue gas after semi-dry deacidification treatment:

[0042] in, This represents the intermediate flue gas state at the next time step output by the semi-dry deacidification model. In this embodiment, the intermediate flue gas state is the flue gas state at the dust collector outlet. It is expressed as:

[0043] In the formula, and These represent the HCl and SO2 concentrations at the dust collector outlet at the next moment. The next moment is the flue gas temperature at the dust collector outlet. This is a semi-dry deacidification model; This is the set of time series input variables for the semi-dry deacidification model.

[0044] (2) Establish a wet deacidification sub-model to predict final emissions:

[0045] in, The next time step of the acid gas concentration in the chimney, output by the wet deacidification model, is expressed as:

[0046] In the formula, and These are the calculated concentrations of HCl and SO2 in the chimney at the next time step; This is a wet deacidification model; This is the set of time series input variables for the wet deacidification model.

[0047] In one embodiment, a semi-dry deacidification model and wet deacidification model Machine learning methods are used to build the network, including but not limited to Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN), Transformer network, or Extreme Gradient Boosting Tree (XGBoost) network.

[0048] Step 4: Construct a reinforcement learning control model based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The reinforcement learning control model includes a state space, action space, state transition, and reward function.

[0049] Step 4.1, State Space Definition:

[0050] in, For the amount of waste fed, For furnace temperature, Main steam flow rate, The oxygen content of the flue gas at the economizer outlet. Calculate the concentration of HCl in the chimney. The concentration of SO2 in the chimney is calculated.

[0051] Step 4.2, Define the action space: The action space is defined as the control increment of the deacidifying agent flow rate at both ends of the semi-dry and wet processes:

[0052] in, The semi-dry slurry flow rate control increment generated by the agent at time t represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle. The increment for wet caustic soda flow control generated by the agent at time t represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle. Here, the slurry flow rate refers to the slaked lime slurry flow rate in the semi-dry deacidification system, and the caustic soda flow rate refers to the NaOH solution flow rate in the wet deacidification system. The corresponding actual control quantity is updated as follows:

[0053]

[0054] in, and These are the slurry flow rate and caustic soda flow rate at the next moment, respectively. and These are the slurry flow rate and caustic soda flow rate at the current moment, respectively.

[0055] Step 4.3, State Transition: After a control action is applied to the training environment, the system state transitions as follows:

[0056] Among them, the state transition function The model consists of the semi-dry deacidification model and the wet deacidification model established in step 3, and the non-deacidification related state variables are updated by combining historical operating data.

[0057] Step 4.4, Reward Function Construction: With flue gas pollutant emissions and deacidifying agent consumption as optimization objectives, a bi-objective optimization function and a reward function are constructed; at the same time, a control smoothing penalty term is introduced to constrain action changes and improve system stability.

[0058] Step 4.4.1, Bi-objective optimization function. The bi-objective optimization function is defined as follows:

[0059] in, For the comprehensive objective function, For the emission objective function, Let the objective function be the consumption of deacidifying agent. and These are the weighting coefficients.

[0060] (1) Emission target function: The emission target function consists of two items: HCl and SO2.

[0061] in, As a penalty item for emissions, As a penalty item for HCl emissions, SO2 emission penalties and These are the corresponding weights. The penalty functions for HCl emissions and SO2 emissions are as follows:

[0062]

[0063] in, and Execute control actions respectively The calculated concentrations of HCl and SO2 in the chimney at the next moment were then obtained from the environmental prediction model. and The emission limits for HCl and SO2 are set according to the local laws and regulations of the power plant. and These are the safety boundary concentrations (below the emission limits) for HCl and SO2, respectively, set according to power plant regulations considering both environmental and economic factors. and These are the penalty coefficients for exceeding emission standards for HCl and SO2, respectively.

[0064] (2) Objective function for deacidifying agent consumption:

[0065] in, This is a deacidifying agent consumption item. and These are the semi-dry slurry flow rate and wet caustic soda flow rate, respectively, determined by the agent at the current moment for the next moment. This is the unit slurry cost coefficient. This is the unit cost coefficient for caustic soda.

[0066] Step 4.4.2: Control the smoothing penalty term.

[0067] To avoid drastic fluctuations in the control quantity, a penalty term for control change is introduced:

[0068] in, The semi-dry slurry flow rate control increment generated by the agent at time t represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle. The increment of wet caustic soda flow control generated by the agent at time t represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle.

[0069] Step 4.4.3: Define the reward function.

[0070] Based on the bi-objective optimization function and control smoothing penalty items The reward function is defined as follows:

[0071] in, Execute the action at time t The immediate reward value is calculated afterward. The function is a bi-objective optimization function (obtained from step 4.4.1). To control the smoothing penalty term (obtained from step 4.4.2), To control the weighting coefficients of the smoothing term. In one embodiment, the weights... , , The initial values ​​are set to 0.5, 0.3, and 0.2 respectively, and can be adjusted according to the actual needs of the power plant. The emission function weight settings satisfy: .

[0072] Step 5: Training the reinforcement learning control model based on the DDPG algorithm.

[0073] The Deep Deterministic Policy Gradient (DDPG) algorithm includes an Actor network, a Critic network, a target network, and an experience replay pool. The Actor network is the policy network, used to output continuous control actions; the Critic network is the value evaluation network, used to evaluate state-action value; the target network is used for stable training; and the experience replay pool is used to store interaction data. All networks employ multi-layer fully connected feedforward neural networks to implement nonlinear mapping.

[0074] Initialize the network parameters and experience replay pool before training. The specific steps are as follows: Step 5.1, Initialization.

[0075] 1) Initialize Actor network parameters ; 2) Initialize Critic network parameters ; 3) Initialize the target Actor network parameters Make it equal to ; 4) Initialize the target Critic network parameters Make it equal to ; 5) Initialize the experience replay pool .

[0076] Step 5.2, Interactive Training Process.

[0077] In each training round, proceed as follows: (1) State input and action output. Collect the current state from the environment. The data is then input into the Actor network to generate the control decision for the current moment.

[0078] in, For an agent at time t, based on state Output control decisions. To explore noise, it is used to enhance the ability to explore movements in the early stages of training, and the noise intensity is gradually reduced as the number of training iterations increases; For the Actor network in state The deterministic action output below.

[0079] (2) Environmental interaction and sample storage. This involves controlling actions... Applying to the training environment to obtain the state at the next moment. and instant rewards This forms an interactive sample. And store it in the experience replay pool. .

[0080] (3) Critic network update. Random sampling from the experience replay pool. N These samples constitute a mini-batch training data. For each sample, the target Actor network is first used to generate the target action based on the state at the next time step, and then the target Critic network is used to calculate the target Q-value.

[0081] in, For the first i The target Q value corresponding to each sample; For instant rewards; This is a discount factor used to characterize the importance of future rewards; For the target Critic network; For the target Actor network; For the first i The next state of the sample. The loss function of the Critic network is:

[0082] in, Let be the loss function of the Critic network; For the Critic network, the value of the current state-action pair is estimated. N The batch size is the number of samples. The loss function is minimized. Update Critic network parameters .

[0083] (4) Actor Network Update. After the Critic network is updated, its value evaluation results are used to calculate the policy gradient and update the Actor network parameters, so that the actions output by the Actor are gradually optimized in the direction of improving the state-action value. The policy gradient is approximately expressed as:

[0084] in, The gradient of the Actor network parameters; This represents the gradient of the Critic network with respect to the action. This represents the gradient of the Actor network with respect to its own parameters.

[0085] (5) Update the target network parameters using a soft update method. To improve training stability, the target network parameters are updated using a soft update method:

[0086]

[0087] in, For soft update coefficients, satisfying ; and These are the parameters for the main Critic network and the main Actor network, respectively. and These are the parameters for the target Critic network and the target Actor network, respectively.

[0088] (6) Repeat steps (1) to (5) above until the strategy converges or the preset number of training rounds is reached.

[0089] In steps 4 and 5, other weights and coefficients are compared through multiple combinations of weights and coefficients in the constructed simulation environment. The weights and coefficients that minimize the consumption of deacidifying agent and ensure stable control under emission constraints are selected.

[0090] Step 6: Implement online control.

[0091] The trained Actor network is deployed in the deacidification system, based on the current state. Output control action Based on this, the flow rates of semi-dry slurry and wet caustic soda in the next control cycle are adjusted to achieve real-time optimized control of HCl and SO2 emissions.

[0092] Collect the current system status in each control cycle. Then, it is input into the trained Actor network to calculate the control action. :

[0093] in, Based on the current state The control actions generated in the current control cycle are used to determine the control instructions for the next control cycle; The trained Actor network; These are the optimal Actor network parameters after training. Based on the control actions, the semi-dry slurry flow rate and wet caustic soda flow rate are updated for the next time step and applied to the field actuators to achieve real-time adjustment and optimized control of HCl and SO2 emissions.

[0094] This invention constructs an environmental prediction model and combines it with reinforcement learning algorithms to achieve intelligent optimization and adjustment of slurry flow rate and caustic soda flow rate. Under the premise of ensuring stable control process, it achieves stable compliance of acid gas emissions and effectively reduces the consumption of deacidifying agent.

[0095] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A dual-objective control method for a waste incineration desulfurization system based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Collect operating and emission parameters of the waste incineration power plant and preprocess the collected data; Step 2: Based on the preprocessed data, perform variable selection and construct a set of time series input variables; Step 3: Construct a semi-dry-wet desulfurization cascade prediction model based on time series samples as an environmental prediction model. Input the time series of input variables into the environmental prediction model to predict the intermediate flue gas state after semi-dry desulfurization treatment and predict the final emissions. Step 4: Using the control increments of semi-dry slurry flow rate and wet caustic soda flow rate as control actions, construct a dual-objective reward function that takes into account both acid gas emission control and deacidifying agent consumption optimization, and use a deep deterministic policy gradient algorithm to construct a reinforcement learning control model. Step 5: Train the constructed reinforcement learning control model; Step 6: Deploy the trained reinforcement learning control model in the actual deacidification system for online control.

2. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 1, characterized in that, In step 3, the environmental prediction model is a semi-dry-wet deacidification tandem prediction model, which includes a semi-dry deacidification sub-model and a wet deacidification sub-model. The semi-dry deacidification sub-model is used to predict the intermediate flue gas state after semi-dry deacidification treatment, and the output is the intermediate flue gas state at the next moment; the wet deacidification sub-model is used to predict the final emission, and the output is the acid gas concentration in the chimney at the next moment.

3. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning control model described in step 4 includes a state space, an action space, a state transition, and a reward function.

4. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 1, characterized in that, Step 4 specifically includes: Step 4.1, State space definition, the state space includes waste feed rate, furnace temperature, main steam flow rate, economizer outlet flue gas oxygen content, chimney HCl equivalent concentration, and chimney SO2 equivalent concentration; Step 4.2, Action Space Definition: The action space includes the semi-dry slurry flow rate control increment and the wet caustic soda flow rate control increment. The semi-dry slurry flow rate control increment represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle; the wet caustic soda flow rate control increment represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle. Step 4.3, State Transition: The state transition refers to the transition of the system state according to the state transition function after the control action is applied to the training environment. The state transition function is composed of a semi-dry deacidification sub-model and a wet deacidification sub-model, and updates the non-deacidification related state variables in combination with historical operating data. Step 4.4: Reward function construction. The reward function is a dual-objective optimization function with flue gas pollutant emissions and deacidifying agent consumption as optimization objectives. At the same time, a control smoothing penalty term is introduced to constrain action changes and improve system stability. The reward function is constructed based on the constructed dual-objective optimization function and the control smoothing penalty term. The dual-objective optimization function includes an HCl emission penalty function, an SO2 emission penalty function, and a deacidifying agent consumption objective function.

5. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning control model in step 5 is trained using a deep deterministic policy gradient algorithm. The deep deterministic policy gradient algorithm includes an Actor network, a Critic network, a target network, and an experience replay pool. The Actor network outputs continuous actions, the Critic network evaluates state-action value, the target network is used for stable training, and the experience replay pool is used to store interaction data. All of these networks use multi-layer fully connected feedforward neural networks to achieve nonlinear mapping.

6. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 4, characterized in that, The actual control quantities corresponding to the action space in step 4.2 are the slurry flow rate and caustic soda flow rate at the next moment, which are respectively the semi-dry slurry flow rate control increment and wet caustic soda flow rate control increment generated by the agent at time t, plus the slurry flow rate and caustic soda flow rate at the current moment.

7. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 4, characterized in that, Step 4.4 involves constructing the reward function, which includes the following steps: Step 4.4.1: Define the bi-objective optimization function as follows: in, It is a bi-objective optimization function. For the emission objective function, Let the objective function be the consumption of deacidifying agent. and These are the weighting coefficients; Step 4.4.2: To avoid drastic fluctuations in the control quantity, a penalty term for control change is introduced: in, The semi-dry slurry flow rate control increment generated by the agent at time t represents the adjustment range of the semi-dry slurry flow rate from the current control cycle to the next control cycle. The wet caustic soda flow rate control increment generated by the agent at time t represents the adjustment range of the wet caustic soda flow rate from the current control cycle to the next control cycle. Step 4.4.3: Based on the dual-objective optimization function and control smoothing penalty items The reward function is defined as follows: in, Execute the action at time t The immediate reward value is calculated afterward. It is a bi-objective optimization function. To control the smoothing penalty term, To control the weighting coefficients of the smoothing term.

8. The dual-objective control method for a waste incineration desulfurization system based on reinforcement learning according to claim 7, characterized in that, The emission objective function in step 4.4.1 consists of two items: HCl and SO2. in, As a penalty item for emissions, As a penalty item for HCl emissions, SO2 emission penalties and The corresponding weights are as follows; the HCl emission penalty function and the SO2 emission penalty function are as follows: in, and These are the calculated concentrations of HCl and SO2 in the chimney at the next moment after the control action is executed, obtained from the environmental prediction model. and The emission limits for HCl and SO2 are set according to the local laws and regulations of the power plant. and These are the safety boundary concentrations (below the emission limits) for HCl and SO2, respectively, set according to power plant regulations considering both environmental and economic factors. and These are the penalty coefficients for exceeding emission standards for HCl and SO2, respectively.

9. A computer device comprising a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, characterized in that: When the computer program is executed by the processor, it causes the processor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor implements the method as described in any one of claims 1 to 8.

Images