A combustion optimization method for thermal power units

Abstract

The present invention provides a combustion optimization method for thermal power units, which takes the optimal boiler efficiency and pollutant emission as the optimization control targets, uses the improved asynchronous advantage actor-critic algorithm as the control algorithm, and trains through the state design and reward design of the thermal power unit combustion system. The improved asynchronous advantage actor-critic algorithm has a fast convergence speed and stronger learning ability. It can effectively find the best control strategy for the complex problem of multi-variable constraints and multi-objective optimization of boiler combustion optimization, and at the same time considers the problem of wall temperature control, which can effectively improve the safety of unit operation.

Description

Technical Field

[0001] The invention relates to the field of automatic control of thermal power units, in particular to a combustion optimization method for thermal power units. Background Art

[0002] With the continuous growth of the new energy industry, wind power and solar energy have gradually changed the current power grid structure. Due to the instability of new energy, the flexibility transformation of each unit is one of the main directions. The main technical difficulty of flexibility transformation is how to make large-capacity coal-fired power units deeply peak-shift to ultra-low load, which puts higher requirements on the combustion stability of large-capacity coal-fired power units. Boiler combustion optimization is to reduce the generation of NOx by means of graded air distribution and coordinated coal distribution. At the same time, it also takes into account economic indicators such as CO emission concentration and boiler efficiency and safety indicators such as prevention of high-temperature steam pipe wall overheating. It is a complex problem of multi-field coupling, multi-variable constraints, and multi-objective optimization. Under deep peak-shaving operation, the decrease in load may lead to instability of boiler combustion, ineffective operation of the denitrification system, and overheating of the boiler steam-water system pipe wall. How to effectively ensure boiler efficiency and NOx emissions is an important research issue in combustion optimization under deep peak-shaving.

[0003] Boiler combustion conditions are more complicated and involve more restrictions. Currently, swarm intelligence optimization algorithms may not be able to find the optimal solution. However, deep reinforcement learning can directly use the state description of the environment and action evaluation to dynamically adjust the strategy to meet the ultimate control goal. However, sometimes learning directly through environmental feedback may result in low learning efficiency.

[0004] Based on the above-mentioned problems, the present invention proposes to improve the actor-critic algorithm with asynchronous advantages, add a Dyna structure to increase the algorithm training efficiency, dynamically adjust the learning proportion of the virtual environment, and carry out state design and reward design of the algorithm for the boiler combustion system, which can reduce the time required for training while considering multiple constraints. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies mentioned in the above background technology and provide a method for optimizing combustion of a thermal power plant, which is specifically implemented by the following scheme:

[0006] The present invention provides a method for optimizing combustion of a thermal power unit, the steps of which are as follows:

[0007] Step 1: Design the algorithm’s state and reward according to the boiler equipment conditions;

[0008] Step 2: Design the network structure and basic parameters of the asynchronous advantage actor-critic algorithm;

[0009] Step 3: Establish a multivariable transfer function model of the boiler combustion system;

[0010] Step 4: Establish an asynchronous advantage actor-critic algorithm based on the virtual environment model according to the transfer function model in step 3, and perform training;

[0011] The algorithm state design in step 1 includes the set value, actual value, adjustment amount and deviation required by the algorithm. For the boiler combustion system, the set value includes the carbon monoxide CO sp , exhaust gas temperature T e,sp and NOx concentration setting value NO x,sp ; The adjustment amount is the total air volume D AIR 、Nth layer burner coal quantity D B,n , the opening degree of the primary air door V at the i-th layer f,i , the opening degree of the secondary air door V on the jth layer s,j , k-th layer burnout damper opening V c,k and burner swing angle A f ,; Actual value of carbon monoxide CO, exhaust gas temperature T e and NOx concentration NO x , e Cx ,e Te ,e NOx They are the carbon monoxide deviation, exhaust gas temperature deviation, NOx concentration deviation, and the safety margin ΔT between the actual maximum wall temperature and the over-temperature value of the water-cooled wall. P , the state S of the coordinated control system at time t t :

[0012] S t ={CO sp ,T e,sp ,NO x,sp ,D AIR ,D B,n ,V f,i ,V s,j ,V c,k ,A f ,CO,T e ,NO x ,e CO ,e Te ,e NOx ,ΔT P}.

[0013] The algorithm reward design in step 1 is divided into a continuous reward item and a control amount change rate limit item. The continuous reward is:

[0014] e t is the weighted deviation at time t, K1 is the weight of the continuous reward item;

[0015] The control variable change rate limit item is:

[0016] K2 is the weight of the control variable change rate limit item;

[0017] The asynchronous advantage actor-critic algorithm based on the virtual environment model in step 4 adds a Dyna structure to the transfer function model in step 3 in each thread, so that the algorithm is trained in the transfer function model while learning in the real environment to improve the learning efficiency, and the virtual environment learning dynamic weight μ is increased, and the parameter update is:

[0018] θ′ a ←θ′ a +με a dθ′ a ,θ′ v ←θ′ v +με v dθ′ v

[0019] in In the above formula, Rm is the cumulative reward in the virtual environment model. When Rm<0, Rm is set to zero. rm is the maximum reward given by the virtual environment model at each step. j is the number of repeated executions in the virtual environment model. μ decreases as the number of global parameter updates increases and the cumulative rewards in each thread increase. θ′ a and θ′ v is the global shared parameter in the virtual environment, and T is the global shared count.

[0020] Beneficial Effects

[0021] The method described in the present invention optimizes the boiler combustion system using the actor-critic algorithm with asynchronous advantages, which can overcome its multi-objective and multi-coupling problems. At the same time, the Dyna structure is added to improve the learning efficiency of the algorithm, making the algorithm converge faster, and the algorithm training is simple, which can be realized in engineering practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the combustion system of a thermal power unit.

[0023] Figure 2 This is the boiler combustion optimization training structure diagram of the present invention. Specific implementation plan

[0024] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0025] Figure 1 This is a schematic diagram of the boiler combustion system of a thermal power unit. It is a six-input and three-output system. The inputs are total air volume, fuel amount at each layer (or total fuel amount), primary air door opening at each layer, secondary air door opening at each layer, burnout air opening at each layer, and burner swing angle. The outputs are carbon monoxide concentration, NOx concentration, and exhaust gas temperature.

[0026] Figure 2 This is the training result of the boiler combustion system algorithm based on the deep reinforcement learning algorithm, which includes the algorithm's reward design, state design, network parameter setting, and virtual environment setting. First, the algorithm's state and reward are designed, and then the state and reward values ​​are input into the algorithm. The state will be input into the virtual environment and the actual algorithm in Dyna at the same time. After the algorithm outputs the action, it is integrated to obtain the actual adjustment amount at the current moment. The adjustment amount acts on the actual boiler combustion system to obtain the state and reward at the next moment, and repeated learning is performed until the algorithm converges.

[0027] The main steps are as follows:

[0028] Step 1: Design the algorithm’s state and reward according to the boiler equipment conditions;

[0029] Step 2: Design the network structure and basic parameters of the asynchronous advantage actor-critic algorithm;

[0030] Step 3: Establish a multivariable transfer function model of the boiler combustion system;

[0031] Step 4: Establish an asynchronous advantage actor-critic algorithm based on the virtual environment model according to the transfer function model in step 3, and perform training;

[0032] The algorithm state design includes the set value, actual value, adjustment amount and deviation required by the algorithm. For the boiler combustion system, the set value includes the carbon monoxide CO sp , exhaust gas temperature T e,sp and NOx concentration setting value NO x,sp ; The adjustment amount is the total air volume D AIR 、Nth layer burner coal quantity D B,n , the opening degree of the primary air door V at the i-th layer f,i , the opening degree of the secondary air door V on the jth layer s,j , k-th layer burnout damper opening V c,k and burner swing angle A f ,; Actual value of carbon monoxide CO, exhaust gas temperature T e and NOx concentration NO x , e Cx ,e Te ,e NOx They are the carbon monoxide deviation, exhaust gas temperature deviation, NOx concentration deviation, and the safety margin ΔT between the actual maximum wall temperature and the over-temperature value of the water-cooled wall. P , the state S of the coordinated control system at time t t :

[0033] S t ={CO sp,T e,sp ,NO x,sp ,D AIR ,D B,n ,V f,i ,V s,j ,V c,k ,A f ,CO,T e ,NO x ,e CO ,e Te ,e NOx ,ΔT P}

[0034] The algorithm reward design in step 1 is divided into a continuous reward item and a control amount change rate limit item. The continuous reward is:

[0035]

[0036] where e t is the weighted deviation at time t, [λ1,λ2,λ3] T is the deviation weight matrix, which is used to adjust the proportions between the three adjustment targets, and K1 is the weight of the continuous reward item;

[0037] The control variable change rate limit item is:

[0038] K2 is the weight of the control variable change rate limit item;

[0039] The network structure of the asynchronous advantage actor-critic algorithm in step 2 is a 4-layer fully connected layer. The input layer of the critic network contains state information and reward information, with a total of 20 nodes, two intermediate layers with 80 nodes each, and an output layer with 1 node; the input layer of the actor network contains state information and the output of the critic network, with a total of 20 nodes, the intermediate hidden layer is the same as the critic network, and the output layer has 3 nodes. The basic parameter is the global maximum update number T max , maximum number of thread updates t max , actor network learning rate ε a , critic network learning rate ε v , the number of virtual environment model repetitions n, thread update discount factor γ and global update frequency F update , the specific settings are shown in the following table:

[0040]

[0041]

[0042] Generally, the speed of algorithm convergence can be determined based on the boiler structure and design parameters, and the above parameters can be adjusted accordingly.

[0043] The multivariable transfer function model of the boiler combustion system in step 3 is used as a virtual environment for auxiliary training. The accuracy requirement is not high. Generally, a disturbance experiment is conducted at a certain load point to directly establish a 6-input 3-output boiler combustion system transfer function model. This model is only used as a virtual environment in Dyna, which will improve the convergence speed of the algorithm in the early stage without affecting the final accuracy.

[0044] The asynchronous advantage actor-critic algorithm based on the virtual environment model in step 4 adds a Dyna structure to the transfer function model in step 3 in each thread, so that the algorithm is trained in the transfer function model while learning in the real environment to improve learning efficiency, and increases the dynamic weight μ of virtual environment learning. Then the parameter update is:

[0045] θ′ a ←θ′ a +με a dθ′ a ,θ′ v ←θ′ v +με v dθ′ v

[0046] in

[0047] In the above formula, Rm is the cumulative reward in the virtual environment model. When Rm<0, Rm is set to zero. rm is the maximum reward given by the virtual environment model at each step. j is the number of repeated executions in the virtual environment model. μ decreases as the number of global parameter updates increases and the cumulative rewards in each thread increase. θ′ a and θ′ v is the global shared parameter in the virtual environment, and T is the global shared count.

Claims

1. A method for optimizing combustion of a thermal power unit, characterized in that: The specific steps are as follows: Step 1: Design the algorithm’s state and reward according to the boiler equipment conditions; Step 2: Design the network structure and basic parameters of the asynchronous advantage actor-critic algorithm; Step 3: Establish a multivariable transfer function model of the boiler combustion system; Step 4: Establish an asynchronous advantage actor-critic algorithm based on the virtual environment model according to the transfer function model in step 3 and perform training; The state design in step 1 is divided into set value, actual value, adjustment amount and deviation, wherein the set value includes the carbon monoxide amount CO sp , exhaust gas temperature T e,sp and NOx concentration setting value NO x,sp ; The adjustment amount is the total air volume D AIR 、Nth layer burner coal quantity D B,n , the opening degree of the primary air door V at the i-th layer f,i , the opening degree of the secondary air door V on the jth layer s,j , k-th layer burnout damper opening V c,k and burner swing angle A f ,; the actual value is carbon monoxide CO, exhaust gas temperature T e and NOx concentration NO x ,;e Cx ,e Te ,e NOx They are the carbon monoxide deviation, exhaust gas temperature deviation, NOx concentration deviation, and the safety margin ΔT between the actual maximum wall temperature and the over-temperature value of the water-cooled wall. P , the state S of the coordinated control system at time t t : S t ={CO sp ,T e,sp , NO x,sp ,D AIR ,D B,n ,V f,i ,V s,j ,V c,k ,A f ,CO,T e , NO x ,have been CO ,have been Te ,have been NOx ,ΔT P }; The reward design in step 1 is divided into a continuous reward item and a control amount change rate limit item. The continuous reward is: e t is the weighted deviation at time t, K1 is the weight of the continuous reward item; The control variable change rate limit item is: K2 is the weight of the control variable change rate limit item; The asynchronous advantage actor-critic algorithm based on the virtual environment model in step 4 adds a Dyna structure to the transfer function model in step 3 in each thread, so that the algorithm is trained in the transfer function model while learning in the real environment to improve the learning efficiency, and the virtual environment learning dynamic weight μ is increased, and the parameter update is: i a ′←θ a ′+with a dth a ′,θ v ′←θ v ′+with v dth v ′ in In the above formula, Rm is the cumulative reward in the virtual environment model. When Rm<0, Rm is set to zero. rm is the maximum reward given by the virtual environment model at each step. j is the number of repeated executions in the virtual environment model. μ decreases with the increase in the number of global parameter updates and the increase in the cumulative rewards in each thread. θ′ a and θ′ v is the global shared parameter in the virtual environment, and T is the global shared count.

Images