SCR (Selective Catalytic Reduction) denitration optimization control system of thermal power generating unit

Abstract

The invention discloses a thermal power generating unit SCR denitration optimization control system, and the system comprises a current operation state parameter obtaining unit which is used for obtaining a plurality of current operation state parameters of a thermal power generating unit; the vector standardization unit is used for obtaining a current state vector of the thermal power generating unit after the plurality of operation state parameters are subjected to standardization processing; the model input and output unit is used for receiving the current state vector as the input of a denitration optimization model and executing the trained optimal control strategy so as to select the optimal control action of the thermal power generating unit; wherein the denitration optimization model is obtained after a time sequence sub-model and a reinforcement learning sub-model are subjected to stage training; according to the method, the denitration optimization control model is constructed, and the second training data after data enhancement is learned, so that the denitration optimization system can fully consider the historical operation state and trend when executing operation control, and the operation stability and long-term efficiency of the denitration system are improved.

Description

Technical Field

[0001] The invention relates to the field of denitration optimization control, and in particular to an SCR denitration optimization control system for a thermal power unit. Background Art

[0002] The SCR (Selective Catalytic Reduction) denitrification system of thermal power units is an important flue gas denitrification technology. Its purpose is to reduce the concentration of nitrogen oxides (NOx) in emissions and reduce pollution to the atmospheric environment through the chemical reaction of catalysts and ammonia. However, in actual operation, the control and optimization of the denitrification system faces many technical problems. One of the key issues is how to accurately control the operating control parameters such as the ammonia injection amount and the catalyst state under a complex and dynamically changing operating environment to achieve efficient removal of NOx and minimize ammonia escape.

[0003] The patent document with patent publication number CN116417094A discloses an intelligent ammonia injection denitrification prediction system based on a time series neural network model, which solves the problems that the current denitrification ammonia injection system adopts fixed ammonia distribution and cannot adapt to the variability of denitrification inlet flue gas NOx under the flexible peak regulation of the unit.

[0004] However, the existing technology for training flue gas desulfurization models based on historical training data either focuses on the temporal regularity of the data or focuses on the prediction regularity between the operating state and the operation control in the data; in fact, the change between the two parameters of the SCR denitrification system of thermal power units on a longer time scale is a complex dynamic process. For example, the above is based on the prediction of the temporal neural network model, but the training data is not locally disturbed to generate training data that not only focuses on the regularity between parameters in the short term, but also captures the longer time scale. Therefore, the development of a denitrification optimization control model that can combine the direct regularity between the operating state parameters and the operation control parameters and the long-term temporal regularity has become a key technical problem to improve the efficiency and stability of the denitrification system. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an SCR denitrification optimization control system for a thermal power unit, which generates training data through local disturbance to solve the technical problems raised in the background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] A thermal power unit SCR denitration optimization control system, the control system comprising:

[0008] A current operating state parameter acquisition unit is used to acquire several current operating state parameters of the thermal power unit;

[0009] A vector normalization unit is used to obtain the current state vector of the thermal power unit after normalizing a number of operating state parameters;

[0010] A model input and output unit, used to receive the current state vector as the input of the denitrification optimization model, and execute the trained optimal control strategy to select the optimal control action of the thermal power unit;

[0011] Among them, the denitrification optimization model is obtained after the time series sub-model and the reinforcement learning sub-model are trained in stages;

[0012] Among them, the optimal control action of the denitrification optimization model is characterized by: the optimal operating control parameters of the thermal power unit in the SCR denitrification optimization.

[0013] In some embodiments, the stage training steps of the denitrification optimization model include:

[0014] S1. Iterate the time series sub-model using the first training data to obtain an action prediction model; wherein the first training data is characterized as: a historical training set including N historical training samples;

[0015] S2. Locally perturb and number the first training data to obtain a random state set including a random state vector;

[0016] S3, inputting the random state set including the random state vector into the action prediction model to obtain a random action set including the random action vector;

[0017] S4, combining the random state set and the random action set to obtain second training data after data enhancement;

[0018] S5. Use the second training data to iteratively reinforce the learning sub-model to obtain a denitrification optimization model.

[0019] In some embodiments, the step of iterating the time series sub-model using the first training data includes:

[0020] S1-1, obtaining a historical training set containing N historical training samples;

[0021] S1-2, inputting a number of historical training samples into the time series sub-model in batches for self-supervised iterative training until the historical training set is traversed several times;

[0022] S1-3, after traversing the historical training set for several times, deriving the temporal sub-model of the current parameters as the action prediction model. In some embodiments, the step of obtaining the historical training set containing N historical training samples includes:

[0023] S1-1-1. Collect N operating status parameters and corresponding N operation control parameters in the historical database of the thermal power unit in a fixed-size time window according to the time sequence; and assign unique time sequence numbers to the operating status parameters and the corresponding operation control parameters;

[0024] S1-1-2, standardize the N operating state parameters and operation control parameters that have been assigned sequential numbers and construct them into sequential state vectors and sequential action vectors respectively;

[0025] S1-1-3, using the time series state vector as input feature and the time series action vector as target label to construct historical training samples;

[0026] S1-1-4. Construct N historical training samples into a historical training set; wherein each historical training sample has an independent time series number.

[0027] In some embodiments, the step of performing local perturbation numbering on the first training data includes:

[0028] S2-1, sorting the time sequence state vectors according to the time sequence numbers to obtain a time sequence state set including N time sequence state vectors; wherein the time sequence state vectors include M operating state parameters of the thermal power unit;

[0029] S2-2, according to a preset attribute arrangement rule, assigning continuous attribute numbers to a number of operating state parameters of each time series state vector in the time series state set, and sorting them according to the attribute numbers to obtain time series state vectors with continuous attribute numbers;

[0030] S2-3, substitute the N time series state vectors into the local perturbation formula to obtain N*M random state vectors and the corresponding post-perturbation numbers;

[0031] The local perturbation formula is:

[0032]

[0033] Where i is the timing number of the timing state vector, j is the attribute number of the operating state parameter, N is the total number of timing numbers, M is the total number of attribute numbers, k is the offset that changes with the value of N and is used to control the local disturbance range; ((i+j×k-1)modN)+1 represents the post-disturbance number generated within the timing number range, which ranges from 1 to N cycles; X' i,j is a random state vector generated by local disturbance of the jth operating state parameter in the i-th time series state vector;

[0034] S2-4, sort the N*M random state vectors according to the perturbation number, and define them as a random state set; wherein the random state set S is expressed as: S = {X' i,j∣i∈{1,2,…,N},j∈{1,2,…,M}}.

[0035] In some embodiments, the step of inputting a random state set including a random state vector into an action prediction model to obtain a random action set including a random action vector includes:

[0036] S3-1, inputting the random state vectors in the random state set into the action prediction model in sequence;

[0037] S3-2, the action prediction model performs forward propagation and outputs the corresponding random action vector;

[0038] S3-3, repeat S3-1 and S3-2 until all random state vectors are input, and construct a random action set containing all random action vectors;

[0039] The expression of random action set A is: A={A' i,j ∣i∈{1,2,…,N},j∈{1,2,…,M}};

[0040] Among them, A' i,j Denotes the corresponding random state vector X' i,j A random action vector.

[0041] In some embodiments, a step of inputting a plurality of historical training samples in batches into the time series sub-model for self-supervised iterative training is as follows:

[0042] S1-2-1, taking the time series state vector as the feature variable, performing forward propagation to obtain the time series action prediction value;

[0043] S1-2-2, calculating the mean square error loss of the temporal action prediction value and the temporal action vector according to the loss function;

[0044] The expression of the loss function is:

[0045] in, represents the mean square error loss between the temporal action prediction value and the temporal action vector, N is the number of samples of the temporal state vector, and Y i is a temporal action vector constructed by several operation control parameters in the historical database. It is the predicted value of the time-series action based on the output of the time-series state vector;

[0046] S1-2-3, calculate the parameter gradient of the mean square error loss relative to the timing sub-model;

[0047] S1-2-4, using the gradient descent algorithm to update the model parameters to obtain updated model parameters;

[0048] S1-2-5. Execute S1-2-1 and S1-2-2 using the updated model parameters until the mean square error loss reaches the preset mean square error loss threshold.

[0049] In some embodiments, the step of combining the random state set and the random action set to obtain the second training data after data enhancement includes:

[0050] S4-1. Sort each random state vector in the random state set by its perturbation number to obtain a sorted random state set.

[0051] S4-2, according to the order of the perturbation numbers, match the sorted random state set with the corresponding random actions in the random action set to construct a state-action tuple;

[0052] S4-3. Combine all state-action pairs to form second training data containing N*M state-action pairs.

[0053] In some embodiments, the step of iteratively reinforcing the learning sub-model using the second training data includes:

[0054] S5-1. Initialize the model parameters of the reinforcement learning sub-model and define the immediate reward function r(S,A) and the Q-value function Q(S,A);

[0055] Among them, S represents the random state vector X' in the second training data i,j Defined as the state of the reinforcement learning sub-model, A represents the random action vector X' in the second training data i,j Defined as the action of the reinforcement learning sub-model, the Q-value function is used to evaluate the Q-value of the state-action binary;

[0056] S5-2, select the state-action pair (X' i,j ,A' i,j );

[0057] S5-3, in the current random state vector X' i,j Next, execute the random action vector A' i,j ;

[0058] S5-4, when executing random action vector A' i,j After that, get the immediate reward r(X' i,j ,A' i,j );

[0059] The expression of immediate reward is:

[0060] r(X' i,j ,A' i,j )=-α·(NOx out )2 -β·(NH 3,escape ) 2 ;

[0061] Among them, NOx out Indicates NOx emission concentration, used to characterize the concentration of nitrogen oxides that are not reduced in the SCR reaction, NH 3,escape Indicates NH 3, Escape concentration, used to characterize the residual ammonia concentration after the SCR reaction;

[0062] S5-5. According to the order of the perturbation numbers, select the next adjacent random state vector from the sorted random state set as the next state X' i+1,j ;

[0063] S5-6, according to the selected state action tuple (X' i,j ,A' i,j ), immediate reward r(X' i,j ,A' i,j ) and the next state X' i+1,j , update the Q value;

[0064] The Q value update is expressed as:

[0065] Q new ←Q(X i,j ,A i,j ;θ)+α(r(X′ i,j ,A i,j )+γmax a' Q(X′ i+1,j ,a;θ)-Q(X i,j ,A i,j ;θ));

[0066] Among them, θ represents the model parameter set; a' represents all possible actions, which are used to calculate the next state X' i+1,j The Q value of the optimal action under a' Q(X' i+1,j ,a';θ);max a' Q(X' i+1,j ,a';θ) means that in the next X' i+1,j The Q value of the optimal action selected under the condition; α is used to control the step size of each Q value update, ranging from 0 to 1, r(X' i,j ,A' i,j ) means in state X' i,j Next take action A' i,j The immediate feedback obtained after the reward is given, γ represents the discount factor, which indicates the influence of future rewards on the current Q value, and the value ranges from 0 to 1. The closer γ is to 1, the more attention is paid to long-term rewards, and the closer it is to 0, the more attention is paid to immediate rewards.new In state X' i,j Next take action A' i,j The updated Q value, Q(X' i,j ,A' i,j ; θ) means that under the model parameter θ, in state X' i,j Next take action A' i,j Q value before update;

[0067] S5-7. Repeat S5-2 to S5-6 until the convergence condition is reached to obtain the denitrification optimization model.

[0068] The present invention provides an SCR denitration optimization control system for a thermal power unit, which has the following beneficial effects:

[0069] The present invention constructs a denitrification optimization control model and learns the second training data after data enhancement. In this way, on the basis of obtaining the direct laws between the operating state parameters and the operational control parameters, it can also capture the temporal laws on a long time scale, thereby solving the problem that traditional control models are difficult to adapt to complex dynamic environments. The denitrification optimization system can fully consider the historical operating status and trends when performing operational control, thereby improving the operating stability and long-term efficiency of the denitrification system.

[0070] On the other hand, the present invention enhances the first training data through a local disturbance mechanism, so that the second training data after data enhancement can not only retain the time series relationship of some operating state parameter data of the thermal power unit in the denitrification optimization control, but also introduce exploratory data to expand the training data set without losing the time series information.

[0071] On the other hand, the present invention uses the random state vector and random action vector constructed by the operating state parameters in the denitrification optimization control as the state-action binary group, and uses them to train the reinforcement learning model, and then learns the optimal operation control strategy of the thermal power unit in the denitrification optimization control, so that the model can adaptively select the optimal operation control parameters during operation, realize the control of NOx concentration and ammonia escape concentration, reduce the use of ammonia, reduce costs, and improve denitrification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a structural block diagram of an SCR denitration optimization control system for a thermal power unit according to the present invention;

[0073] Figure 2 This is a control flow chart of a thermal power unit SCR denitration optimization control system of the present invention;

[0074] Figure 3 This is a modeling flow chart of the denitration optimization model of the present invention;

[0075] Figure 4Schematic diagram of constructing the random state vector of the present invention. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] First, the prior art and related concepts involved in the embodiments of the present invention are described:

[0078] SCR: A technology used in the denitrification process of thermal power units to reduce nitrogen oxide (NOx) emissions in flue gas. By injecting ammonia (or urea) to react with nitrogen oxides, harmful nitrogen oxides are converted into harmless nitrogen and water.

[0079] Temporal sub-model: The temporal sub-model is a model that can capture the temporal dependencies in the data and is used for self-supervised learning in the pre-training stage.

[0080] Reinforcement learning sub-model: A model used to learn the optimal control strategy. During the training process, by interacting with the environment, the reinforcement learning sub-model continuously adjusts the strategy based on the immediate reward signal and learns the optimal control actions to be taken under different operating conditions.

[0081] Example 1: Please refer to Figure 1 to Figure 2 The present invention provides a thermal power unit SCR denitration optimization control system, comprising:

[0082] A current operating state parameter acquisition unit is used to acquire several current operating state parameters of the thermal power unit;

[0083] A vector normalization unit is used to obtain the current state vector of the thermal power unit after normalizing a number of operating state parameters;

[0084] A model input and output unit, used to receive the current state vector as the input of the denitrification optimization model, and execute the trained optimal control strategy to select the optimal control action of the thermal power unit;

[0085] Among them, the denitrification optimization model is obtained after the time series sub-model and the reinforcement learning sub-model are trained in stages;

[0086] Among them, the optimal control action of the denitrification optimization model is characterized by: the optimal operating control parameters of the thermal power unit in the SCR denitrification optimization.

[0087] This embodiment constructs a denitrification optimization model by using joint training of a timing sub-model and a reinforcement learning sub-model, and uses the denitrification optimization model to select optimal operating control parameters from current operating state parameters to achieve optimized control of the SCR denitrification system of a thermal power unit.

[0088] Example 2: Participation Figures 1 to 4 The technical solution of this embodiment 2 is different from that of embodiment 1 in that a stage training modeling method of a denitration optimization model is also disclosed, and the modeling method includes:

[0089] S1. Iterate the time series sub-model using the first training data to obtain an action prediction model; wherein the first training data is characterized as: a historical training set including N historical training samples;

[0090] S2. Locally perturb and number the first training data to obtain a random state set including a random state vector;

[0091] S3, inputting the random state set including the random state vector into the action prediction model to obtain a random action set including the random action vector;

[0092] S4, combining the random state set and the random action set to obtain second training data after data enhancement;

[0093] S5. Use the second training data to iteratively reinforce the learning sub-model to obtain a denitrification optimization model.

[0094] This embodiment gradually improves the control performance of the denitration optimization model on the SCR system of the thermal power unit by training the timing sub-model and the reinforcement learning sub-model in stages, ensuring the capture of complex state-action relationships.

[0095] Furthermore, S1 specifically includes:

[0096] S1-1, obtaining a historical training set containing N historical training samples;

[0097] S1-2, inputting a number of historical training samples into the time series sub-model in batches for self-supervised iterative training until the historical training set is traversed several times;

[0098] S1-3. After traversing the historical training set several times, the temporal sub-model of the current parameters is derived as the action prediction model.

[0099] In this embodiment, although the operating state parameters and the operation control parameters have no obvious temporal connection at a single moment, in the historical data of a longer time scale, the operating state parameters and the operation control parameters may show certain temporal regularities. In other words, the operating state and control operation of the thermal power unit may form a certain trend or dependency over time. For example, the change of the operation control parameters may gradually affect the operating state of the system after a period of time. This potential temporal characteristic may be implicit in the data in the long-term data.

[0100] At the same time, the first training data forms a random state vector after local perturbation, which not only retains a certain temporal relationship, but also introduces unknown exploration properties of the data. The temporal sub-model can make full use of this structured temporal relationship to capture temporal regularities that may exist in a longer time scale, and explore the direct regularities between the random state vector and the random action vector.

[0101] Moreover, in actual applications, the operating state of thermal power units will change over time, which means that historical operating states and control operations may affect current decisions. The time series model can learn these complex dynamic relationships, not just the static operating states and direct laws of operation control.

[0102] Therefore, this embodiment selects the timing sub-model as a pre-trained model that captures the regularities between the operating state parameters and the operation control parameters, while capturing the timing regularities in a longer time scale.

[0103] This embodiment obtains an action prediction model by batch training the timing sub-model, thereby being able to construct a model to mine potential timing rules from a longer time scale and effectively enhance the prediction of future operations.

[0104] Furthermore, S1-1 specifically includes:

[0105] S1-1-1. Collect N operating status parameters and corresponding N operation control parameters in the historical database of the thermal power unit in a fixed-size time window according to the time sequence; and assign unique time sequence numbers to the operating status parameters and the corresponding operation control parameters;

[0106] Among them, the operating status parameters include: flue gas temperature, nitrogen oxide concentration, ammonia injection amount, oxygen concentration, exhaust gas flow rate, catalyst bed pressure difference, ammonia slip concentration;

[0107] The operating control parameters include: ammonia injection rate, ammonia injection rate, ammonia injection distribution position, ammonia injection pressure, flue gas recirculation ratio, catalyst inlet temperature, flue gas oxygen content, and ammonia to nitrogen oxide molar ratio;

[0108] S1-1-2. After standardizing N operating status parameters and operation control parameters that have been assigned timing numbers, they are constructed into timing state vectors and timing action vectors respectively; the representation state vectors and action vectors both have timing numbers.

[0109] S1-1-3, using the time series state vector as input feature and the time series action vector as target label to construct historical training samples;

[0110] S1-1-4. Construct N historical training samples into a historical training set; wherein each historical training sample has an independent time series number.

[0111] This embodiment constructs standardized time-series state vectors and action vectors to accurately capture the operating status and operational control characteristics of the thermal power unit, thereby providing historical data for subsequent training.

[0112] Furthermore, S2 specifically includes:

[0113] S2-1, sorting the time sequence state vectors according to the time sequence numbers to obtain a time sequence state set including N time sequence state vectors; wherein the time sequence state vectors include M operating state parameters of the thermal power unit;

[0114] S2-2, according to a preset attribute arrangement rule, assigning continuous attribute numbers to a number of operating state parameters of each time series state vector in the time series state set, and sorting them according to the attribute numbers to obtain time series state vectors with continuous attribute numbers;

[0115] For example, the attribute arrangement rules need to be defined according to the key control parameters of the SCR (selective catalytic reduction) denitration system of the thermal power unit. The purpose of the SCR denitration system is to reduce the emission of nitrogen oxides (NOx), so the attribute arrangement rules should include operating state parameters related to the denitration process, pollutant control, airflow management, etc. In addition, general parameters (such as temperature and pressure) are also necessary because they affect the efficiency and operating stability of the catalytic reaction.

[0116] In the SCR denitration system of the thermal power unit of this embodiment, the following attribute arrangement rules can be defined to ensure that the operating state parameters in each state vector conform to the operating logic:

[0117] Flue gas temperature: Temperature is a key factor in the SCR denitrification reaction. The catalyst works effectively only within a specific temperature range. Too high or too low temperature will affect the denitrification efficiency.

[0118] Nitrogen oxide concentration: reflects the concentration of NOx pollutants before entering the SCR system and is used to determine the required degree of denitrification reaction.

[0119] Ammonia injection amount: The core parameter for controlling the SCR reaction. The ammonia injection amount needs to be precisely adjusted according to the NOx concentration and airflow conditions to avoid excess or insufficient injection.

[0120] Oxygen concentration: The oxygen content affects the chemical balance of the reaction process and has a direct impact on the efficiency of the catalyst.

[0121] Exhaust flow rate: The flow rate affects the reaction time and the contact efficiency between the gas and the catalyst. Too fast a flow rate may result in incomplete reduction of NOx, affecting the denitrification efficiency.

[0122] Catalyst bed pressure difference: used to monitor the status of the catalyst. If the pressure difference is too large, it may indicate that the catalyst is clogged or needs to be replaced.

[0123] Ammonia escape concentration: This parameter is used to monitor the escape of ammonia. Ammonia escape is an important indicator of the denitrification system. Excessive ammonia escape may cause secondary pollution.

[0124] Therefore, in this embodiment, the attribute numbering rule is to assign numbers to attributes in the following order to ensure that all timing state vectors remain consistent.

[0125] Attribute No. 1: Flue gas temperature; Attribute No. 2: Nitrogen oxide concentration; Attribute No. 3: Ammonia injection amount; Attribute No. 4: Oxygen concentration; Attribute No. 5: Exhaust flow rate; Attribute No. 6: Catalyst bed pressure difference; Attribute No. 7: Ammonia slip concentration.

[0126] S2-3, substitute the N time series state vectors into the local perturbation formula to obtain N*M random state vectors and the corresponding post-perturbation numbers;

[0127] The local perturbation formula is:

[0128]

[0129] Where i is the timing number of the timing state vector, j is the attribute number of the operating state parameter, N is the total number of timing numbers, M is the total number of attribute numbers, k is the offset that changes with the value of N and is used to control the local disturbance range; ((i+j×k-1)modN)+1 represents the post-disturbance number generated within the timing number range, which ranges from 1 to N cycles; X' i,j is a random state vector generated by local disturbance of the jth operating state parameter in the i-th time series state vector;

[0130] For example, when N>100, k∈[1,5] is taken; assuming that the total number of time series numbers N=120, the total number of attribute numbers M=7, the current time series number i=50, and the current attribute number j=3, then a random integer k is taken, such as k=4, and the calculation process of the perturbed number is:

[0131] ((i + j × k - 1) mod N) + 1 = ((50 + 3 × 4 - 1) % 120) + 1 = (50 + 12 - 1) % 120 + 1 = 61;

[0132] Therefore, the third running state parameter of the local perturbation random state vector X' 50,3 comes from the third running state parameter in the time series state vector with time series number 61, that is, X' 50,3 = X 61,3 . That is, by introducing a certain degree of randomness to select the offset k, the current time series number i is perturbed.

[0133] Similarly, when 10 < N ≤ 100, take k ∈ [1, 10]; assume the total number of time series numbers N = 80, the total number of attribute numbers M = 7, the current time series number i = 25, and the current attribute number j = 4. Take a random integer k, such as k = 7, then the calculation process of the perturbed number is:

[0134] ((i + j × k - 1) mod N) + 1 = ((25 + 4 × 7 - 1) % 80) + 1 = (25 + 28 - 1) % 80 + 1 = 52, that is, X' 25,4 = X 52,4 .

[0135] Similarly, when N ≤ 10, take k = 1; assume the total number of time series numbers N = 8, the total number of attribute numbers M = 5, the current time series number i = 3, and take k = 1, then the calculation process of the perturbed number is:

[0136] ((i + j × k - 1) % N) + 1 = ((3 + 2 × 1 - 1) % 8) + 1 = (3 + 2 - 1) % 8 + 1 = 5, that is, X' 3,2 = X 5,2 .

[0137] S2-4. Sort the N * M random state vectors according to the perturbed numbers and define them as the random state set; among them, the expression of the random state set S is:

[0138] S = {X′ i,j | i ∈ {1, 2,..., N}, j ∈ {1, 2,..., M}};

[0139] Exemplarily, if the current random state vector is X' i,j , then the random state vector immediately following it in the random state set S is X' i+1,j ; if i = N, it means that the last time series number of the random state set has been reached, and return to the initial position or terminate the iteration.

[0140] This embodiment generates a random state set with exploratory and continuous characteristics through attribute arrangement rules and local disturbances, thereby providing diversified training data.

[0141] Further, S3 specifically includes:

[0142] S3-1, inputting the random state vectors in the random state set into the action prediction model in sequence;

[0143] S3-2, the action prediction model performs forward propagation and outputs the corresponding random action vector;

[0144] S3-3, repeat S3-1 and S3-2 until all random state vectors are input, and construct a random action set containing all random action vectors;

[0145] The expression of random action set A is:

[0146] A={A′ i,j ∣i∈{1,2,…,N},j∈{1,2,…,M}};

[0147] Among them, A' i,j Denotes the corresponding random state vector X' i,j A random action vector.

[0148] This embodiment generates a corresponding random action vector by inputting a random state vector into the action prediction model, thereby constructing a random action set matching different working conditions and providing training data for the reinforcement learning sub-model.

[0149] Furthermore, S1-2 specifically includes:

[0150] S1-2-1, taking the time series state vector as the feature variable, performing forward propagation to obtain the time series action prediction value;

[0151] S1-2-2, calculating the mean square error loss of the temporal action prediction value and the temporal action vector according to the loss function;

[0152] The expression of the loss function is:

[0153] in, represents the mean square error loss between the temporal action prediction value and the temporal action vector, N is the number of samples of the temporal state vector, and Y i is a temporal action vector constructed by several operation control parameters in the historical database. It is the predicted value of the time-series action based on the output of the time-series state vector;

[0154] S1-2-3, calculate the parameter gradient of the mean square error loss relative to the timing sub-model;

[0155] S1-2-4, using the gradient descent algorithm to update the model parameters to obtain updated model parameters;

[0156] For example, the gradient descent algorithm refers to iteratively adjusting the model parameters to minimize the loss function. It updates the parameters along the gradient direction of the loss function, gradually reducing the loss and eventually converging to a smaller loss value, so that the model fits the data better.

[0157] S1-2-5. Execute S1-2-1 and S1-2-2 using the updated model parameters until the mean square error loss reaches the preset mean square error loss threshold.

[0158] This embodiment inputs training samples in batches and performs a self-supervised learning process to adjust the parameters of the time series sub-model so that the model can more accurately fit the relationship between the operating status of the thermal power unit and the operational control.

[0159] Furthermore, S4 specifically includes:

[0160] S4-1. Sort each random state vector in the random state set by its perturbation number to obtain a sorted random state set.

[0161] S4-2, according to the order of the perturbation numbers, match the sorted random state set with the corresponding random actions in the random action set to construct a state-action tuple;

[0162] S4-3. Combine all state-action pairs to form second training data containing N*M state-action pairs.

[0163] Exemplarily, the post-disturbance numbering of this embodiment is not continuous, but it is unique and independent. Specifically, since the post-disturbance numbering is calculated by the local perturbation formula, the post-disturbance numbering will be offset by the disturbance (through the action of j and k), so it will not be strictly arranged in a natural sequence, which can break the continuity of the original state and allow the model to perform richer exploration between different states.

[0164] Furthermore, S5 specifically includes:

[0165] S5-1. Initialize the model parameters of the reinforcement learning sub-model and define the immediate reward function r(S,A) and the Q-value function Q(S,A);

[0166] Among them, S represents the random state vector X' in the second training data i,j Defined as the state of the reinforcement learning sub-model, A represents the random action vector A' in the second training data i,j Defined as the action of the reinforcement learning sub-model, the Q-value function is used to evaluate the Q-value of the state-action binary;

[0167] S5-2, select the state-action pair (X' i,j ,A' i,j );

[0168] S5-3, in the current random state vector X' i,j Next, execute the random action vector A' i,j ;

[0169] S5-4, when executing random action vector A' i,j After that, get the immediate reward r(X' i,j ,A' i,j );

[0170] The expression of immediate reward is:

[0171] r(X' i,j ,A' i,j )=-α·(NOx out ) 2 -β·(NH 3,escape ) 2 ;

[0172] Among them, NOx out Indicates NOx emission concentration, used to characterize the concentration of nitrogen oxides that are not reduced in the SCR reaction, NH 3,escape Indicates NH 3, Escape concentration, used to characterize the residual ammonia concentration after the SCR reaction;

[0173] S5-5. According to the order of the perturbation numbers, select the next adjacent random state vector from the sorted random state set as the next state X' i+1,j ;

[0174] S5-6, according to the selected state action tuple (X' i,j ,A' i,j ), immediate reward r(X' i,j ,A' i,j ) and the next state X' i+1,j , update the Q value;

[0175] The Q value update is expressed as:

[0176] Q new ←Q(X′ i,j ,A′ i,j ;θ)+α(r(X′ i,j ,A′ i,j )+γmax a' Q(X′ i+1,j ,a;θ)-Q(X′ i,j ,A i,j;θ));

[0177] Among them, θ represents the model parameter set; a' represents all possible actions, which are used to calculate the next state X' i+1,j The Q value of the optimal action under a' Q(X' i+1,j ,a';θ);max a' Q(X' i+1,j ,a';θ) means that in the next X' i+1,j The Q value of the optimal action selected under the condition; α is used to control the step size of each Q value update, ranging from 0 to 1, r(X' i,j ,A' i,j ) means in state X' i,j Next take action A' i,j The immediate feedback obtained after the event, γ represents the discount factor, which indicates the influence of future rewards on the current Q value, and the value ranges from 0 to 1; the closer γ is to 1, the more emphasis is placed on long-term rewards, and the closer it is to 0, the more emphasis is placed on immediate rewards; that is, in state X' i+1,j The action with the highest Q value selected next indicates the highest expected reward that can be obtained in the next state; Q new In state X' i,j Next take action A' i,j The updated Q value, Q(X' i,j ,A' i,j ; θ) means that under the model parameter θ, in state X' i,j Next take action A' i,j Q value before update;

[0178] S5-7. Repeat S5-2 to S5-6 until the convergence condition is reached to obtain the denitrification optimization model.

[0179] Exemplarily, the convergence condition is ΔQ<∈, that is, the Q value change is less than a set threshold ∈, or the maximum number of training times is obtained, indicating that the model parameters of the reinforcement learning sub-model at this time have reached convergence. The reinforcement learning sub-model with this model indicates that it has learned the optimal control strategy to output the optimal action, which is the denitrification optimization model.

[0180] After inputting several operating state parameters of the current thermal power unit, the denitrification optimization module executes the optimal control strategy and selects the current optimal control action; that is, outputs the optimal operating control parameters of the thermal power unit in SCR denitrification optimization.

[0181] Exemplarily, during the training phase, this embodiment selects flue gas temperature, nitrogen oxide concentration, ammonia injection rate, oxygen concentration, exhaust gas flow rate, catalyst bed pressure difference and ammonia slip concentration. Then, at least these operating status parameters need to be input into the denitrification optimization model.

[0182] Of course, in different design schemes, the core operating state parameters, such as nitrogen oxide concentration and ammonia injection volume, can be selected during the training phase, while general parameters or parameters that do not change for a long time or have a low change range can be omitted or preset. For example, the catalyst bed pressure difference usually does not change much, so it can be tested regularly or set to a fixed value; the ammonia escape concentration mainly monitors the escape of ammonia, which can be preset or collected infrequently.

[0183] In reinforcement learning training, through Q-value updating and immediate reward calculation, the reinforcement learning sub-model is guided to gradually converge to the optimal control strategy, realizing the adaptive optimization control capability of the denitrification system under variable working conditions.

[0184] The present invention uses a denitrification optimization control model training method, which can not only obtain the direct laws between the operating state parameters and the operational control parameters, but also capture the timing laws on a long time scale; it solves the problem that traditional control models are difficult to adapt to complex dynamic environments, so that the denitrification optimization system can fully consider the historical operating status and trends when performing operational control, thereby improving the operating stability and long-term efficiency of the denitrification system.

[0185] On the other hand, the present invention introduces a local disturbance mechanism so that the training data can not only retain the time series relationship of some operating state parameter data of the thermal power unit in the denitrification optimization control, but also introduce exploratory data to expand the training data set without losing the time series information to improve the robustness.

[0186] On the other hand, the present invention uses the random state vector and random action vector constructed by the operating state parameters in the denitrification optimization control as the state-action binary group, and uses them to train the reinforcement learning model, and then learns the optimal operation control strategy of the thermal power unit in the denitrification optimization control, so that the model can adaptively select the optimal operation control parameters during operation, realize the control of NOx concentration and ammonia escape concentration, reduce the use of ammonia, reduce costs, and improve denitrification efficiency.

[0187] In summary, the present invention adopts a staged training strategy of self-supervised training based on a time series model and reinforcement learning during the training process, so that the final denitration optimization model is not only suitable for dynamic working condition adjustment, but also can flexibly adjust the operating parameters according to actual conditions. It not only effectively improves the denitration efficiency and ammonia utilization rate of the denitration system, but also significantly improves the operational stability of the denitration optimization control system.

[0188] The above embodiments can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0189] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division of a waterway underwater terrain change analysis system and method. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A thermal power unit SCR denitrification optimization control system, characterized in that: include: A current operating state parameter acquisition unit is used to acquire several current operating state parameters of the thermal power unit; A vector standardization unit, used for obtaining a current state vector of the thermal power unit after standardizing the plurality of operating state parameters; A model input and output unit, used for receiving a current state vector as an input of a denitration optimization model, and executing a trained optimal control strategy to select an optimal control action of the thermal power unit; Wherein, the denitrification optimization model is obtained after the time series sub-model and the reinforcement learning sub-model are trained in stages; The optimal control action of the denitration optimization model is characterized by: the optimal operating control parameters of the thermal power unit in the SCR denitration optimization.

2. The SCR denitration optimization control system for a thermal power unit according to claim 1 is characterized in that: The stage training steps of the denitrification optimization model include: S1. Iterate the time series sub-model using the first training data to obtain an action prediction model; wherein the first training data is characterized as: a historical training set including N historical training samples; S2. Locally perturb and number the first training data to obtain a random state set including a random state vector; S3, inputting the random state set including the random state vector into the action prediction model to obtain a random action set including the random action vector; S4, combining the random state set and the random action set to obtain second training data after data enhancement; S5. Use the second training data to iteratively reinforce the learning sub-model to obtain the denitration optimization model.

3. The SCR denitration optimization control system for a thermal power unit according to claim 2 is characterized in that: The step of iterating the time series sub-model using the first training data includes: S1-1, obtaining a historical training set containing N historical training samples; S1-2, inputting a number of historical training samples into the time series sub-model in batches for self-supervised iterative training until the historical training set is traversed several times; S1-3. After traversing the historical training set several times, derive the temporal sub-model of the current parameters as the action prediction model.

4. The SCR denitration optimization control system for a thermal power unit according to claim 3 is characterized in that: The steps of obtaining a historical training set containing N historical training samples include: S1-1-1, collecting N operating status parameters and corresponding N operation control parameters in the historical database of the thermal power unit in a fixed-size time window according to the time sequence; and assigning unique time sequence numbers to the operating status parameters and the corresponding operation control parameters; S1-1-2, standardize the N operating state parameters and operation control parameters that have been assigned sequential numbers and construct them into sequential state vectors and sequential action vectors respectively; S1-1-3, using the time series state vector as an input feature and the time series action vector as a target label to construct the historical training sample; S1-1-4. Construct N historical training samples into the historical training set; wherein any historical training sample has an independent time series number.

5. The SCR denitration optimization control system for a thermal power unit according to claim 2 is characterized in that: The step of performing local perturbation numbering on the first training data comprises: S2-1, sorting the time sequence state vectors according to the time sequence numbers to obtain a time sequence state set including N time sequence state vectors; wherein the time sequence state vectors include M operating state parameters of the thermal power unit; S2-2, according to a preset attribute arrangement rule, assigning continuous attribute numbers to a number of operating state parameters of each time series state vector in the time series state set, and sorting them according to the attribute numbers to obtain time series state vectors with continuous attribute numbers; S2-3, substitute the N time series state vectors into the local perturbation formula to obtain N*M random state vectors and the corresponding post-perturbation numbers; The local perturbation formula is: X' i,j =X ((i+j×k-1)modN)+1,j ,in Where i is the timing number of the timing state vector, j is the attribute number of the operating state parameter, N is the total number of timing numbers, M is the total number of attribute numbers, k is the offset that changes with the value of N and is used to control the local disturbance range; ((i+j×k-1)modN)+1 represents the post-disturbance number generated within the timing number range, which ranges from 1 to N cycles; X' i,j is a random state vector generated by local disturbance of the jth operating state parameter in the i-th time series state vector; S2-4, sort the N*M random state vectors according to the numbers after the disturbance, and define them as the random state set; wherein the expression of the random state set S is: S = {X' i,j |i∈{1,2,…,N},j∈ { 1,2,…,M}}.

6. The SCR denitration optimization control system for a thermal power unit according to claim 2, characterized in that: The step of inputting a random state set including a random state vector into an action prediction model to obtain a random action set including a random action vector comprises: S3-1, inputting the random state vectors in the random state set into the action prediction model in sequence; S3-2, the action prediction model performs forward propagation and outputs a corresponding random action vector; S3-3, repeat S3-1 and S3-2 until all random state vectors are input, and construct a random action set containing all random action vectors; The expression of the random action set A is: A={A' i,j ∣i∈{1,2,…,N},j∈{1,2,…,M}}; Among them, A' i,j Denotes the corresponding random state vector X' i,j A random action vector.

7. The SCR denitration optimization control system for a thermal power unit according to claim 2, characterized in that: The step of inputting a plurality of historical training samples in batches into the time series sub-model for self-supervised iterative training includes: S1-2-1, using the time series state vector as a feature variable, performing forward propagation to obtain a time series action prediction value; S1-2-2, calculating the mean square error loss of the temporal action prediction value and the temporal action vector according to the loss function; The expression of the loss function is: in, represents the mean square error loss between the temporal action prediction value and the temporal action vector, N is the number of samples of the temporal state vector, and Y i is a time sequence action vector constructed by several operation control parameters in the historical database, It is the predicted value of the time-series action based on the output of the time-series state vector; S1-2-3, calculate the parameter gradient of the mean square error loss relative to the timing sub-model; S1-2-4, using the gradient descent algorithm to update the model parameters to obtain updated model parameters; S1-2-5. Execute S1-2-1 and S1-2-2 using the updated model parameters until the mean square error loss reaches a preset mean square error loss threshold.

8. The SCR denitration optimization control system for a thermal power unit according to claim 2, characterized in that: The step of combining the random state set and the random action set to obtain the second training data after data enhancement includes: S4-1. Sort each random state vector in the random state set according to the perturbation number to obtain a sorted random state set. S4-2, according to the order of the perturbation numbers, match the sorted random state set with the corresponding random actions in the random action set to construct a state-action tuple; S4-3. Combine all state-action pairs to form the second training data containing N*M state-action pairs.

9. The SCR denitration optimization control system for a thermal power unit according to claim 2, characterized in that: The step of iteratively reinforcing the learning sub-model using the second training data comprises: S5-1. Initialize the model parameters of the reinforcement learning sub-model and define the immediate reward function r(S,A) and the Q-value function Q(S,A); Among them, S represents the random state vector X' in the second training data i,j Defined as the state of the reinforcement learning sub-model, A represents the random action vector A' in the second training data i,j Defined as the action of the reinforcement learning sub-model, the Q-value function is used to evaluate the Q-value of the state-action binary; S5-2, select a state-action pair (X') from the second training data i,j ,A' i,j ); S5-3, in the current random state vector X' i,j Next, execute the random action vector A' i,j ; S5-4, when executing random action vector A' i,j After that, get the immediate reward r(X' i,j ,A' i,j ); The expression of the instant reward is: r(X' i,j ,A' i,j )=-α·(NOx out ) 2 -β·(NH 3,escape ) 2 ; Among them, NOx out Indicates NOx emission concentration, which is used to characterize the concentration of nitrogen oxides that are not reduced in the SCR reaction. 3,escape Indicates NH3, escape concentration, used to characterize the residual ammonia concentration after SCR reaction; S5-5. According to the order of the perturbation numbers, select the next adjacent random state vector from the sorted random state set as the next state X' i+1,j ; S5-6, according to the selected state action tuple (X' i,j ,A' i,j ), immediate reward r(X' i,j ,A' i,j ) and the next state X' i+1,j , update the Q value; The Q value update is expressed as: Q new ←Q(X′ i,j ,A′ i,j ;θ)+α(r(X′ i,j ,A i,j )+γmax a' Q(X′ i+1,j ,a;θ)-Q(X′ i,j ,A i,j ;i)); Among them, θ represents the model parameter set; a' represents all possible actions, which are used to calculate the next state X' i+1,j The Q value of the optimal action under ' Q(X' i+1,j ,a';θ);max a' Q(X' i+1,j ,a';θ) means that in the next X' i+1,j The Q value of the optimal action selected under the condition; α is used to control the step size of each Q value update, ranging from 0 to 1, r(X' i,j ,A' i,j ) means in state X' i,j Next take action A' i,j The immediate feedback obtained after the reward is given, γ represents the discount factor, which indicates the influence of future rewards on the current Q value, and the value ranges from 0 to 1. The closer γ is to 1, the more emphasis is placed on long-term rewards, and the closer it is to 0, the more emphasis is placed on immediate rewards. Q new In state X' i,j Next take action A' i,j The updated Q value, Q(X' i,j ,A' i,j ; θ) means that under the model parameter θ, in state X' i,j Next take action A' i,j The Q value before update; S5-7. Repeat S5-2 to S5-6 until the convergence condition is reached to obtain the denitration optimization model.

Images