A boiler wall temperature increment prediction method of multi-time graph convolution bidirectional transformer

Abstract

The present application provides a boiler wall temperature increment prediction method of a multi-time graph convolutional bidirectional Transformer, and the steps of the boiler wall temperature increment prediction method of the multi-time graph convolutional bidirectional Transformer are as follows: first, data is preprocessed, then the preprocessed data is analyzed by Pearson correlation coefficient, and features with high correlation with output temperature are screened out; the data of boiler 1 is defined as data set 1, and the data of boiler 2 is defined as data set 2; data set 1 is used to train a convolutional bidirectional Transformer network, then based on the method of incremental learning, data set 2 is continuously input to the incremental convolutional bidirectional Transformer to obtain a prediction value and update the network parameters. The boiler wall temperature increment prediction method of the multi-time graph convolutional bidirectional Transformer can accurately predict the temperature of the boiler wall of data set 1, and without retraining the network, a higher accuracy can be achieved on data set 2, saving computing resources and improving energy utilization.

Description

Technical Field

[0001] This invention belongs to the fields of power system thermal power generation, artificial intelligence, time series, image convolution and incremental learning, and relates to a method for predicting boiler wall temperature increments using a multi-temporal graph convolutional bidirectional Transformer, which is applicable to boiler wall temperature prediction in power system thermal power generation. Background Technology

[0002] Furnace wall temperature is affected by a variety of complex factors. Existing patents, such as the one for predicting the furnace outer wall temperature of a high-temperature reheater in an ultra-supercritical coal-fired boiler (application number 2020107666428, dated August 3, 2020) and the one published by Linfei Yin et al. in Energy, Vol. 238, p. 121657 in 2022, are both improved methods based on a single model. These methods have the problem that a single model cannot capture complex time series data well.

[0003] Furthermore, the patent application No. 2022112992233 dated October 24, 2022, entitled "A Method for Predicting Boiler Wall Temperature Using Multi-Time Graph Convolutional Attention Networks," and the paper "Integrated Adversarial Long Short-Term Memory Deep Networks for Reheater Tube Temperature Forecasting of Ultra-Supercritical Turbo-Generators" published by Linfei Yin et al. in Applied Soft Computing, Vol. 142, p. 110347, 2023, both utilize historical data for training and lack the ability to dynamically update parameters. When using the two trained models mentioned above to predict the wall temperature of other boilers, the performance will significantly decrease, requiring the retraining of a network adapted to the new boiler, increasing computational load and lacking adaptability.

[0004] Therefore, this invention proposes a method for predicting boiler wall temperature increments using a multi-time graph convolutional bidirectional Transformer to address the problems that single models cannot handle complex time series data well and that old models cannot adapt to new data. Summary of the Invention

[0005] A method for incremental prediction of boiler wall temperature using multi-temporal graph convolutional bidirectional Transformer is proposed. This method combines convolutional networks, bidirectional Transformer, and incremental learning for boiler wall temperature prediction. The method improves the model's generality and has the ability to accurately predict boiler wall temperature, enabling efficient and safe boiler operation. The steps in its application are as follows:

[0006] Step (1): Collect data on boiler wall temperature, coal feed rate, pipe inlet steam temperature, damper opening, water valve opening, generator power and total air volume, and then preprocess the data by deleting abnormal data and replacing missing data with the average of the data before and after.

[0007] Step (2): Use the Pearson correlation coefficient method to filter features of the collected relevant data, retaining highly correlated features and removing low-correlation features. The correlation coefficient is... for:

[0008] (1)

[0009] in, For the first One feature input; for The average value; for The average value;

[0010] The correlation coefficient ranges from -1 to 1, with positive values ​​indicating positive correlation and negative values ​​indicating negative correlation. Based on the above correlation coefficient calculation results, 0.6 is selected as the benchmark value. Features with correlation coefficients less than 0.6 are removed, and the filtered data form a new dataset. The data of boiler 1 is defined as dataset 1, and the data of boiler 2 is defined as dataset 2.

[0011] Step (3): Map dataset 1 and dataset 2 to grayscale values ​​in the range [0, 255], where Time of the first Gray values ​​of each feature for:

[0012] (2)

[0013] in, for The first moment One feature; For the first The minimum value of each feature; For the first The maximum value of each feature;

[0014] The mapped gray values ​​are arranged in chronological order, and a fixed-length row of data is regarded as a map block. Multiple map blocks constitute a two-dimensional map.

[0015] Step (4): Input the graph formed by dataset 1 into a one-dimensional convolutional neural network:

[0016] (3)

[0017] in, The first in the atlas The output after convolving a graph block; For the first Features of each spectral block; It is a one-dimensional convolution matrix; The length of the map block;

[0018] Step (5): Input the convolutional graph into a bidirectional Transformer for embedding and encoding:

[0019] (4)

[0020] in, For position encoding functions; For time steps; Dimensions of the input data; For the dimension of embedding; It is a sine function;

[0021] Step (6): Embed the encoded matrix The input is fed into multiple different self-attention systems, and the self-attention systems obtain the query matrix through three linear transformations. Key-value matrix Sum matrix :

[0022] (5)

[0023] (6)

[0024] (7)

[0025] in, , and Query matrix Key-value matrix Sum matrix The weights; It is a linear mapping function;

[0026] Step (7): Using the query matrix obtained above Key-value matrix Sum matrix Calculate the output of self-attention The outputs of multiple self-attention methods are concatenated to obtain the output of multi-head attention. :

[0027] (8)

[0028] (9)

[0029] in, It is a normalized exponential function; Key-value matrix transpose; Key-value matrix The dimension; This is a concatenation function; Weights for multi-head attention; , and They are the first, the second, and the third, respectively. Each self-attention output, among which The number of self-attention outputs;

[0030] Step (8): The encoder consists of multiple sub-layers. Each sub-layer undergoes summation and layer normalization operations to output the multi-head attention. The output of the sublayer is obtained through summation and layer normalization. :

[0031] (10)

[0032] in, For layer normalization function;

[0033] Step (9): Output the sub-layer The input is fed into the feedforward neural network, and then summation and layer normalization operations are performed again to obtain the encoder output. :

[0034] (11)

[0035] (12)

[0036] in, It is a feedforward neural network; For activation functions;

[0037] Step (10): Output the encoder The input is fed into the linear layer to obtain the hidden state of the first Transformer in the first layer. :

[0038] (13)

[0039] Step (11): The hidden state of each Transformer in the first layer will be used as the input of each Transformer in the next layer. Then, after the same calculation as the first layer Transformer, the predicted value of the convolutional bidirectional Transformer is obtained. ;

[0040] Step (12): Calculate the network's loss function and continuously update the parameters. After reaching the specified number of iterations, end the training to obtain the convolutional bidirectional Transformer network trained on dataset 1, where the loss function uses the root mean square error. :

[0041] (14)

[0042] in, This represents the number of iterations used to train the network in dataset 1; For the first dataset 1 One predicted value; For the first data set 1 One true value;

[0043] Step (13): To accurately predict the furnace wall temperature of dataset 2 while ensuring high accuracy of dataset 1 on the incremental bidirectional convolutional Transformer network, an extended network structure is used in incremental learning to construct the incremental bidirectional convolutional Transformer network. The incremental bidirectional convolutional Transformer network is based on the bidirectional convolutional Transformer network trained on dataset 1, with another network having the same parameters as the bidirectional convolutional Transformer network connected in parallel. The parameters of the bidirectional convolutional Transformer network trained on dataset 1 are frozen and do not participate in the parameter update of the incremental bidirectional convolutional Transformer network. When dataset 2 is input into the incremental bidirectional convolutional Transformer network, only the parameters of the newly added parallel network are updated. Then, the outputs of the two bidirectional convolutional Transformers are fed into a fully connected layer to obtain the predicted value of the incremental bidirectional convolutional Transformer network. .

[0044] The present invention has the following advantages and effects compared with the prior art:

[0045] (1) The existing patent application No. 2020107666428 filed on August 3, 2020, on a method for predicting the outer wall temperature of a high-temperature reheater in an ultra-supercritical coal-fired boiler, and the multi-feature-scale fusion temporal convolution networks for metal temperature forecasting of ultra-supercritical coal-fired power plant reheater tubes published by linfei yin et al. in Energy, Vol. 238, p. 121657 in 2022, cannot handle complex time series data well. However, this invention combines convolutional networks and Transformers. Convolutional networks extract local features, while Transformers capture global dependencies, which can process information in time series more comprehensively and improve prediction accuracy.

[0046] (2) The existing patent application No. 2022112992233, filed on October 24, 2022, on a method for predicting boiler wall temperature using a multi-temporal graph convolutional attention network, and the paper "Integrated adversarial long short-term memory deep networks for reheater tube temperature forecasting of ultra-supercritical turbo-generators" published by linfei yin et al. in Applied Soft Computing, Vol. 142, p. 110347 in 2023, have the problem of not being able to adapt to new data. However, the incremental learning proposed in this invention can directly apply the trained model to new data and continuously update the parameters without retraining, thus saving computational resources while having good prediction accuracy.

[0047] (3) The boiler wall temperature increment prediction method proposed in this invention using a multi-time graph convolutional bidirectional Transformer accurately predicts the boiler wall temperature, providing a guarantee for the efficient and safe operation of coal-fired boilers. Attached Figure Description

[0048] Figure 1 This is a framework diagram for predicting boiler wall temperature using the method of the present invention.

[0049] Figure 2 This is a diagram of the bidirectional convolutional Transformer framework of the method of this invention.

[0050] Figure 3 This is a diagram of the incremental learning framework of the method of this invention. Detailed Implementation

[0051] This invention proposes a method for predicting boiler wall temperature increments using a multi-time graph convolutional bidirectional Transformer, which is described in detail below with reference to the accompanying drawings:

[0052] Figure 1 This is a framework diagram for predicting boiler wall temperature using the method of this invention. First, the collected data is preprocessed, outlier data is removed, missing data is replaced with the mean of the preceding and following data, and features are filtered using the Pearson correlation coefficient. A correlation coefficient benchmark of 0.6 is selected, and features with correlation coefficients less than 0.6 are removed. Data from boiler 1 is designated as dataset 1, and data from boiler 2 is designated as dataset 2. Dataset 1 is used to train a bidirectional convolutional Transformer network. The root mean square error of the loss function between the predicted and actual values ​​is calculated, and the parameters are continuously updated. After reaching a specified number of iterations, training ends, resulting in the trained bidirectional convolutional Transformer network for dataset 1. Dataset 2 is input into an incremental bidirectional convolutional Transformer network, which updates the network parameters in real time and outputs the prediction results. .

[0053] Figure 2 This is a diagram of the convolutional bidirectional Transformer framework of the method of this invention. The convolutional bidirectional Transformer extracts features from the data through one-dimensional convolution before feeding them into the bidirectional Transformer for prediction. First, dataset 1 and dataset 2 are mapped to gray values ​​in the range [0, 255]. The mapped gray values ​​are arranged in chronological order, and a fixed-length row of data is considered a map patch; multiple map patches constitute a two-dimensional map. Then, the map formed from dataset 1 is input into the first-layer bidirectional Transformer after one-dimensional convolution and ReLU activation. After the input data is fed into the Transformer, embedding and positional encoding are performed first, and then the data is fed into multiple different self-attention layers. Each self-attention layer undergoes three linear transformations to obtain a query matrix. Key-value matrix Sum matrix The query matrix is ​​calculated using multiple attention mechanisms. Key-value matrix Sum matrix The output of multi-head attention is obtained by splicing. Furthermore, the output of multi-head attention is the same as the input of the Transformer. The dimensions are the same. The output of multi-head attention is the same as the output of positional encoding. The sums form a residual connection, and then layer normalization is performed to obtain the output of the first sub-layer. The output of the first sub-layer is input into the feedforward neural network, passing through two linear layers and a ReLU activation function to obtain the output of the feedforward neural network. The output is then added to the output of the first sublayer, and then layer normalization is performed again to obtain the encoder output. Finally, the encoder outputs... After passing through a linear layer, we obtain the hidden state of the first-layer Transformer. The hidden states of each Transformer in the first layer serve as inputs to each Transformer in the next layer, fully considering information from the previous time step. The second Transformer undergoes the same computations as the first layer, yielding the final output of the bidirectional convolutional Transformer. .

[0054] Figure 3 This is a diagram of the incremental learning framework of the method of this invention. First, dataset 1 is input into a bidirectional convolutional Transformer for training. After a specified number of iterations, training is completed, resulting in a trained bidirectional convolutional Transformer network. To accurately predict the furnace wall temperature of dataset 2 while ensuring good performance of dataset 1 on the incremental bidirectional convolutional Transformer network, the method of extending the network structure in incremental learning is used to construct the incremental bidirectional convolutional Transformer network. The incremental bidirectional convolutional Transformer network is based on the bidirectional convolutional Transformer network trained on dataset 1, with another network having the same parameters as the original bidirectional convolutional Transformer network connected in parallel. The parameters of the original bidirectional convolutional Transformer network trained on dataset 1 are frozen and do not participate in the parameter updates of the incremental bidirectional convolutional Transformer. Dataset 2 is input into the incremental bidirectional convolutional Transformer network, and only the parameters of the newly added parallel network are updated. Then, the outputs of the two bidirectional convolutional Transformers are fed into a fully connected layer to obtain the predicted value of the incremental bidirectional convolutional Transformer network.

[0055] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for predicting boiler wall temperature increments using a multi-temporal graph convolutional bidirectional Transformer, characterized in that, This method combines convolutional networks, bidirectional Transformers, and incremental learning for boiler wall temperature prediction. This approach improves the model's generality and enables accurate prediction of boiler wall temperature, thus ensuring efficient and safe boiler operation. The steps involved in its application are as follows: Step (1): Collect data on boiler wall temperature, coal feed rate, pipe inlet steam temperature, damper opening, water valve opening, generator power and total air volume, and then preprocess the data by deleting abnormal data and replacing missing data with the average of the data before and after. Step (2): Use the Pearson correlation coefficient method to filter features of the collected relevant data, retaining highly correlated features and removing low-correlation features. The correlation coefficient is... for: (1) in, For the first One feature input; for The average value; for The average value; The correlation coefficient ranges from -1 to 1, with positive values ​​indicating positive correlation and negative values ​​indicating negative correlation. Based on the above correlation coefficient calculation results, 0.6 is selected as the benchmark value. Features with correlation coefficients less than 0.6 are removed, and the filtered data form a new dataset. The data of boiler 1 is defined as dataset 1, and the data of boiler 2 is defined as dataset 2. Step (3): Map dataset 1 and dataset 2 to grayscale values ​​in the range [0, 255], where Time of the first Gray values ​​of each feature for: (2) in, for The first moment One feature; For the first The minimum value of each feature; For the first The maximum value of each feature; The mapped gray values ​​are arranged in chronological order, and a fixed-length row of data is regarded as a map block. Multiple map blocks constitute a two-dimensional map. Step (4): Input the graph formed by dataset 1 into a one-dimensional convolutional neural network: (3) in, For the first in the atlas The output after convolving a graph block; For the first Features of each spectral block; It is a one-dimensional convolution matrix; The length of the map block; Step (5): Input the convolutional graph into a bidirectional Transformer for embedding and encoding: (4) in, For position encoding functions; For time steps; Dimensions of the input data; For the dimension of embedding; It is a sine function; Step (6): Embed the encoded matrix The input is fed into multiple different self-attention systems, and the self-attention systems obtain the query matrix through three linear transformations. Key-value matrix Sum matrix : (5) (6) (7) in, , and Query matrix Key-value matrix Sum matrix The weights; It is a linear mapping function; Step (7): Using the query matrix obtained above Key-value matrix Sum matrix Calculate the output of self-attention The outputs of multiple self-attention methods are concatenated to obtain the output of multi-head attention. : (8) (9) in, It is a normalized exponential function; Key-value matrix transpose; Key-value matrix The dimension; This is a concatenation function; Weights for multi-head attention; , and They are the first, the second, and the third, respectively. Each self-attention output, among which The number of self-attention outputs; Step (8): The encoder consists of multiple sub-layers. Each sub-layer undergoes summation and layer normalization operations to output the multi-head attention. The output of the sublayer is obtained through summation and layer normalization. : (10) in, For layer normalization function; Step (9): Output the sub-layer The input is fed into the feedforward neural network, and then summation and layer normalization operations are performed again to obtain the encoder output. : (11) (12) in, It is a feedforward neural network; For activation functions; Step (10): Output the encoder The input is fed into the linear layer to obtain the hidden state of the first Transformer in the first layer. : (13) Step (11): The hidden state of each Transformer in the first layer will be used as the input of each Transformer in the next layer. Then, after the same calculation as the first layer Transformer, the predicted value of the convolutional bidirectional Transformer is obtained. ; Step (12): Calculate the network's loss function and continuously update the parameters. After reaching the specified number of iterations, end the training to obtain the convolutional bidirectional Transformer network trained on dataset 1, where the loss function uses the root mean square error. : (14) in, This represents the number of iterations used to train the network in dataset 1; For the first dataset 1 One predicted value; For the first dataset 1 One true value; Step (13): To accurately predict the furnace wall temperature of dataset 2 while ensuring high accuracy of dataset 1 on the incremental bidirectional convolutional Transformer network, an extended network structure is used in incremental learning to construct the incremental bidirectional convolutional Transformer network. The incremental bidirectional convolutional Transformer network is a network with the same parameters as the bidirectional convolutional Transformer network trained on dataset 1, connected in parallel. The parameters of the bidirectional convolutional Transformer network trained on dataset 1 are frozen and do not participate in the parameter update of the incremental bidirectional convolutional Transformer. When dataset 2 is input into the incremental bidirectional convolutional Transformer network, only the parameters of the newly added parallel network are updated. Then, the outputs of the two bidirectional convolutional Transformers are fed into a fully connected layer to obtain the predicted value of the incremental bidirectional convolutional Transformer network. .

Images