A grid-based PM2.5 concentration prediction method based on transformer-attention recurrent network

Abstract

The present invention discloses a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network. The prediction method introduces attention in the transformer into PredRNN for the spatial correlation and temporal fluctuation of PM2.5 concentration, and proposes a combined transformer attention recurrent prediction network. The core unit MH-LSTM not only contains hidden states H and cell states C, but also designs a new memory state M to increase the information extraction of spatiotemporal information flow in the network. The attention in the transformer adopts global parallel computing, which can process a wider range of spatial information, capture global spatial correlation, and cope with the volatility of PM2.5 concentration in time. At the same time, the multi-head attention mechanism can learn multiple contextual information, reduce the influence of random parameters, and increase the accuracy of spatiotemporal sequence PM2.5 concentration image prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of environmental data prediction, and specifically is a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network. Background Art

[0002] The prediction of PM2.5 concentration and diffusion has a strong interdisciplinary nature and has always been a hot topic in the fields of environment, meteorology, mathematics, geography and computer science. Due to the diversity and complexity of pollutant components, there is often a highly nonlinear relationship between pollution indicators. It is difficult to establish an accurate prediction model using traditional mathematical model methods, and a large amount of data collection and analysis of movement mechanisms are required. In addition, the interference of real-time changes in meteorological conditions increases the difficulty of PM2.5 prediction. The problem of pollutant concentration prediction is a spatiotemporal sequence problem, and future trends are predicted based on analysis of historical data changes. Existing PM2.5 concentration prediction algorithms mainly use RNN, LSTM and other networks to numerically predict the PM2.5 concentration of a site, while gridded PM2.5 concentration data is not only a time series problem, but also contains regional spatial information. LSTM and other networks cannot achieve unified modeling of spatiotemporal information and eliminate the interference of time information fluctuations. Summary of the invention

[0003] In view of the defects of the above-mentioned prior art, the present invention patent designs a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network. The prediction method of the present invention makes full use of the temporal and spatial information of the pollutants themselves to realize PM2.5 concentration prediction, which is helpful for the effective prevention and supporting measures of PM2.5 concentration.

[0004] The technical solution of the present invention to solve the technical problem is to design a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network, characterized in that the prediction method includes the following steps:

[0005] Step 1: Collect the monitoring data of PM2.5 concentration of monitoring stations within the geographical range of 105°-125° east longitude and 30°-50° south latitude to obtain the original micaps grid data of PM2.5 concentration; the original micaps grid data of PM2.5 concentration is a time-space sequence vector Z = [Z1, Z2, Z3, ... Z t …,Z T-1 ,Z T ], It represents the PM2.5 concentration value monitored by all monitoring stations within the region at the tth sampling time point. H represents the number of grid points in longitude, and W represents the number of grid points in latitude. The values ​​of H and W are both 81. The monitoring data of PM2.5 concentration at the monitoring station is sampled at intervals of 3 hours, and the accuracy of micaps grid point data is 0.25*0.25.

[0006] Step 2: The spatiotemporal sequence vector of the original micaps grid data of PM2.5 concentration obtained in step 1 is first interpolated using the Kriging interpolation method to supplement the PM2.5 concentration data at the missing time, and the outliers are rationalized by threshold quality control. Finally, normalization is performed to obtain the historical spatiotemporal sequence data;

[0007] Step 3: According to the seasonal characteristics of PM2.5 concentration in the region, select the data from October to March of the following year from the historical spatiotemporal series data as a data sample, slice each data sample using a sliding window of 48 frames, and each slice consists of 48 consecutive frames; slice data of no less than 2 data samples are used as the training set, and slice data of no less than one data sample is used as the validation set;

[0008] Step 3: Build a joint transformer attention recurrent network, which is composed of an MH-LSTM unit array, where the horizontal MH-LSTM unit is the time dimension, each MH-LSTM unit in the horizontal direction represents a sampling time point, and each MH-LSTM unit in each row in the horizontal direction shares parameters with other MH-LSTM units in the row; the vertical MH-LSTM unit is the space dimension, each MH-LSTM unit in each column does not share parameters with other MH-LSTM units in the column, and the MH-LSTM units in the same column correspond to the same sampling time point;

[0009] In the spatial dimension, the information interaction equation between L MH-LSTM units in a stacked architecture is:

[0010]

[0011] Where: f(·) represents the processing of an MH-LSTM unit, is the input data of the first layer MH-LSTM unit at the tth sampling time point; is the hidden state of the first layer MH-LSTM unit at the t-1th sampling time point, is the memory state of the L-th layer MH-LSTM unit at the t-1-th sampling time point, is the cell state of the first layer MH-LSTM unit at the t-1th sampling time point; is the cell state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the memory state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the hidden state of the MH-LSTM unit of the lth layer at the tth sampling time point, and the value range of l is [1,L];

[0012] The information processing equation of the MH-LSTM unit is as follows:

[0013]

[0014] In formula (1) and formula (2), * is the convolution operation, ⊙ is the matrix multiplication, mha(·) represents the transformer attention mechanism calculation; X t , is the input of the MH-LSTM unit, i t , i′ t is the input gate, f t , f′ t For the forget gate, t is the output gate, tanh and σ are two activation functions, c t is the transitional cell state, m t It is a transitional memory state; is the data prediction value at the t+1th sampling time point;

[0015] The parameters that need to be trained in the joint transformer attention recurrent network are all W and all b in formula (1) and formula (2), as well as the number of layers L of MH-LSTM units in the stacked architecture and the number of attention layers in the attention mechanism; where W zr is a 1×1 parameter matrix, the rest of W are 3×3 parameter matrices, and b is the offset parameter;

[0016] Step 4: Train the joint transformer attention recurrent network model

[0017] Step 4.1: First, initialize the joint transformer attention recurrent network, set the hidden state H, cell state C, and memory state M of each layer of MH-LSTM units at the sampling time point before the first sampling time point to 0, and randomly assign values ​​to all parameters that need to be trained;

[0018] Step 4.2: Input the first 24 consecutive frames of the 48 consecutive frames of the first slice data in the training set into the network model initialized in step 4.1, and use the data prediction value output at the 24th sampling time point as the input at the 25th sampling time point to obtain the data prediction value at the 26th sampling time point. According to this method, the data prediction values ​​of the last 24 consecutive frames are obtained; the data prediction values ​​of the last 24 consecutive frames obtained according to the first 24 consecutive frames in the first slice data and the true values ​​of the last 24 consecutive frames in the slice data are respectively denormalized, and the Huber function is used as the loss function to calculate the prediction loss of the last 24 consecutive frames of the slice data:

[0019]

[0020] In the formula, Y=(y1,…,y t ,…,y 24 ), where y t is the true value of PM2.5 concentration at the t-th sampling time point, is the predicted value of PM2.5 concentration at the t-th sampling time point, δ is the loss parameter, and its value is The mean of

[0021] According to formula (6), the prediction loss of the first slice data is obtained, and back propagation is performed based on the prediction loss to update all parameters that need to be trained. The optimization algorithm uses the Adam optimizer to adjust the values ​​of all parameters that need to be trained to complete the training of a slice data; according to the parameter values ​​of the current network model and the hidden state H, cell state C and memory state M of each layer at the last sampling time point in the previous slice data, the first 24 consecutive frames of the next slice data are input into the network model to obtain the data prediction values ​​of the next 24 consecutive frames; then the prediction loss of the current slice data is calculated, and back propagation is performed again based on the prediction loss to update all parameters that need to be trained; each time the network model processes a slice data, the network parameters will be adjusted once until all slice data in the training set are processed, then the network model completes one training, and the hidden state H, cell state C and memory state M and parameters of the current network model are saved;

[0022] Step 4.3: Input the slice data in the validation set into the network model that has completed one training. According to the saved hidden state H, cell state C, memory state M and parameters, the data prediction values ​​of the next 24 consecutive frames are obtained from the first 24 consecutive frames of each slice data in the validation set. Denormalize the data prediction values ​​and the true values ​​of the next 24 consecutive frames of each slice data in the validation set, and calculate the average MAE' and the average RMSE' of the MAE at each sampling time point under the network model respectively.

[0023] Step 4.4: The hidden state H, cell state C, memory state M and parameters of the network model when completing one training in step 4.2 are used as the initialization conditions of the network model, and the slice data in the training set is input into the network model again, and the operation of step 4.2 is repeated to complete the second training of the network model, and the hidden state H, cell state C, memory state M and parameters of the network model obtained from the second training are used as the initialization conditions of the network model for the next training, until the number of training times reaches the preset value, and the training of the network model is completed; each time the network model completes one training, the operation of step 4.3 is performed once to obtain the MAE' and RMSE' under the network model; the MAE' and RMSE' obtained by each execution of step 4.3 are compared, and the network model corresponding to the minimum value of MAE' and RMSE' is selected as the effective network model;

[0024] Step 5: Obtain the monitoring data of PM2.5 concentration at 24 sampling points three days before the day to be predicted, and process it according to the method described in step 2 to obtain slice data at 24 sampling points; input the slice data of the 24 sampling points into the effective network model selected in step 4.4 to obtain the data prediction values ​​of 24 sampling points in the next three days; perform denormalization on the data prediction values ​​of the last 24 sampling points to obtain the PM2.5 concentration prediction values ​​for the day to be predicted and the next two days.

[0025] Compared with the prior art, the beneficial effect of the present invention is that the prediction method of the present invention introduces the attention in the transformer into the PredRNN for the spatial correlation and temporal fluctuation of PM2.5 concentration, and proposes a joint transformer attention recurrent prediction network (TARNN). The core unit MH-LSTM not only contains hidden states H and cell states C, but also designs a new memory state M to increase the information extraction of spatiotemporal information flow in the network, and can transmit more long-term information to improve the prediction accuracy. The attention in the Transformer adopts global parallel computing, which can process a wider range of spatial information, capture global spatial correlation, and cope with the volatility of PM2.5 concentration in time. At the same time, the multi-head attention mechanism can learn multiple contextual information and reduce the influence of random parameters, thereby increasing the accuracy of spatiotemporal sequence PM2.5 concentration image prediction. The test results show that the prediction method of the present invention has good practical prospects and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a structural diagram of a joint transformer attention recurrent network of an embodiment of the prediction method of the present invention.

[0027] Figure 2 It is a structure and principle diagram of the MH-LSTM unit of the combined transformer attention recurrent network of an embodiment of the prediction method of the present invention.

[0028] Figure 3 It is the structure diagram of the transformer attention mechanism.

[0029] Figure 4 Attention model diagram for transformer attention mechanism.

[0030] Figure 5 This is a line chart of PM2.5 concentration monitored by a monitoring station in Shijiazhuang from 2017 to 2021.

[0031] Figure 6 This is a line chart of PM2.5 concentration monitored by a certain monitoring station in Xingtai from 2017 to 2021. DETAILED DESCRIPTION

[0032] The present invention provides a grid-based PM2.5 concentration prediction method (hereinafter referred to as prediction method) combined with a transformer attention recurrent network, and the prediction method comprises the following steps:

[0033] Step 1: Collect the monitoring data of PM2.5 concentration of monitoring stations within the geographical range of 105°-125° east longitude and 30°-50° south latitude to obtain the original micaps grid data of PM2.5 concentration; the original micaps grid data of PM2.5 concentration is a time-space sequence vector Z = [Z1, Z2, Z3, ... Z t …,Z T-1 ,Z T ], represents the PM2.5 concentration value monitored by all monitoring stations within the region at the tth sampling time point, H represents the number of grid points in longitude, and W represents the number of grid points in latitude. In this embodiment, the values ​​of H and W are both 81; the monitoring data of the PM2.5 concentration of the monitoring station is sampled at intervals of 3 hours, and the accuracy of the micaps grid point data is 0.25*0.25;

[0034] Step 2: The spatiotemporal series vector of the original micaps grid data of PM2.5 concentration obtained in step 1 is first interpolated using the Kriging interpolation method to supplement the PM2.5 concentration data at the missing time, and the outliers are rationalized by threshold quality control. Finally, normalization (using the min-max normalization method to normalize the data between 0 and 1) is used to obtain the historical spatiotemporal series data.

[0035] Step 3: According to the seasonal characteristics of PM2.5 concentration in the region, select the data from October to March of the following year from the historical spatiotemporal series data as a data sample, and slice each data sample using a sliding window of 48 frames. Each slice consists of 48 consecutive frames. One frame corresponds to one sampling time point. The slice data of at least 2 data samples is used as the training set, and the slice data of at least one data sample is used as the validation set.

[0036] Step 3: Build a joint transformer attention recurrent network (TARPN, see Figure 1 ), the joint transformer attention recurrent network is composed of an array of MH-LSTM units, wherein the horizontal MH-LSTM units are time dimensions, each MH-LSTM unit in the horizontal direction represents a sampling time point, and each MH-LSTM unit in each row in the horizontal direction shares (i.e., is the same as) the parameters of other MH-LSTM units in the row; the vertical MH-LSTM units are space dimensions, each MH-LSTM unit in each column does not share parameters with other MH-LSTM units in the column, and the MH-LSTM units in the same column correspond to the same sampling time point.

[0037] For the t-1th sampling time point, the L MH-LSTM units in a column corresponding to the sampling time point extract the input data layer by layer, and the MH-LSTM unit of the Lth layer outputs highly abstract features. This is then mapped back to PM 2.5 The concentration value is predicted and the memory state of the MH-LSTM unit in the Lth layer is Used to initialize the starting memory state of the first layer MH-LSTM unit at the t-th sampling time point;

[0038] In the spatial dimension, the information interaction equation between L MH-LSTM units in a stacked architecture is:

[0039]

[0040] Where: f(·) represents the processing of an MH-LSTM unit, is the input data of the first layer MH-LSTM unit at the tth sampling time point. is the hidden state of the first layer MH-LSTM unit at the t-1th sampling time point, is the memory state of the L-th layer MH-LSTM unit at the t-1-th sampling time point, is the cell state of the first layer MH-LSTM unit at the t-1th sampling time point. is the cell state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the memory state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the hidden state of the MH-LSTM unit of the lth layer at the tth sampling time point, and the value range of l is [1,L].

[0041] For each MH-LSTM unit (its structure see Figure 2 ), it uses ConvLSTM with joint transformer attention when processing spatiotemporal data to reduce the interference of temporal information volatility and enhance the ability to capture spatial context information. The information processing equation of the MH-LSTM unit is as follows:

[0042]

[0043] In formula (1) and formula (2), * is the convolution operation, ⊙ is the matrix multiplication, mha(·) represents the transformer attention mechanism calculation; X t , is the input of the MH-LSTM unit, i t , i′ t is the input gate, f t , f′ t For the forget gate, t is the output gate, tanh and σ are two activation functions, c t is the transitional cell state, m t It is the transitional memory state. It is the data prediction value at the t+1th sampling time point.

[0044] The parameters that need to be trained in the joint transformer attention recurrent network are all W and all b in formula (1) and formula (2), as well as the number of MH-LSTM units in the stacked architecture, and the number of attention layers in the attention mechanism. zr is a 1×1 parameter matrix, the rest of W are 3×3 parameter matrices, and b is the offset parameter.

[0045] Figure 2 Middle: X t , is the input of the MH-LSTM unit, i t , i′ t is the input gate, f t , f′ t For the forget gate, t is the output gate, tanh and σ are two activation functions, c t is the transitional cell state, mt For the transitional memory state, is matrix multiplication, ⊕ is matrix addition, and MHA is the transformer attention mechanism calculation.

[0046] Transformer attention mechanism structure diagram Figure 3 (The Linear in the figure represents the linear layer, also called the fully connected layer, Linear Layer) As shown in the form, given an input sequence H = [h1, ..., h n ], constructing the hidden state of the output layer by focusing on the state of the input layer. Specifically, the input layer H∈R n*d First convert to query vector Q∈R n*d , key vector K∈R n*d The sum vector V∈R n*d , as shown in Formula 3.

[0047]

[0048] Among them, {W Q , W K , W V}∈R d*d is the trainable parameter matrix, and d is the dimension of the input state. Output layer O∈R n*d It is expressed by formula (4):

[0049]

[0050] Here h is the number of attention layers, and ATT(·) is the attention model, such as Figure 4 , which can be expressed as the following formula:

[0051]

[0052] in is the proportional factor. In this embodiment, d k is 24 (i.e., the input sequence length of the network model. When the transformer attention mechanism is applied, the proportional factor can be adjusted according to the different input sequence lengths).

[0053] Step 4: Train the joint transformer attention recurrent network model

[0054] Step 4.1: First, initialize the joint transformer attention recurrent network, set the hidden state H, cell state C, and memory state M of each layer of MH-LSTM units at the sampling time point before the first sampling time point to 0, and randomly assign values ​​to all parameters that need to be trained.

[0055] Step 4.2: Input the first 24 consecutive frames of the 48 consecutive frames of the first slice data in the training set into the network model initialized in step 4.1, and use the data prediction value output at the 24th sampling time point as the input at the 25th sampling time point to obtain the data prediction value at the 26th sampling time point. According to this method, the data prediction values ​​of the last 24 consecutive frames are obtained; the data prediction values ​​of the last 24 consecutive frames obtained according to the first 24 consecutive frames in the first slice data and the true values ​​of the last 24 consecutive frames in the slice data are respectively denormalized, and the Huber function is used as the loss function to calculate the prediction loss of the last 24 consecutive frames of the slice data:

[0056]

[0057] In the formula, Y=(y1,…,y t ,…,y 24 ), where y t is the true value of PM2.5 concentration at the t-th sampling time point, is the predicted value of PM2.5 concentration at the t-th sampling time point, δ is the loss parameter, and its value is The mean of the two. In essence, Huber loss is the absolute error, but when the error is very small, it becomes the mean square error. The error is reduced to a certain level and becomes the mean square error, which is controlled by the loss parameter δ (delta). When Huber loss is between [0-δ, 0+δ], it is equivalent to MSE, and when it is between [-∞, δ] and [δ, +∞], it is MAE.

[0058] According to formula (6), the prediction loss of the first slice data is obtained, and back propagation is performed based on the prediction loss to update all parameters that need to be trained. The optimization algorithm uses the Adam optimizer to adjust the values ​​of all parameters that need to be trained to complete the training of a slice data; according to the parameter values ​​of the current network model and the hidden state H, cell state C and memory state M of each layer at the last sampling time point in the previous slice data, the first 24 consecutive frames of the next slice data are input into the network model to obtain the data prediction values ​​of the next 24 consecutive frames; then the prediction loss of the current slice data is calculated, and back propagation is performed again based on the prediction loss to update all parameters that need to be trained; each time the network model processes a slice data, the network parameters will be adjusted once until all slice data in the training set are processed, then the network model completes one training, and the hidden state H, cell state C and memory state M and parameters of the current network model are saved;

[0059] The specific training process of a slice data is as follows: the slice data at the t-1 sampling time point is input into the MH-LSTM unit of the first layer at the t-1 sampling time point, and according to the hidden state H, cell state C, memory state M and parameters of each layer of MH-LSTM units at the t-2 sampling time point of the network model, referring to formula (1) and formula (2), the t-1 sampling time point of each layer is obtained. in The value of is the data prediction value at the t-th sampling time point; then the slice data at the t-th sampling time point and According to formula (1) and formula (2), we can get That is, the data prediction value at sampling time point t+1; according to the memory state M, hidden state H and cell state C of each layer at the previous sampling time point and the slice data corresponding to the current sampling time point, the memory state M, cell state C and hidden state H of each layer at the current sampling time point are obtained according to formula (1) and formula (2), and the data prediction value of the next sampling time point is obtained, and the above process is repeated continuously, and the data prediction value of the output at the 24th sampling time point is used as the input at the 25th sampling time point to obtain the data prediction value at the 26th sampling time point. Referring to this method, the data prediction values ​​of the last 24 consecutive frames in a slice data are obtained.

[0060] Step 4.3: Input the slice data in the validation set into the network model that has completed one training. According to the saved hidden state H, cell state C, memory state M and parameters, the data prediction values ​​of the next 24 consecutive frames are obtained from the first 24 consecutive frames of each slice data in the validation set. The data prediction values ​​of the next 24 consecutive frames of each slice data in the validation set are denormalized with the true values, and the average value MAE' of MAE (mean absolute error) and the average value RMSE' of RMSE (root mean square error) at each sampling time point under the network model are calculated.

[0061] Step 4.4: Use the hidden state H, cell state C, memory state M and parameters of the network model when completing one training in step 4.2 as the initialization conditions of the network model, and input the slice data in the training set into the network model again, repeat the operation of step 4.2 to complete the second training of the network model, and use the hidden state H, cell state C, memory state M and parameters of the network model obtained from the second training as the initialization conditions of the network model for the next training, until the number of training times reaches the preset value and the training of the network model is completed; each time the network model completes one training, execute the operation of step 4.3 once to obtain the MAE' and RMSE' under the network model; compare the MAE' and RMSE' obtained by each execution of step 4.3, and select the network model corresponding to the minimum value of MAE' and RMSE' as the valid network model.

[0062] Step 5: Obtain the monitoring data of PM2.5 concentration at 24 sampling points three days before the day to be predicted, and process it according to the method described in step 2 to obtain slice data at 24 sampling points; input the slice data of the 24 sampling points into the effective network model selected in step 4.4 to obtain the data prediction values ​​of 24 sampling points in the next three days; perform denormalization on the data prediction values ​​of the last 24 sampling points to obtain the PM2.5 concentration prediction values ​​for the day to be predicted and the next two days.

[0063] This embodiment selects slice data from October 2017 to March 2018 and from October 2018 to March 2019 within the regional scope as training sets, slice data from October 2019 to March 2020 as validation sets, and slice data from October 2020 to March 2021 as test sets. The slice data in the test set are input into the effective network model, and the data prediction values ​​of the subsequent 24 consecutive frames are obtained from the first 24 consecutive frames of each slice data in the test set; the data prediction values ​​of the subsequent 24 consecutive frames of each slice data in the test set are denormalized with the true values, and the MAE' and RMSE' under the network model are calculated, and the results are shown in Table 1.

[0064] The MAE' and RMSE' involved in this patent are respectively the average values ​​of the MAE and RMSE of the PM2.5 concentration at each sampling time point of the last 24 sampling time points of each slice data. One slice data corresponds to 48 sampling time points, i.e., 6 days. The data prediction values ​​at each sampling time point of the last three days are obtained in sequence based on the slice data of the first three days. The PM2.5 concentration data is obtained by denormalizing the true value and the predicted value. Then, the MAE and RMSE of each sampling time point of the last three days can be obtained by further calculation (the PM2.5 concentration data at each sampling time point is a spatiotemporal series); the MAE and RMSE of each sampling time point of the last three days obtained based on each slice data are added and averaged, i.e., the values ​​of MAE' and RMSE' of each sampling time point are obtained. For example, when calculating the MAE' of the first sampling time point of the predicted first day, the MAE of the slice data at the sampling time point can be obtained by calculating the predicted value and the true value of the first sampling time point of the first day predicted by each slice data, that is, the 25th sampling time point; the MAE of each slice data at the sampling time point is added and averaged to obtain the MAE' of the sampling time point. The MAE' and RMSE' of each sampling time point are calculated by referring to this method.

[0065] Table 1 MAE' and RMSE' of the test set

[0066]

[0067]

[0068] When evaluating a prediction type network model, in addition to considering the evaluation index, the prediction time span also needs to be considered. Generally speaking, the longer the prediction time span, the lower the prediction accuracy. According to the evaluation results of the network model shown in Table 1, the prediction accuracy of the prediction method of the present invention is within an acceptable range when the prediction time span is 3 days, which proves that the prediction method of the present invention has good practical prospects and generalization ability.

[0069] It should be noted that the above embodiments are only used to illustrate the pollutant prediction description of the patent of the present invention, rather than limiting the patent of the present invention. Those skilled in the art should understand that the technical solution of the present design can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0070] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network, characterized in that: The prediction method includes the following steps: Step 1: Collect the monitoring data of PM2.5 concentration of monitoring stations within the geographical range of 105°-125° east longitude and 30°-50° south latitude to obtain the original micaps grid data of PM2.5 concentration; the original micaps grid data of PM2.5 concentration is a time-space sequence vector Z = [Z1, Z2, Z3, ... Z t …,Z T-1 ,Z T ], It represents the PM2.5 concentration value monitored by all monitoring stations within the region at the tth sampling time point. H represents the number of grid points in longitude, and W represents the number of grid points in latitude. The values ​​of H and W are both 81. The monitoring data of PM2.5 concentration at the monitoring station is sampled at intervals of 3 hours, and the accuracy of micaps grid point data is 0.25*0.

25. Step 2: The spatiotemporal sequence vector of the original micaps grid data of PM2.5 concentration obtained in step 1 is first interpolated using the Kriging interpolation method to supplement the PM2.5 concentration data at the missing time, and the outliers are rationalized by threshold quality control. Finally, normalization is performed to obtain the historical spatiotemporal sequence data; Step 3: According to the seasonal characteristics of PM2.5 concentration in the region, select the data from October to March of the following year from the historical spatiotemporal series data as a data sample, slice each data sample using a sliding window of 48 frames, and each slice consists of 48 consecutive frames; slice data of no less than 2 data samples are used as the training set, and slice data of no less than one data sample is used as the validation set; Step 3: Build a joint transformer attention recurrent network, which is composed of an MH-LSTM unit array, where the horizontal MH-LSTM unit is the time dimension, each MH-LSTM unit in the horizontal direction represents a sampling time point, and each MH-LSTM unit in each row in the horizontal direction shares parameters with other MH-LSTM units in the row; the vertical MH-LSTM unit is the space dimension, each MH-LSTM unit in each column does not share parameters with other MH-LSTM units in the column, and the MH-LSTM units in the same column correspond to the same sampling time point; In the spatial dimension, the information interaction equation between L MH-LSTM units in a stacked architecture is: Where: f(·) represents the processing of an MH-LSTM unit, is the input data of the first layer MH-LSTM unit at the tth sampling time point; is the hidden state of the first layer MH-LSTM unit at the t-1th sampling time point, is the memory state of the L-th layer MH-LSTM unit at the t-1-th sampling time point, is the cell state of the first layer MH-LSTM unit at the t-1th sampling time point; is the cell state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the memory state of the MH-LSTM unit in the lth layer at the tth sampling time point, is the hidden state of the MH-LSTM unit of the lth layer at the tth sampling time point, and the value range of l is [1,L]; The information processing equation of the MH-LSTM unit is as follows: In formula (1) and formula (2), * is the convolution operation, ⊙ is the matrix multiplication, mha(·) represents the transformer attention mechanism calculation; X t , is the input of the MH-LSTM unit, i t , i′ t is the input gate, f t , f′ t For the forget gate, t is the output gate, tanh and σ are two activation functions, c t is the transitional cell state, m t It is a transitional memory state; is the data prediction value at the t+1th sampling time point; The parameters that need to be trained in the joint transformer attention recurrent network are all W and all b in formula (1) and formula (2), as well as the number of layers L of MH-LSTM units in the stacked architecture and the number of attention layers in the attention mechanism; where W zr is a 1×1 parameter matrix, the rest of W are 3×3 parameter matrices, and b is the offset parameter; Step 4: Train the joint transformer attention recurrent network model Step 4.1: First, initialize the joint transformer attention recurrent network, set the hidden state H, cell state C, and memory state M of each layer of MH-LSTM units at the sampling time point before the first sampling time point to 0, and randomly assign values ​​to all parameters that need to be trained; Step 4.2: Input the first 24 consecutive frames of the 48 consecutive frames of the first slice data in the training set into the network model initialized in step 4.1, and use the data prediction value output at the 24th sampling time point as the input at the 25th sampling time point to obtain the data prediction value at the 26th sampling time point. According to this method, the data prediction values ​​of the last 24 consecutive frames are obtained; the data prediction values ​​of the last 24 consecutive frames obtained according to the first 24 consecutive frames in the first slice data and the true values ​​of the last 24 consecutive frames in the slice data are respectively denormalized, and the Huber function is used as the loss function to calculate the prediction loss of the last 24 consecutive frames of the slice data: In the formula, Y=(y1,…,y t ,…,y 24 ), where y t is the true value of PM2.5 concentration at the t-th sampling time point, is the predicted value of PM2.5 concentration at the t-th sampling time point, δ is the loss parameter, and its value is The mean of According to formula (6), the prediction loss of the first slice data is obtained, and back propagation is performed based on the prediction loss to update all parameters that need to be trained. The optimization algorithm uses the Adam optimizer to adjust the values ​​of all parameters that need to be trained to complete the training of a slice data; according to the parameter values ​​of the current network model and the hidden state H, cell state C and memory state M of each layer at the last sampling time point in the previous slice data, the first 24 consecutive frames of the next slice data are input into the network model to obtain the data prediction values ​​of the next 24 consecutive frames; then the prediction loss of the current slice data is calculated, and back propagation is performed again based on the prediction loss to update all parameters that need to be trained; each time the network model processes a slice data, the network parameters will be adjusted once until all slice data in the training set are processed, then the network model completes one training, and the hidden state H, cell state C and memory state M and parameters of the current network model are saved; Step 4.3: Input the slice data in the validation set into the network model that has completed one training. According to the saved hidden state H, cell state C, memory state M and parameters, the data prediction values ​​of the next 24 consecutive frames are obtained from the first 24 consecutive frames of each slice data in the validation set. Denormalize the data prediction values ​​and the true values ​​of the next 24 consecutive frames of each slice data in the validation set, and calculate the average MAE' and the average RMSE' of the MAE at each sampling time point under the network model respectively. Step 4.4: The hidden state H, cell state C, memory state M and parameters of the network model when completing one training in step 4.2 are used as the initialization conditions of the network model, and the slice data in the training set is input into the network model again, and the operation of step 4.2 is repeated to complete the second training of the network model, and the hidden state H, cell state C, memory state M and parameters of the network model obtained from the second training are used as the initialization conditions of the network model for the next training, until the number of training times reaches the preset value, and the training of the network model is completed; each time the network model completes one training, the operation of step 4.3 is performed once to obtain the MAE' and RMSE' under the network model; the MAE' and RMSE' obtained by each execution of step 4.3 are compared, and the network model corresponding to the minimum value of MAE' and RMSE' is selected as the effective network model; Step 5: Obtain the monitoring data of PM2.5 concentration at 24 sampling points three days before the day to be predicted, and process it according to the method described in step 2 to obtain slice data at 24 sampling points; input the slice data of the 24 sampling points into the effective network model selected in step 4.4 to obtain the data prediction values ​​of 24 sampling points in the next three days; perform denormalization on the data prediction values ​​of the last 24 sampling points to obtain the PM2.5 concentration prediction values ​​for the day to be predicted and the next two days.

2. According to claim 1, a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network is characterized in that: The specific training process of a slice data is as follows: the slice data at the t-1 sampling time point is input into the MH-LSTM unit of the first layer at the t-1 sampling time point, and according to the hidden state H, cell state C, memory state M and parameters of each layer of MH-LSTM units at the t-2 sampling time point of the network model, referring to formula (1) and formula (2), the t-1 sampling time point of each layer is obtained. in The value of is the data prediction value at the t-th sampling time point; then the slice data at the t-th sampling time point and According to formula (1) and formula (2), we can get That is, the data prediction value at sampling time point t+1; according to the memory state M, hidden state H and cell state C of each layer at the previous sampling time point and the slice data corresponding to the current sampling time point, the memory state M, cell state C and hidden state H of each layer at the current sampling time point are obtained according to formula (1) and formula (2), and the data prediction value of the next sampling time point is obtained, and the above process is repeated continuously, and the data prediction value of the output at the 24th sampling time point is used as the input at the 25th sampling time point to obtain the data prediction value at the 26th sampling time point. Referring to this method, the data prediction values ​​of the last 24 consecutive frames in a slice data are obtained.

3. According to claim 1, a grid-based PM2.5 concentration prediction method combined with a transformer attention recurrent network is characterized in that: MAE' and RMSE' are the average values ​​of MAE and RMSE of PM2.5 concentration at each sampling time point of the last 24 sampling time points of each slice data respectively; one slice data corresponds to 48 sampling time points, i.e. 6 days, and the data prediction values ​​of each sampling time point of the last three days are obtained in turn according to the slice data of the first three days, and the PM2.5 concentration data is obtained by denormalizing the true value and the predicted value, and then the MAE and RMSE of each sampling time point of the last three days are obtained by further calculation; the MAE and RMSE of each sampling time point of the last three days obtained according to each slice data are added and averaged respectively, that is, the values ​​of MAE' and RMSE' of each sampling time point are obtained.

Images