PM2.5 concentration prediction method based on VMD-TCN-BiGRU-SA model

Abstract

The invention provides a PM2.5 (Particulate Matter 2.5) concentration prediction method based on a VMD-TCN-BiGRU-SA model. The method comprises the following steps: acquiring a PM2.5 concentration data set and preprocessing the data set; calculating the Spearman correlation coefficient of the related characteristics in the pre-processed PM2.5 concentration data set and the PM2.5 concentration; variational mode decomposition is carried out on PM2.5 concentration data in the data set after the features which are poor in correlation with the PM2.5 concentration are removed, and n intrinsic mode function components IMF and a residual component RES are obtained; all components obtained through variational mode decomposition are combined into the preprocessed PM2.5 concentration data set; dividing the merged data set into a training set and a test set; a PM2.5 concentration prediction model of the TCN-BiGRU-SA is constructed; the model comprises a TCN network, a BiGRU network and a sparse attention module which are sequentially connected in series. Inputting the training set into the model; and inputting the test set into the trained model to obtain a PM2.5 concentration prediction result. According to the method, the prediction result of the PM2.5 concentration is more accurate, and the generalization ability of the model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of PM2.5 concentration prediction, and in particular to a PM2.5 concentration prediction method based on a VMD-TCN-BiGRU-SA model. Background Art

[0002] As my country pays more and more attention to environmental protection, the public's requirements for air quality are also increasing. Scientifically managing urban air quality and protecting the atmospheric environment have become a key task for environmental protection departments at all levels. At present, PM2.5 concentration is an important indicator for measuring urban air quality. Therefore, building an accurate PM2.5 concentration prediction model has become a research hotspot in the field of environmental science in recent years.

[0003] With the in-depth research and continuous development in fields such as artificial intelligence and data mining, artificial neural networks have been widely used in the prediction field, especially in PM2.5 concentration prediction, due to their advantages such as fast convergence and high prediction accuracy when processing nonlinear data prediction.

[0004] However, in the related art, on the one hand, most methods do not consider the complexity of PM2.5 concentration data and possible noise, do not perform necessary modal decomposition on the data signal, and directly use the original data for prediction. If the original data complexity of the PM2.5 concentration to be predicted is high, the prediction result will be inaccurate.

[0005] In the related art, on the other hand, most methods use one-dimensional convolutional neural networks as the initial feature extractor. One-dimensional convolutional neural networks are a deep learning model that specializes in processing one-dimensional sequence data. They are mainly used to process serialized data such as audio and text. Compared with traditional fully connected neural networks, one-dimensional convolutional neural networks can more effectively capture local relationships in sequence data, so they are widely used in speech recognition, natural language processing, time series prediction and other fields. However, one-dimensional convolutional neural networks have limited ability to extract features of future time series, which will lead to poor performance of the model.

[0006] In addition, in the related technologies, methods based on recurrent neural networks are more common. Recurrent neural networks are a type of neural network with feedback connections that can be used to store and restore patterns. They are mainly used to process sequence data, such as time series data or natural language text. They have memory capabilities and can use time information or context information in the sequence for learning and prediction. The LSTM network, a variant of the recurrent neural network, has been widely used in time series prediction technology. However, the LSTM network has some defects, including the need for a large amount of training data, long training time, and easy overfitting.

[0007] Therefore, we urgently need a more reliable PM2.5 concentration prediction method with higher prediction accuracy to solve the above-mentioned technical problems existing in the above-mentioned related technologies. Summary of the invention

[0008] The present invention aims to address the shortcomings of existing PM2.5 concentration prediction technologies and solve the problems of low accuracy in PM2.5 concentration prediction and weak generalization ability of the model.

[0009] To this end, the purpose of the present invention is to propose a PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model.

[0010] In order to achieve the above-mentioned purpose, the technical solution of the present invention provides a PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model. The prediction method comprises: step S1: obtaining a PM2.5 concentration data set and preprocessing the PM2.5 concentration data set; step S2: calculating the Spearman correlation coefficient between the relevant features in the preprocessed PM2.5 concentration data set and the PM2.5 concentration, so as to eliminate the features with poor correlation with the PM2.5 concentration from the preprocessed PM2.5 concentration data set; step S3: performing variational mode decomposition on the PM2.5 concentration data in the PM2.5 concentration data set processed by step S2 to obtain n intrinsic mode function components IMF and a residual component RES; step S4: merging the various components obtained by variational mode decomposition into the preprocessed PM2.5 concentration data set; step S5: performing the variational mode decomposition according to the preset The merged PM2.5 concentration data set is divided into a training set and a test set in proportion; step S6: constructing a PM2.5 concentration prediction model of TCN-BiGRU-SA; wherein the PM2.5 concentration prediction model of TCN-BiGRU-SA includes a TCN network, a BiGRU network and a sparse attention module which are sequentially connected in series; step S7: inputting the training set into the PM2.5 concentration prediction model of TCN-BiGRU-SA to train the PM2.5 concentration prediction model of TCN-BiGRU-SA; step S8: inputting the test set into the trained PM2.5 concentration prediction model of TCN-BiGRU-SA to obtain a prediction result of PM2.5 concentration.

[0011] Preferably, the mathematical expression corresponding to the Spearman correlation coefficient is:

[0012]

[0013] In formula (1), N is the total amount of data; x i is the value of the i-th data variable x; y iis the value of the i-th data variable y; is the mean value of variable x; is the mean value of variable y; ρ is x i With y i The Spearman correlation coefficient between the two.

[0014] Preferably, the mathematical expression corresponding to the variational mode decomposition of the PM2.5 concentration data in the PM2.5 concentration data set after the processing in step S2 is:

[0015]

[0016] In formula (2), K is the number of components; u k (t) is the single component signal after decomposition; ω k is the center frequency of each single component signal; It means to find partial derivative; δ(t) is the Dirac function; * is the convolution operator; f(t) is the original input signal; st is the abbreviation of subject to, and st means the constraint condition is.

[0017] Preferably, the TCN network TCN is composed of a plurality of residual blocks stacked together; each residual block comprises: a first causal expansion convolution layer, a first WeightNorm layer, a first activation function layer, a first Dropout layer, a second causal expansion convolution layer, a second WeightNorm layer, a second activation function layer, and a second Dropout layer connected in sequence to each other; and the input of the first causal expansion convolution layer is subjected to a data addition operation with the output of the second Dropout layer through an l*1 convolution operation.

[0018] Preferably, the BiGRU network comprises: an input layer, a GRU backward layer, a GRU forward layer, and an output layer connected to each other; the GRU backward layer and the GRU forward layer each comprise: a plurality of GRU networks connected to each other;

[0019] Among them, the mathematical expression corresponding to the calculation process of the BiGRU network is:

[0020]

[0021] In formula (3), w t and v t They are respectively represented as the forward hidden layer state corresponding to the BiGRU network at time t and the reverse hidden state The corresponding weight; b t Represents the bias corresponding to the hidden layer state at time t;

[0022] The input of the current neuron and the output of the previous neuron are used as the input of the GRU network, and the output of the current neuron is calculated by the GRU network; the mathematical expression corresponding to the calculation process of the GRU network is:

[0023] r t =σ(W rx x t +W rh h t-1 +b r )

[0024] z t =σ(W zx x t +W zh h t-1 +b z )

[0025]

[0026]

[0027] In formula (4), x t is the input vector input to the memory unit at time t, i.e., the input of this neuron; h t is the output value of the neuron this time; h t-1 is the output of the previous neuron; z t is the update gate; r t To reset the gate; is the candidate output value of the current neuron; W is the weight matrix; b is the bias vector; ⊙ is the dot multiplication operation at the corresponding position of the element; σ and tanh represent the activation function.

[0028] Preferably, the sparse attention module comprises an input layer, a hidden layer, and an output layer connected to each other; the attention weight coefficient x of the sparse attention module is i The mathematical expression is:

[0029] x i =utanh(ωh i +b) (5)

[0030] In formula (5), u and ω are weight coefficients; b is the bias; h i is the output of the hidden layer;

[0031] And use softmax to adjust the attention weight coefficient x i Normalize, the mathematical expression corresponding to the normalization process is:

[0032]

[0033] In formula (6), a i It is represented as the attention weight corresponding to the output of the i-th hidden layer; e j Represented as the index of the j-th element.

[0034] Preferably, the PM2.5 concentration data set is preprocessed, specifically including: removing outlier data values ​​in the PM2.5 concentration data set; and filling in the removed data values ​​and missing values ​​in the PM2.5 concentration data set using linear interpolation.

[0035] Beneficial effects of the present invention:

[0036] (1) The present invention provides a new PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model. Specifically, the prediction method combines variational mode decomposition, temporal convolutional network, bidirectional gated recurrent unit and sparse attention mechanism, making the prediction result of PM2.5 concentration more accurate, making up for the low prediction accuracy of traditional PM2.5 concentration prediction methods, and providing strong data support for environmental protection.

[0037] (2) The new PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model provided by the present invention can effectively overcome the problem of inaccurate prediction results caused by the high complexity of the original data of PM2.5 concentration, the problem of limited feature extraction ability of one-dimensional convolutional neural network for future time series, and the problem that LSTM network requires a large amount of training data, takes a long time to train, and is prone to overfitting by combining variational mode decomposition, TCN network, BiGRU network, and sparse attention mechanism. It reduces the time and resources required for model training, enhances the accuracy of model prediction, and improves the generalization ability of the model.

[0038] Additional aspects and advantages of the invention will become apparent from the following description, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic flow chart of a PM2.5 concentration prediction method based on a VMD-TCN-BiGRU-SA model according to an embodiment of the present invention is shown;

[0040] Figure 2 A decomposition result signal diagram of decomposing PM2.5 concentration data using variational mode decomposition according to an embodiment of the present invention is shown;

[0041] Figure 3 A schematic block diagram showing the structure of a TCN-BiGRU-SA model according to an embodiment of the present invention is shown;

[0042] Figure 4 A functional structure diagram of a TCN network used in an embodiment of the present invention is shown;

[0043] Figure 5 A functional structure diagram of a GRU network used in an embodiment of the present invention is shown;

[0044] Figure 6 A functional structure diagram of a BiGRU network used in one embodiment of the present invention is shown;

[0045] Figure 7 A schematic diagram showing the functional structure of an attention module used in one embodiment of the present invention is shown;

[0046] Figure 8 A fitting diagram of PM2.5 concentration prediction based on the VMD-TCN-BiGRU-SA model according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0049] Figure 1 FIG. 1 is a schematic flow chart of a PM2.5 concentration prediction method based on a VMD-TCN-BiGRU-SA model according to an embodiment of the present invention. Figure 1 As shown, the PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model includes:

[0050] Step S1: Obtain a PM2.5 concentration data set and preprocess the PM2.5 concentration data set;

[0051] Step S2: Calculate the Spearman correlation coefficient between the relevant features in the preprocessed PM2.5 concentration data set and the PM2.5 concentration, so as to eliminate the features with poor correlation with the PM2.5 concentration from the preprocessed PM2.5 concentration data set;

[0052] Step S3: performing variational mode decomposition on the PM2.5 concentration data in the PM2.5 concentration data set processed in step S2 to obtain n intrinsic mode function components IMF and one residual component RES;

[0053] Step S4: merging the components obtained by variational mode decomposition into the preprocessed PM2.5 concentration dataset;

[0054] Step S5: Divide the merged PM2.5 concentration data set into a training set and a test set according to a preset ratio;

[0055] Step S6: constructing a PM2.5 concentration prediction model of TCN-BiGRU-SA;

[0056] Step S7: inputting the training set into the PM2.5 concentration prediction model of TCN-BiGRU-SA to train the PM2.5 concentration prediction model of TCN-BiGRU-SA;

[0057] Step S8: Input the test set into the trained TCN-BiGRU-SA PM2.5 concentration prediction model to obtain the prediction result of PM2.5 concentration.

[0058] In this embodiment, the PM2.5 concentration prediction model of TCN-BiGRU-SA includes a TCN network, a BiGRU network and a sparse attention module which are sequentially connected in series.

[0059] In this embodiment, step S4 is to merge the components obtained by variational mode decomposition into the data set. Specifically, n intrinsic mode function components IMF and one residual component RES after variational mode decomposition are all merged into the data set to form a new data set, and then the new data set is used for training and testing.

[0060] In this embodiment, step S5 is to divide the merged data set into a training set and a test set. The ratio of deep learning to divide the data set is generally 8:2 (ratio of training set to test set). Therefore, this embodiment divides the merged PM2.5 concentration data set into a training set and a test set according to the preset ratio of 8:2.

[0061] In this embodiment, the present invention provides a new PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model. Specifically, the prediction method combines variational mode decomposition, temporal convolutional network, bidirectional gated recurrent unit and sparse attention mechanism, making the prediction result of PM2.5 concentration more accurate, making up for the low prediction accuracy of traditional PM2.5 concentration prediction methods, and providing strong data support for environmental protection.

[0062] In this embodiment, the new PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model provided by the present invention, by combining variational mode decomposition, TCN network, BiGRU network, and sparse attention mechanism, can effectively overcome the problem of inaccurate prediction results caused by the high complexity of the original data of PM2.5 concentration, the problem of limited feature extraction ability of one-dimensional convolutional neural network for future time series, and the problem that LSTM network requires a large amount of training data, long training time, and easy to overfit. It reduces the time and resources required for model training, enhances the accuracy of model prediction, and improves the generalization ability of the model.

[0063] In one embodiment of the present invention, the mathematical expression corresponding to the Spearman correlation coefficient is:

[0064]

[0065] In formula (1), N is the total amount of data; x i is the value of the i-th data variable x; y i is the value of the i-th data variable y; is the mean value of variable x; is the mean value of variable y; ρ is x i With y i The Spearman correlation coefficient between the two.

[0066] In one embodiment of the present invention, the mathematical expression corresponding to the variational mode decomposition of the PM2.5 concentration data in the PM2.5 concentration data set after the processing in step S2 is:

[0067]

[0068] In formula (2), K is the number of components; u k (t) is the single component signal after decomposition; ω k is the center frequency of each single component signal; It means to find partial derivative; δ(t) is the Dirac function; * is the convolution operator; f(t) is the original input signal; st is the abbreviation of subject to, and st means the constraint condition is.

[0069] In one embodiment of the present invention, the TCN network TCN is composed of a plurality of residual blocks stacked together; each residual block includes: a first causal expansion convolution layer, a first WeightNorm layer, a first activation function layer, a first Dropout layer, a second causal expansion convolution layer, a second WeightNorm layer, a second activation function layer, and a second Dropout layer connected in sequence to each other; and the input of the first causal expansion convolution layer is subjected to a data addition operation with the output of the second Dropout layer through a 1*1 convolution operation.

[0070] In one embodiment of the present invention, the BiGRU network includes: an input layer, a GRU backward layer, a GRU forward layer, and an output layer connected to each other; the GRU backward layer and the GRU forward layer each include: a plurality of GRU networks connected to each other;

[0071] Among them, the mathematical expression corresponding to the calculation process of the BiGRU network is:

[0072]

[0073] In formula (3), w t and v t They are respectively represented as the forward hidden layer state corresponding to the BiGRU network at time t and the reverse hidden state The corresponding weight; b t Represents the bias corresponding to the hidden layer state at time t;

[0074] The input of the current neuron and the output of the previous neuron are used as the input of the GRU network, and the output of the current neuron is calculated by the GRU network; the mathematical expression corresponding to the calculation process of the GRU network is:

[0075] r t =σ(W rx x t +W rh h t-1 +b r )

[0076] z t =σ(W zx x t +W zh h t-1 +b z )

[0077]

[0078]

[0079] In formula (4), xt is the input vector input to the memory unit at time t, i.e., the input of this neuron; h t is the output value of the neuron this time; h t-1 is the output of the previous neuron; z t is the update gate; r t To reset the gate; is the candidate output value of the current neuron; W is the weight matrix; b is the bias vector; ⊙ is the dot multiplication operation at the corresponding position of the element; σ and tanh represent the activation function.

[0080] In one embodiment of the present invention, the sparse attention module includes an input layer, a hidden layer, and an output layer connected to each other; the attention weight coefficient x of the sparse attention module is i The mathematical expression is:

[0081] x i =utanh(ωh i +b) (5)

[0082] In formula (5), u and ω are weight coefficients; b is the bias; h i is the output of the hidden layer;

[0083] And use softmax to adjust the attention weight coefficient x i Normalize, the mathematical expression corresponding to the normalization process is:

[0084]

[0085] In formula (6), a i It is represented as the attention weight corresponding to the output of the i-th hidden layer; e j Represented as the index of the j-th element.

[0086] In one embodiment of the present invention, the PM2.5 concentration data set is preprocessed, specifically including: removing each outlier data value in the PM2.5 concentration data set; and filling each removed data value and missing value in the PM2.5 concentration data set using linear interpolation.

[0087] The following is a specific example to illustrate the PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model of the present invention. The specific implementation steps of the PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model are as follows:

[0088] (1) Step S1: Obtain a PM2.5 concentration data set and pre-process the PM2.5 concentration data set. The PM2.5 concentration data set used in this specific embodiment comes from the national urban air quality real-time publishing platform of the China National Environmental Monitoring Center. The samples are 43,800 pieces of air quality monitoring data in Shenyang from 0:00 on January 2, 2010 to 23:00 on December 31, 2014. The data set sampling time interval is 1 hour, and the data set includes environmental data such as PM2.5 concentration, Celsius temperature, humidity, air pressure, combined wind direction, cumulative wind speed, precipitation, etc.

[0089] The PM2.5 concentration data set is preprocessed, specifically including: removing outlier data values ​​in the PM2.5 concentration data set; and using linear interpolation to fill in the removed data values ​​and missing values ​​in the PM2.5 concentration data set.

[0090] (2) Step S2: Calculate the Spearman correlation coefficient between the relevant features in the preprocessed PM2.5 concentration data set and the PM2.5 concentration, so as to eliminate the features with poor correlation with the PM2.5 concentration from the preprocessed PM2.5 concentration data set.

[0091] The mathematical expression corresponding to the Spearman correlation coefficient is:

[0092]

[0093] In formula (1), N is the total amount of data; x i is the value of the i-th data variable x; y i is the value of the i-th data variable y; is the mean value of variable x; is the mean value of variable y; ρ is x i With y i The Spearman correlation coefficient between the two.

[0094] The Spearman correlation coefficient is calculated based on the preprocessed PM2.5 concentration data set, and then the seven key features with coefficients higher than 0.6, including dew point, Celsius temperature, humidity, atmospheric pressure, cumulative wind speed, snowfall and rainfall, are used as input features of the PM2.5 concentration prediction model. The calculation results of the Spearman correlation coefficient are shown in Table 1.

[0095] Table 1 Spearman correlation coefficient calculation results

[0096]

[0097] (3) Step S3: Perform variational mode decomposition on the PM2.5 concentration data in the PM2.5 concentration data set processed in step S2 to obtain n intrinsic mode function components IMF and one residual component RES.

[0098] The mathematical expression corresponding to the variational mode decomposition of the PM2.5 concentration data in the PM2.5 concentration data set processed in step S2 is:

[0099]

[0100] In formula (2), K is the number of components; u k (t) is the single component signal after decomposition; ω k is the center frequency of each single component signal; It means to find partial derivative; δ(t) is the Dirac function; * is the convolution operator; f(t) is the original input signal; st is the abbreviation of subject to, and st means the constraint condition is.

[0101] Variational mode decomposition is an adaptive, non-recursive signal processing method that can decompose complex non-periodic signals into a series of sub-signals with specific center frequencies and limited bandwidths, namely intrinsic mode functions (IMFs). Variational mode decomposition shows higher robustness in terms of sampling and noise. PM2.5 concentration data can be regarded as a nonlinear signal mixed with a lot of noise. Variational mode decomposition is used to decompose the unstable components into multiple relatively stable components to improve the prediction accuracy of the TCN-BiGRU-SA model. The PM2.5 concentration data in the data set is decomposed using variational mode decomposition. The signal diagram of the decomposition result is shown in the figure below. Figure 2 shown.

[0102] (4) Step S4: merge the components obtained by variational mode decomposition into the preprocessed PM2.5 concentration data set. Specifically, the n intrinsic mode function components IMF and one residual component RES after variational mode decomposition are all merged into the data set to form a new data set, and then the new data set is used for training and testing. The fields contained in the data set after the components are merged are shown in Table 2.

[0103] Table 2 Field table of the dataset after component integration

[0104]

[0105] (5) Step S5: Divide the merged PM2.5 concentration dataset into a training set and a test set according to a preset ratio. Step S5 is to divide the merged dataset into a training set and a test set. The ratio of deep learning to divide the dataset is generally 8:2 (ratio of training set to test set). Therefore, in this specific embodiment, the merged PM2.5 concentration dataset is divided into a training set and a test set according to a preset ratio of 8:2.

[0106] (6) Step S6: Construct the PM2.5 concentration prediction model of TCN-BiGRU-SA. ​​Specifically, the PM2.5 concentration prediction model of TCN-BiGRU-SA consists of three modules. Figure 3 As shown in the figure, the PM2.5 concentration prediction model of TCN-BiGRU-SA includes a TCN network, a BiGRU network and a sparse attention module that are connected in series. The first module is the TCN network. As a sequence modeler, TCN is mainly used to capture long-term dependencies and temporal correlations in sequence data. TCN is composed of several stacked residual blocks. Each residual block contains two causal expansion convolutional layers, a batch normalization layer and an activation function layer. By introducing skip connections, some inputs skip the intermediate convolution operations. After the 1*1 convolution operation, it is directly added to the data after the convolution operation in the middle, which can effectively avoid problems such as gradient explosion caused by stacking deep networks. The specific structure of the TCN module is shown in the figure. Figure 4 As shown. Figure 4 As shown, the TCN network TCN is composed of several residual blocks stacked together; each residual block includes: a first causal expansion convolution layer, a first WeightNorm layer, a first activation function layer, a first Dropout layer, a second causal expansion convolution layer, a second WeightNorm layer, a second activation function layer, and a second Dropout layer connected in sequence; and the input of the first causal expansion convolution layer is added to the output of the second Dropout layer through a 1*1 convolution operation.

[0107] The second module is the BiGRU network, which is an improved GRU network that combines the information features of both the forward and backward directions and can fully learn the time series features of the past and future. Figure 6 As shown, the BiGRU network includes: an input layer, a GRU backward layer, a GRU forward layer, and an output layer connected to each other; the GRU backward layer and the GRU forward layer both include: a plurality of GRU networks connected to each other;

[0108] Among them, the mathematical expression corresponding to the calculation process of the BiGRU network is:

[0109]

[0110] In formula (3), w t and v t They are respectively represented as the forward hidden layer state corresponding to the BiGRU network at time t and the reverse hidden state The corresponding weight; b t Represents the bias corresponding to the hidden layer state at time t;

[0111] The input of the current neuron and the output of the previous neuron are used as the input of the GRU network, and the output of the current neuron is calculated through the GRU network;

[0112] like Figure 5 As shown, the mathematical expression corresponding to the calculation process of the GRU network is:

[0113] r t =σ(W rx x t +W rh h t-1 +b r )

[0114] z t =σ(W zx x t +W zh h t-1 +b z )

[0115]

[0116]

[0117] In formula (4), x t is the input vector input to the memory unit at time t, i.e., the input of this neuron; h t is the output value of the neuron this time; h t-1 is the output of the previous neuron; z t is the update gate; r t To reset the gate; is the candidate output value of the current neuron; W is the weight matrix; b is the bias vector; ⊙ is the dot multiplication operation at the corresponding position of the element; σ and tanh represent the activation function.

[0118] The third module is the sparse attention module. Sparse attention is an optimized attention mechanism that reduces the amount of computation and memory consumption by only calculating the similarity between the query vector and some key vectors. The core idea of ​​this mechanism is selective attention, that is, only paying attention to and calculating the relationship between those elements that have a greater impact on the performance of the model, while ignoring those elements with less impact. Figure 7As shown, the sparse attention module includes an input layer, a hidden layer, and an output layer connected to each other; the attention weight coefficient x of the sparse attention module i The mathematical expression is:

[0119] x i =utanh(ωh i +b) (5)

[0120] In formula (5), u and ω are weight coefficients; b is the bias; h i is the output of the hidden layer;

[0121] And use softmax to adjust the attention weight coefficient x i Normalize, the mathematical expression corresponding to the normalization process is:

[0122]

[0123] In formula (6), a i It is represented as the attention weight corresponding to the output of the i-th hidden layer; e j Represented as the index of the j-th element.

[0124] (7) Step S7: Input the training set into the PM2.5 concentration prediction model of TCN-BiGRU-SA to train the PM2.5 concentration prediction model of TCN-BiGRU-SA. ​​Input the training set into the model for training to obtain the optimal model (i.e., the trained PM2.5 concentration prediction model of TCN-BiGRU-SA). Generally speaking, a relatively good effect can be achieved after training the model for about 20 rounds.

[0125] (8) Step S8: Input the test set into the trained TCN-BiGRU-SA PM2.5 concentration prediction model to obtain the prediction result of PM2.5 concentration.

[0126] Finally, the prediction performance index of the PM2.5 concentration prediction model of TCN-BiGRU-SA is calculated to evaluate the prediction performance of the PM2.5 concentration prediction model of TCN-BiGRU-SA. ​​This specific example tests the CNN-BiLSTM-AT model, the CNN-BiGRU-AT model, and the VMD-TCN-BiGRU-SA model, and calculates the mean absolute error (MAE), root mean square error (RMSE), and fitting coefficient (R 2 ) are used to evaluate the prediction performance of the model of this method. Among them, the mathematical expression corresponding to the mean absolute error (MAE) is:

[0127]

[0128]

[0129] In formulas (7), (8) and (9), n is the number of samples, y is i is the actual value, is the predicted value, The calculation results of the prediction performance indicators of the three models are shown in Table 3. The smaller the MAE and RMSE of the above three indicators, the better the prediction performance of the model. 2 The closer it is to 1, the better the prediction performance of the model.

[0130] Table 3 Calculation results of prediction performance indicators of three models

[0131]

[0132] The results show that the PM2.5 concentration prediction model TCN-BiGRU-SA provided by the present invention is the best among the three. The specific reasons are as follows:

[0133] 1. The previous two models did not process and decompose the PM2.5 concentration data, but directly input the data into the initial model for training. The original data contained noise, which made the prediction effect of the model not very good. The model provided by the present invention first performed variational mode decomposition, which has better noise resistance, so that the prediction effect has been significantly improved compared with the previous two methods.

[0134] 2. Both of the previous two models use convolutional neural networks (CNN) as the initial feature extractor, but the one-dimensional convolutional neural network has limited feature extraction capabilities for future time series, which leads to poor performance of the model. The model provided by the present invention uses a temporal convolutional network (TCN) as the initial feature extractor. TCN can capture patterns and long-term dependencies of different scales in the sequence, thereby better modeling long-term temporal correlations, making the prediction effect significantly improved compared to the previous two methods.

[0135] 3. The previous two models both use the self-attention mechanism to capture the global dependencies in the sequence and extract key information. The self-attention mechanism only considers the similarity between each position and other positions when calculating the weights, without directly considering the global consistency, which will lead to poor performance of the model. The model provided by the present invention uses the sparse attention mechanism (SA) to capture the global dependencies in the sequence and extract key information. SA only focuses on and calculates the relationship between those elements that have a greater impact on the performance of the model, while ignoring those elements with less impact, so that the prediction effect is significantly improved compared with the previous two methods.

[0136] The fitting diagram of the PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model of the present invention is as follows: Figure 8As shown. Figure 8 As shown, the method of the present invention can prevent the occurrence of overfitting.

[0137] The present invention provides a new combined model for PM2.5 concentration prediction, which combines variational mode decomposition (VMD), temporal convolutional network (TCN), bidirectional gated recurrent unit (BiGRU) and sparse attention mechanism (SA). VMD is used to decompose unstable and noisy PM2.5 concentration data to reduce the impact of noise on the prediction results; TCN network is used to process time series data and capture long-term dependencies and temporal correlations in the sequence; BiGRU network is used to extract deeper features of the sequence, including capturing long-term dependencies and contextual information in the sequence data; sparse attention mechanism is used to capture global dependencies in the sequence and extract key information, and improve the performance and generalization ability of the model by weighted combination of features at different positions in the sequence.

[0138] In summary, the method of the present invention can effectively improve the prediction accuracy and prediction performance of the traditional PM2.5 concentration prediction model, while also preventing the occurrence of overfitting, and the model has good generalization ability.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A PM2.5 concentration prediction method based on VMD-TCN-BiGRU-SA model, characterized in that: include: Step S1: Acquire a PM2.5 concentration data set and preprocess the PM2.5 concentration data set; Step S2: Calculate the Spearman correlation coefficient between the relevant features in the preprocessed PM2.5 concentration data set and the PM2.5 concentration, so as to eliminate the features with poor correlation with the PM2.5 concentration from the preprocessed PM2.5 concentration data set; Step S3: performing variational mode decomposition on the PM2.5 concentration data in the PM2.5 concentration data set processed in step S2 to obtain n intrinsic mode function components IMF and one residual component RES; Step S4: merging the components obtained by variational mode decomposition into the preprocessed PM2.5 concentration data set; Step S5: Divide the merged PM2.5 concentration data set into a training set and a test set according to a preset ratio; Step S6: constructing a PM2.5 concentration prediction model of TCN-BiGRU-SA; wherein the PM2.5 concentration prediction model of TCN-BiGRU-SA includes a TCN network, a BiGRU network and a sparse attention module connected in series in sequence; Step S7: inputting the training set into the PM2.5 concentration prediction model of the TCN-BiGRU-SA to train the PM2.5 concentration prediction model of the TCN-BiGRU-SA; Step S8: Input the test set into the trained TCN-BiGRU-SA PM2.5 concentration prediction model to obtain the prediction result of PM2.5 concentration.

2. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to claim 1, characterized in that: The mathematical expression corresponding to the Spearman correlation coefficient is: In formula (1), N is the total amount of data; x i is the value of the i-th data variable x; y i is the value of the i-th data variable y; is the mean value of variable x; is the mean value of variable y; ρ is x i With y i The Spearman correlation coefficient between the two.

3. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to claim 1, characterized in that: The mathematical expression corresponding to the variational mode decomposition of the PM2.5 concentration data in the PM2.5 concentration data set processed in step S2 is: In formula (2), K is the number of components; u k (t) is the single component signal after decomposition; ω k is the center frequency of each single component signal; It means to find partial derivative; δ(t) is the Dirac function; * is the convolution operator; f(t) is the original input signal; st is the abbreviation of subject to, and st means the constraint condition is.

4. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to claim 1, characterized in that: The TCN network TCN is composed of a plurality of residual blocks stacked together; each residual block includes: a first causal expansion convolution layer, a first WeightNorm layer, a first activation function layer, a first Dropout layer, a second causal expansion convolution layer, a second WeightNorm layer, a second activation function layer, and a second Dropout layer connected in sequence; and the input of the first causal expansion convolution layer is subjected to a data addition operation with the output of the second Dropout layer through a 1*1 convolution operation.

5. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to claim 1, characterized in that: The BiGRU network includes: an input layer, a GRU backward layer, a GRU forward layer, and an output layer connected to each other; the GRU backward layer and the GRU forward layer each include: a plurality of GRU networks connected to each other; Among them, the mathematical expression corresponding to the calculation process of the BiGRU network is: In formula (3), w t and v t They are respectively represented as the forward hidden layer state corresponding to the BiGRU network at time t and the reverse hidden state The corresponding weight; b t Represents the bias corresponding to the hidden layer state at time t; The input of the current neuron and the output of the previous neuron are used as the input of the GRU network, and the output of the current neuron is calculated through the GRU network; The mathematical expression corresponding to the calculation process of the GRU network is: r t =σ(W rx x t +W rh h t-1 +b r ) z t =σ(W zx x t +W zh h t-1 +b z ) In formula (4), x t is the input vector input to the memory unit at time t, i.e., the input of this neuron; h t is the output value of the neuron this time; h t-1 is the output of the previous neuron; z t is the update gate; r t To reset the gate; is the candidate output value of the current neuron; W is the weight matrix; b is the bias vector; ⊙ is the dot multiplication operation at the corresponding position of the element; σ and tanh represent the activation function.

6. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to claim 1, characterized in that: The sparse attention module includes an input layer, a hidden layer, and an output layer connected to each other; the attention weight coefficient x of the sparse attention module i The mathematical expression is: x i =utanh(ωh i +b) (5) In formula (5), u and ω are weight coefficients; b is the bias; h i is the output of the hidden layer; And use softmax to adjust the attention weight coefficient x i Normalize, the mathematical expression corresponding to the normalization process is: In formula (6), a i It is represented as the attention weight corresponding to the output of the i-th hidden layer; e j Represented as the index of the j-th element.

7. The PM2.5 concentration prediction method based on the VMD-TCN-BiGRU-SA model according to any one of claims 1 to 6, characterized in that: The PM2.5 concentration dataset is preprocessed, specifically including: Each outlier data value in the PM2.5 concentration data set is removed; and each removed data value and missing value in the PM2.5 concentration data set is filled using a linear interpolation method.

Images