CNN-AM-BiLSTM residual network-based ozone concentration prediction method

Abstract

The invention belongs to the technical field of deep learning, and particularly relates to an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network. Comprising the following steps: acquiring historical conventional pollutant and meteorological factor data of a monitoring station and carrying out data cleaning; carrying out feature extraction on the input multi-dimensional data by the convolutional neural network; a multi-head attention mechanism is constructed to help the model to better pay attention to important features, so that the feature representation capability is enhanced; the BiLSTM performs time modeling on the extracted features, so that the long-term dependency relationship in the time sequence data is better captured; a residual network is constructed to relieve the gradient disappearance problem, so that the model better learns nonlinear mapping of input data; the characteristics output by the BiLSTM are mapped to the target output space through the full connection layer, the invention provides a practical and effective method for ozone concentration prediction, the ozone concentration can be accurately predicted, and effective technical support is provided for atmospheric environment treatment.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and specifically is an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network. Background Art

[0002] Ozone is not only one of the atmospheric pollutants, but high concentrations of ozone can also cause sudden severe pollution weather, causing harm to public health and the environment.

[0003] The existing ozone prediction methods are based on single factor prediction and short-term ozone prediction. Ozone is a secondary pollutant, so the accuracy of single factor prediction needs to be improved; the prediction accuracy of a single LSTM model needs to be improved. Summary of the invention

[0004] In order to better predict ozone and give full play to the role of deep learning in the field of air pollution monitoring, the present invention proposes an ozone concentration prediction method based on CNN-AM-BiLSTM residual network. The present invention extracts features from the monitoring data of conventional pollutants and meteorological factors of a monitoring station through the CNN module, optimizes the features extracted by the multi-head attention mechanism, and performs time modeling on the extracted features by BiLSTM. The gradient of the residual network link disappears, and finally the ozone concentration in the future can be more accurately predicted.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0006] A method for predicting ozone concentration based on CNN-AM-BiLSTM residual network, comprising the following steps:

[0007] 1) Obtain multi-dimensional historical monitoring data from monitoring stations and perform data cleaning;

[0008] 2) Construct a feature extraction model based on convolutional neural network and perform feature extraction on the cleaned data;

[0009] 3) Construct an ozone concentration prediction model and use the extracted features to predict ozone concentration.

[0010] In the step 1), data cleaning is performed, specifically:

[0011] Outlier detection is performed on the acquired multi-dimensional historical monitoring data, the detected outliers are regarded as missing values, and the missing values ​​are filled using the Lagrange interpolation method.

[0012] The step 2) comprises the following steps:

[0013] 2.1) Use convolutional neural network to extract features from the cleaned data;

[0014] 2.2) Construct a multi-head attention mechanism to filter the features extracted by the convolutional neural network.

[0015] The calculation formula of the convolutional layer of the convolutional neural network in step 2.1) is:

[0016]

[0017] Among them, Z i,j Represents the value of the convolutional layer output feature map at position (i, j), X i,j Represents the value of the input data at position (i, j), W m represents the weight of the mth convolution kernel, b is the bias term, and M represents the size of the convolution kernel.

[0018] The multi-head attention mechanism model in step 2.2) is:

[0019]

[0020] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Q, K, V are all equal to B, B is the feature sequence output by the convolutional neural network, d k is the dimension of the attention head, is the scaled attention weight used to balance the gradient magnitude.

[0021] The step 3) comprises the following steps:

[0022] 3.1) Use Bi-LSTM to perform temporal modeling on the extracted features;

[0023] 3.2) Use the residual network to optimize the features of Bi-LSTM output;

[0024] 3.3) The output features of the residual network are mapped to the target output space through the fully connected layer to obtain the ozone concentration prediction result.

[0025] The output of the residual network in step 3.2) is:

[0026] Output=F(C)+C

[0027] Among them, F(C) is the output of the residual block, and C is the feature obtained by the BiLSTM module.

[0028] An ozone concentration prediction device based on CNN-AM-BiLSTM residual network, comprising:

[0029] The data acquisition module is used to obtain the multi-dimensional historical monitoring data of the monitoring station and perform data cleaning on it;

[0030] The feature extraction module is used to build a feature extraction model based on a convolutional neural network and extract features from the cleaned data;

[0031] The concentration prediction module is used to build an ozone concentration prediction model and use the extracted features to predict ozone concentration.

[0032] An ozone concentration prediction system based on a CNN-AM-BiLSTM residual network comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network when executing the computer program.

[0033] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network is implemented.

[0034] The present invention has the following beneficial effects and advantages:

[0035] 1. The present invention provides a practical and effective method for predicting ozone concentration, which can more accurately predict the ozone concentration in the monitoring station area in the future, and provides effective technical support for atmospheric environmental governance.

[0036] 2. The present invention is based on a CNN model and a BiLSTM model, and continuously optimizes feature extraction through a multi-head attention mechanism and a residual network, thereby improving the prediction accuracy of ozone. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a basic flow chart of an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network of the present invention;

[0038] Figure 2 This is the structural diagram of the CNN-AM-BiLSTM residual network. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0040] In order to better improve the prediction accuracy of ozone, an appropriate deep learning model is selected to train the data, so as to learn the distribution characteristics and laws of the data, thereby solving the problems encountered by traditional methods in ozone concentration research. This paper proposes an ozone concentration prediction method based on CNN-AM-BiLSTM residual network.

[0041] like Figure 1 and Figure 2As shown in FIG. 1 , the ozone concentration prediction method based on the CNN-AM-BiLSTM residual network includes the following steps:

[0042] Step 1: Obtain historical conventional pollutants (O3, SO2 NO2, NO, NO X , PM2.5, PM10, CO and AQI) and meteorological factor data (temperature, humidity, wind direction, air pressure and wind speed). One piece of data can be obtained every hour, and a total of 8760 pieces of data for one year can be obtained.

[0043] Step 2: Clean the acquired monitoring data, mainly dealing with outliers and missing values, and dividing the data into training sets and test sets;

[0044] Step 3: CNN extracts features from the input multi-dimensional data;

[0045] Step 4: Build a multi-head attention mechanism (AM) to help the model better focus on important features and ignore unimportant features, thereby enhancing the ability of feature representation;

[0046] Step 5: BiLSTM performs temporal modeling on the extracted features, which can consider both past and future information at the same time, thereby better capturing long-term dependencies in time series data;

[0047] Step 6: Build a residual network to alleviate the gradient vanishing problem, so that the model can better learn the nonlinear mapping of input data;

[0048] Step 7: Map the features of the residual network output to the target output space through a fully connected layer.

[0049] The data cleaning method specifically includes: performing outlier detection on the acquired historical conventional pollutant and meteorological factor data, treating the detected outliers as missing values, and then using the Lagrange interpolation method to fill in the missing values;

[0050] The CNN model is specifically as follows: when a one-dimensional convolutional neural network (CNN) processes multivariate and multi-step information, the input data is usually a three-dimensional tensor with a shape of (batch_size, input_channels, input_sequence_length). Among them, batch_size represents the number of input samples, input_channels represents the number of feature channels of the input data, and input_sequence_length represents the length of the time series.

[0051] The calculation formula of the convolutional layer is as follows:

[0052]

[0053] Among them, Z i,j Represents the value of the convolutional layer output feature map at position (i, j), X i,j Represents the value of the input data at position (i, j), W m represents the weight of the mth convolution kernel, b is the bias term. M represents the size of the convolution kernel;

[0054] The multi-head attention mechanism model is specifically as follows: the sequence obtained through the CNN module is B, where Q, K, and V are all equal to B.

[0055]

[0056] Where Q is the query matrix, K is the key matrix, and V is the value matrix. k is the dimension of the attention head, It is to scale the attention weights to balance the gradient size;

[0057] The Bi-LSTM model is specifically as follows: The calculation formula of the forward LSTM at time step t is as follows:

[0058] ft=σ(Wf[ht-1,xt]+bf)

[0059] it=σ(Wi·[ht-1,xt]+bi)

[0060]

[0061]

[0062] ot=σ(Wo·[ht-1,xt]+bo)

[0063] ht=ot⊙tanh(Ct)

[0064] Among them, f t It is the forget gate, t is the input gate, is a candidate memory cell, σ represents the sigmoid function, ⊙ represents element-wise multiplication, ht represents the hidden state of the forward LSTM at time step t, Ct represents the memory cell, and x t represents the input data, the input at time step t, [h t-1 ,x t ] means concatenating the hidden state of the previous moment and the current input together, W and b represent the weight and bias parameters respectively.

[0065] The calculation formula of the reverse LSTM at time step t is similar to that of the forward LSTM, except that the order of the input data is opposite to that of the forward LSTM. That is, in the reverse LSTM, x trepresents the input data at time step t, [h t+1 ,x t ] means concatenating the hidden state of the next moment and the current input together.

[0066] Combining the calculations in the above two directions, the output of the bidirectional LSTM at time step t is:

[0067]

[0068] in, represents the hidden state of the forward LSTM at time step t, Represents the hidden state of the reverse LSTM at time step t. The bidirectional LSTM concatenates the forward and reverse hidden states together, taking into account both past and future information, thereby improving the modeling ability of time series data.

[0069] The residual network is specifically: the feature obtained by the BiLSTM module is C, and the output of the residual block is F(C). The final output of the residual network is as follows:

[0070] Output=F(X)+X

[0071] Among them, F represents two fully connected layers and two activation functions.

Claims

1. A method for predicting ozone concentration based on CNN-AM-BiLSTM residual network, characterized in that: The following steps are involved: 1) Obtain multi-dimensional historical monitoring data from monitoring stations and perform data cleaning; 2) Construct a feature extraction model based on convolutional neural network and perform feature extraction on the cleaned data; 3) Construct an ozone concentration prediction model and use the extracted features to predict ozone concentration.

2. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 1 is characterized in that: In the step 1), data cleaning is performed, specifically: Outlier detection is performed on the acquired multi-dimensional historical monitoring data, the detected outliers are regarded as missing values, and the missing values ​​are filled using the Lagrange interpolation method.

3. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 1 is characterized in that: Step 2) The following steps are involved: 2.1) Use convolutional neural network to extract features from the cleaned data; 2.2) Construct a multi-head attention mechanism to filter the features extracted by the convolutional neural network.

4. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 3 is characterized in that: The calculation formula of the convolutional layer of the convolutional neural network in step 2.1) is: Among them, Z i,j Represents the value of the convolutional layer output feature map at position (i, j), X i,j Represents the value of the input data at position (i, j), W m represents the weight of the mth convolution kernel, b is the bias term, and M represents the size of the convolution kernel.

5. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 3 is characterized in that: The multi-head attention mechanism model in step 2.2) is: Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Q, K, V are all equal to B, B is the feature sequence output by the convolutional neural network, d k is the dimension of the attention head, is the scaled attention weight used to balance the gradient magnitude.

6. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 1 is characterized in that: The step 3) comprises the following steps: 3.1) Use Bi-LSTM to perform temporal modeling on the extracted features; 3.2) Use the residual network to optimize the features of Bi-LSTM output; 3.3) The output features of the residual network are mapped to the target output space through the fully connected layer to obtain the ozone concentration prediction result.

7. The ozone concentration prediction method based on CNN-AM-BiLSTM residual network according to claim 6 is characterized in that: The output of the residual network in step 3.2) is: Output=F(C)+C Among them, F(C) is the output of the residual block, and C is the feature obtained by the BiLSTM module.

8. An ozone concentration prediction device based on CNN-AM-BiLSTM residual network, characterized in that: include: The data acquisition module is used to obtain the multi-dimensional historical monitoring data of the monitoring station and perform data cleaning on it; The feature extraction module is used to build a feature extraction model based on a convolutional neural network and extract features from the cleaned data; The concentration prediction module is used to build an ozone concentration prediction model and use the extracted features to predict ozone concentration.

9. An ozone concentration prediction system based on CNN-AM-BiLSTM residual network, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement an ozone concentration prediction method based on a CNN-AM-BiLSTM residual network as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the ozone concentration prediction method based on the CNN-AM-BiLSTM residual network as described in any one of claims 1 to 7 is implemented.

Images