Ozone concentration prediction method based on GCN-BiLSTM-Attention space-time neural network

Abstract

The invention discloses an ozone concentration prediction method based on a GCN-BiLSTM-Attention space-time neural network. The ozone concentration prediction method is used for predicting O3 concentration in the next 24 hours. The model fuses a graph convolutional network GCN, an attention mechanism and a bidirectional long short-term memory network BiLSTM so as to learn the space-time dependence of the air pollutant concentration. The model can effectively improve the prediction accuracy of the O3 concentration by integrating meteorological data, geographical location information of a national control site and historical data of concentrations of pollutants such as O3, PM2.5 and CO. A research result shows that compared with a traditional statistical model and a single deep learning model, the GCN-BiLSTM-Attention shows higher accuracy and a lower error rate on a plurality of evaluation indexes.

Description

Technical Field

[0001] The present invention belongs to the field of meteorological forecasting, and specifically relates to an ozone concentration prediction method based on a GCN-BiLSTM-Attention spatiotemporal neural network. Background Art

[0002] In the O3 concentration prediction task, there are three main types of existing time series prediction methods: prediction models based on traditional statistical methods, such as the autoregressive integrated moving average model (ARIMA) and the vector autoregression model (VAR). These models can only use historical data and it is difficult to fully consider various factors such as meteorology that affect O3 concentration, so the prediction accuracy is limited. Prediction models based on machine learning, such as the support vector machine model (SVR) and the random forest model (RF). These models can better handle nonlinear relationships than statistical models, but they are still insufficient in dealing with multivariate and strongly nonlinear O3 concentration prediction problems. Prediction models based on deep learning, such as the long short-term memory network (LSTM), the gated recurrent unit (GRU), and the convolutional neural network (CNN). These models have strong nonlinear modeling capabilities and are the mainstream methods for current O3 concentration prediction, but they have never considered the spatial factors in the data. Summary of the invention

[0003] In view of the shortcomings of the existing technology and the fact that spatial factors have not been considered, the present invention provides an ozone concentration prediction method based on the GCN-BiLSTM-Attention spatiotemporal neural network, which can more accurately predict the O3 concentration in multiple areas.

[0004] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: an ozone concentration prediction method based on a GCN-BiLSTM-Attention spatiotemporal neural network, comprising the following steps:

[0005] Step 1: Obtain historical pollutant concentration data, geographical location information of national control stations, and meteorological data, and perform data preprocessing;

[0006] Step 2: Build an ozone concentration prediction model based on the GCN-BiLSTM-Attention spatiotemporal neural network. Build spatiotemporal data based on the preprocessed data and use it as the input data of the model. Use the label data as the output data of the model. Perform predictive learning iterative training on the model and reversely adjust the network model parameters to obtain an ideal model, so that the ideal model can predict the O3 concentration in different areas in the next 24 hours.

[0007] Step 3: Collect historical pollutant concentration data of the test site, geographical location information of the national control site, and meteorological data in real time for data preprocessing, and input them into the ideal model to automatically predict the O3 concentration of the test site in the next 24 hours.

[0008] The historical pollutant concentration data include PM 2.5 、O3、PM 10 , CO, SO2, NO2; the meteorological data include temperature and wind speed; the geographical location information of the national control site is the latitude and longitude data of the site.

[0009] The data preprocessing comprises the following steps:

[0010] Step 1.1, performing data cleaning and normalization processing on pollutant concentration data and meteorological data; the data cleaning includes filling missing values ​​and removing outliers;

[0011] Step 1.2: Construct an adjacency matrix based on the geographical location information of the national control stations;

[0012] The method of filling missing values ​​includes: for the time series data corresponding to each pollutant and meteorological field, removing non-numeric values ​​and using interpolation methods to interpolate data: a. For data missing for 3 consecutive hours or less, a linear interpolation method is used; b. For data missing for 4 to 6 consecutive hours, a third-order spline interpolation method is used, and it is ensured that the interpolated data is between the upper threshold and the lower limit; c. For missing values ​​for more than 6 consecutive hours, the mean pollutant concentration of the hour is used; the process of removing outliers is to use the 3σ detection method to remove outliers in the data.

[0013] The constructing of the adjacency matrix comprises:

[0014] The adjacency matrix is ​​defined to represent the distance between any two sites. The distance between any two sites is calculated using the K nearest neighbor algorithm. If the distance is less than a threshold, the corresponding elements of the two sites in the adjacency matrix are set to 1 to indicate that they are adjacent, otherwise they are set to 0.

[0015] The normalization process is as follows: normalizing the pollutant concentration data, meteorological data and adjacency matrix data after data cleaning to between [0, 1] according to the following formula;

[0016]

[0017] Where X represents the original value of meteorological pollutants, x scaler represents the normalized value, X max ,X min Indicates the maximum and minimum values ​​of the feature.

[0018] The spatiotemporal data is in the form of [N, T, D], where N is the station number, T is the sampling cycle time, and D is the normalized pollutant concentration and meteorological data at each sampling time;

[0019] Among them, the O3 concentration data [N, 24, O3] of the last 24-hour time window of each station is used as the prediction label of the station.

[0020] The ozone concentration prediction model structure based on the GCN-BiLSTM-Attention spatiotemporal neural network includes: a graph convolutional network GCN, a bidirectional long short-term memory network BiLSTM, an attention mechanism, and a spatiotemporal data feature fully connected layer;

[0021] The spatiotemporal data [N, T, D] and the normalized adjacency matrix are input into the ozone concentration prediction model; the spatial features are extracted through the graph convolutional network GCN to obtain time series data with spatial features; then the temporal features of the time series data with spatiotemporal features are extracted using BiLSTM, and the hidden state of each time point is output; these hidden states are processed using the attention mechanism; finally, the spatiotemporal features are concatenated through the fully connected layer to obtain the ozone concentration prediction result.

[0022] The graph convolutional network GCN processes the input adjacency matrix and extracts the spatial latitude and longitude features Z of the site;

[0023]

[0024] in, represents the normalized adjacency matrix, X l Represents the feature matrix of the node, W 0 , W 1 Represents the trainable parameters of the neural network;

[0025] The bidirectional long short-term memory network BiLSTM processes the input pre-processed pollutant concentration and meteorological data to extract time features;

[0026] Further learning deep spatial features and temporal features through the attention mechanism;

[0027] The deep time features and time features are flattened to obtain the final features of the spatiotemporal data of the current site [1, N*F]; where 1 represents the current site, N represents the number of nodes, and F represents the node embedding representation features obtained through model learning, which includes O3 concentration.

[0028] The predictive learning iterative training of the model includes: dividing the model input data into a training set, a validation set and a test set, training with the training set data, verifying the model accuracy with the validation set data, and predicting the O3 concentration with the test set data.

[0029] The present invention has the following beneficial effects and advantages:

[0030] 1. Integrate multi-source heterogeneous data input, including historical data on pollutant concentrations, geographic location information of national control stations and meteorological data, and comprehensively consider influencing factors.

[0031] 2. Use a dual-branch architecture to process different types of input data separately, fully explore their respective features, and finally perform feature fusion to improve the model's expression ability.

[0032] 3. GCN is introduced to capture the spatial correlation of sites, and a gating mechanism based on the attention mechanism is used to adaptively learn the feature importance of different GNC layers.

[0033] 4. The overall architecture is flexible and scalable, with strong nonlinear modeling capabilities, and its prediction accuracy and robustness are better than existing traditional models and single deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the ozone concentration prediction method based on the GCN-BiLSTM-Attention spatiotemporal neural network of the present invention;

[0035] Figure 2 This is a schematic diagram of the BiLSTM structure;

[0036] Figure 3 This is a comparison diagram of the principles of CNN and GCN;

[0037] Figure 4 It is a schematic diagram of the characteristics of the gating mechanism fusing multiple layers of GCN convolutional layers;

[0038] Figure 5 It is a schematic diagram of the fully connected output layer. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0041] like Figure 1 As shown, a method for predicting ozone concentration based on GCN-BiLSTM-Attention spatiotemporal neural network includes the following steps:

[0042] Step 1) Get PM separately 2.5 、O3、PM 10 , CO, SO2, NO2 six kinds of pollutant historical concentration data, national control station location information and temperature, wind speed two kinds of meteorological data, and pre-processing; including the following steps:

[0043] Step 1.1) O3 concentration, PM 2.5 , CO and other pollutant concentration data and meteorological data are cleaned, including filling missing values ​​and removing outliers.

[0044] Specifically, for each pollutant recorded (O3 concentration, PM 2.5 concentration, CO concentration, etc.) and meteorological fields (temperature, wind speed, etc.), obtain the time series data of the field, first remove the non-numeric values ​​(NaN values) and use the following interpolation method to interpolate: 2.5 、O3、PM 10 For the six pollutants, CO, SO2, and NO2, if the data is missing for 3 consecutive hours or less, the linear interpolation method is used; if the data is missing for 4 to 6 consecutive hours, the third-order spline interpolation method is used, and the interpolation data should be between the upper and lower limits; if the data is missing for more than 6 consecutive hours, the interpolation error of the above method is large, and the average pollutant concentration of that hour is used. The 3σ detection method is then used to remove outliers in the data.

[0045] Step 1.2) Construct an adjacency matrix based on the geographical location information of the national control stations; specifically:

[0046] Generate an adjacency matrix: The row and column numbers in the adjacency matrix are arranged according to the sequence number of the site, and each element represents the distance between any two sites. Customize a distance parameter, use the K nearest neighbor algorithm to calculate the distance between any two sites, and determine whether the distance between the two sites is less than or equal to the distance parameter. If so, set the corresponding elements of the two sites in the adjacency matrix to 1, indicating that they are adjacent; if not, set the corresponding elements to 0, indicating that they are not adjacent. After completing the above operations, an adjacency matrix representing the distance between any two sites is obtained. The adjacency matrix represents the directed self-loop graph between the nodes. For the initial attributes of the edges, the inverse of the distance between the nodes is used for assignment, that is, the edge weight attribute between the more adjacent nodes is larger, and the edge weight between the farther nodes is smaller.

[0047] Step 1.3) O3, PM 2.5 , CO and other pollutant concentration data and meteorological data are preprocessed; specifically:

[0048] For each field in the data, the maximum and minimum normalization is used to normalize the data in the field to between [0,1]. The specific formula is shown in (1), where X represents the meteorological and pollutant values ​​to be normalized, x scaler represents the normalized value, X max ,X min Indicates the maximum and minimum values ​​of the feature.

[0049]

[0050] Step 2) Build an O3 concentration prediction model based on the GCN-BiLSTM-Attention spatiotemporal neural network, and build model input data based on the preprocessed data. Take the model input data as input and the label data as output, perform predictive learning iterative training, and reversely adjust the network model parameters to obtain the ideal model, so that the ideal model can predict the O3 concentration of stations in different regions in the next 24 hours.

[0051] Step 2.1) constructing spatiotemporal data as model input data; including the following steps:

[0052] Step 2.1.1) Time dimension data; specifically:

[0053] At each station, sensors record PM 2.5 、O3、PM 10 , CO, SO2, NO2 six pollutant concentration values ​​and temperature, wind speed two meteorological data values, the cycle is 1 hour. Many features will be recorded at the same time, these features are extended in the time dimension, we will get the multivariate time series data of each node, the form is [T, D], where T is the time, D is the number of features at each moment. For a single node, the time dimension is divided into intervals of 24 hours in the time window, and the single node data is obtained in the form of [1, 24, D], where 1 represents the current node, 24 represents the 24 moments in history, and D represents the eight feature values ​​recorded at each moment.

[0054] Step 2.1.2) Spatial dimension data; specifically:

[0055] For each node, the time dimension is divided into 24-hour time windows to obtain single node data in the form of [1,24,D], where 1 represents a node, 24 represents 24 historical moments, and D represents the number of features recorded at each moment. To add spatial dimension information, all node data in the study area are spliced ​​in the first dimension. For example, if there are N nodes, each node is [1,24,D]. By splicing the first dimension, spatiotemporal data in the form of [N,24,D] can be obtained.

[0056] Step 2.1.3) Predict label data; specifically:

[0057] For the selected predicted site, the O3 concentration data within the last 24-hour time window is used as the predicted label of the node, with a shape of [1,24]. Taking all nodes into consideration, label data of the shape of [N,24] is obtained.

[0058] Step 2.2) Construct an O3 concentration prediction model based on the GCN-BiLSTM-Attention spatiotemporal neural network;

[0059] Step 2.2.1) BiLSTM temporal feature extraction method; specifically:

[0060] In terms of the feature extraction method of the time dimension, the BiLSTM neural network model is used. BiLSTM effectively maintains and updates the internal state of the network by introducing three key gating mechanisms - input gate, forget gate and output gate, so as to capture the long-term dependencies in time series data. At the same time, BiLSTM will perform the same operation on the time series from the reverse direction to obtain the forward and backward time relationship features. The key to BiLSTM is its hidden state at each moment. This hidden state is a comprehensive compression of the information of all moments before the current moment. Therefore, BiLSTM will output two hidden states at each moment, one representing the forward sequence relationship and the other representing the reverse sequence relationship. The overall output is the concatenation of the two hidden states. Its specific structure is as follows Figure 2 , where X t and h t Respectively represent the input and output hidden states at that moment.

[0061] Step 2.2.2) Attention mechanism of feature fusion method between time points; specifically:

[0062] BiLSTM can only compress the features of the time series unidirectionally from front to back or from back to front, and faces the problem of information forgetting caused by gradient vanishing. Therefore, the present invention introduces an attention mechanism based on BiLSTM, which can comprehensively consider and weight the information at all times and avoid the problem of information forgetting. Specifically, the hidden state of BiLSTM at all times is first obtained, and then the attention mechanism Attention is used to fuse the information at all times.

[0063] Step 2.2.3) Spatial feature extraction method GCN; specifically:

[0064] GCN is used to extract spatial information from spatiotemporal data. Specifically, information between different nodes is exchanged with each other. The reason is that the prediction of O3 often has spatiotemporal correlation, and the O3 concentration detection values ​​in different regions often affect each other, especially in the spatiotemporal region of the city. Therefore, the present invention takes this spatiotemporal factor into consideration in the modeling factor. The core idea of ​​GCN is that for each node, its feature information is obtained from all its neighboring nodes, including its own features, and then this information is integrated through an aggregation function (such as the average function), such as Figure 3 shown.

[0065] The specific calculation formula is as follows (1). represents the normalized adjacency matrix, represents the degree matrix with self-loops, X l Represents the feature matrix of the node, W 0 , W 1 Represents the trainable parameters of a neural network.

[0066]

[0067] in, represents the normalized adjacency matrix, represents the degree matrix with self-loops, X l Represents the feature matrix of the node, W 0 , W 1 Represents the trainable parameters of the neural network;

[0068] Step 2.2.4) Gating mechanism of feature fusion method between spatial nodes; specifically:

[0069] The present invention uses GCN to extract spatial features. Each time a GCN convolution is performed, information is transferred from the node along the edge once. At least three GCN convolutions are required before the information of any node in the graph can contact and fuse with each other. Similarly, multiple information transfers also face the problem of information forgetting. Therefore, the present invention introduces a gating mechanism to fuse information from different GCN layers, which can alleviate the problem of information forgetting and improve the prediction performance of the model. The specific principle is as follows: Figure 4 shown.

[0070] Step 2.2.5) Spatiotemporal data feature flattening method; specifically:

[0071] After extracting spatiotemporal data features through GCN and BiLSTM-Attention, a tensor of the form [1, N, F] is obtained, where N represents the number of nodes and F represents the node embedding representation features learned by the model. In order to input the features into the fully connected layer for modeling, [1, N, F] is flattened to [1, N*F], which is the final feature of the input spatiotemporal data.

[0072] Step 2.2.6) 24-hour long-term O3 pollutant prediction network layer based on flattened space-time data characteristics; specifically:

[0073] Definition of spatiotemporal prediction problem of O3 concentration: For the multi-step prediction problem of O3 pollutant concentration, suppose there are P historical time steps of pollutant observation values ​​x=(X1,X2,......,X p )∈R Q×N The goal of the present invention is to predict the target value y=(Y p+1 ,Y p+2 ,......,Y p+Q )∈R Q×N , then the pollutant prediction process can be expressed as (2), such as Figure 5 As shown, the pollutant forecast value for the next 24 hours is output at one time.

[0074] Y p+1 ,Y p+2 ,……,Y p+Q =f(G;X1,X2,......,X p )(2)

[0075] Among them, G is the entire ozone concentration prediction model.

[0076] Step 2.2.7) Model data stream transmission process; specifically:

[0077] The original data is first passed through GCN to extract the spatial features of the original data, and the original data is converted into time series data with spatial features. Then, BiLSTM is used to extract the temporal features of the time series data with spatiotemporal features. BiLSTM outputs the hidden state at each time point, and the attention mechanism is used to process these hidden states. Finally, the fully connected layer is used to integrate the spatiotemporal features to obtain the ozone concentration prediction result.

[0078] As shown in Table 1, a method for predicting ozone concentration based on the GCN-BiLSTM-Attention spatiotemporal neural network is presented. The specific parameters of the model used in the method are as follows:

[0079] Table 1 Model parameter structure table

[0080]

[0081] The above is a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network is characterized by: The following steps are involved: Step 1: Obtain historical pollutant concentration data, geographical location information of national control stations, and meteorological data, and perform data preprocessing; Step 2: Build an ozone concentration prediction model based on the GCN-BiLSTM-Attention spatiotemporal neural network. Build spatiotemporal data based on the preprocessed data and use it as the input data of the model. Use the label data as the output data of the model. Perform predictive learning iterative training on the model and reversely adjust the network model parameters to obtain an ideal model, so that the ideal model can predict the O3 concentration in different areas in the next 24 hours. Step 3: Collect historical pollutant concentration data of the test site, geographical location information of the national control site, and meteorological data in real time for data preprocessing, and input them into the ideal model to automatically predict the O3 concentration of the test site in the next 24 hours.

2. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The historical pollutant concentration data include PM 2.5 、O3、PM 10 , CO, SO2, NO2; the meteorological data include temperature and wind speed; the geographical location information of the national control site is the latitude and longitude data of the site.

3. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The data preprocessing comprises the following steps: Step 1.1, performing data cleaning and normalization processing on pollutant concentration data and meteorological data; the data cleaning includes filling missing values ​​and removing outliers; Step 1.2: Construct an adjacency matrix based on the geographical location information of the national control stations.

4. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 3 is characterized in that: The method of filling missing values ​​includes: for the time series data corresponding to each pollutant and meteorological field, removing non-numeric values ​​and using interpolation methods to interpolate data: a. For data missing for 3 consecutive hours or less, a linear interpolation method is used; b. For data missing for 4 to 6 consecutive hours, a third-order spline interpolation method is used, and it is ensured that the interpolated data is between the upper threshold and the lower limit; c. For missing values ​​for more than 6 consecutive hours, the mean pollutant concentration of the hour is used; the process of removing outliers is to use the 3σ detection method to remove outliers in the data.

5. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 3 is characterized in that: The constructing of the adjacency matrix comprises: The adjacency matrix is ​​defined to represent the distance between any two sites. The distance between any two sites is calculated using the K nearest neighbor algorithm. If the distance is less than a threshold, the corresponding elements of the two sites in the adjacency matrix are set to 1 to indicate that they are adjacent, otherwise they are set to 0.

6. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The normalization process is as follows: normalizing the pollutant concentration data, meteorological data and adjacency matrix data after data cleaning to between [0, 1] according to the following formula; Where X represents the original value of meteorological pollutants, x scaler represents the normalized value, X max ,X min Indicates the maximum and minimum values ​​of the feature.

7. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The spatiotemporal data is in the form of [N, T, D], where N is the station number, T is the sampling cycle time, and D is the normalized pollutant concentration and meteorological data at each sampling time; Among them, the O3 concentration data [N, 24, O3] of the last 24-hour time window of each station is used as the prediction label of the station.

8. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The ozone concentration prediction model structure based on the GCN-BiLSTM-Attention spatiotemporal neural network includes: a graph convolutional network GCN, a bidirectional long short-term memory network BiLSTM, an attention mechanism, and a spatiotemporal data feature fully connected layer; The spatiotemporal data [N, T, D] and the normalized adjacency matrix are input into the ozone concentration prediction model; the spatial features are extracted through the graph convolutional network GCN to obtain time series data with spatial features; then the temporal features of the time series data with spatiotemporal features are extracted using BiLSTM, and the hidden state of each time point is output; these hidden states are processed using the attention mechanism; finally, the spatiotemporal features are concatenated through the fully connected layer to obtain the ozone concentration prediction result.

9. The ozone concentration prediction method based on GCN-BiLSTM-Attention spatiotemporal neural network according to claim 1 is characterized in that: The predictive learning iterative training of the model includes: dividing the model input data into a training set, a validation set and a test set, training with the training set data, verifying the model accuracy with the validation set data, and predicting the O3 concentration with the test set data.

Images