Deep learning atmosphere prediction method based on time-frequency domain decomposition and spatio-temporal information extraction

Abstract

The invention discloses a deep learning atmosphere prediction method based on time-frequency domain decomposition and spatio-temporal information extraction. The deep learning atmosphere prediction method comprises the following steps: 1) collecting hour data of atmospheric pollutants and meteorological variables of a plurality of stations; 2) decomposing the data into sub-signal sequences with the same length under different frequency domains by adopting stationary wavelet transform (SWT), and meanwhile, fusing date and season time factors into the sequences to preliminarily form a data set required by the model; 3) constructing a graph data set of a topological structure by the data set in the step 2) according to the geographic position of the monitoring station; the method comprises the steps of (1) establishing a GCN, LSTM and FC, (2) establishing an encoder based on a time-space attention mechanism, and enhancing the spatial-temporal information weight of data, (3) extracting the time characteristics and spatial information of the data by a decoder established by GCN, LSTM and FC, and forming prediction output, and (4) taking RMSE, MAE and MAPE as evaluation parameters, The method can mine the propagation law of pollutants in the atmosphere, and improve the atmospheric prediction precision.

Description

Technical Field

[0001] The present invention relates to deep learning and environmental cross technology, and specifically to a deep learning atmosphere prediction method based on time-frequency domain decomposition and spatiotemporal information extraction. Background Art

[0002] With the continuous advancement of urbanization and industrialization in my country, the problem of air pollution has become increasingly serious, posing a major threat to public health and social development. In response to this problem, many atmospheric monitoring stations have been established to monitor the atmospheric environment and collect data, accumulating a large amount of complex and highly nonlinear time series data. Based on these data, scientists have developed a variety of atmospheric prediction methods to support the protection of the atmospheric environment. However, since the diffusion of atmospheric pollutants is closely related to spatiotemporal factors, existing atmospheric prediction methods mainly focus on a single monitoring station, ignoring the spatial correlation between monitoring stations in the region and lacking spatial analysis of atmospheric changes. This limitation significantly weakens the model's ability to understand and predict overall atmospheric changes. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies of the prior art and provide a deep learning atmospheric prediction method based on time-frequency domain decomposition and spatiotemporal information extraction. This method can explore the propagation laws of pollutants in the atmosphere and improve the accuracy of atmospheric prediction.

[0004] The technical solution for achieving the purpose of the present invention is:

[0005] A deep learning atmospheric prediction method based on time-frequency domain decomposition and spatiotemporal information extraction includes the following steps:

[0006] 1) Data collection: PM, an atmospheric pollutant variable, at a set of monitoring stations 2.5 、PM 10 , CO, NO2, SO2, O3 and meteorological variables Wind speed, Temperature, Humidity, Rain, Pressure are collected, and the number of monitoring stations is not less than 2, including:

[0007] 1-1) Delete the data less than zero and missing data for the whole day in the collected atmospheric pollutant variables and meteorological variables;

[0008] 2-1) Fill in the missing data of each local area based on the average values ​​of the collected atmospheric pollutant data and meteorological variable data;

[0009] 3-1) The Laida criterion is used to eliminate abnormal data in atmospheric data. By calculating the deviation coefficient of the data, it is determined whether it exceeds the set threshold. The threshold is set to 3. If the deviation coefficient exceeds the threshold, it is determined as an abnormal value and eliminated;

[0010] 2) Establishing a data set: Using the stationary wavelet transform (SWT), the data collected in step 1) is decomposed into sub-signal sequences of equal length in different frequency domains. At the same time, the date and seasonal time factors are integrated into the sequence, and the atmospheric time series data is separated into high-frequency components representing short-term drastic changes and low-frequency components representing long-term trends, thus initially forming the data set required by the model;

[0011] 3) Constructing a graph network structure: In the study area, each monitoring site is regarded as a node in the graph, and the adjacency relationship between monitoring points is used as the edge of the graph to construct an adjacency matrix. The weight of the edge is the inverse of the distance between the monitoring sites to construct a weight matrix, thereby constructing a graph network structure. In the graph network structure, the adjacency matrix and the weight matrix represent the spatial information of each monitoring site;

[0012] 4) Constructing an encoder: Construct an encoder based on the time-space attention mechanism to encode the decomposed multi-scale high- and low-frequency variables, enhance the time-space relationship weight of the data, and make the model more accurate in extracting key features;

[0013] 5) Obtain prediction output: The decoder built based on GCN, LSTM and FC extracts the temporal features and spatial information of the data to form a prediction output, specifically:

[0014] 1-5) The decoder constructed based on GCN, LSTM and FC extracts the spatiotemporal feature information of time series data. The spatial information is first mined through the GCN part through graph convolution operation and coupled to the input of the next stage. The formula for extracting spatial information is as follows;

[0015]

[0016] Among them, H represents the feature matrix of the graph convolution layer; l is the number of graph convolution layers; W (l) is the weight matrix; It is the adjacency matrix A plus the identity matrix I; yes The degree matrix of ; σ is the activation function;

[0017] 2-5) In the LSTM part, the time information is extracted from the time series data coupled with spatial information. The formula for extracting spatial information is as follows;

[0018] f t =σ(w f [h t-1 ,x t ]+b f ),

[0019] i t =σ(w i [h t-1 ,x t ]+bi ),

[0020] o t =σ(w o [h t-1 ,x t ]+b o ),

[0021]

[0022] h t =o t *tanh(C t ),

[0023] where f t 、i t , o t , are the outputs of the forget gate, input gate, output gate and candidate memory unit, respectively. f 、w i 、w o are the weight matrices of the three gates, w c is the weight matrix of the candidate memory unit, b f , b i , b o , b c is the bias term, C t and h t Represent memory unit state update and hidden state update respectively;

[0024] 3-5) Finally, a fully connected network is used to combine the feature information and output the final prediction result. The output formula is as follows:

[0025]

[0026] Among them, H i is the extracted spatiotemporal feature representation, W1, W2, b1, b2 are the weights and biases of the fully connected layer, σ is the activation function, is the predicted value;

[0027] 6) Evaluation: RMSE, MAE, and MAPE are used as evaluation parameters. During the model training process, a combination of L2 regularization weight decay, packet loss, and early stopping methods is used to suppress overfitting. The Adam optimizer is used to optimize the parameters of the deep learning model, and the Random Forest, SVR, RNN, LSTM, and Transformer five benchmark models are selected for comparison and verification. The smaller the evaluation parameter, the better the prediction effect. The evaluation parameter formula is as follows:

[0028]

[0029] Where n is the total number of samples, y i is the time series of monitoring, y′ i is the predicted time series.

[0030] Compared with existing technologies, this technology is based on an intelligent hybrid model of deep learning and time-frequency domain decomposition methods to explore the impact of the spatiotemporal evolution of atmospheric pollutants in the region during the transmission process, which not only improves the prediction performance, but also enhances the interpretability of atmospheric prediction results; at the same time, an encoder Encoder built based on the spatiotemporal attention mechanism and a Decoder designed by combining GCN, LSTM and fully connected network FC are proposed to achieve accurate capture of the spatiotemporal information of atmospheric pollutants.

[0031] This method can explore the propagation patterns of pollutants in the atmosphere and improve the accuracy of atmospheric forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the process of the embodiment method;

[0033] Figure 2 It is a schematic diagram of the process of decomposing time series data by stationary wavelet transform in an embodiment;

[0034] Figure 3 This is the structure diagram of Encoder;

[0035] Figure 4 This is the structure diagram of the Decoder. DETAILED DESCRIPTION

[0036] The content of the present invention is further described below in conjunction with the drawings and embodiments, but the present invention is not limited thereto.

[0037] Example:

[0038] like Figure 1 As shown, a deep learning atmospheric prediction method based on time-frequency domain decomposition and spatiotemporal information extraction includes the following steps:

[0039] 1) Data collection: PM, an atmospheric pollutant variable, at a set of monitoring stations 2.5 、PM 10 , CO, NO2, SO2, O3 and meteorological variables Wind speed, Temperature, Humidity, Rain, Pressure are collected, and the number of monitoring stations is not less than 2, including:

[0040] 1-1) Delete the data less than zero and missing data for the whole day in the collected atmospheric pollutant variables and meteorological variables;

[0041] 2-1) Fill in the missing data of each local area based on the average values ​​of the collected atmospheric pollutant data and meteorological variable data;

[0042] 3-1) The Laida criterion is used to eliminate abnormal data in atmospheric data. By calculating the deviation coefficient of the data, it is determined whether it exceeds the set threshold. The threshold is set to 3. If the deviation coefficient exceeds the threshold, it is determined as an abnormal value and eliminated;

[0043] In this case, Guilin, located in the Guangxi Zhuang Autonomous Region of China, is selected as the research area. Guilin is located in southern China, with karst terrain and a mid-subtropical humid monsoon climate. It has abundant precipitation, large changes in wind direction and speed, and a complex climate. Atmospheric turbulence shows obvious differences in different seasons and time periods, resulting in obvious high- and low-frequency fluctuations. At the same time, it is adjacent to the industrial city of Liuzhou, and the propagation of air pollutants is diverse. Therefore, the atmospheric environmental data of Guilin City has the conditions for time-frequency domain decomposition, and also provides an ideal data basis for building a graph neural network model. The monitoring data can be used to verify the effectiveness of this research model.

[0044] The research data in this case comes from the multi-point continuous online monitoring station of Guilin Ecological Environment Center Station. The monitoring sites include the monitoring station, No. 8 Middle School, Longyin Road Primary School, Overseas Chinese Tourism Economic Zone, Forestry Institute, Yaoshan Campus of University of Electronic Science and Technology of China, Lingui District Environmental Protection Bureau, Entrepreneurship Building, Tourism College and Dabu Central School, a total of 10 sites, as shown in Table 1. Table 1 lists the pollutant variables and meteorological variables in the basic data set. The pollutant variables include particulate matter with a diameter of less than or equal to 2.5 microns in the atmosphere (PM 2.5 ), pollutant variables include particulate matter in the atmosphere with a diameter less than or equal to 10 microns (PM 10 ), carbon monoxide (CO), nitrogen dioxide (NO2), sulfur dioxide (SO2) and ozone (O3) parameters. Meteorological variables include wind speed, temperature, humidity, rainfall and pressure. The hourly data from 2020 to 2023 are used as the basic training data set for model training and testing. The data set is divided into 60% training set, 20% test set and 20% validation set. Finally, the study selected PM 2.5 As the subject of evaluation:

[0045] Table 1. Variable data

[0046]

[0047] 2) Establish a data set: Use the stationary wavelet transform (SWT) to decompose the data collected in step 1) into sub-signal sequences of equal length in different frequency domains. At the same time, the date and seasonal time factors are integrated into the sequence, and the atmospheric time series data is separated into high-frequency components representing short-term drastic changes and low-frequency components representing long-term trends, thus initially forming the data set required by the model. Atmospheric environmental data have problems such as time correlation being masked and data entanglement. As a data time-frequency domain decomposition technology, the stationary wavelet transform has a good effect in the frequency division of time series data, overcomes the size limitation of the fixed window, and extracts all and local information in the time-frequency domain. In this example, the stationary wavelet transform is used to decompose atmospheric pollutants and meteorological data, extract low-frequency and high-frequency components on different time scales, and capture the changing patterns of pollutants in short-term fluctuations and long-term trends, so that the data can reflect more characteristics and the prediction results can be clearer and more accurate. Figure 2 As shown, Figure 2 This is a flow chart of the decomposition of time series data by stationary wavelet transform. First, input the data set, then decompose the long time series data by stationary wavelet transform, then extract the corresponding high and low frequency components, analyze them, and finally output the results. The formula is as follows:

[0048]

[0049] Where x[n] represents the original pollutant data, CA1[k] and CD1[k] are the low-frequency and high-frequency parts after the first decomposition, CA2[k] and CD2[k] are the low-frequency and high-frequency parts after the second decomposition of CA1[k], h[n] and g[n] represent the low-pass and high-pass filter coefficients, k is the time index, and n is each point in the data;

[0050] 3) Constructing a graph network structure: In the study area, each monitoring station is regarded as a node in the graph, and the adjacency relationship between monitoring points is used as the edge of the graph to construct an adjacency matrix. The weight of the edge is the inverse of the distance between the monitoring stations to construct a weight matrix, thereby constructing a graph network structure. In the graph network structure, the adjacency matrix and the weight matrix represent the spatial information of each monitoring station. In the atmospheric study area, this example regards each monitoring station as a node in the graph network, and constructs an adjacency matrix based on the actual adjacency relationship of the geographical location. In the adjacency matrix, a value of 1 indicates that the two monitoring stations have a connection relationship in the graph structure network. On this basis, according to the adjacency matrix, weights are assigned to the connecting edges of adjacent stations. The size of the weight is set to the inverse of the straight-line distance between the two stations on the actual map. By combining the adjacency matrix and the weight matrix, the graph network structure in the study area is constructed. Among them, the adjacency matrix represents the spatial adjacency relationship of the monitoring stations, and the weight matrix reflects the degree of influence between adjacent stations. The two together describe the spatial information of each monitoring station;

[0051] 4) Constructing an encoder: Construct an encoder based on the time-space attention mechanism to encode the decomposed multi-scale high- and low-frequency variables, enhance the time-space relationship weights of the data, and make the model more accurate in extracting key features. Although the wavelet transformed data solves the data entanglement problem, the time information and spatial features are difficult to be directly extracted by the network model. It is necessary to enhance the time-space relationship weights and convert them into feature representations suitable for the model. Therefore, this example uses an encoder based on the time-space attention mechanism to encode the decomposed data. The encoder structure is as follows: Figure 3 As shown, in the encoder, firstly, the time and space feature mapping is performed through the linear layer to generate the query (Q) and key (K) matrices. Then, the time and space correlation matrix (A) is obtained by multiplying the transpose of Q and K. A is normalized and Tanh activated to make the weight more accurate. Then, the adjusted A is multiplied by the data matrix (V) to highlight the high correlation variables. Finally, the weights are weighted and summed to enhance the time and space relationship weights. The time and space relationship weights of the data processed by the encoder are enhanced. The core time-space attention mechanism formula of the encoder is as follows:

[0052]

[0053] where H′ t is the time weight, is the spatial weight, T is the number of time steps, and H′ is the comprehensive representation after weighted summation;

[0054] 5) Obtain prediction output: The decoder built based on GCN, LSTM and FC extracts the temporal features and spatial information of the data to form a prediction output, specifically:

[0055] 1-5) The decoder constructed based on GCN, LSTM and FC extracts the spatiotemporal feature information of time series data. The spatial information is first mined through the GCN part through graph convolution operation and coupled to the input of the next stage. The formula for extracting spatial information is as follows;

[0056]

[0057] Among them, H represents the feature matrix of the graph convolution layer; l is the number of graph convolution layers; W (l) is the weight matrix; It is the adjacency matrix A plus the identity matrix I; yes The degree matrix of ; σ is the activation function;

[0058] 2-5) In the LSTM part, the time information is extracted from the time series data coupled with spatial information. The formula for extracting spatial information is as follows;

[0059] f t =σ(wf [h t-1 ,x t ]+b f ),

[0060] i t =σ(w i [h t-1 ,x t ]+b i ),

[0061] o t =σ(w o [h t-1 ,x t ]+b o ),

[0062]

[0063] h t =o t *tanh(C t ),

[0064] where f t 、i t , o t , are the outputs of the forget gate, input gate, output gate and candidate memory unit, respectively. f 、w i 、w o are the weight matrices of the three gates, w c is the weight matrix of the candidate memory unit, b f , b i , b o , b c is the bias term, C t and h t Represent memory unit state update and hidden state update respectively;

[0065] 3-5) Finally, a fully connected network is used to combine the feature information and output the final prediction result. The output formula is as follows:

[0066]

[0067] Among them, H i is the extracted spatiotemporal feature representation, W1, W2, b1, b2 are the weights and biases of the fully connected layer, σ is the activation function, is the predicted value. After the spatiotemporal relationship weights are enhanced, the data will be decoded by the decoder built based on GCN, LSTM and FC to extract the spatiotemporal feature information and make the final prediction output. The decoder structure is as follows: Figure 4As shown in the figure, GCN improves the accuracy and adaptability of model prediction by performing convolution operations on the nodes of the graph, integrating the feature information of neighboring nodes, and processing complex spatial relationships and dynamic data related to pollutant diffusion. It also extracts the pollutant diffusion relationship between each monitoring site, explores the transmission impact between adjacent sites, and reveals the transmission path of pollutants between sites. LSTM is a special recursive neural network that can avoid the problem of gradient disappearance or explosion, retain long-term information, and includes input layer, hidden layer and output layer. LSTM is used to extract the time dependence of pollutant concentrations at each monitoring site, and capture the changing trend and long-term and short-term dependencies of pollutants at different times. Finally, the above information is integrated through the FC network and the final prediction result is output;

[0068] 6) Evaluation: RMSE, MAE, and MAPE are used as evaluation parameters. During the model training process, this example uses a combination of L2 regularization weight decay, packet loss, and early stopping to suppress overfitting. The Adam optimizer is used to optimize the parameters of the deep learning model, and the Random Forest, SVR, RNN, LSTM, and Transformer five benchmark models are selected for comparison and verification. The smaller the evaluation parameter, the better the prediction effect. The evaluation parameter formula is as follows:

[0069]

[0070] Where n is the total number of samples, y i is the time series of monitoring, y′ i is the predicted time series.

[0071] Experimental verification:

[0072] The method in this example is a deep learning atmospheric prediction method based on time-frequency domain decomposition and spatiotemporal information extraction, namely SGLNet. This method is compared with five benchmark models: Random Forest, SVR, RNN, LSTM, and Transformer. Table 2 shows the prediction accuracy of the six models for PM2.5 at two sites and the same time step. Since the machine learning model cannot effectively capture the complex relationship in time series data, the prediction accuracy of Random Forest and SVR is low and the error is large. RNN, LSTM, and Transformer, as time series models, perform better than non-time series models in machine learning. Among them, compared with the RNN model, LSTM and Transformer effectively solve the problems of gradient explosion and disappearance, so the prediction performance of the latter two is better than RNN. This method is better than LSTM and Transformer in prediction performance. This is because the SGLNet model in this example method can separate hidden video signals, extract hidden correlations in complex time series data, and extract spatial information and time information at the same time, and show good applicability.

[0073] Table 2. Prediction comparison results

[0074]

[0075]

[0076] , GCN can automatically learn the spatial relationship between sites through the adjacency matrix and graph structure, extract the feature similarity between adjacent nodes through convolution operations, capture the local changes in pollutant concentrations, and aggregate the feature vectors of each node and its neighbors. After multi-layer feature aggregation, the overall diffusion trend of pollutants is extracted. The number of sites will affect the ability of GCN to extract spatial features. Table 3 shows the performance of the GCN model under different numbers of sites:

[0077] Table 3. Quantitative results of the number of sites

[0078]

[0079] As can be seen from the table, as the number of sites increases, the prediction performance of the GCN model is significantly improved, specifically, the RMSE, MAE, and SMAPE values ​​gradually decrease, which indicates that more sites provide richer spatial information, enabling the model to more accurately capture the complex relationship between sites and extract the spatial characteristics of pollutants. Therefore, GCN is particularly suitable for multi-site research environments;

[0080] Based on the results, this method provides a new method for the prediction of sequence data and model behavior analysis on atmospheric environmental quality, provides a scientific basis for the accurate prediction of atmospheric pollutant concentrations, pollution source tracing and comprehensive management of environmental quality, and can assist in the formulation of more effective pollution prevention and control measures. At the same time, this method is suitable for the prediction of atmospheric environmental quality at multiple monitoring stations in the city. The experimental results show that this method effectively improves the prediction accuracy by applying the stationary wavelet transform to decompose time series data and adopting the spatiotemporal feature extraction method. It can be seen that this method has important practical value in real-time monitoring and forecasting of atmospheric prediction, can provide theoretical support for alleviating atmospheric pollution, help reduce costs and protect the ecological environment.

Claims

1. A deep learning atmospheric prediction method based on time-frequency domain decomposition and spatiotemporal information extraction, characterized in that: The steps include: 1) Data collection: PM, an atmospheric pollutant variable, at a set of monitoring stations 2.5 、PM 10 , CO, NO2, SO2, O3 and meteorological variables Wind speed, Temperature, Humidity, Rain, Pressure are collected, and the number of monitoring stations is not less than 2, including: 1-1) Delete the data less than zero and missing data for the whole day in the collected atmospheric pollutant variables and meteorological variables; 2-1) Fill in the missing data of each part according to the average value of the collected atmospheric pollutant data and meteorological variable data; 3-1) The Laida criterion is used to eliminate abnormal data in atmospheric data. By calculating the deviation coefficient of the data, it is determined whether it exceeds the set threshold. The threshold is set to 3. If the deviation coefficient exceeds the threshold, it is determined as an abnormal value and eliminated; 2) Establishing a data set: Using the stationary wavelet transform (SWT), the data collected in step 1) is decomposed into sub-signal sequences of equal length in different frequency domains. At the same time, the date and seasonal time factors are integrated into the sequence, and the atmospheric time series data is separated into high-frequency components representing short-term drastic changes and low-frequency components representing long-term trends, thus initially forming the data set required by the model; 3) Constructing a graph network structure: In the study area, each monitoring site is regarded as a node in the graph, and the adjacency relationship between monitoring points is used as the edge of the graph to construct an adjacency matrix. The weight of the edge is the inverse of the distance between the monitoring sites to construct a weight matrix, thereby constructing a graph network structure. In the graph network structure, the adjacency matrix and the weight matrix represent the spatial information of each monitoring site; 4) Constructing an encoder: Construct an encoder based on the time-space attention mechanism to encode the decomposed multi-scale high- and low-frequency variables and enhance the time-space relationship weight of the data; 5) Obtain prediction output: The decoder built based on GCN, LSTM and FC extracts the temporal features and spatial information of the data to form a prediction output, specifically: 1-5) The decoder constructed based on GCN, LSTM and FC extracts the spatiotemporal feature information of time series data. The spatial information is first mined through the GCN part through graph convolution operation and coupled to the input of the next stage. The formula for extracting spatial information is as follows; Among them, H represents the feature matrix of the graph convolution layer; l is the number of graph convolution layers; W (l) is the weight matrix; It is the adjacency matrix A plus the identity matrix I; yes The degree matrix of ; σ is the activation function; 2-5) In the LSTM part, the time information is extracted from the time series data after coupling the spatial information. The formula for extracting the spatial information is as follows; f t =σ(w f [h t-1 ,x t ]+b f ), i t =σ(w i [h t-1 ,x t ]+b i ), the t =σ(w o [h t-1 ,x t ]+b o ), h t =o t *tanh(C t ), where f t 、i t , o t , are the outputs of the forget gate, input gate, output gate and candidate memory unit, respectively. f 、w i 、w o are the weight matrices of the three gates, w c is the weight matrix of the candidate memory unit, b f 、b i 、b o 、b c is the bias term, C t and h t Represent memory unit state update and hidden state update respectively; 3-5) Finally, a fully connected network is used to combine the feature information and output the final prediction result. The output formula is as follows: Among them, H i is the extracted spatiotemporal feature representation, W1, W2, b1, b2 are the weights and biases of the fully connected layer, σ is the activation function, is the predicted value; 6) Evaluation: RMSE, MAE, and MAPE are used as evaluation parameters. During the model training process, a combination of L2 regularization weight decay, packet loss, and early stopping methods is used to suppress overfitting. The Adam optimizer is used to optimize the parameters of the deep learning model, and the Random Forest, SVR, RNN, LSTM, and Transformer five benchmark models are selected for comparison and verification. The evaluation parameter formula is as follows: Where n is the total number of samples, y i is the time series of monitoring, y′ i is the predicted time series.

Images