A long-time series PM2.5 prediction method and system based on spatiotemporal attention

Abstract

The present invention belongs to the field of PM2.5 time series prediction, and is a long-time series PM2.5 prediction method and system based on spatiotemporal attention. The method comprises the following steps: obtaining output and performing preprocessing; inputting preprocessed data into a feature extraction network for feature extraction; connecting and fusing features extracted from different sites using a spatial attention network; obtaining past features from features processed by the spatial attention network through a multi-layer bidirectional LSTM; extracting future features from known future feature data corresponding to a time period to be predicted through a neural network, and connecting the future features to obtain a prediction result; iteratively training the network until convergence using a loss function that takes into account standard deviation fluctuations and average errors of the data; inputting the data of the site to be tested into a PM2.5 prediction network based on spatiotemporal attention that has completed training, and outputting a prediction result; the present invention can accurately perform long-time series PM2.5 prediction.

Description

Technical Field

[0001] The present invention belongs to the field of PM2.5 time series prediction, and in particular relates to a long time series PM2.5 prediction method and system based on spatiotemporal attention. Background Art

[0002] PM2.5 (particles with an aerodynamic diameter of less than 2.5 μm), as the main pollutant in the atmosphere, has attracted widespread attention in the field of forecasting due to its negative impact on ambient air quality, public health and socioeconomic development. In particular, PM2.5 concentration prediction is of great significance for controlling and reducing air pollution, helping the government to make effective early warning decisions and reminding the public to travel in a healthy manner. Therefore, more accurate prediction results of PM2.5 concentration predictions produced by effective forecasting models will become increasingly important. However, since PM2.5 is affected by external factors such as weather, traffic volume, wind direction, wind speed and other meteorological factors, complex time entanglements have emerged, making PM2.5 predictions still challenging in long time series.

[0003] PM2.5 concentration prediction methods are mainly divided into two categories: physical model method and data-driven method. Research based on physical model methods such as CMAQ model and WRF / Chem model are widely used in air quality prediction. This physical model method is usually based on professional knowledge such as the physical change process of atmospheric pollutants, and constructs a pollutant concentration change model from a professional perspective. The main advantage of the physical model-based method is that it is widely used, and the operating laws and principles of the interaction of various factors under certain environmental conditions can be clearly seen. However, due to the obvious variation of PM2.5 components, the extremely complex physical propagation process, and the lack of corresponding knowledge and conjectures, it is difficult for us to comprehensively consider all situations that occur. At the same time, the environment in different regions is very different, such as wind direction and weather climate, local industrial facility construction, deployment location and density, and changes in traffic volume, which will make the transmission and reaction of air pollutants very different in different places. Therefore, compared with the physical model method, the statistical method based on data-driven modeling is simple and has good performance.

[0004] In the prior art, however, due to the dynamic characteristics of the atmospheric environment, the recurrent neural network can process any input sequence, thereby ensuring the learning ability of the time series, and is particularly suitable for simulating the time evolution of the distribution of atmospheric pollutants. However, when the traditional RNN lag time is long, problems such as gradient disappearance and gradient explosion will occur. The long short term memory network can alleviate this problem to a certain extent.

[0005] Recently, the use of composite model methods has also become a trend. Some methods use convolutional neural networks to mine the nonlinear spatial correlation of data, thereby further improving the performance of the model. Many researchers use neural networks based on the CNN-LSTM multi-layer structure to learn the intrinsic spatiotemporal correlation of air pollution time series data. However, the CNN network is a two-dimensional convolution, and this method destroys the original structural information of the data and ignores the temporal correlation.

[0006] Existing PM2.5 concentration prediction methods ignore the dynamic impact of the spatiotemporal state of different sites on future PM2.5 concentrations, and most methods cannot effectively simulate the spatiotemporal dependence of PM2.5 concentrations at the same time. At the same time, the simultaneous processing of the site's own characteristics and the spatial connections between sites will lead to errors in feature extraction. At the same time, due to the diversity of environmental characteristics in different regions, how to adaptively extract features based on data characteristics, learn the spatial correlation between different sites, and capture complex PM2.5 cycle patterns at the same time is a problem that needs to be solved urgently. Summary of the invention

[0007] To solve the above problems, the present invention proposes an adaptive long-time series PM2.5 prediction method and system based on spatiotemporal attention mechanism, the method comprising the following steps:

[0008] Obtain different pollutant concentration data and meteorological factor data, normalize them and fill in missing values;

[0009] The preprocessed meteorological data of different stations are input into their corresponding feature extraction networks for feature extraction;

[0010] Use the spatial attention network to connect and fuse the features extracted from different sites;

[0011] The processed features are put into a multi-layer bidirectional LSTM to obtain the positive and reverse trends of the data, which are used to extract the complex periodic characteristics of PM2.5 concentration;

[0012] Take the known future feature data, use the feature extraction network to obtain future features through the embedding layer, connect the past features, and finally output the regression results through the linear layer;

[0013] The network is iteratively trained until convergence using a loss function that takes into account the standard deviation fluctuations and mean error of the data;

[0014] The data of the site to be tested is input into the trained PM2.5 prediction network based on spatiotemporal attention, and the prediction results are output.

[0015] Furthermore, the pre-processed meteorological data of different stations are input into the corresponding feature extraction network for feature extraction, including:

[0016] FEN(f)=GLU(μ0)+μ1;

[0017] μ0=tanh(w0f+b0);

[0018] μ1=w1f+b1;

[0019] Among them, FEN(f) is the feature extraction network; f is the preprocessed meteorological data of different stations; GLU() is the gated linear network; w0 is the feature weight; b0 is the bias term; w1 is the feature weight; b1 is the bias term.

[0020] Furthermore, the process of extracting nonlinear features by the gated linear network is expressed as:

[0021] GLU(μ0)=(σ(w1*μ0+b2)⊙(w1*μ0+b3));

[0022] Among them, GLU (μ0) is the nonlinear feature extracted according to the input data; w1 is the hidden feature weight; b2 and b3 are bias terms; ⊙ is the dot product; σ() represents the sigmoid function.

[0023] Furthermore, the spatial attention network is used to connect and fuse the features extracted from different sites, that is, the extracted features are input into the feedforward neural network to obtain the feature factor of the site, which is expressed as:

[0024] h0=w tar h tar +b tar ;

[0025] α i =concat(h i ,h0);

[0026] The characteristic factors of the target site are concatenated with the characteristic factors of other sites respectively, and the concatenated values ​​are used to calculate the attention weight through the hyperbolic tangent function activation function, which is expressed as:

[0027]

[0028] The attention weight of each site is calculated by softmax and attention weight, expressed as:

[0029]

[0030] Among them, h0 is the influence weight of the target feature; w tar is the spatial weight of the target site; b taris the spatial offset of the target site; h i is a characteristic factor of a non-target site; α i For h0 and h i The concatenated features; Represents α i The j-th dimension feature of i is the feature weight; b i is the spatial offset of the ith site; represents the importance weight of the j-th dimension of air station i, represents the spatial attention weight of the jth dimension of air station i; L is the characteristic dimension of the station, exp represents the exponential function; h tar is the target site characteristic sequence.

[0031] Furthermore, the loss function considering the standard deviation fluctuation and mean error of the data is expressed as:

[0032]

[0033] Among them, Loss is the loss function; MSE is the mean square error in parameter estimation; Std * is the standard deviation of the predicted sequence; Std is the standard deviation of the true sequence; w2 is the L2 regularization parameter, expressed as λ is the regularization parameter, w i is the parameter of the ith neural network; M is the number of parameters in the neural network.

[0034] Furthermore, the mean square error in parameter estimation is the expected value of the square of the difference between the parameter estimate and the true value of the parameter, expressed as:

[0035]

[0036] Where N is the number of sites; is the parameter estimate; y (j) The true value of the parameter.

[0037] Furthermore, the standard deviation calculation process of the predicted sequence and the actual sequence includes:

[0038]

[0039]

[0040] Where N is the number of sites; is the predicted sequence point value; *- is the mean of the predicted sequence; y (j) is the actual sequence point value; y - is the true series mean.

[0041] The present invention also proposes a system for long-time series PM2.5 prediction based on spatiotemporal attention, which is used to implement a method for long-time series PM2.5 prediction based on spatiotemporal attention. The system includes a time series data acquisition module, a feature extraction module, a spatial attention network, a multi-layer bidirectional LSTM, a time series feature extraction module, a feature connection module and a prediction module; wherein:

[0042] The time series data acquisition module is used to obtain pollutant concentration data and meteorological factor data at different sites, including historical data and real-time data, and train the system based on historical data; the real-time data is input into the trained system for real-time prediction;

[0043] The feature extraction module is used to extract the features of the data obtained by the time series data acquisition module. The feature extraction module inputs the pre-processed meteorological data of different stations into its corresponding feature extraction network for feature extraction; and compares the features of the target station, i.e. the station to be predicted, with the features of other stations.

[0044] The spatial attention network is used to obtain the attention weight of each auxiliary site. The site to be predicted is taken as the target site and other sites as auxiliary sites. The features obtained by the feature extraction module are used through a feedforward neural network to obtain the feature factor of the site. The feature factor of the target site is concatenated with the feature factor of each auxiliary site, and the attention weight is calculated through the hyperbolic tangent function activation function. The attention weight of each site is calculated through softmax and attention weight.

[0045] Multi-layer bidirectional LSTM is used to extract the periodic features of the output features of the spatial attention network;

[0046] The time series feature extraction module is used to obtain known future feature data, that is, the season of the time period to be predicted and the upcoming holiday information, and use embedding operations to convert the collected information into a dimensional vector, and extract its features through a neural network;

[0047] The prediction module is used to obtain the features output by the multi-layer bidirectional LSTM and the features output by the time series feature extraction module, and then perform linear weighting to obtain the regression prediction result, which is used as the output of the prediction system.

[0048] Compared with the prior art, the beneficial technical effects of the present invention include:

[0049] (1) Compared with other PM2.5 time series prediction models, the present invention has a high precision effect and can perform feature extraction and spatiotemporal attention parameters on data sets with any feature dimension input.

[0050] (2) An adaptive feature selection network was designed, which can dynamically capture the linear and nonlinear features of the data and adaptively determine the fitting complexity of the model according to the characteristics of different data sets, thereby improving the flexibility of the model.

[0051] (3) A new attention mechanism is designed for the model to achieve accurate spatial interpretation. The attention mechanism can simultaneously obtain the attention weights between the target site and the auxiliary sites. The algorithm adaptively weights different feature states in different regions and captures the complex dynamic relationship between each auxiliary time series and the target time series.

[0052] (4) The present invention adds a time feature extraction module after the spatial feature extraction, which can capture the positive and reverse time trends of the data and can be used to extract complex periodic features.

[0053] (5) The present invention proposes a time series feature enhancement module that utilizes known future data. The module enhances the model's perceptual field of view based on known time series data such as holiday cycles, and finally connects past high-dimensional latent feature data with future latent features to obtain multi-scale time series feature data.

[0054] (6) The present invention proposes an error measurement index that not only utilizes the absolute error between the predicted value and the actual observed value, but also can measure the degree of fluctuation of long time series data and can comprehensively reflect the difference between the predicted value and the actual observed value. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a long time series PM2.5 prediction method based on spatiotemporal attention mechanism according to an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the structure of a site adaptive feature extraction module according to an embodiment of the present invention;

[0057] Figure 3 It is a structural schematic diagram of a site space attention weight acquisition module according to an embodiment of the present invention;

[0058] Figure 4 It is a structural diagram of an LSTM module according to an embodiment of the present invention;

[0059] Figure 5 It is a schematic diagram of the structure of a long-time series PM2.5 prediction network based on the spatiotemporal attention mechanism according to an embodiment of the present invention;

[0060] Figure 6 A schematic diagram of a training process according to an embodiment of the present invention;

[0061] Figure 7 The figure is an application effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] The present invention proposes an adaptive long-time series PM2.5 prediction method based on spatiotemporal attention mechanism, comprising the following steps:

[0064] Obtain different pollutant concentration data and meteorological factor data, normalize them and fill in missing values;

[0065] The preprocessed meteorological data of different stations are input into their corresponding feature extraction networks for feature extraction;

[0066] Use the spatial attention network to connect and fuse the features extracted from different sites;

[0067] The processed features are put into a multi-layer bidirectional LSTM to obtain the positive and reverse trends of the data, which are used to extract the complex periodic characteristics of PM2.5 concentration;

[0068] Take the known future feature data, use the feature extraction network to obtain future features through the embedding layer, connect the past features, and finally output the regression results through the linear layer;

[0069] The network is iteratively trained until convergence using a loss function that takes into account the standard deviation fluctuations and mean error of the data;

[0070] The data of the site to be tested is input into the trained PM2.5 prediction network based on spatiotemporal attention, and the prediction results are output.

[0071] The long-time series PM2.5 prediction method based on spatiotemporal attention provided by the present invention can be applied to the following scenarios:

[0072] 1. Weather forecast scenario

[0073] In response to the demand for climate forecasting, technical assistance can be provided to relevant institutions. The PM2.5 time series forecasting model provided by this application can be called in the background to extract the PM2.5 forecast results for a period of time in the future for the caller.

[0074] 2. Travel reminder scenarios

[0075] For example, when a user needs to make a travel plan, in order to reduce disruption to the trip caused by sudden weather conditions, the user can be provided with air quality conditions for a period of time in the future to assist the user in making travel plans.

[0076] 3. Air warning scenarios

[0077] It can assist relevant government departments in making reliable decisions supported by data. For possible air pollution warnings, cities and individuals can respond in advance, such as restricting transportation, temporarily closing heavily polluting factories, restricting outdoor activities of some people (with respiratory and other related diseases), and individuals wearing masks in advance, etc. to reduce possible damage.

[0078] For ease of understanding, this embodiment proposes a specific implementation method of a long-time series PM2.5 prediction method based on a spatiotemporal attention mechanism, such as Figure 1 As shown, including:

[0079] 101. Obtain pollutant concentration data and meteorological factor data from different stations and perform preprocessing operations on them.

[0080] Specifically, the existing air quality data sets of Xi'an and Beijing are used for corresponding preprocessing. In the embodiment of the present invention, the air quality data set of Beijing is taken as an example. The data set consists of three parts: air quality characteristics, meteorological characteristics and time characteristics. The air quality data of this research data set comes from UCI public data and. We selected hourly air quality records of 12 monitoring stations in Beijing from January 1, 2010 to December 31, 2014, totaling 43,824 records. Each air quality example contains six pollutants, that is, the pollutant concentration data includes: PM2.5, PM10, NO2, CO, O3 and SO2. At the same time, meteorology is recorded, that is, the meteorological factor data includes 7 attributes: time, weather, temperature, pressure, humidity, wind speed and wind direction.

[0081] These data sets have large fluctuations in both mean and variance. For the data set, 60%, 10%, and 30% are selected as training, validation, and test sets in chronological order. For consecutive missing data, linear interpolation is performed based on the previous and next data to fill in the missing data. This embodiment also adds time information including year, month, day, and day of the week to each data block.

[0082] Finally, before the dataset is input into the network, the feature data of different scales are normalized to ensure that each feature is treated equally by the classifier and to avoid the situation where small data features in absolute values ​​are overly affected by large data features.

[0083] 102. The pre-processed meteorological data from different stations are input into the feature extraction network for feature extraction.

[0084] The preprocessed training time series data samples are input into the feature extraction network (Feature Extraction Network). First, the time series data samples from different sites will enter their respective feature extraction networks as training samples, where the input f is the multi-dimensional time series features of a single site. For possible category features, we use embedding operations to convert the classification features into dimensional vectors to unify the data. First, a linear activation layer is used to obtain the nonlinear feature data of the data set, and then the gated linear network GLU is used to adaptively select the nonlinear features mapped to the hidden layer, further amplify important features, and suppress some features that may have no effect. The formula of GLU is as follows:

[0085] GLU(μ0)=(σ(w1*μ0+b2)⊙(w1*μ0+b3));

[0086] At the same time, in order to enable the model to adaptively select the degree of fitting complexity, the input is also residually connected, so that the initial input passes through the linear layer to obtain simple linear features, and finally the extracted linear features are added to the nonlinear features for adaptive feature selection. The overall structure is as follows:

[0087] μ0=tanh(w0f+b0)

[0088] μ1=w1f+b1

[0089] FEN(f)=GLU(μ0)+μ1

[0090] 103. Use the spatial attention network to connect and fuse the features extracted from different sites.

[0091] In this embodiment, the spatial attention extraction network is used to obtain the spatial attention of the features of each site after feature extraction. First, the feature sequence of the target site is passed through a shallow perceptron, i.e., a feedforward neural network (FNN), to obtain the target feature influence weight h0, where h tar is the target site feature sequence, and then the feature sequences of all sites are connected with the target site feature factor to obtain the sequence feature α i ,include:

[0092] h0=w tar h tar +b tar

[0093] α i =concat(h i ,h0)

[0094] Among them, w tar is the spatial weight of the target site; btar is the spatial offset of the target site.

[0095] Then, let the sequence features pass through a shallow multi-layer perception and a hyperbolic tangent activation function (tanh) to obtain the following attention weights:

[0096]

[0097] Next, the spatial attention weight of each site is estimated using the softmax formula:

[0098]

[0099] in is the normalized attention weight, which represents the attention weight given to different features of different sites, where i represents the site subscript, j represents the j-th dimension feature of the site, and L represents the number of features in each site.

[0100] 104. Put the processed features into a multi-layer bidirectional LSTM to obtain the positive and reverse trends of the data to extract the complex periodic characteristics of PM2.5 concentration.

[0101] Thanks to the complex gating mechanism, LSTM can selectively store and discard information in long time series data. Therefore, compared with traditional RNN, it can effectively alleviate the problems of gradient explosion and gradient disappearance in long time series dependency modeling. The figure shows the basic structure of the LSTM unit module. It is composed of memory cells c responsible for storing historical information. t The current hidden state h t The forget gate f t It is by h t 、x t Determined from the previous moment c t-1 The input gate is used to determine the input x t and the previous hidden state h t-1 How many messages to get from the current c t Get and output h t , the formula is as follows:

[0102] f t =σ g (W f x t +U f h t-1 +b f )

[0103] i t =σ g (W i x t+U i h t-1 +b i )

[0104] o t =σ g (W o x t +U o h t-1 +b o )

[0105] c t =f t ⊙c t-1 +i t ⊙σ c (W c x t +U c h t-1 +b c )

[0106] h t =o t ⊙tanh(c t )

[0107] Among them, σ g () represents the sigmoid function; W f represents the weight of the forget gate; x t Indicates the current input; U f represents the weight of the forget gate; b f W represents the forget gate bias; i represents the input gate weight; U i represents the input gate weight; b i represents the input gate weight; W o represents the output gate weight; U o represents the output gate weight; b0 represents the output gate bias; i t represents the result of the input gate; σ c () represents the sigmoid function. In this embodiment, the subscripts g and c of the sigmoid function only indicate that the sigmoid function is used for certain areas, such as σ g () is the sigmoid function of the forget gate, input gate, and output gate, σ c () represents the sigmoid function of the current cell; W c Represents the current cell state weight; U c represents the current cell state weight; b c Indicates the current cell state bias.

[0108] As a periodic time simulator, the Bi-LSTM module can capture the forward and reverse trends of the data and can be used to extract the periodic characteristics of PM2.5 concentration. Therefore, we use a multi-layer bidirectional LSTM to extract the time characteristics of the network. Compared with the use of a unidirectional LSTM to obtain the periodic characteristics of time series data, it carries richer feature information and can capture the forward and backward trends of periodic data at the same time. This can more efficiently utilize the time series information of the data, thereby enabling more accurate prediction of time series data.

[0109] 105. The known future feature data is passed through the embedding layer and then the feature extraction network is used to obtain the future features, the past features are connected, and finally the regression results are output through the linear layer.

[0110] Affected by human natural activities and meteorological conditions, PM2.5 has obvious multi-scale periodicity in many regions around the world. Known future feature data include seasons, upcoming holidays, weekends, etc. Unlike continuous digital features such as PM2.5, these future time period features can be obtained in advance. They have a direct time correspondence with the predicted results and play an auxiliary role in the final prediction results.

[0111] In order to unify the numerical features and categorical features, an embedding operation is used to convert the categorical features into a dimensional vector. This operation is similar to word embedding in natural language processing tasks and can be trained through the network. In this way, the categorical features have "semantic" meaning and can be directly input into the neural network. After the future time features that have undergone the embedding operation are extracted, they are connected and fused with the processed past time series features to obtain the regression prediction results.

[0112] 106. Using a loss function that takes into account the standard deviation fluctuation and mean error of the data, the network is iteratively trained until convergence.

[0113] In this invention, we use a loss function containing L2 weight regularization to update the parameters in the model, which can prevent the deep network from overfitting and avoid the situation where a certain feature in the model is better than the model's prediction performance. The loss function can be defined as follows:

[0114]

[0115] The MSE is defined as follows: the mean square error in parameter estimation refers to the expected value of the square of the difference between the parameter estimate and the true value of the parameter, where N is the number of prediction samples.

[0116]

[0117] Std *, Std is defined as follows. Standard deviation is also called standard deviation or experimental standard deviation. It is most commonly used in probability statistics as a measure of the degree of statistical distribution. Standard deviation is the arithmetic square root of variance. Standard deviation can reflect the degree of dispersion of a data set. Two sets of data with the same mean may not have the same standard deviation. Therefore, in order to simultaneously reflect the difference in data fluctuations between the predicted sequence and the actual sequence, we add the standard deviation to the formula, where Std * represents the predicted sequence, Std represents the real sequence, and y - represents the mean of the real series; y *- represents the mean of the forecast sequence; y (j) Represents the value of the jth element in the real sequence; Represents the value of the jth element in the predicted sequence. Each element in the predicted sequence and the true sequence is a site.

[0118]

[0119]

[0120] w2 is L2 regularization, and the definition of w2 is expressed as:

[0121]

[0122] Among them, λ is the regularization parameter, w i is the weight parameter of the ith neural network, and M is the number of parameters in the neural network. The purpose of setting w2 is to limit the number of parameters to be too large or too large to avoid making the model more complex. For example, when using a polynomial model, if the model contains high-order polynomials, the model may be too complex and prone to overfitting. Therefore, in order to prevent overfitting, the weight of the high-order part can be limited to 0, which is equivalent to converting from a high-order form to a low-order form.

[0123] Therefore, when the length of the predicted time series sample is N, the loss function can also be written as follows.

[0124]

[0125] 107. Input the site data to be tested into the trained PM2.5 prediction network based on spatiotemporal attention, and output the prediction results.

[0126] In an embodiment of the present invention, the time series data to be measured can be input into a neural network, and the prediction length can be dynamically selected according to the actual prediction needs. At the same time, the past regression results can be output from the past time series data, or the future time feature network can be combined to output the connection fusion results of the past and future features. This model can be applied to any application involving multivariate spatiotemporal sequences, not only to deal with PM2.5, but also to provide spatiotemporal interpretability for the prediction.

[0127] In some other embodiments of the present invention, the present invention also provides a system for long-time series PM2.5 prediction based on spatiotemporal attention, the system comprising:

[0128] The time series data acquisition module is used to obtain pollutant concentration data and meteorological factor data at different stations.

[0129] The data preprocessing module is used to preprocess the pollutant concentration data and meteorological factor data of different stations.

[0130] The neural network training module based on spatiotemporal attention inputs the preprocessed training time series data samples into the feature extraction network, allowing the initial input to obtain linear features while extracting nonlinear features, and finally performing adaptive feature selection. Then, the spatial attention weights of the features of each site after feature extraction are obtained using the spatial attention extraction network to obtain the attention weights of each site. Then, a multi-layer bidirectional LSTM is used to extract the time features of the network, which can simultaneously capture the forward and backward trends of periodic data, and use the feature extraction network to obtain future features after the known future feature data is embedded in the layer, connect the past features, and finally output the regression results through the linear layer; the loss function that considers the standard deviation fluctuation and mean error of the data is used to iteratively train the neural network until convergence;

[0131] The output module is used to output the prediction results of the time series data of the site to be tested.

[0132] In some embodiments, the present invention can use the ADAM optimizer for training adjustment. After multiple rounds of training, the neural network tends to be stable and the iterative training ends. The training process is as follows: Figure 6 As shown:

[0133] After obtaining the pollutant concentration data and meteorological factor data of different stations, the data set is preprocessed;

[0134] Construct a neural network model based on spatiotemporal attention;

[0135] Train the neural network using the dataset and perform multiple iterations;

[0136] Compare the output of the neural network with the true value of the time series to find the loss until the loss becomes stable;

[0137] At this point, the training is finished and the trained neural network model is saved.

[0138] The trained neural network model is as follows: Figure 5 As shown in Figure 2, the adaptive feature extraction module is as follows: Figure 2 As shown in Fig. 2, the spatial attention feature extraction module is as follows: Figure 3 As shown in Fig. Figure 4 shown.

[0139] In some embodiments, the neural network training module includes a past feature network module and a future feature network module, wherein the past feature network module is composed of an adaptive feature extraction module, a spatial attention feature extraction module, and a temporal attention extraction module, and this network module is used to extract past features; and the future feature network module is composed of a feature embedding module and a feature extraction module.

[0140] The adaptive feature extraction module allows each site to initially input and obtain linear features while extracting nonlinear features, and finally performs adaptive feature selection. The spatial attention feature extraction module is used to obtain the spatial attention weights of each site. The temporal attention extraction module is used to obtain the complex periodic patterns of the data. The future feature network module is based on the regression results of past features, and further regression prediction is performed after integrating future temporal features.

[0141] Figure 7 This is the time series prediction result diagram of the present invention. After obtaining the original past and future time series data, the data is preprocessed by filling missing values, normalizing data, aligning data, and cleaning data to make the input more stable and reliable; the processed data is input into the PM2.5 time series prediction network based on spatiotemporal attention for feature extraction and time series prediction. From the image, it can be seen that the final output prediction result including predicted values ​​and true values.

[0142] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A long-time series PM2.5 prediction method based on spatiotemporal attention, characterized in that: The following steps are involved: Obtain different pollutant concentration data and meteorological factor data, normalize them and fill in missing values; The pre-processed meteorological data of different stations are input into their corresponding feature extraction networks for feature extraction, including: FEN(f)=GLU(μ0)+μ1; μ0=tanh(w0f+b0); μ1=w1f+b1; Among them, FEN(f) is the feature extraction network; f is the preprocessed meteorological data of different stations; GLU() is the gated linear network; w0 is the feature weight; b0 is the bias term; w1 is the feature weight; b1 is the bias term; The process of extracting nonlinear features by gated linear network is expressed as: GLU(μ0)=(σ(w1*μ0+b2)⊙(w1*μ0+b3)); Among them, GLU (μ0) is the nonlinear feature extracted according to the input data; w1 is the hidden feature weight; b2 and b3 are bias terms; ⊙ is the dot product; σ() represents the sigmoid function; The spatial attention network is used to connect and fuse the features extracted from different sites, that is, the extracted features are input into the feedforward neural network to obtain the feature factor of the site, which is expressed as: <h2 style=";text-align:left;direction:ltr">h0=w<h2 style=";text-align:left;direction:ltr"> tar <h2 style=";text-align:left;direction:ltr"> h<h2 style=";text-align:left;direction:ltr"> tar <h2 style=";text-align:left;direction:ltr"> +b<h2 style=";text-align:left;direction:ltr"> tar <h2 style=";text-align:left;direction:ltr"> ; α i =concat(h i (h0); The characteristic factors of the target site are concatenated with the characteristic factors of other sites respectively, and the concatenated values ​​are used to calculate the attention weight through the hyperbolic tangent function activation function, which is expressed as: The attention weight of each site is calculated by softmax and attention weight, expressed as: Among them, h0 is the impact weight of the target feature; w tar is the spatial weight of the target site; b tar is the spatial offset of the target site; h i is a characteristic factor of a non-target site; α i For h0 and h i The concatenated features; Represents α i The j-th dimension feature of i is the feature weight; b i is the spatial offset of the ith site; represents the importance weight of the j-th dimension of air station i, represents the spatial attention weight of the jth dimension of air station i; L is the characteristic dimension of the station, exp represents the exponential function; h tar is the characteristic sequence of the target site; The features processed by the spatial attention network are passed through a multi-layer bidirectional LSTM to obtain the past features; The known future feature data corresponding to the time period to be predicted, that is, the season and holiday information corresponding to the time period to be predicted, is converted into a dimensional vector through the embedding layer, and the future features are extracted through the neural network; After connecting the past features with the future features, the regression results are output through the linear layer to obtain the prediction results; The network is iteratively trained until convergence using a loss function that takes into account the standard deviation fluctuations and average error of the data; the loss function that takes into account the standard deviation fluctuations and average error of the data is expressed as: Among them, Loss is the loss function; MSE is the mean square error in parameter estimation; Std * is the standard deviation of the predicted sequence; Std is the standard deviation of the true sequence; w2 is L2 regularization, expressed as λ is the regularization parameter, w i is the parameter of the ith neural network; M is the number of parameters in the neural network; The data of the site to be tested is input into the trained PM2.5 prediction network based on spatiotemporal attention, and the prediction results are output.

2. According to claim 1, a long-time series PM2.5 prediction method based on spatiotemporal attention is characterized in that: The mean square error in parameter estimation is the expected value of the square of the difference between the parameter estimate and the true value of the parameter, expressed as: Where N is the number of sites; is the parameter estimate; y (j) The true value of the parameter.

3. According to the long-term PM2.5 prediction method based on spatiotemporal attention according to claim 1, it is characterized in that: The standard deviation calculation process of the predicted sequence and the actual sequence includes: Where N is the number of sites; is the value of the jth element in the predicted sequence; y *- is the mean of the predicted sequence; y (j) is the value of the jth element in the real sequence; y - is the true series mean.

4. A system for long-term PM2.5 prediction based on spatiotemporal attention, characterized in that: The system is used to implement any one of the long-time series PM2.5 prediction methods based on spatiotemporal attention described in claims 1 to 3, and the system includes a time series data acquisition module, a feature extraction module, a spatial attention network, a multi-layer bidirectional LSTM, a time series feature extraction module, a feature connection module and a prediction module; wherein: The time series data acquisition module is used to obtain pollutant concentration data and meteorological factor data at different sites, including historical data and real-time data, and train the system based on historical data; the real-time data is input into the trained system for real-time prediction; The feature extraction module is used to extract the features of the data obtained by the time series data acquisition module. The feature extraction module inputs the pre-processed meteorological data of different stations into its corresponding feature extraction network for feature extraction; and compares the features of the target station, i.e. the station to be predicted, with the features of other stations. The spatial attention network is used to obtain the attention weight of each auxiliary site. The site to be predicted is taken as the target site and other sites as auxiliary sites. The features obtained by the feature extraction module are used through a feedforward neural network to obtain the feature factor of the site. The feature factor of the target site is concatenated with the feature factor of each auxiliary site, and the attention weight is calculated through the hyperbolic tangent function activation function. The attention weight of each site is calculated through softmax and attention weight. Multi-layer bidirectional LSTM is used to extract the periodic features of the output features of the spatial attention network; The time series feature extraction module is used to obtain known future feature data, that is, the season of the time period to be predicted and the upcoming holiday information, and use embedding operations to convert the collected information into a dimensional vector, and extract its features through a neural network; The prediction module is used to obtain the features output by the multi-layer bidirectional LSTM and the features output by the time series feature extraction module, and then perform linear weighting to obtain the regression prediction result, which is used as the output of the prediction system.

Images