PM based on multi-scale spatio-temporal attention network 2.5 Prediction method

Abstract

The application relates to a PM 2.5 Prediction method belongs to the field of air quality prediction. The method comprises the following steps: S1: obtaining air quality data for a period of time, and preprocessing the air quality data to obtain a feature data set; S2: constructing a space-time hybrid model MCMANet, including a time module for capturing the time characteristics of the air quality data; a space module for obtaining the spatial characteristics of the air quality data; and a prediction module for predicting the future PM 2.5 concentration by fusing the time characteristics and the space characteristics; S3: inputting a training set into the MCMANet model for training until the model outputs the most accurate PM 2.5 concentration prediction value; and S4: inputting the preprocessed air quality data at the current moment into the trained MCMANet model to predict the future PM 2.5 concentration. The method can comprehensively improve the accuracy and reliability of PM 2.5 prediction.

Description

Technical Field

[0001] This invention belongs to the field of air quality prediction, and involves atmospheric remote sensing, deep learning, and PM2.5. 2.5 Prediction techniques, specifically involving a PM based on a multi-scale spatiotemporal attention network. 2.5 Prediction methods. Background Technology

[0002] PM 2.5 Air quality refers to particulate matter with an aerodynamic diameter of 2.5 micrometers or less. Due to its small size, it can adsorb a variety of toxic and harmful pollutants. Air quality forecasting technology has developed along two core paths: traditional forecasting methods and deep learning methods, forming a multi-layered technical system covering numerical simulation, statistical analysis, and data-driven approaches.

[0003] Traditional prediction methods are based on numerical simulation models, statistical models, and classical machine learning.

[0004] Common numerical simulation models include the Community Multiscale Air Quality (CMAQ), the Comprehensive Air Quality Model with Extensions (CAMx), the Weather Research and Forecasting Model with Chemistry (WRF-Chem), and the Nested Air Quality Prediction Model System (NAQPMS). However, the application of traditional numerical simulation models relies heavily on extensive prior domain knowledge and suffers from insufficient generalization performance.

[0005] Common statistical models include Autoregressive Moving Average (ARMA), Autoregressive Integrated Moving Average (ARIMA), and Multivariable Linear Regression (MLR). However, statistical models struggle to effectively capture the complex nonlinear dynamic characteristics of data.

[0006] In the field of air quality forecasting, widely used classic machine learning models include Support Vector Regression (SVR) and Random Forest Regression (RFR). However, these machine learning models do not take into account the spatial characteristics of air pollutants and are difficult to handle large-scale spatiotemporal data flexibly.

[0007] Deep learning methods, based on pure temporal models, have further developed into hybrid models that integrate spatiotemporal information.

[0008] Mainstream time-series prediction models include Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Networks (CNN), and Transformer, which can be used for air quality prediction tasks at different scales, including short, medium, and long term. However, traditional recurrent neural networks such as LSTM and GRU suffer from long-sequence memory forgetting defects when processing long-term air quality data, making it difficult to retain early key temporal features. Attention-based time-series models only focus on the numerical correlation of a single time step, ignoring the concentration change trends over continuous local periods, and cannot effectively characterize the temporal evolution patterns of pollutants.

[0009] In the technical system of spatial information modeling, representative model architectures mainly include CNNs, Graph Convolutional Networks (GCNs), and attention mechanisms. However, existing methods do not employ a combination of multi-scale convolution and max pooling to construct multi-scale temporal features, relying solely on a single time scale for modeling. This fails to simultaneously consider the fine-grained instantaneous fluctuations and coarse-grained long-term periodic characteristics of pollutant concentrations. Modeling spatial dependencies between stations at only a single time step severs the continuity of the temporal dimension, failing to uncover the dynamic spatiotemporal correlations of pollutants between stations over continuous periods, and artificially fragmenting spatiotemporal interaction characteristics.

[0010] Therefore, a new PM is urgently needed. 2.5 Prediction methods are used to address the shortcomings of existing air quality prediction technologies. Summary of the Invention

[0011] In view of this, the purpose of this invention is to provide a PM based on a multi-scale spatiotemporal attention network. 2.5The prediction method addresses the following issues: (1) It addresses the shortcomings of traditional prediction models in expressing complex nonlinear relationships and capturing complex spatiotemporal dependencies. (2) It addresses the problems of single-time-granular modeling and missing local temporal semantics in existing models, overcomes the defects of long-term memory forgetting in recurrent networks and neglecting local time-period trends in a single time step, and improves the model's ability to model the continuous temporal evolution patterns of pollutants. (3) It addresses the technical problem of lacking multi-scale temporal feature modeling in existing methods, constructs multi-scale temporal features by combining multi-scale convolution and pooling, realizes the simultaneous extraction of fine-grained fluctuations and coarse-grained long-term periodic features of pollutants, and makes up for the inherent limitations of single-scale modeling. (4) It addresses the problems of single-time-step modeling of spatial features and separation of spatiotemporal characteristics in traditional methods, breaks the limitation of single-time-step spatial modeling, models the dynamic spatiotemporal dependencies between sites in a continuous time period, and truly restores the spatiotemporal characteristics of pollutants.

[0012] This invention comprehensively improves the accuracy and reliability of PM2.5 prediction by constructing a prediction model that integrates multi-scale time series fusion, dynamic spatiotemporal correlation, and multi-pollutant synergy, specifically achieving the following objectives: (1) By fully utilizing the temporal and spatial information of monitoring stations, it realizes the prediction of PM2.5 within the study area. 2.5 (2) Model the complex spatiotemporal relationships of air quality data using a multi-scale convolutional max-pooling attention network (MCMANet) to improve the model’s ability to process and adapt to large-scale spatiotemporal data. (3) Improve the temporal feature extraction mechanism, retain key temporal information, fully capture the evolution pattern of pollutants over continuous periods, and make up for the shortcomings of missing local temporal semantics. (4) Construct a multi-scale temporal feature extraction and fusion system to simultaneously capture the fine-grained fluctuations and coarse-grained long-term periodic features of pollutants, breaking through the limitations of single-scale modeling. (5) Realize spatiotemporal feature joint modeling, explore the dynamic spatial dependence of monitoring stations over continuous periods, and capture the spatiotemporal features of pollutants.

[0013] To achieve the above objectives, the present invention provides the following technical solution: A PM based on multi-scale spatiotemporal attention network 2.5 The prediction method specifically includes the following steps: S1: Obtain air quality data for a period of time (including Air Quality Index (AQI) and concentrations of six pollutants: PM2.5). 2.5 PM 10 The dataset contains NO2, CO, O3, and SO2, which are preprocessed to obtain a feature dataset, which is divided into a training set, a test set, and a validation set.

[0014] S2: Construct a spatiotemporal hybrid model (MCMANet), including a time module, a spatial module, and a prediction module; the time module is used to effectively capture the temporal characteristics of air quality data; the spatial module is used to acquire the spatial characteristics of air quality data; the prediction module predicts future PM levels by fusing the time and spatial characteristics. 2.5 concentration.

[0015] S3: Input the training set into the MCMANet model for training until the model outputs the most accurate PM. 2.5 Concentration prediction; S4: Input the preprocessed air quality data of the current moment into the trained MCMANet model to predict the future PM2.5 levels. 2.5 concentration.

[0016] Furthermore, in step S1, the air quality data is preprocessed, specifically including: data normalization, and missing values ​​are filled using linear interpolation.

[0017] Further, in step S2, the time module specifically involves: inputting the preprocessed data into the time module according to different features for initial mapping; extracting time dimension features from the mapped data through the LFE module and adding position encoding to the time dimension features; then performing multi-head attention on the time dimension at the same scale, and then obtaining fused time features through the MSA module; wherein, the LFE module represents the local feature extraction module, and the MSA module represents the multi-scale time information aggregation module.

[0018] Furthermore, in step S2, the LFE module specifically includes: performing multi-scale convolution and max pooling operations in the time dimension, and then fusing the features of convolution and max pooling at the same scale through a learnable weight w.

[0019] Furthermore, in step S2, the MSA module specifically includes: aggregating the features after multi-head attention on the same time dimension using an attention mechanism, then concatenating the multi-scale features, generating normalized weights for three scales using MLP, summing them according to the dynamic weights, and outputting the fused time features.

[0020] Furthermore, in step S2, the spatial module specifically includes: splicing and fusing the time features output by the time module, performing multi-head attention calculation between stations in the spatial dimension, and obtaining spatiotemporal information through residual connection and normalization.

[0021] Furthermore, in step S2, the prediction module specifically includes: taking the spatiotemporal information processed by the time module and the spatial module, and implementing PM through a two-dimensional convolution and a fully connected layer. 2.5 Concentration prediction.

[0022] Furthermore, when training the MCMANet model, when the model outputs the most accurate PM... 10 When predicting the concentrations of NO2, CO, O3, or SO2, the output in step S4 will be the future concentration of the corresponding pollutant.

[0023] The beneficial effects of this invention are as follows: (1) A multi-scale dual-path temporal feature extraction system was constructed, which broke through the limitations of single-scale modeling and accurately captured PM. 2.5 This study examines both fine-grained instantaneous fluctuations and coarse-grained long-term periodic characteristics of pollutant concentration. A dual-path feature extraction branch is constructed using convolutional kernels with strides of 3, 6, and 12, along with corresponding max-pooling. Learnable gating weights adaptively fuse the detailed features of convolution and the global features of pooling. This approach preserves abrupt changes in pollutant concentration while capturing periodic evolution patterns, effectively addressing the shortcomings of traditional time-series models that lack local temporal semantics and significantly improving the ability to model complex time-series patterns.

[0024] (2) A scale-independent multi-head self-attention plus query-based global temporal aggregation mechanism is proposed, which fully preserves key temporal information and enhances the ability to capture long-sequence dependencies. This invention configures independent projection and multi-head self-attention layers for each time scale, avoiding mutual interference between features at different scales; at the same time, a learnable temporal aggregation query vector is introduced, which dynamically focuses on the time segment that contributes the most to the prediction through similarity calculation, realizing refined global aggregation of temporal features, and further improving the model's accuracy in depicting the continuous evolution of pollutants.

[0025] (3) Multi-pollutant collaborative time-series modeling was realized, making full use of the correlation between air pollutants to improve PM2.5 levels. 2.5 Prediction accuracy. This invention improves PM prediction accuracy. 2.5 PM 10 Five key atmospheric indicators—AQI, NO2, CO, and PM2.5—were analyzed for multi-scale time-series feature extraction. Then, a fusion layer was used to integrate multivariate information, uncovering the co-evolutionary patterns of different pollutants during their formation, transport, and dissipation. This approach compensates for the limitations of the single-variable PM2.5 analysis. 2.5 The inability to utilize cross-correlation information in prediction significantly improves the model's accuracy in predicting complex pollution processes.

[0026] (4) A dynamic multi-head self-attention mechanism at the station dimension was constructed to accurately capture the dynamic spatial dependencies between monitoring stations. After the fusion of multivariate features, the present invention performs multi-head self-attention calculation along the station dimension. It does not require the pre-definition of a fixed spatial adjacency matrix and can automatically learn the spatial relationships between stations, effectively characterizing the spatial characteristics of pollutants. It realizes the joint modeling of temporal and spatial features and comprehensively improves the model's ability to process and adapt to large-scale spatiotemporal air quality data.

[0027] (5) The MCMANet model constructed in this invention is used in PM 2.5 It exhibits excellent performance in prediction tasks, with a prediction accuracy (R²). 2 The value is 0.950. The method of this invention can provide high-precision and high-reliability technical support for regional air quality early warning and atmospheric pollution prevention and control decision-making.

[0028] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a diagram of the overall architecture of the MCMANet model of the present invention; Figure 2 Here is a structural diagram of the Local Feature Extraction (LFE) module; Figure 3 This is a structural diagram of the Multi-Scale Temporal Information Aggregation (MSA) module; Figure 4 For PM 2.5 Concentration prediction flowchart; Figure 5 For PM 2.5 A comparison chart of actual and predicted concentration values. Detailed Implementation

[0030] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0031] Please see Figures 1-5 This invention provides a PM based on a multi-scale spatiotemporal attention network. 2.5 The prediction method specifically includes the following steps: Step 1: Obtain Resources: Hourly air quality data from the Environmental Monitoring Center. The data includes the Air Quality Index (AQI) and concentrations of six pollutants: PM2.5.2.5 PM 10 NO2, CO, O3, and SO2. The data is preprocessed, and missing values ​​are filled using linear interpolation.

[0032] Step 2: Input the preprocessed feature dataset from Step 1 into the spatiotemporal mixture model (MCMANet) to predict the future PM. 2.5 Concentration. The input sample dimension information is (b,t,s,f), where b represents the batch size, t represents the time step length, and s and f represent the number of monitoring stations and the number of features, respectively.

[0033] The MCMANet model can fully utilize historical temporal and spatial information surrounding air quality monitoring stations. The overall architecture of the MCMANet model is as follows: Figure 1 As shown, it mainly consists of three modules: a temporal module for effectively capturing the temporal dependence of air quality data; a spatial module for acquiring the spatial relationships of air quality data; and finally, a prediction module to fuse the temporal and spatial features to predict future PM2.5 levels. 2.5 Concentration. This spatiotemporal hybrid modeling method effectively improves PM concentration. 2.5 Accuracy of concentration prediction.

[0034] (1) Time model Attention mechanism is the basic computational module of MCMANet. Its core principle is to adaptively allocate weights by measuring the correlation between elements in a time series, thereby capturing the global dependency features of the sequence. Its formula is (1):

[0035] in, , and These represent the query, key, and value, respectively. For the dimensions of query and key.

[0036] Multi-head attention consists of multiple attentions, and its formula is (2):

[0037] in, ; , ,and All are learnable parameter matrices. Indicates the original data dimension. , Indicates the number of parallel attentions. .

[0038] The processed data is input independently into the time module of the model according to different features, at which point f=1. Initial mapping is then performed on the data, and the mapped data... The dimension d is 32-dimensional. For example... Figure 2 As shown, multi-scale convolution and max pooling operations are performed in the time dimension, and their formulas are (3) and (4).

[0039]

[0040]

[0041] in, To represent different scales, this model uses three scales for modeling: s1=3, s2=6, and s3=12. This represents the results obtained after convolution at different scales; This represents the result obtained after max pooling at different scales. Features from convolution and max pooling at the same scale are fused using a learnable weight w, as shown in formula (5). Positional encoding is added along the time dimension. The formula is (6).

[0042]

[0043]

[0044] in, This represents the result of convolution and max pooling at the same scale, dynamically fused with learnable weights. This indicates a learnable positional encoding.

[0045] like Figure 3 As shown, multi-head attention and convergent attention are performed on the same time dimension to fully extract and aggregate time information, as shown in formulas (7) and (8).

[0046]

[0047]

[0048] in, This represents the data after multi-head attention has been implemented over the time dimension;

[0049] This represents data aggregated along the time dimension; This represents the time-aggregated information that can be learned.

[0050] Multi-scale features are concatenated, and MLP generates normalized weights for three scales. The weights are then summed according to the dynamic weights to output the fused features, as shown in formula (9).

[0051]

[0052] in, This represents the output feature after weighted fusion of multi-scale features.

[0053] (2) Spatial module After modeling each pollutant separately using the time module, the output fusion feature X of each variable is obtained. PM2.5 ,…, X CO After concatenating and fusing the variables (formula (10)), multi-head attention calculation is performed between sites in the site dimension (i.e., spatial dimension), and spatiotemporal information is obtained through residual connection and normalization. The formula is (11).

[0054]

[0055]

[0056] in, This indicates the characteristics of the individual pollutants after they have been spliced ​​and fused together.

[0057] (3) Prediction module The features processed by the time and space modules are used to predict pollutant concentrations through two-dimensional convolution and fully connected layers, as shown in formula (12).

[0058]

[0059] in, t represents the prediction result. pre This indicates the length of the prediction time step.

[0060] Step 3, Verification Experiment: Experimental data: Air quality data for multiple regions from January 1, 2022 to December 31, 2024, obtained from the National Environmental Monitoring Center. The data includes the Air Quality Index (AQI) and concentrations of six pollutants: PM2.5. 2.5 (Unit: μg / m³) 3 PM 10 (Unit: μg / m³) 3 NO2 (unit: μg / m³) 3 CO (unit: mg / m³) 3 O3 (unit: μg / m 3 ) and SO2 (unit: μg / m 3 ).

[0061] Experimental parameters: batch size is 64, historical time step is 24, prediction step is 1, loss function is mean squared error, Adam optimizer is used, learning rate is 0.0001, number of iterations is 100, and early stopping mechanism is used.

[0062] Experimental results: Figure 5 Showcase PM 2.5 The comparison between the actual and predicted concentration values ​​shows that the MCMANet model proposed in this invention is effective in PM2.5 concentration analysis. 2.5 It exhibits excellent performance in prediction tasks, with a prediction accuracy (R²). 2 The value is 0.950.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A PM based on a multi-scale spatiotemporal attention network 2.5 The prediction method is characterized by, The method includes the following steps: S1: Acquire air quality data over a period of time, preprocess it to obtain a feature dataset, which is divided into a training set, a test set, and a validation set; S2: Construct a spatiotemporal hybrid model MCMANet, including a time module, a spatial module, and a prediction module; the time module is used to capture the temporal characteristics of air quality data; the spatial module is used to acquire the spatial characteristics of air quality data; the prediction module predicts future PM levels by fusing the time and spatial characteristics. 2.5 concentration; S3: Input the training set into the MCMANet model for training until the model outputs the most accurate PM. 2.5 Concentration prediction; S4: Input the preprocessed air quality data of the current moment into the trained MCMANet model to predict the future PM2.5 levels. 2.5 concentration.

2. The PM based on a multi-scale spatiotemporal attention network as described in claim 1 2.5 The prediction method is characterized by, In step S1, the air quality data is preprocessed, specifically including: data normalization, and missing values ​​are filled using linear interpolation.

3. The PM based on a multi-scale spatiotemporal attention network as described in claim 1 2.5 The prediction method is characterized by, In step S2, the time module specifically involves: inputting the preprocessed data into the time module according to different features for initial mapping; extracting time dimension features from the mapped data through the LFE module and adding position encoding to the time dimension features; then performing multi-head attention on the time dimension at the same scale, and then obtaining fused time features through the MSA module; wherein, the LFE module represents the local feature extraction module, and the MSA module represents the multi-scale time information aggregation module.

4. The PM based on a multi-scale spatiotemporal attention network as described in claim 3 2.5 The prediction method is characterized by, In step S2, the LFE module specifically includes: performing multi-scale convolution and max pooling operations in the time dimension, and then fusing the features of convolution and max pooling at the same scale through a learnable weight.

5. The PM based on a multi-scale spatiotemporal attention network as described in claim 3 2.5 The prediction method is characterized by, In step S2, the MSA module specifically includes: aggregating the features after multi-head attention on the same scale time dimension using an attention mechanism, then concatenating the multi-scale features, generating normalized weights for three scales using MLP, summing them according to the dynamic weights, and outputting the fused time features.

6. The PM based on a multi-scale spatiotemporal attention network as described in claim 1 2.5 The prediction method is characterized by, In step S2, the spatial module specifically includes: splicing and fusing the time features output by the time module, performing multi-head attention calculation between stations in the spatial dimension, and obtaining spatiotemporal information through residual connection and normalization.

7. The PM based on a multi-scale spatiotemporal attention network as described in claim 1 2.5 The prediction method is characterized by, In step S2, the prediction module specifically includes: taking the spatiotemporal information processed by the time module and the spatial module, and implementing PM through a two-dimensional convolution and a fully connected layer. 2.5 Concentration prediction.

8. The PM based on a multi-scale spatiotemporal attention network as described in claim 1 2.5 The prediction method is characterized by, In step S1, the air quality data includes the air quality index and the concentrations of six pollutants, namely PM2.

5. 2.5 PM 10 NO2, CO, O3 and SO2.

9. The PM based on a multi-scale spatiotemporal attention network as described in claim 8 2.5 The prediction method is characterized by, When training the MCMANet model, the model output is the most accurate PM. 10 When predicting the concentrations of NO2, CO, O3, or SO2, the output in step S4 is the predicted future concentration of the corresponding pollutant.

Images