PM2.5 concentration prediction method fusing RF-RFE and TCN-BiLSTM-Attention

Abstract

The invention relates to a PM2.5 concentration prediction method based on RF-RFE and TCN-BiLSTM-Attention, and belongs to the technical field of atmospheric environment monitoring. According to the method, firstly, PM2.5 concentration monitoring data and data of related influence factors such as weather, other pollutants and time characteristics are preprocessed, recursive feature elimination (RFE) is combined with random forest (RF) for feature screening, and a time sequence input sequence is constructed; extracting local time sequence features of the sequence through a time convolutional network (TCN), and capturing bidirectional long-range time sequence dependence features of the sequence by using a bidirectional long-short-term memory network (BiLSTM); an attention mechanism is introduced to endow a BiLSTM output feature vector sequence with a differentiation weight, and key time sequence feature contribution is highlighted; and finally, outputting a PM2.5 concentration predicted value through a full connection layer, and completing model training and optimization based on the training set and the verification set. According to the method, the advantages of TCN, BiLSTM and Attention mechanisms are combined, the precision and stability of PM2.5 prediction are improved, and the method can be widely applied to atmospheric pollution early warning and environmental governance decision making.

Description

Technical Field

[0001] This invention belongs to the field of air quality prediction, specifically, it designs a PM prediction method based on RF-RFE, TCN, BiLSTM, and attention mechanisms. 2.5 Concentration prediction methods. Background Technology

[0002] With the rapid advancement of industrialization and urbanization, air pollution has become an increasingly prominent problem, particularly fine particulate matter (PM2.5). 2.5 PM2.5 is a core pollutant affecting air quality, and its concentration changes are directly related to human health, the ecological environment, and the safety of public transportation. 2.5 With its small particle size and large specific surface area, PM2.5 easily adsorbs harmful substances such as heavy metals and volatile organic compounds. Long-term exposure can induce various diseases of the respiratory and cardiovascular systems, posing a serious threat to public health. Meanwhile, PM2.5... 2.5 Sudden increases in PM2.5 concentrations can also lead to reduced visibility, traffic congestion, and other problems, disrupting the normal functioning of the social economy. Therefore, achieving PM2.5 control is crucial. 2.5 Accurate and efficient concentration prediction is of great practical significance and application value for issuing early air quality warnings, formulating pollution prevention and control measures, protecting public health, and ensuring the orderly operation of cities.

[0003] Due to the importance of air quality forecasting, numerous studies have been conducted. Looking back at the research history, the methods have generally evolved through four stages: traditional statistical models, machine learning, deep learning, and combined models. Early studies primarily used classic statistical methods such as ARIMA, SARIMA, and multiple linear regression (MLR), extrapolating from the linear or seasonal characteristics of historical concentration series. These models were simple in structure and highly interpretable, but they did not address the issue of PM2.5 in the atmospheric system. 2.5 Concentration changes are influenced by a combination of factors, including meteorological conditions, pollution source emissions, and topography, making it difficult for such models to characterize the complex nonlinear and high-dimensional interactions within the atmospheric system. While machine learning algorithms such as Support Vector Regression (SVR), Random Forest (RF), and Gradient Boosting Tree (GBDT) have been introduced, significantly improving the accuracy of short- to medium-term predictions, they remain limited by feature engineering and shallow structures. With increased computing power and the accumulation of large amounts of atmospheric environmental data, deep learning models such as Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), ConvLSTM (Convolutional Long Short-Term Memory), TCN, and attention mechanisms have been introduced into PM2.5 systems. 2.5 In the field of prediction, the spatiotemporal dependencies in pollutant concentration changes have been further explored. However, the accuracy and robustness of single deep learning models remain limited. Therefore, combining different models to complement each other's strengths has become a research hotspot in the field.

[0004] Application publication number CN118366564A, a future PM in cities 2.5 Concentration prediction methods and systems, including: constructing PM2.5 concentration prediction systems using Pearson correlation coefficients. 2.5 The correlation coefficient matrix between air pollutant concentration and other gas concentrations and meteorological characteristics is used to evaluate the linear relationship between input variables and determine the input features of the LSTM network model and the TCN combined model with spatiotemporal attention mechanism. The spatiotemporal attention mechanism is introduced to extract the temporal dependencies between each time step in the urban pollutant concentration sequence dataset and the relationships between the input features at each time step. The LSTM network model and the TCN combined model with spatiotemporal attention mechanism are constructed. The proposed model is used to extract and fuse input features. After model training parameter tuning, air pollutant concentration and meteorological data are used as inputs to the trained prediction model for prediction. The root mean square error of the two models is used to adaptively inversely weighted fuse the prediction results of the two network models to obtain the final prediction result.

[0005] The above invention proposes an effective PM 2.5 While predictive methods exist, they still have the following limitations: Pearson correlation coefficient primarily reflects the linear relationship between variables, while PM... 2.5 There may be non-linear relationships with other variables (such as temperature and PM). 2.5 Concentration often exhibits a non-linear relationship, which may prevent the identification of complex interactions between multiple variables and miss potential key feature combinations; LSTM and Attention-TCN are simply parallelized: the two models are trained independently and then fused, but the parallel architecture leads to independent extraction of the two types of features without deep fusion, failing to fully utilize the complementary capabilities of the models; unidirectional LSTM can only capture temporal dependencies from a single "history → future" direction, while PM 2.5 Concentration changes exhibit long-range, two-way dependence: current concentrations are not only cumulatively affected by historical pollution emissions and meteorological conditions, but their evolution trends also imply a response to future diffusion potential (e.g., persistent high humidity and stable weather patterns suggest that diffusion capacity will remain limited in the coming hours). Two-way LSTM can capture such time-series dynamics more comprehensively by modeling both forward and reverse sequences.

[0006] Based on this, the present invention proposes a new PM 2.5 Concentration prediction model: A TCN-BiLSTM-Attention model combining RFE-RF. This model integrates a Temporal Convolutional Network (TCN), a Bidirectional Long Short-Term Memory Network (BiLSTM), and an attention mechanism, and combines RFE-RF for feature selection, thereby reducing model complexity and improving prediction accuracy. Specifically, TCN effectively extracts historical PM through multi-layer causal dilated convolutions. 2.5This model integrates the local time-series characteristics and medium- to long-term dependencies of multiple covariates (such as temperature, humidity, wind speed, air pressure, SO2, NO2, CO, etc.); BiLSTM further models the dynamic interactions and nonlinear evolution among these variables; and the attention mechanism, through adaptive weight allocation, highlights the contribution weights of key features and important time steps in the current prediction. By combining the advantages of different deep learning models, this model achieves PM... 2.5 High-precision prediction of concentration. Summary of the Invention

[0007] This invention aims to solve the problems of the prior art. It proposes a PM based on RF-RFE and TCN-BiLSTM-Attention. 2.5 Concentration prediction method. The technical solution of this invention is as follows:

[0008] A PM based on RF-RFE and TCN-BiLSTM-Attention 2.5 Concentration prediction method, which includes the following steps:

[0009] S1, Get Historical PM 2.5 Concentration monitoring data, along with corresponding meteorological data, other pollutant concentration data, and time characteristic data, constitute a multi-source input dataset. Preprocessing of the multi-source input dataset, including missing value imputation, outlier handling, feature filtering, and normalization, is performed to construct a standardized time-series input sequence.

[0010] S2, Local Temporal Feature Extraction: The temporal input sequence constructed in step S1 is input into the temporal convolutional network TCN, and the local temporal features of the temporal input sequence are extracted through the temporal convolutional network TCN.

[0011] S3, Bidirectional long-range temporal dependency feature capture: The local temporal features extracted in step S2 are input into the Bidirectional Long Short-Term Memory (BiLSTM) network. The BiLSTM network captures the bidirectional long-range temporal dependency features of the sequence and outputs a feature vector.

[0012] S4, Key Feature Weight Allocation: Introduce a time step attention mechanism to calculate the weights of the feature vector sequence output from step S3, assign differentiated weights to features at different time steps, and highlight the contribution of key temporal features.

[0013] S5, Predicted value output: The feature vector processed in step S4 is input into the fully connected layer, and the PM of the next time step is output through the fully connected layer. 2.5 The concentration was predicted, and the predicted values ​​were then normalized to restore them.

[0014] Furthermore, in step S1:

[0015] S11, Input data: The other pollutant concentration data includes hourly concentration data of PM10, SO2, NO2, O2 and CO, and the meteorological data includes hourly data of temperature, air pressure, humidity, rainfall, wind direction and wind speed.

[0016] S12, Handling missing and outlier values: For missing values ​​in the dataset that account for less than or equal to 10% and whose data trend is stable, linear interpolation is used for imputation; box plots are used to identify outliers.

[0017] S13, Feature Selection: Other pollutant variables and meteorological variables are evaluated by combining Recursive Feature Elimination (RFE) with Random Forest (RF) method; based on the feature importance score of the RF model, the least important features are gradually eliminated by the RFE algorithm until a preset number of key features are retained;

[0018] S14, Normalization: The Min-Max normalization method is used to map all feature values ​​to the [0,1] interval, constructing a time-series input sequence with uniform dimension and standardized distribution. The normalization formula is as follows:

[0019]

[0020] in, This represents the original data. For the normalized data, This represents the maximum value in the input data. This represents the minimum value in the input data. The data scale is adjusted by a linear ratio, and the original data is mapped to the [0,1] interval by a linear transformation.

[0021] S15, Constructing a Standard Time Series Input Sequence: A fixed-length sliding window of length T is used to slice the original multivariate time series data. The window moves forward sequentially at 1-hour intervals to achieve rolling updates of the input samples; where PM is the value at the next future time step. 2.5 Concentration is used as a prediction label, and the model learns and infers based solely on current and historical observations.

[0022] Furthermore, step S2 specifically includes:

[0023] The standard time series input sequence constructed in step S1 is input into the TCN. The input of the TCN is a three-dimensional tensor with the shape (T, D), where T represents the number of time steps in the historical observation window and D represents the number of multi-dimensional environmental variables contained in each time step. The TCN extracts multi-scale local time series features and generates context-enhanced sequence representations with long-range awareness capabilities, providing input for subsequent global dependency modeling of bidirectional LSTM.

[0024] Furthermore, the TCN module uses a one-dimensional network architecture and applies a zero-padded strategy to the front of the input sequence. It employs four causal convolutional layers, each containing an independent convolutional kernel and a specific dilation rate, and enables residual connections. The TCN dilated convolution calculation formula is as follows:

[0025]

[0026] Where u is the kernel size; f is the filter coefficient of the TCN; v is the kernel index; and z is the variable value. i-dv To perform convolution calculations on historical pollutant and meteorological data, d is the dilation rate, and F(i) is the dilation convolution output value at the i-th time step.

[0027] Furthermore, in step S3, the BiLSTM module adopts a two-layer stacked bidirectional LSTM structure: the number of hidden units in each direction of the first bidirectional LSTM is 64, the number of hidden units in the second layer is 32, and a Dropout layer with a dropout rate of 0.2 is inserted in the middle; all LSTM layers output complete time series, and by splicing forward and backward hidden states, the layer-by-layer deepening modeling of bidirectional temporal dependencies is realized.

[0028] Furthermore, in step S4, the attention mechanism module is used to perform weighted fusion of the temporal features output by the bidirectional long short-term memory network. The feature tensor output by the BiLSTM module is transposed along the feature dimension and the time dimension to obtain a tensor of shape (batch_size, D, T). A fully connected layer is used to perform a linear transformation on the transposed tensor along the time step dimension, and the output dimension is equal to the number of time steps. The hyperbolic tangent tanh nonlinear activation function and softmax normalization function are applied sequentially to the linear transformation result to generate an attention weight matrix along the time step dimension. The generated attention weights are adjusted to the same time step-feature dimension arrangement as the input sequence, and the attention weights are multiplied element-wise with the original input feature tensor to obtain the weighted attention feature representation.

[0029] Furthermore, in step S5, a "linear transformation + nonlinear activation" is performed on the hidden state of the BiLSTM at each time step, mapping the original hidden state to an unnormalized attention score to measure the importance of that time step. The attention score e t The calculation expression is as follows:

[0030]

[0031] Among them, h tis the hidden state at time step t output by BiLSTM; W is the weight matrix, b represents the bias vector, and tanh is the hyperbolic tangent function used to introduce nonlinearity;

[0032] The unnormalized attention scores are converted into probabilistic weights, ensuring that the sum of the weights over all time steps is 1. The calculation expression is as follows:

[0033]

[0034] Among them, a tj In order to obtain the output of the j-th time step, the model assigns importance weights to the information at the j-th time step; This represents the attention score at the j-th time step corresponding to the t-th output time.

[0035] Finally, using the normalized attention weights as coefficients, the hidden states of the BiLSTM at all time steps are weighted and summed to obtain the output that incorporates global time step information. The calculation expression is as follows:

[0036]

[0037] Among them, S t Let be the feature vector after weighted fusion by the attention mechanism at time step t.

[0038] An electronic device includes a memory and a processor, the memory storing program code executable by the processor, wherein the processor executes a PM fusion of RF-RFE and TCN-BiLSTM-Attention as described in any one of the invention. 2.5 Concentration prediction methods.

[0039] A computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform a PM that integrates RF-RFE and TCN-BiLSTM-Attention as described in any one of the claims. 2.5 Concentration prediction methods.

[0040] The advantages and beneficial effects of this invention are as follows:

[0041] 1) RF-RFE Feature Screening (Step S1 of claim): Selecting multi-dependency statistical correlation (such as Pearson coefficient) or fixed threshold filtering is difficult to capture the nonlinear contribution relationship between features and concentration. This invention effectively screens features that contribute to PM by introducing a recursive feature elimination mechanism based on random forests. 2.5 Concentration prediction has a key feature that contributes significantly. By eliminating redundant and noisy features, the input dimension is reduced, and the model training efficiency and generalization ability are improved.

[0042] 2) TCN-BiLSTM-Attention Cascade Fusion (steps S2-S4 of claims): Existing methods mostly employ a single TCN / BiLSTM or a "parallel dual-model" design. This invention, however, adopts a cascaded architecture of "TCN (local temporal sequence) → BiLSTM (long-range bidirectional dependency) → Attention (focusing on key time steps)". The TCN module uses causal dilated convolution to capture local temporal patterns and multi-scale meteorological fluctuations; the BiLSTM module models long-range bidirectional dependencies; and the attention mechanism adaptively focuses on key historical moments in the pollution accumulation or diffusion process. These three components work together to achieve PM2.5 control. 2.5 Refined modeling of the nonlinear variation law of concentration;

[0043] Experiments on the Yining City air pollution and meteorological dataset show that the present invention outperforms the benchmark model in all four indices: MAE, RMSE, MAPE, and R².

[0044] The core technical solution (feature selection + fusion model) does not require PM targeting specific regions. 2.5 The data core architecture has been redesigned, requiring only the adaptation of normalized parameters and feature dimensions of local data to achieve hourly PM data for different cities and monitoring stations. 2.5 Accurate concentration prediction. Attached Figure Description

[0045] Figure 1 This invention provides a preferred embodiment of a PM that integrates RF-RFE and TCN-BiLSTM-Attention. 2.5 Flowchart of concentration prediction method.

[0046] Figure 2 This is a diagram of the TCN-BiLSTM-Attention combined model architecture described in this invention.

[0047] Figure 3 This is a time-series trend diagram of each feature in the dataset described in this invention.

[0048] Figure 4 For each feature in the dataset described in this invention, the PM 2.5 Predicted feature importance ranking chart.

[0049] Figure 5 This is a scatter plot comparing the predicted and true values ​​of the TCN-BiLSTM-Attention combined model described in this invention on the test set.

[0050] Figure 6This is a line graph comparing the predicted and true values ​​of the TCN-BiLSTM-Attention combined model described in this invention on the test set. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0052] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0053] A PM based on RF-RFE and TCN-BiLSTM-Attention 2.5 Concentration prediction methods include the following steps:

[0054] S1, Data Collection: Air pollution data is acquired from the National Air Quality Release Platform, and meteorological data is acquired from the China Meteorological Administration. The resulting dataset is characterized by PM2.5 levels. 2.5 PM 10 The variables include six air pollutants: SO2, CO, NO2, and O3, as well as six meteorological variables: temperature, atmospheric pressure, humidity, wind speed, wind direction, and rainfall.

[0055] S2, Data Preprocessing: Linear interpolation was used to impute missing values ​​in the dataset, thus preserving the temporal trend and fluctuation characteristics of the data, avoiding the introduction of additional errors, and ensuring the validity of the sample data. Box plots were used to identify outliers, but no outliers were found, so they were not processed.

[0056] S3, Feature Selection: A combination of Recursive Feature Elimination (RFE) and Random Forest (RF) is used to select features from the preprocessed dataset. First, the importance of each feature is calculated and ranked using RF. Then, based on this importance ranking, RFE iteratively eliminates redundant features, ultimately selecting the features most relevant to the PM. 2.5 The feature set that makes a significant contribution to concentration prediction reduces the computational complexity of the model and improves prediction accuracy.

[0057] S4, Constructing Standard Time Series Input: The original time series data is sliced ​​using a sliding window of length N. Each window contains multivariate observations at N consecutive time steps, and its corresponding label is the PM at the next time step after the window ends. 2.5 Concentration, thereby transforming unsupervised time series data into supervised learning samples.

[0058] S5, Train the model and make predictions: Train the TCN-BiLSTM-Attention model using the supervised learning dataset obtained in S4, and use the trained model to make predictions on the test set, and then inversely normalize the prediction results.

[0059] The linear interpolation method in S2 is based on the valid data points adjacent to the missing value, and completes the missing value through linear fitting. The calculation formula is as follows:

[0060]

[0061] Where y is the completed value of a feature at time point x calculated by linear interpolation, x is the sampling time point corresponding to the missing value, x1 and x2 are the two most recent valid sampling points before and after x, and y1 and y2 are the monitoring values ​​of the corresponding features at time points x1 and x2.

[0062] The feature selection method in S3, which combines RFE with RF, includes the following calculation process:

[0063] (1) RF quantifies the impact of each feature on PM by measuring the change in out-of-bag error (OOB). 2.5 The predicted contribution is determined during RF training, where each decision tree is trained using only a subset of samples; samples not used in training are referred to as "out-of-bag samples." The importance score of the j-th feature is I. j Calculate using the following formula:

[0064]

[0065] Among them, I j For the j-th feature pair PM 2.5 Importance value of concentration prediction, I j The higher the value, the more effective the feature is in reducing PM2.5. 2.5 The greater the contribution of prediction error, T represents the total number of decision trees in the RF, and OOB. t For the t-th decision tree, the PM of the samples outside the bag 2.5 Mean squared error (MSE) of concentration prediction. To shuffle the values ​​of the j-th feature, the t-th decision tree performs PM calculations on the out-of-bag samples. 2.5 Mean squared error of concentration prediction;

[0066] (2) The core of RFE is based on I j Sort → Iteratively remove low-importance features → Stop when the termination condition is met. The termination condition formula is as follows:

[0067]

[0068] Among them, S k The subset of features retained after the k-th iteration, k minScore(S) is the preset minimum feature count threshold. k ) is a feature subset S k After training the RF model, the PM on the validation set 2.5 Predicted score. ε is the performance change threshold;

[0069] The time-series data, after feature filtering, is used to construct standard time-series input data to train the TCN-BiLSTM-Attention prediction model. The TCN-BiLSTM-Attention model structure is designed as follows:

[0070] (1) PM extraction using TCN layer 2.5 The localized, sudden pollution characteristics of PM2.5 concentrations (such as short-term concentration fluctuations caused by industrial emissions or peak traffic) are addressed by using causal convolution constraints to prevent temporal backward leakage, thus adapting to PM2.5 concentration characteristics. 2.5 Predicted temporal characteristics. TCN is an improvement on convolutional neural networks, introducing dilated causal convolution and residual connection structures. Dilated causal convolution effectively expands the receptive field, enhancing the ability to model historical information, while ensuring that the current output depends only on the historical and current inputs, strictly satisfying temporal causality. The formula for calculating the dilated convolution in TCN is as follows:

[0071]

[0072] Where u is the kernel size; f is the filter coefficient of the TCN; v is the kernel index; and z is the variable value. i-dv To perform convolution calculations on historical data of other pollutants and meteorological data, d represents the dilation rate, and F(i) is the dilation convolution output value at the i-th time step;

[0073] (2) Based on the output of the TCN layer, a BiLSTM layer is introduced to capture PM. 2.5 The global forward-backward temporal dependence of concentration (such as the cumulative effect of meteorological factors in the previous 24 hours) is calculated using a two-way structure, which overcomes the limitation of a single LSTM that can only capture temporal patterns in one direction. The calculation process is as follows:

[0074] (3) Output the forward LSTM (History → Current) and Backward LSTM Output (Current → Future), Two-way integrated output , Feature splicing. The output of the BiLSTM at time step t is a combination of global temporal features that integrate the "history → present → future" time series.

[0075] Furthermore, the BiLSTM module includes a two-layer stacked structure composed of forward and backward LSTM units. To improve model robustness, a random dropout layer is set between the output of the first BiLSTM layer and the input of the second BiLSTM layer. This layer randomly masks some neuron activation values ​​with a preset probability during training, thereby effectively reducing the co-fitness between neurons and preventing the model from overfitting on limited environmental data.

[0076] (4) After the BiLSTM module, an attention mechanism is introduced to assign adaptive weights to each time step in the feature vector sequence output by the BiLSTM, so as to dynamically enhance the attention to PM. 2.5 The concentration prediction has a significant impact on the characterization of key historical moments. The calculation process of the attention mechanism is as follows:

[0077] For each time step, the hidden state of the BiLSTM undergoes a "linear transformation + non-linear activation," mapping the original hidden state to an unnormalized attention score used to measure the importance of that time step. The attention score e t The calculation expression is as follows:

[0078]

[0079] Among them, h t is the hidden state at time step t output by BiLSTM; W is the weight matrix, b represents the bias vector, and tanh is the hyperbolic tangent function used to introduce nonlinearity;

[0080] Next, the unnormalized attention scores are converted into probabilistic weights, ensuring that the sum of the weights at all time steps is 1. The calculation expression is as follows:

[0081]

[0082] Among them, a tj In order to obtain the output at each time step, the model assigns importance weights to the information at the j-th time step, which represent the attention score at the t-th output time corresponding to the j-th time step;

[0083] Finally, using the normalized attention weights as coefficients, the hidden states of the BiLSTM at all time steps are summed in a weighted manner to obtain the output that incorporates information from all time steps (time steps with higher weights contribute more to the final output). The calculation expression is as follows:

[0084]

[0085] Among them, S t Let be the feature vector after weighted fusion by the attention mechanism at time step t.

[0086] Furthermore, after the attention layer output, the weighted sequence feature tensor is first converted into a one-dimensional vector through a Flatten() layer, and then connected to a fully connected layer. This fully connected layer takes the one-dimensional vector as input, introduces a ReLU activation function to enhance the non-linear expressive power of the features, and outputs a value similar to the PM. 2.5 The predicted concentration values ​​were finally converted to PM2.5 using inverse normalization to obtain a more accurate physical value. 2.5 Predicted concentration values.

[0087] This invention proposes an electronic device, including a memory and a processor. The memory stores program code executable by the processor, which executes a PM based on a hybrid model of RF-RFE feature selection and TCN-BiLSTM-Attention. 2.5 Concentration prediction methods.

[0088] This invention proposes a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement a PM based on RF-RFE and TCN-BiLSTM-Attention. 2.5 Concentration prediction methods.

[0089] like Figure 1 As shown, this invention provides a PM based on RF-RFE and TCN-BiLSTM-Attention. 2.5 The concentration prediction method, described in detail below with reference to a specific embodiment, includes the following steps:

[0090] S1, Data Collection: The dataset used contains hourly data on six major air pollutants and six meteorological variables for Yining City from January 1, 2020 to January 1, 2021. The air pollution data comes from the National Air Quality Release Platform (https: / / air.cnemc.cn:18007 / ), and the meteorological data comes from the China Meteorological Administration (CMA). http: / / www.nmc.cn Trend charts of each variable in the dataset are shown below. Figure 4 As shown.

[0091] Furthermore, in this dataset, the air pollution variable is: PM2.5 2.5 PM 10 The meteorological variables are: SO2, NO2, CO, O3; temperature (temp), atmospheric pressure (press), humidity (humd), wind speed (ws), wind direction (wd), and rainfall (rain1h).

[0092] S2, Data Preprocessing: This includes missing value imputation, outlier handling, feature selection, normalization, constructing a standard time series input, and dataset partitioning. The specific process is as follows:

[0093] Missing Values ​​and Outlier Handling: For a small number of missing values ​​in the dataset, due to PM 2.5 Hourly changes in environmental variables such as concentration and temperature are typically gradual, with small differences between adjacent values, conforming to the assumption of linear interpolation that "adjacent values ​​change linearly." Therefore, linear interpolation is used for filling in these gaps. Box plots were used to identify outliers, but no data was found to be inconsistent with the norm, so no outlier processing was performed. The formula for linear interpolation is as follows:

[0094]

[0095] Where y is the completed value of a feature at time point x calculated by linear interpolation, x is the sampling time point corresponding to the missing value, x1 and x2 are the two most recent valid sampling points before and after x, and y1 and y2 are the monitoring values ​​of the corresponding features at time points x1 and x2.

[0096] Feature selection: Extract all candidate variable features from the dataset to form an initial feature set as shown below:

[0097]

[0098] A Random Forest Regressor is used as the feature importance estimator. This estimator learns the relationship between input features and PM (Productivity Model). 2.5 Mapping relationships between concentration labels and quantifying the role of various features in predicting PM2.5 2.5 The contribution of each feature, and the importance of each feature, such as Figure 3 As shown, its key hyperparameter settings are as follows:

[0099] The number of trees (n_estimators) is 300, the maximum depth (max_depth) is 15, and the out-of-bag error is used for model validation.

[0100] Furthermore, the recursive feature elimination iteration is performed, as follows:

[0101] (1) Train a random forest regressor using the current feature set and calculate the importance score for each feature. The importance score is based on Mean Decrease Impurity, reflecting the contribution of a feature to reducing prediction error during decision tree splitting;

[0102] (2) Remove the feature with the lowest importance score and update the feature set;

[0103] Repeat steps (1)-(2) until the size of the feature set is reduced to the preset target value k. Since the model constructed by selecting features has the highest determination coefficient (R²) on the test set when k = 8, k = 8 in this invention.

[0104] Normalization: Since the data does not conform to a normal distribution, the Min-Max normalization method is used to process the dataset after feature selection, mapping all feature values ​​to the [0,1] interval to construct a time-series input sequence with uniform dimensions and standardized distribution. The normalization formula is as follows:

[0105]

[0106] in, This represents the original data. For the normalized data, This represents the maximum value in the input data. This method represents the minimum value in the input data. It does not change the relative trend of the original data, but only adjusts the data scale by linear proportion and maps the original data to the [0,1] interval by linear transformation.

[0107] Constructing a standard time-series input: The dataset contains n environmental monitoring variables at consecutive time stamps (n=12 in this invention), including the target variable PM. 2.5 Concentration and other covariates (such as temperature, humidity, wind speed, etc.) (etc.), with a time resolution of 1 hour, denoted as:

[0108]

[0109] in, Let N represent the observation vector at hour t, and N be the total number of samples.

[0110] A fixed history window length T (T = 15 in this invention) is selected to construct the input context for each training sample;

[0111] Next, the window is moved forward sequentially at 1-hour intervals to perform rolling slices on the original sequence, generating a set of input-label pairs:

[0112] Input sample: Corresponding tags: , i = 1, 2 ,…, N−T

[0113] Among them, y i PM is the next time step after the end of the window (i.e., t = i + T). 2.5 Concentration value is used as the target label for supervised learning.

[0114] Throughout the construction process, each input sample X i It only includes observations at time step t ≤ i + T - 1, while the label y i The future value corresponds to t = i + T, therefore the model cannot access any future information during the training and inference phases.

[0115] Dataset partitioning: Time series prediction should partition data chronologically: first use the earlier data set for training, then use the later data set for validation and testing. This ensures that the model encounters past data during training, while using future data for validation and testing. This partitioning method helps the model capture the temporal dependencies and evolution patterns of time series data. Therefore, the dataset is divided into training, validation, and test sets in a 70%:15%:15% ratio according to chronological order.

[0116] S3, Construct and train the TCN-BiLSTM-Attention model: The TCN-BiLSTM-Attention model structure is as follows Figure 2 As shown, the model is trained using the training set described in S2. The input to the model is a three-dimensional tensor with shape (T, D), where T represents the number of time steps in the historical observation window, and D represents the number of multidimensional environmental variables included in each time step.

[0117] Furthermore, the model consists of three parts:

[0118] The TCN module employs multi-layer causal dilated convolution, with each layer containing 64 filters and a kernel size of 3. The dilation coefficients increase in increments of [1, 2, 4, 8] to form a receptive field covering the entire historical window length (e.g., 15 hours) to capture local temporal patterns and multi-scale meteorological fluctuations.

[0119] The formula for calculating the dilated convolution of TCN is as follows:

[0120]

[0121] Where u is the convolution kernel size; f is the filter coefficient of the TCN, and z i The input data is v, where v is the kernel index and z is z. i-dv To perform convolution calculations on historical data of other pollutants and meteorological data, d represents the dilation rate, and F(i) is the dilation convolution output value at the i-th time step;

[0122] The BiLSTM module cascades a bidirectional LSTM layer after the TCN output to process the forward and reverse time series information respectively, modeling the long-range dependence of pollution concentration changes. Specifically, the forward LSTM output... (History → Current) and Backward LSTM Output (Current → Future), Two-way integrated output , For feature splicing, This is the output of the BiLSTM at time step t;

[0123] Furthermore, the first hidden layer of the BiLSTM has 64 neurons, and the second hidden layer has 24 neurons. A Dropout layer (with a dropout rate of 20%) is added between the hidden layers of the BiLSTM to avoid overfitting of the model.

[0124] Attention module: An attention mechanism is applied after the output of the second BiLSTM hidden layer to assign adaptive weights to each time step in the feature vector sequence output by the BiLSTM. The calculation process of the attention mechanism is as follows:

[0125] For each time step, the hidden state of the BiLSTM undergoes a "linear transformation + non-linear activation," mapping the original hidden state to an unnormalized attention score used to measure the importance of that time step. The attention score e t The calculation expression is as follows:

[0126]

[0127] Among them, h t is the hidden state at time step t output by BiLSTM; W is the weight matrix, b represents the bias vector, and tanh is the hyperbolic tangent function used to introduce nonlinearity;

[0128] Next, the unnormalized attention scores are converted into probabilistic weights, ensuring that the sum of the weights at all time steps is 1. The calculation expression is as follows:

[0129]

[0130] Among them, a tj In order to obtain the output of the j-th time step, the model assigns importance weights to the information at the j-th time step;

[0131] Finally, using the normalized attention weights as coefficients, the hidden states of the BiLSTM at all time steps are summed in a weighted manner to obtain the output that incorporates information from all time steps. The calculation expression is as follows:

[0132]

[0133] Among them, S t Let be the feature vector after weighted fusion by the attention mechanism at time step t.

[0134] Furthermore, after the attention layer output, the weighted sequence feature tensor is first converted into a one-dimensional vector through a Flatten() layer, and then connected to a fully connected layer. This fully connected layer takes the one-dimensional vector as input, introduces a ReLU activation function to enhance the non-linear expressive power of the features, and outputs a value similar to the PM. 2.5 Predicted concentration values.

[0135] The compilation and training of the model are configured as follows:

[0136] Optimizer: Adam optimizer is used; Loss function: Mean squared error (MSE) is used as the loss function to measure the deviation between the predicted and the true values, with the objective of minimizing the loss function value; Learning rate is set to 6E-4; Early stopping strategy: An early stopping callback function is set to prevent model overfitting, and the monitoring metric is the validation set loss. Training stops when the model's error on the validation set does not decrease after 25 epochs; Weight restoration: After training stops, the model weights are restored to the optimal values ​​when the validation set loss is achieved; Training epochs: 150; Batch size: 128.

[0137] S4, Model Prediction and Evaluation: Evaluate the model performance on the test set using the trained TCN-BiLSTM-Attention model, denormalize the prediction results, and compare them with the true values.

[0138] Furthermore, the inverse normalization formula is as follows:

[0139]

[0140] in, This represents the original data. For the normalized data, Represents the maximum value in the sequence. This represents the minimum value in the sequence;

[0141] Figure 5 and Figure 6 The figures shown are a scatter plot and a line graph comparing the predicted results with the actual values ​​using the method of this invention. To evaluate the performance of the method, root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²) were used. 2 Four indicators. Their calculation formulas are as follows:

[0142]

[0143]

[0144]

[0145]

[0146] in, Indicates the sample size of the test set. Represents the PM of the i-th sample 2.5 In actual concentration, Represents the PM of the i-th sample 2.5 Predicted concentration values, Indicates actual PM 2.5 Average concentration.

[0147] To verify the performance of the method of the present invention and the operation of each module, an ablation comparison experiment was conducted. The method proposed in this invention was compared with TCN, BiLSTM, TCN-BiLSTM, BiLSTM-Attention, 1DCNN-LSTM-Attention, and TCN-BiLSTM-Attention models, as shown in Table 1.

[0148] Table 1: Comparison of prediction accuracy of different models

[0149]

[0150] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0151] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0152] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A PM based on RF-RFE and TCN-BiLSTM-Attention 2.5 Concentration prediction method, characterized in that, Includes the following steps: S1, Get Historical PM 2.5 Concentration monitoring data, along with corresponding meteorological data, other pollutant concentration data, and time characteristic data, constitute a multi-source input dataset. Preprocessing of the multi-source input dataset, including missing value imputation, outlier handling, feature filtering, and normalization, is performed to construct a standardized time-series input sequence. S2, Local Temporal Feature Extraction: The temporal input sequence constructed in step S1 is input into the temporal convolutional network TCN, and the local temporal features of the temporal input sequence are extracted through the temporal convolutional network TCN. S3, Bidirectional long-range temporal dependency feature capture: The local temporal features extracted in step S2 are input into the Bidirectional Long Short-Term Memory (BiLSTM) network. The BiLSTM network captures the bidirectional long-range temporal dependency features of the sequence and outputs a feature vector. S4, Key Feature Weight Allocation: Introduce a time step attention mechanism to calculate the weights of the feature vector sequence output from step S3, assign differentiated weights to features at different time steps, and highlight the contribution of key temporal features. S5, Predicted value output: The feature vector processed in step S4 is input into the fully connected layer, and the PM of the next time step is output through the fully connected layer. 2.5 The concentration was predicted, and the predicted values ​​were then normalized to restore them.

2. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 1 2.5 Concentration prediction method, characterized in that, In step S1: S11, Input data: The other pollutant concentration data includes hourly concentration data of PM10, SO2, NO2, O2 and CO, and the meteorological data includes hourly data of temperature, air pressure, humidity, rainfall, wind direction and wind speed. S12, Handling missing and outlier values: For missing values ​​in the dataset that account for less than or equal to 10% and whose data trend is stable, linear interpolation is used for imputation; box plots are used to identify outliers. S13, Feature Selection: Other pollutant variables and meteorological variables are evaluated by combining Recursive Feature Elimination (RFE) with Random Forest (RF) method; based on the feature importance score of the RF model, the least important features are gradually eliminated by the RFE algorithm until a preset number of key features are retained; S14, Normalization: The Min-Max normalization method is used to map all feature values ​​to the [0,1] interval, constructing a time-series input sequence with uniform dimension and standardized distribution. The normalization formula is as follows: in, This represents the original data. For the normalized data, This represents the maximum value in the input data. This represents the minimum value in the input data. The data scale is adjusted by a linear ratio, and the original data is mapped to the [0,1] interval by a linear transformation. S15, Constructing a Standard Time Series Input Sequence: A fixed-length sliding window of length T is used to slice the original multivariate time series data. The window moves forward sequentially at 1-hour intervals to achieve rolling updates of the input samples; where PM is the value at the next future time step. 2.5 Concentration is used as a prediction label, and the model learns and infers based solely on current and historical observations.

3. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 1 2.5 Concentration prediction method, characterized in that, Step S2 specifically includes: The standard time series input sequence constructed in step S1 is input into the TCN. The input of the TCN is a three-dimensional tensor with the shape (T, D), where T represents the number of time steps in the historical observation window and D represents the number of multi-dimensional environmental variables contained in each time step. The TCN extracts multi-scale local time series features and generates context-enhanced sequence representations with long-range awareness capabilities, providing input for subsequent global dependency modeling of bidirectional LSTM.

4. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 3 2.5 Concentration prediction method, characterized in that, The TCN module uses a one-dimensional network architecture and applies a zero-padded strategy to the front of the input sequence. It employs four causal convolutional layers, each containing an independent convolutional kernel and a specific dilation rate, and enables residual connections. The TCN dilated convolution calculation formula is as follows: Where u is the kernel size; f is the filter coefficient of the TCN; v is the kernel index; and z is the kernel index. i-dv To perform convolution calculations on historical pollutant and meteorological data, d is the dilation rate, and F(i) is the dilation convolution output value at the i-th time step.

5. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 1 2.5 Concentration prediction method, characterized in that, In step S3, the BiLSTM module adopts a two-layer stacked bidirectional LSTM structure: the first layer of bidirectional LSTM has 64 hidden units in each direction, the second layer has 32, and a Dropout layer with a dropout rate of 0.2 is inserted in the middle; all LSTM layers output complete time series, and by splicing forward and backward hidden states, the layer-by-layer deepening modeling of bidirectional temporal dependencies is achieved.

6. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 1 2.5 Concentration prediction method, characterized in that, In step S4, the attention mechanism module is used to perform weighted fusion of the temporal features output by the bidirectional long short-term memory network. The feature tensor output by the BiLSTM module is transposed along the feature dimension and the time dimension to obtain a tensor of shape (batch_size, D, T). A fully connected layer is used to perform a linear transformation on the time step dimension of the transposed tensor, and the output dimension is equal to the number of time steps. The hyperbolic tangent tanh nonlinear activation function and softmax normalization function are applied to the linear transformation result in sequence to generate the attention weight matrix in the time step dimension. The generated attention weights are adjusted to the same time step-feature dimension arrangement as the input sequence, and the attention weights are multiplied element-wise with the original input feature tensor to obtain the weighted attention feature representation.

7. The PM based on RF-RFE and TCN-BiLSTM-Attention as described in claim 6 2.5 Concentration prediction method, characterized in that, In step S5, a "linear transformation + nonlinear activation" is performed on the hidden state of the BiLSTM at each time step, mapping the original hidden state to an unnormalized attention score to measure the importance of that time step. The attention score e t The calculation expression is as follows: Among them, h t is the hidden state at time step t output by BiLSTM; W is the weight matrix, b represents the bias vector, and tanh is the hyperbolic tangent function used to introduce nonlinearity; The unnormalized attention scores are converted into probabilistic weights, ensuring that the sum of the weights over all time steps is 1. The calculation expression is as follows: Among them, a tj In order to obtain the output of the j-th time step, the model assigns importance weights to the information at the j-th time step; This represents the attention score at the j-th time step corresponding to the t-th output time. Finally, using the normalized attention weights as coefficients, the hidden states of the BiLSTM at all time steps are weighted and summed to obtain the output that incorporates global time step information. The calculation expression is as follows: Among them, S t Let be the feature vector after weighted fusion by the attention mechanism at time step t.

8. An electronic device comprising a memory and a processor, the memory storing program code executable by the processor, characterized in that, The processor executes the program code to perform a PM that integrates RF-RFE and TCN-BiLSTM-Attention as described in any one of claims 1 to 7. 2.5 Concentration prediction methods.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that: When executed by the processor, the computer instruction performs a PM that integrates RF-RFE and TCN-BiLSTM-Attention as described in any one of claims 1 to 7. 2.5 Concentration prediction methods.

Images