PM2.5 concentration prediction method fusing STL-VMD decomposition mechanism and IBKA-TCN-ITransform deep learning model

Abstract

The invention provides a PM2.5 concentration prediction method fusing an STL-VMD decomposition mechanism and an IBKA-TCN-ITransform deep learning model, and the PM2.5 concentration prediction method comprises the following steps of: firstly, carrying out deep learning on the STL-VMD decomposition mechanism; firstly, pollutants and meteorological data are acquired and preprocessed, and then features are screened by using Pearson's correlation coefficients. STL-VMD decomposition is carried out on PM2.5 data to obtain a plurality of intrinsic mode components IMF, a K-MEANS algorithm is adopted to obtain four components with different frequencies, and the four components are spliced with other features to construct a sub-data set. According to the method, an independent TCN-ITransform model is adopted for training, and hyper-parameters are optimized through an IBKA algorithm. And finally, fusing a prediction result of the model to obtain a PM2.5 concentration prediction value, and evaluating the model performance by calculating a model performance evaluation index and combining SHAP interpretability analysis. According to the method, the capturing capacity of the time sequence dependence and variable cooperation relation in the PM2.5 data set is enhanced, meanwhile, the focusing capacity of key information is improved, the convergence speed and PM2.5 prediction precision are high, and reliable technical support is provided for air quality early warning.

Description

Technical Field

[0001] This invention relates to the field of air quality prediction technology, and proposes a PM2.5 prediction method that integrates the STL-VMD decomposition mechanism and the IBKA-TCN-ITransformer model. 2.5 Concentration prediction methods. Background Technology

[0002] With the increasing emphasis on ecological and environmental protection in my country and the growing public demand for high-quality air, the scientific and precise management of urban air quality has become a core task for ecological and environmental authorities. PM 2.5 As one of the core indicators characterizing air quality, high-precision hourly forecasts of PM2.5 can directly provide timely support for air quality early warning responses and pollution control decisions, and have significant practical application value for protecting public health and optimizing social production activities. Therefore, constructing high-precision PM2.5 forecasts is crucial. 2.5 Concentration prediction models have become a research hotspot in the interdisciplinary field of environmental science.

[0003] Deep learning networks, due to their ability to capture non-linear data trends, have become a key technology in product development. 2.5 The mainstream technical approach for concentration prediction. However, existing research still has many limitations: most methods only focus on mining the time dependence of data, without fully considering PM2.5 concentration. 2.5 The cooperative dependencies between the model and other feature variables contribute to the gain in prediction accuracy, but this results in insufficient ability of the model to characterize complex influencing mechanisms. Furthermore, existing prediction methods primarily rely on the original PM... 2.5 The sequence was used directly to train the model for predicting labels, but it failed to address PM. 2.5Hierarchical deconstruction of multimodal features such as seasonal, trend, and residual terms inherent in the sequence fails to accurately capture the dynamic evolution of each modal component, resulting in high short-term prediction errors at the hourly scale, which is insufficient to meet the accuracy requirements of practical applications. Furthermore, some prediction schemes lack attention mechanisms or equivalent mechanisms, resulting in a lack of adaptive focusing on key information during feature extraction. This leads to the indiscriminate treatment of high-value and redundant information, making the model lack the ability to represent core features, thus weakening the stability and robustness of the model's prediction results. Regarding hyperparameter optimization, some existing methods rely on empirically-based adjustment strategies. However, when faced with a large number of hyperparameters and a wide value space, this method is not only inefficient in finding the optimal solution but also fails to cover the global optimum. Other prediction methods employ brute-force search algorithms such as grid search or heuristic optimization algorithms to optimize hyperparameters. However, the former is too time-consuming, and the latter is prone to getting trapped in local optima, ultimately causing the accuracy of the trained model to often fall short of expectations. In addition, some existing studies have failed to conduct interpretability analysis on the models, which leads to a trust problem where the models' predictions are reliable but cannot be explained.

[0004] Against this backdrop, there is an urgent need for a new type of PM 2.5 Prediction methods: PM 2.5 Multimodal feature decomposition of time series data is used to accurately capture its multidimensional dynamic patterns. It simultaneously considers the time-dependent characteristics and variable synergies of the data, possesses adaptive focusing capabilities for key information, and can achieve globally optimal configuration of hyperparameter combinations using efficient hyperparameter optimization algorithms, thereby systematically improving hourly PM. 2.5 Accuracy and reliability of concentration prediction. Summary of the Invention

[0005] This invention targets most existing PMs 2.5 Prediction methods failed to adequately capture PM 2.5 The multidimensional dynamic changes in concentration sequences and the co-dependencies among variables lead to technical problems such as low prediction accuracy and time-consuming hyperparameter optimization algorithms that are prone to getting trapped in local optima. This paper proposes a method that integrates the STL-VMD decomposition mechanism and the IBKA-TCN-ITransformer model for hourly PM2.5 concentrations. 2.5 Concentration prediction method. This method introduces the STL-VMD secondary decomposition mechanism to delve deeper into PM2.5 concentration. 2.5 Seasonal, trend, and residual component characteristics of sequence variations are explored. The TCN network is combined with the ITransformer model to capture dual dependencies in the temporal and variable dimensions of the dataset. Simultaneously, the IBKA heuristic hyperparameter optimization algorithm is employed to search the hyperparameter space, improving the ability to retrieve the globally optimal hyperparameter combination.

[0006] To solve the above problems, the present invention specifically includes the following steps in its technical solution: First, obtain a sample containing PM. 2.5 Data sets of other pollutants and meteorological characteristics were collected and preprocessed, with air pollutants including PM2.5. 2.5 Concentration characteristics and API, PM 10 Meteorological pollutants include SO2, NO2, CO, and O3, as well as temperature, dew point, visibility, and average wind speed. The Pearson correlation coefficient formula is used to calculate the pollutants and meteorological characteristics in the preprocessed dataset, which are then compared with PM2.5. 2.5 The correlation coefficients between concentration characteristics were analyzed; features with high correlation coefficients were retained, while those with low correlation coefficients were discarded. Then, for PM... 2.5 The concentration feature sequence was subjected to a two-stage STL-VMD decomposition. The first decomposition used a local weighted regression scatter smoothing (LOWESS)-based STL seasonal trend decomposition on the original PM2.5 concentration. 2.5 The sequence is decomposed into residual, seasonal, and trend terms. A second decomposition performs VMD variational mode decomposition on the three independent components obtained from STL decomposition, further decomposing each component into three independent intrinsic mode function (IMF) components, resulting in a total of nine IMF components: IMF1, IMF2, ..., IMF9. The IMF components are then clustered using the K-MEAMS algorithm and fused based on similarity into four component sequences F with different frequencies. x Then each combined sequence is compared with the data excluding PM. 2.5 Other pollutants and meteorological multivariate features M were fused to construct four subsets: (M, F1), (M, F2), ..., (M, F4). Four independent and structurally consistent IBKA-TCN-ITransformer prediction models were constructed. The four subsets were input into their respective models for training, and the IBKA algorithm was used to optimize the hyperparameters of each model, resulting in four optimized independent prediction models. The prediction results of each model were additively fused and reconstructed to obtain the PM2.5 concentration at the current time. 2.5 The concentration prediction results, and by comparing them with PM2.5 concentrations. 2.5 The R-squared of the model is calculated by comparing concentration observations. 2 The overall performance of the model is evaluated using metrics such as RMSE and MAE. Finally, SHAP interpretability analysis is used to evaluate the contribution of each feature to the model's prediction results, thereby enhancing the reliability of the model's results.

[0007] Each prediction model is structurally composed of a TCN assembly consisting of three TCNRBs cascaded together and an ITransformer model. The ITransformer model uses a multi-head residual attention mechanism instead of the traditional multi-head attention mechanism in the Transformer model. The output of the TCN serves as the input to the ITransformer model, which outputs the prediction result of the IMF component at the current time step. The core mechanism of the dilated causal convolutional residual block (TCNRB) is as follows: feature extraction is performed on the time series through convolution operations. Specifically, the sub-convolutional kernel performs an inner product operation with a vector consisting of N data points selected at intervals determined by the dilation rate up to the current time step. After adding a bias term, the result is processed by ReLU mapping and Dropout to obtain the convolutional feature sequence. The Dilated Casual Conv module in the dilated causal convolutional residual block mainly performs feature extraction, and its core formula is shown below:

[0008] (1)

[0009] In equation (1), k is the kernel window index; d is the dilation rate, which is equal to the dilation coefficient plus one. These are the weight parameters after weight normalization; The cin value of the channel for the input sequence at time (t−k×d); This is the bias term corresponding to the output channel cout. Furthermore, the weights (internal parameters of the convolutional kernel) in the convolutional layer need to be normalized to prevent inconsistent weight values ​​from affecting the training results. This process is completed in the WeightNorm module, and the corresponding mathematical formula is as follows:

[0010] (2)

[0011] Where β is the offset parameter, used to eliminate the impact of weight differences on training; γ is the scaling parameter; Let L2 be the L2 norm of the weight vector. The ReLU activation function is introduced to enhance the nonlinear fitting ability of the model, and the Dropout layer is introduced to reduce the risk of overfitting. The formulas are shown in (3) and (4) respectively.

[0012] (3)

[0013] (4)

[0014] Let cout be the ReLU activation eigenvalue of the cout-th output channel at time t; Let cout be the original convolutional feature value of the cout-th output channel at time t. It is the feature output value of the cout-th output channel at time t after Dropout processing; It is a Bernoulli random variable matrix used to implement the random zeroing operation of the Dropout layer.

[0015] The extracted temporal features are input into the ITransformer model, which structurally includes an embedding layer, a multi-head differential attention mechanism (MUDA), a dropout layer, a layer normalization (LayerNorm), a feed-forward network layer, and a linear transformation layer. A multi-head differential attention mechanism is introduced to replace the traditional multi-head attention mechanism to remove homogeneous noise contained in the first attention feature map. For each attention head, two different weight matrices are used to calculate the softmax(Q1K1) of the input X. T ) and softmax(Q2K2) T Two attention feature maps are used. The first attention feature map is a matrix dominated by effective and valuable information, while the second attention feature map is a matrix dominated by noise. The difference between the two matrices is then used in the operation with V. The final matrix corresponding to a head obtained in this way effectively reduces the impact of homogeneous noise on the final result calculation compared to the matrix obtained by traditional attention mechanisms. The difference operation formula for each head is shown below:

[0016] (5)

[0017] In equation (5), d k The dimension corresponding to each head; Q is a learnable scalar parameter that controls the weights of the two attention maps; K is the query matrix; V is the feature key matrix; and V is the numerical matrix.

[0018] Each model employs the Improved Black-winged Kite Algorithm (IBKA) with multiple strategies during training to optimize three hyperparameters: rolling window size, batch size, and number of TCN channels. The algorithm execution flow includes: first, determining the optimization range for each of the three hyperparameters; then initializing the population size N, the maximum number of iterations M, and setting the algorithm control parameters α and β to 0.7 and 0.4 respectively; during the iterative search process, the algorithm inputs the generated populations into the model, which trains separately based on different populations, returning corresponding performance metrics (such as RMSE), which are then returned to the algorithm for processing. The algorithm converts the performance metrics into fitness values, updates the population based on the individual with the best fitness, and then inputs it back into the model. The process terminates when the maximum number of iterations is reached or the best fitness value no longer improves; otherwise, it repeats the iteration process. Finally, the globally optimal hyperparameters for the entire iterative cycle are output.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention fully considers the time and variable dependencies on PM in its prediction method. 2.5 The significant impact on concentration prediction accuracy is addressed by introducing a series of dilatational causal residual blocks (TCNRBs) with different dilatation rates to achieve PM2.5 prediction accuracy at the current time. 2.5 The precise capture of the temporal dependence features of concentration at different distances (time point intervals) is achieved by applying the multi-head difference attention mechanism of the ITransformer model to mine variable collaborative dependencies, effectively eliminating homogeneous noise in variable collaborative features and improving the adaptive focusing capability of key information. An STL-VMD two-level decomposition mechanism is introduced to analyze PM... 2.5 Characterization of concentration sequences effectively captures PM. 2.5 The multidimensional dynamic changes in concentration were investigated. An improved intelligent optimization algorithm for black-winged kites (IBKA) using a multi-strategy approach was introduced to search for the global optimum of the model's hyperparameters, thereby improving PM2.5 concentration. 2.5 The predictive performance of the prediction model. Attached Figure Description

[0020] Figure 1 This is a detailed flowchart of the prediction method of the present invention.

[0021] Figure 2 This is a graph showing the results of Pearson correlation coefficient analysis between variables in the prediction method of this invention.

[0022] Figure 3 For PM 2.5 Plot of STL seasonal decomposition results for concentration sequences.

[0023] Figure 4 This is a graph showing the VMD decomposition results of the seasonal components from STL decomposition.

[0024] Figure 5 This is a graph showing the VMD decomposition results of the residual components from the STL decomposition.

[0025] Figure 6 This is a VMD decomposition result diagram of the trend component of STL decomposition.

[0026] Figure 7 This is a structural diagram of the IBKA-TCB-ITransformer deep learning model.

[0027] Figure 8 This is a schematic diagram illustrating the feature extraction principle of dilated causal residual convolutional blocks under different dilation rates.

[0028] Figure 9 This is a schematic diagram of the multi-head residual attention mechanism used in the ITransformer model.

[0029] Figure 10 This is a flowchart of the hyperparameter search process for the IBKA heuristic optimization algorithm. Detailed Implementation

[0030] This invention addresses existing PM 2.5 The prediction method has slow convergence speed and fails to capture PM. 2.5 The multidimensional dynamic changes in concentration sequences and the co-dependencies among variables lead to the technical problem of low prediction accuracy. This paper proposes a method that integrates the STL-VMD decomposition mechanism and the IBKA-TCN-ITransformer deep learning model for hourly PM2.5 concentrations. 2.5 Concentration prediction method. This method introduces a basic STL-VMD secondary decomposition mechanism to delve deeper into PM2.5 concentration prediction. 2.5 Seasonal, trend, and residual component characteristics of sequence variations. Three sequentially connected TCNRB blocks with different expansion coefficients are combined with the ITransformer model to capture the dual dependencies between time and variables. Simultaneously, an improved IBKA heuristic hyperparameter optimization algorithm is employed to search the hyperparameter space, improving the retrieval speed of the globally optimal hyperparameter combination. To address the aforementioned problems, this invention's PM... 2.5 See the detailed diagram of the concentration prediction procedure. Figure 1 Specifically, it includes the following steps:

[0031] Step S1: Data Acquisition and Preprocessing. Air pollutant data, specifically PM2.5, was obtained from the China National Environmental Monitoring Centre (https: / / www.cnemc.cn / sssj / ). 2.5 Concentration characteristics and API, PM 10 Characteristics of pollutants such as SO2, NO2, CO, and O3 were obtained from the National Earth System Science Data Center (https: / / auth.geodata.cn / ). Meteorological characteristic data, including temperature, dew point, visibility, and average wind speed, were acquired. Further preprocessing of the acquired data was performed. The specific steps of data preprocessing are as follows:

[0032] a. Outlier Handling: Outliers are identified and handled using the quartile method. This method identifies outliers by calculating the first quartile (Q1) and third quartile (Q3) of the data and the difference between them (IQR). Then, lower and upper limits for outliers are defined. Any data point below the lower limit or above the upper limit is considered an outlier and replaced by the mode.

[0033] b. Missing Value Handling: For missing data, a time-series-based numerical interpolation method is used. This method estimates the missing value based on the time difference between two adjacent data points of the missing field in the time series, fully considering the continuity and sufficiency of time. Therefore, it is suitable for imputing missing values ​​in time series data. The calculation formula is shown below:

[0034] (1)

[0035] In equation (1), y is the estimated value used to fill missing values; t is the time of the missing data point; t1 and t2 are the previous and next adjacent times of the missing point, respectively; y1 and y2 are the actual values ​​corresponding to times t1 and t2, respectively.

[0036] c. Normalization: In order to resolve the differences in the dimensions of the feature variables in the dataset, the dataset is normalized column by column.

[0037] Step S2: Using the Pearson correlation coefficient formula (see Formula 2), correlate the pollutants and meteorological characteristics in the preprocessed dataset with PM2.5. 2.5 The correlation coefficients of concentrations are calculated to clarify the strength of the association between features. Features with high correlation coefficients are retained, while features with low correlation coefficients are removed. The correlation analysis results are as follows: Figure 2 As shown. Then the base

[0038] Based on the correlation analysis results, features are retained or removed.

[0039] (2)

[0040] In equation (2), N is the total amount of data; x i Let y be the value of the i-th data variable x; i Let y be the value of the i-th data variable; The average value of variable x; ρ is the average value of variable y; ρ is the average value of x. i With y i The Pearson correlation coefficient between the two.

[0041] Step S3: For PM 2.5 The concentration feature sequence is decomposed using STL (Synchronous Transformation of Transcriptional Stratification), which separates it into three independent components: seasonal term, trend term, and residual term. Then, based on the Lowest Local Weighted Regression (LOWESS) scatter smoothing method, STL seasonal trend decomposition is performed, further separating it into the residual term T. t Seasonal Item S t and trend item R t Three independent components, STL decomposition effect as follows Figure 3 The corresponding mathematical expression is as follows:

[0042] (3)

[0043] STL continuously optimizes the trend and seasonal terms through multiple iterations. In each iteration, a locally weighted regression scatter smoothing method is used to fit the local trend. The core formula is:

[0044] (4)

[0045] The output is the local trend estimate at time t, W. t For a sliding window centered at time t, For LOWESS weight, This is the seasonally adjusted intermediate sequence.

[0046] Step S4: Perform VMD variational mode decomposition on the three decomposed components. Each component will be further decomposed into three independent intrinsic mode function components (IMFs), resulting in a total of nine PMs with specific characteristics. 2.5 Intrinsic mode function components. The VMD decomposition algorithm uses the augmented Lagrange formula of variational mode decomposition to transform the original PM... 2.5 The sequence is decomposed into multiple independent intrinsic mode function components, and the corresponding mathematical expression is:

[0047] (5)

[0048] in For the augmented Lagrangian function; Let be the eigenmode functions to be decomposed; Corresponding to the center frequency of each IMF; The Lagrange operator is used to reconstruct the original signal; α is the penalty factor that controls the bandwidth of each IMF component; S(t) is the original input signal; It is the Dirac function; For time derivative operators; This is the complex value option for Fourier transform. The results of the decomposition of seasonal, residual, and trend components are shown separately in [link to documentation]. Figure 4 , 5 6.

[0049] Step S5: Cluster the IMF components with high similarity based on the K-MEAMS algorithm, and fuse them into four new feature component sequences F with different frequencies based on similarity. x The algorithm execution process is as follows: First, randomly select K samples from the sample set X as initial cluster centers, and then for each sample x... iCalculate the distance between each vector and all cluster centers, and assign it to the nearest cluster. Calculate the mean of all samples within each cluster and use it as the new cluster center. Finally, determine the convergence of the clustering. The Euclidean distance formula is used to calculate the distance of each vector to the cluster center, and its mathematical expression is as follows:

[0050] (6)

[0051] Where d(x) i ,c k ) represents the Euclidean distance between the i-th IMF eigenmode function sequence and the k-th cluster center sequence; x i,t c represents the value of the i-th IMF component at time t. k,t is the specific value of the center sequence of the k-th cluster at time step t; T is the total length of time steps contained in the IMF sequence.

[0052] Step S6: Combine each fused F... x Sequences and datasets excluding PM 2.5 Other pollutants and meteorological characteristics were combined to construct four data subsets, providing targeted input for subsequent model training and testing. The merged feature field table for the dataset corresponding to the Xth (1≤X≤4) F component is shown below:

[0053] Table 1 Feature fields of the Xth subset

[0054] Data Name unit AQI <![CDATA[μg / m 3 ]]> <![CDATA[PM 10 ]]> <![CDATA[μg / m 3 ]]> SO2 <![CDATA[μg / m 3 ]]> NO2 <![CDATA[μg / m 3 ]]> CO <![CDATA[mg / m 3 ]]> O3 <![CDATA[μg / m 3 ]]> TEMP <![CDATA[ 。 C]]> DEW <![CDATA[ 。 C]]> VISIBILITY m WIND_SPEED m / s <![CDATA[F x ]]> Unitless

[0055] Step S7: Construct four independent and structurally consistent TCN-ITransformer models, with each subset of the dataset being processed by a specific model.

[0056] a. First, each prediction model structurally comprises three cascaded TCNRB assemblies. The output of the assemblies serves as the input to the ITransformer model, which then outputs the prediction result of the F component at the current time. The model's structure diagram is shown below. Figure 7 As shown.

[0057] The principle of Dilated Causal Convolutional Residual Block (TCNRB) is as follows: Figure 8As shown, its core mechanism is as follows: feature extraction of the time series is performed through convolution operation. Specifically, the sub-convolution vector is used to perform an inner product operation with a vector composed of N data points selected at intervals determined by the dilation rate at the current time and before. After adding a bias term, the convolution feature sequence is obtained through ReLU mapping and Dropout processing. The dilation mechanism expands the receptive field of the convolution kernel by selecting non-continuous data to capture long-term dependencies, but multiple superpositions can easily cause data loss. The causal property ensures that the convolution operation only depends on the current and historical data and does not include future information. The residual mechanism fuses and adds the time series feature input sequence and the convolution feature extraction sequence at corresponding positions, effectively alleviating the gradient vanishing problem (i.e., the phenomenon that the gradient values ​​of the early network layers are too small during backpropagation, causing parameter updates to stagnate and unable to learn effectively) and the problem of decreased prediction accuracy caused by the increase in the number of model layers.

[0058] The Dilated Casual Conv module in the dilated causal convolution residual block mainly performs feature extraction, and its core formula is shown below:

[0059] (7)

[0060] Where k is the kernel window index; d is the dilation rate, which is equal to the dilation coefficient plus one; These are the weight parameters after weight normalization; The cin value of the channel for the input sequence at time (t−k×d); This is the bias term corresponding to the output channel cout.

[0061] In addition, the weights (internal parameters of the convolutional kernel) in the convolutional layer need to be normalized to prevent inconsistent weight values ​​from affecting the training results. This process is completed in the WeightNorm module, and the corresponding mathematical formula is as follows:

[0062] (8)

[0063] Where β is the offset parameter, used to eliminate the impact of weight differences on training; γ is the scaling parameter; Let L be the L2 norm of the weight vector. The ReLU activation function is introduced to enhance the nonlinear fitting ability of the model, and the Dropout layer is introduced to reduce the risk of overfitting. The formulas are shown in (9) and (10) respectively.

[0064] (9)

[0065] (10)

[0066] Let cout be the ReLU activation eigenvalue of the cout-th output channel at time t; Let cout be the original convolutional feature value of the cout-th output channel at time t. It is the feature output value of the cout-th output channel at time t after Dropout processing; It is a Bernoulli random variable matrix used to implement the random zeroing operation of the Dropout layer.

[0067] The extracted temporal features are input into the Transformer model and processed through an Embedding layer, a Multi-Head Differential Attention (MUDA) mechanism, a Dropout layer, LayerNorm, a Feed-forward network layer, and a LinearLayer to uncover the collaborative dependencies between features. A MUDA mechanism is introduced to replace the traditional MUDA mechanism; its structural principle is described in [link to documentation]. Figure 9 This is used to remove homogeneous noise contained in the first attention feature map.

[0068] For each attention head, the softmax(Q1K1) is calculated using two different weight matrices on the input X. T ) and softmax(Q2K2) T This process yields two attention maps. The first attention feature map is a matrix dominated by effective and valuable information, while the second attention feature map is a matrix dominated by noise. The difference between the two matrices is then used in the operation with V. This method effectively reduces the impact of similar noise on the final result calculation compared to matrices obtained through traditional attention mechanisms. The difference operation formula for each head is shown below:

[0069] (11)

[0070] In equation (11), d k The dimension corresponding to each head; Q is a learnable scalar parameter that controls the weights of the two attention maps; K is the query matrix; V is the feature key matrix; and V is the numerical matrix.

[0071] b. During training, each model employs a multi-strategy improved Blackwing Kite Intelligent Algorithm (IBKA) to optimize three hyperparameters: scrolling window size, batch size, and number of TCN channels. The IBKA algorithm execution flow is as follows: Figure 10 As shown.

[0072] The algorithm execution flow includes: First, determining the optimization range for each of the three hyperparameters; then initializing the population size N, the maximum number of iterations M, and setting the algorithm control parameters α and β to 0.7 and 0.4 respectively. During the iterative search process, the algorithm inputs the generated population into the model. The model is trained on different populations, returning corresponding performance index values ​​(such as RMSE), which are then returned to the algorithm for processing. The algorithm converts the performance index values ​​into fitness values, updates the population based on the individual with the best fitness, and then inputs it into the model. The process terminates when the maximum number of iterations is reached or the best fitness value no longer improves; otherwise, it repeats the iteration process. Finally, the globally optimal hyperparameters for the entire iterative cycle are output.

[0073] Step S7: The prediction results of the four models are additively fused and reconstructed using Equation 12 to obtain the PM at the current time. 2.5 Concentration prediction results, compared with PM2.5. 2.5 The calculation formulas for the three performance evaluation indicators, R2, RMSE, and MAE, of the concentration observation label value calculation model are shown in formulas 13, 14, and 15.

[0074] (12)

[0075] (13)

[0076] (14)

[0077] (15)

[0078] In equation (12), It is the PM at the current moment. 2.5 Concentration prediction value Let be the model prediction value of the x-th IMF component at the current time. In equations (13), (14), and (15), RMSE represents the average deviation between the predicted value and the actual value; the smaller the value, the more accurate the prediction. 2 It is mainly used to measure how well a regression model fits the data, and its value ranges from 0 to 1. R 2 The closer a value is to 1, the better the model fit. Mean Absolute Error (MAE) is the average of the absolute errors between predicted and observed values, reflecting individual differences in the average value. Where n is the number of samples. For the true value, These are predicted values.

[0079] To verify the accuracy of the method of the present invention, hourly regional air monitoring data from Xianyang City, Shaanxi Province, from January 1, 2021 to January 15, 2021 were used for training tests.

[0080] A comparison is made between the BP neural network model and the TCN-ITransformer model, both of which are based on the absence of PM. 2.5 The results are obtained by training and testing on the original dataset after splitting the sequence. The hyperparameters of the latter, specifically the rolling window, batch size, and number of TCN channels, are 24, 8, and 4, respectively. The STL-TCN-ITransformer model constructs three datasets based on the three components obtained from STL seasonal decomposition, without further decomposition of each component using VMD. Similarly, it uses three independent but structurally identical models for prediction, and then fuses the results. The hyperparameter settings of this model are consistent with those of TCN-ITransformer. To ensure experimental fairness, the hyperparameter initialization settings of the proposed IBKA-TCN-ITransformer deep learning prediction model are consistent with the above models. Compared to the above models, the introduction of the IBKA hyperparameter optimization mechanism and VMD decomposition improves the model's feature mining capability and model performance.

[0081] Considering the limitations of computational time and space costs, the population size in the IBKA hyperparameter optimization algorithm is set to 5, the maximum number of iterations is set to 20, and the algorithm control parameters α and β are set to 0.7 and 0.4, respectively. The scrolling window length, batch size, and number of TCN channels to be optimized are set to [1, 24], [8, 32], and [2, 8], respectively, and the values ​​of the scrolling window, batch size, and number of TCN channels are initialized to 24, 8, and 4, respectively. In addition, the values ​​of hyperparameters that do not participate in the algorithm optimization are set to fixed values. Considering the computational time and space costs, the number of heads in the multi-head residual attention mechanism is set to 2, the number of model iterations corresponding to each hyperparameter combination is set to 100, and an early stopping mechanism is introduced to stop the model training iteration process when the iteration accuracy does not improve for 10 consecutive iterations. The introduction of this mechanism improves the training efficiency of the model.

[0082] The performance metrics of the four models tested are shown in Table 2. It is clear from the table that the feature decomposition and structure enhancement strategies of the models have a significant positive impact on prediction performance. In PM 2.5 In the prediction evaluation system, the smaller the RMSE (Root Mean Square Error) and MAE (Mean Absolute Error) values, the lower the model's prediction error; the closer the R² (Coefficient of Determination) is to 1, the better the model fits the real data. These three indicators together constitute the core basis for judging model performance. The BP model, as a basic neural network, performed the worst among all models, with an RMSE of 25.255, an MAE of 18.167, and an R² of only 0.493. The core reason for this result is that the structural characteristics of the BP network make it unable to effectively capture PM (Predictive Error). 2.5The model suffers from long dependencies in time-series data, and it does not perform any feature decomposition preprocessing on the original data, making it vulnerable to nonlinear and non-stationary PM. 2.5 Sequences are difficult to fit precisely with their complex patterns of change, ultimately leading to low prediction accuracy.

[0083] Table 2 Comparison of Predictive Performance of Different Models

[0084] Model RMSE <![CDATA[R 2 ]]> MAE BP 25.255 0.493 18.167 TCN-ITransformer 25.839 0.759 17.715 STL-TCN-ITransformer 24.449 0.785 15.589 IBKA-STL-VMD-TCN-ITransformer 20.868 0.843 14.220

[0085] The TCN-ITransformer model showed a significant improvement over the BP model, with R² rising to 0.759, indicating a substantial increase in the model's fit to the data. However, its RMSE was 25.839, slightly higher than the BP model. This phenomenon reflects that the combined structure of TCN and Transformer does indeed enhance the model's ability to capture temporal dependencies and better uncover PM (partial time dependence). 2.5 The temporal features in the sequence are analyzed, but due to the lack of feature decomposition preprocessing on the original data, the mixed different modal patterns in the data still increase the difficulty of model fitting, and the error cannot be effectively reduced.

[0086] The STL-TCN-ITransformer model introduces the STL feature decomposition strategy based on the TCN-ITransformer, which decomposes PM... 2.5 The sequence was split into three independent components: trend, seasonality, and residuals. The model performance was further optimized, with RMSE decreasing to 24.449, MAE decreasing to 15.589, and R² increasing to 0.785. This result validates the effectiveness of the feature decomposition strategy, which, by splitting the PM... 2.5 By analyzing the temporal patterns of different modalities in the sequence, the complex overall prediction task is broken down into simpler component prediction tasks, which significantly reduces the difficulty of model fitting and thus improves prediction accuracy.

[0087] The proposed model outperformed all other models, achieving an RMSE of 20.868, a MAE of 14.220, and an R² of 0.843. This model integrates multiple optimization strategies: firstly, it combines STL and VMD dual eigenvalue decomposition. STL is used to extract the macroscopic trend, seasonality, and residual modes, while VMD further decomposes the microscopic frequency components in the residual terms, maximizing the deconstruction of PM. 2.5 The complex characteristics of the sequence are considered; on the other hand, the IBKA algorithm is introduced for hyperparameter optimization, which effectively improves the training efficiency and global optimality of the parameters. The synergistic effect of multiple strategies allows the model to better adapt to PM. 2.5 The fitting ability for complex time series patterns has reached the optimal level.

[0088] In summary, the STL-VMD feature decomposition used in this invention can significantly reduce the difficulty of model fitting by deconstructing the complex features of the data, effectively improving the prediction accuracy of the model. The introduction of heuristic hyperparameter optimization algorithm and multi-head residual attention mechanism further improves the convergence speed and training stability of the model on this basis.

Claims

1. A PM that fuses STL-VMD decomposition mechanism and IBKA-TCN-I Transformer deep learning model 2.5 Concentration prediction method characterized in that The prediction method comprises the following steps: Step S1 : Obtain a dataset containing PM 2.5 and other atmospheric pollutants and meteorological features and pre-process it; Step S2: Calculate the correlation coefficient of the pre-processed air pollutants and meteorological features with PM 2.5 concentration by using Pearson correlation coefficient formula, and features with high correlation coefficient are retained while features with low correlation coefficient are removed. Step S3: PM 2.5 The feature time series is decomposed into residual, seasonal and trend components based on the STL seasonal-trend decomposition procedure based on the locally weighted regression scatterplot smoother (LOWESS). Step S4: VMD variational mode decomposition is performed on the three components after decomposition, and each component will be further decomposed into 3 independent intrinsic mode functions IMFs, a total of 9 PMs with specific characteristics are obtained 2.5 Intrinsic mode function component; Step S5: Fuse the sequences with higher similarity together based on K-MEANS clustering algorithm, and obtain the fused four component sequences F of different frequencies x ; Step S6: combine each fused generated F x sequences with other pollutants, meteorological features in the dataset except PM 2.5 other than PM, weather features, to build 4 subsets of data, providing targeted input for subsequent model training and testing; Step S7: four independent and structurally consistent TCN-ITransformer models are constructed, each sub-dataset is specially trained and learned by a corresponding model, each prediction model is structurally composed of three TCNRB in series and an ITransformer model adopting a multi-head residual attention mechanism, and an IBKA algorithm is used to respectively optimize the hyperparameters of each model to determine the best internal structure of the model; Step S8: fuse the prediction results of the models to reconstruct the PM at the current time 2.5 the prediction results of the concentration Step S9: based on the prediction results, the R2, RMSE, and MAE performance evaluation indexes of the model are calculated to evaluate the performance of the model; Step S10: SHAP explainability analysis is performed on the model to evaluate the contribution of each feature to the prediction results of the model.

2. The PM of fusing STL-VMD decomposition mechanism and IBKA-TCN-I Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that The data preprocessing in step S1 comprises: (1) For the abnormal values in the dataset, the quartile method is used to identify and process the abnormal values. The abnormal values are identified by calculating the first quartile (Q1) and the third quartile (Q3) of the data and the difference (IQR) between them. The lower limit and the upper limit of the abnormal values are defined, and any data point less than the lower limit or greater than the upper limit is considered as an abnormal value and is replaced by the mode; (2) For the missing values in the dataset, the time series-based numerical interpolation method is used for interpolation. This method estimates the missing values based on the time difference between the two adjacent data points of the missing field in the time series, fully considering the continuity and sufficiency of time, so it is suitable for filling the missing values of time series data. The calculation formula is as follows: (1) In formula (1), y is the estimated value used to fill the missing value; t is the time of the missing data point; t1 and t2 are the last adjacent time and the next adjacent time of the missing point, respectively; y1 and y2 are the actual values corresponding to t1 and t2, respectively; (3) The training set and the test set are divided, and then the data set is normalized in column units, mapping the value range of each feature to the same interval, to solve the influence of the dimension difference of each feature variable in the data set on the subsequent prediction accuracy of the model.

3. The method of claim 1, wherein The Pearson correlation coefficient formula is used in step S2 to calculate the correlation coefficient between the PM 2.5 The mathematical expression of the correlation coefficient calculation between other features is as follows: (2) In formula (2), N is the total amount of data; x i is the value of the i-th data variable x; y i is the value of the i-th data variable y; is the average value of variable x; is the average value of variable y; p is the Pearson correlation coefficient value of x i and y i .

4. A PM that fuses an STL-VMD decomposition mechanism and an IBKA-TCNI Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that The STL decomposition mechanism is used to complete step S3 for PM. 2.5 Preliminary decomposition of time series, focusing only on PM 2.5 The concentration time series was subjected to STL decomposition while other feature parameters remained unchanged, thus decomposing it into three independent components: seasonality, trend, and residual. The STL decomposition process employed locally weighted regression scatter smoothing to fit the local trend. The core formula is: (3) In formula (3), is the local trend estimate at time t, W t is a sliding window centered at time t, is the LOWESS weight, is the seasonally adjusted intermediate series.

5. The method according to claim 1, characterized in that the step S4 employs VMD decomposition to independently decompose the seasonal, residual and trend terms obtained from STL decomposition, and each component will be further decomposed into 3 independent intrinsic mode function components (IMF), and the VMD decomposition algorithm employs the augmented Lagrangian formula of variational mode decomposition to split the original PM 2.5 sequence into multiple independent intrinsic mode function components, and the corresponding mathematical expression is as follows: (4) wherein is the augmented Lagrangian function; is the intrinsic mode function to be decomposed; is the center frequency corresponding to each IMF; is the Lagrangian operator to reconstruct the original signal; a is the penalty factor to control the bandwidth of each IMF component; S(t) is the original input signal; is the Dirac function; is the time derivative operator; is the complex-valued option of Fourier transform.

6. A PM that fuses an STL-VMD decomposition mechanism and an IBKA-TCNI Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that In step S5, the K-MEANs clustering algorithm is used to aggregate the high-similarity intrinsic mode components obtained by the STL-VMD secondary decomposition with a fixed number of clusters, to generate four new feature sequences F with independent characteristics x .

7. A PM that fuses an STL-VMD decomposition mechanism and an IBKA-TCNI Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that The TCN network of the model is structurally composed of three series of dilated causal convolution residual blocks. From the input to the output direction, the dilation coefficients of the residual blocks are 1, 2, and 4, respectively. Each residual block is composed of two series of dilated causal convolution extraction modules and a 1×1 convolution layer. Each dilated causal convolution extraction module is composed of a dilated causal convolution layer, a weight normalization layer, a ReLU activation function layer, and a Dropout layer. The output data after the dilated causal convolution extraction operation is added to the original input data processed by the 1×1 convolution layer, and then output. The core formula of the dilated causal convolution residual block feature extraction is as follows: (5) In formula (5), k is a convolution kernel window index; d is an inflation rate, and a value of the inflation rate is equal to an inflation coefficient plus one; is a weight parameter after weight normalization; is a cin value of a channel at a (t-kxd) time of an input sequence; is a bias item corresponding to an output channel cout.

8. A PM that fuses an STL-VMD decomposition mechanism and an IBKA-TCNI Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that: (1)ITransformer model components are composed of Embedding embedding layer, multi-head differential attention mechanism layer (MUDA), layer normalization (LayerNorm), feedforward network layer, layer normalization (LayerNorm), linear transformation layer in series combination from input to output direction; (2)ITransformer model components adopt multi-head differential attention mechanism instead of traditional multi-head attention mechanism; for each attention head, two different weight matrices are used to calculate softmax(Q1K1 T ) and softmax(Q2K2 T ) respectively for the input X, obtaining two attention maps, where the first attention feature map is a matrix dominated by effective valuable information, and the second attention feature map is a matrix dominated by noise; the two matrices are operated by difference before being operated with the shared matrix V, and the differential operation formula corresponding to each head is as follows: (6) In formula (6), d k is a dimension corresponding to each head; is a learnable scalar parameter to control the weight of two attention maps; Q is a query matrix; K is a feature key matrix; V is a numerical matrix.

9. A PM of fusing STL-VMD decomposition mechanism and IBKA-TCN-I Transformer deep learning model according to claim 1 2.5 Concentration prediction method characterized in that The IBKA heuristic optimization algorithm is used to optimize the hyperparameters of the combined model, including the rolling window size, the training batch size, and the TCN output channel number. The optimization process is as follows: (1) First, determine the optimization interval range of each of the three hyperparameters; (2) Then initialize the population size N, the maximum number of iterations M, and set the algorithm control parameters α and β to 0.7 and 0.4 respectively; (3) The iterative search process algorithm inputs the generated population into the model, which is trained based on different populations and returns the corresponding performance indicator values (such as RMSE) for processing by the algorithm; (4) The algorithm converts the performance indicator values into fitness values and updates the population based on the best fitness individual, then inputs it into the model; (5) When the maximum number of iterations is reached or the best fitness value no longer improves, the termination is judged, otherwise the iteration process is repeated; (6) Finally, the global optimal hyperparameter combination is locked by iterating the model corresponding to each sub-data set multiple times to improve the model performance.

Images