An SO4 based on deep cross-network feature enhancement and ensemble learning 2- Concentration-driven factor analysis methods

Abstract

This invention relates to an SO4 based on deep cross-network feature enhancement and ensemble learning. 2‑ The concentration-driving factor analysis method includes the following steps: collecting time-series data on atmospheric pollutant concentrations and meteorological parameters in the target area, performing preprocessing and quality control; constructing a DCN V2 model integrating linear cross-networks, exponential cross-networks, and deep neural networks to achieve nonlinear high-order cross-feature enhancement of the original features; constructing a two-layer Stacking ensemble learning model with XGBoost, LightGBM, and CatBoost as heterogeneous base models and linear regression as the meta-model, and performing global optimization through Bayesian optimization; and applying the SHAP method to quantify the marginal contribution and direction of each feature to identify SO4. 2‑ Concentration-critical driving factors. This invention integrates optimization strategies such as GPU acceleration, early stopping mechanism, automatic task detection, parallel training, and backup training to automate, streamline, and reproducibly improve the analysis process, while ensuring the physical interpretability of the driving factor analysis, thus providing scientific and technological support for air pollution prevention and control decision-making.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of atmospheric environmental science and environmental informatics, specifically relating to a method for predicting and attributing atmospheric pollutant concentrations by integrating cutting-edge deep learning, ensemble learning, and interpretable artificial intelligence technologies. In particular, this invention provides a systematic analysis and modeling method for atmospheric sulfate concentrations, utilizing DCN V2 for high-order feature enhancement, combining a stacking ensemble learning framework to construct a high-precision prediction model, and applying the Shapley and Interpretation (SHAP) method to quantify the contributions of driving factors. This method aims to solve the challenges of complex atmospheric secondary inorganic salt formation mechanisms and the difficulty in balancing prediction accuracy and interpretability in traditional models, providing a scientific and reliable technical tool for the accurate analysis of air pollution sources and the formulation of control strategies. Background Technology

[0002] Sulfates in the atmosphere ( ) is a secondary inorganic aerosol ( The core component of ) is for fine particulate matter ( It has a significant impact on the formation of polluted weather, the decrease in regional visibility, changes in radiation forcing, and human respiratory health. It mainly originates from its gaseous precursor, sulfur dioxide (… In the atmosphere, it undergoes a series of homogeneous and heterogeneous chemical transformations. These transformations are affected by the concentration of precursors (such as...). nitrogen oxides ,ammonia ), oxidant levels (such as ozone) hydroxyl radicals It involves complex nonlinear regulation by multiple factors, including meteorological conditions (such as temperature, humidity, radiation, and wind speed) and aerosol pH. Therefore, accurate prediction... Understanding atmospheric concentrations and scientifically identifying their key drivers is crucial for comprehending atmospheric chemical processes, evaluating the effectiveness of pollution control measures, and developing efficient air pollution prevention and control strategies.

[0003] Currently, in response to Concentration prediction and attribution analysis are mainly divided into four categories, all of which have obvious technical shortcomings:

[0004] (1) Numerical simulation methods based on atmospheric chemical transport models (CTM): These methods (such as WRF-Chem, CMAQ, and GEOS-Chem) simulate the generation, transport, and deposition of pollutants by solving a set of partial differential equations describing atmospheric physicochemical processes. Although the mechanisms are clear, they have drawbacks such as high computational costs, large uncertainties in the input emission inventory, and difficulty in parameterizing complex nonlinear processes. Furthermore, the model results are usually difficult to directly provide a quantitative contribution to specific driving factors.

[0005] (2) Empirical modeling methods based on statistics: including multiple linear regression (MLR), principal component regression (PCR), and generalized additive model (GAM). These methods establish statistical relationships based on historical observation data and have high computational efficiency. However, they often assume that there are linear or additive relationships between variables, making it difficult to effectively capture the complex nonlinear interactions that are common in the atmospheric environment, resulting in limited prediction accuracy and generalization ability in complex pollution scenarios.

[0006] (3) Prediction methods based on traditional machine learning: algorithms such as Random Forest (RF), Support Vector Machine (SVM), and Gradient Boosting Decision Tree (GBDT) have been widely used in air quality prediction. Compared with statistical models, these methods can handle nonlinear relationships better, but as a single "black box" model, their prediction performance has a bottleneck, and the interpretability of the model is usually poor. Although preliminary attribution can be made through feature importance (such as Gini importance in RF and feature gain in GBDT), the reliability of its explanation and its consistency with the real physical mechanism are often questioned.

[0007] (4) Deep learning methods: In recent years, deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN) and their variants have been introduced into the environmental field, demonstrating powerful nonlinear fitting and sequence feature learning capabilities. However, pure deep learning models usually require a large amount of data, are prone to overfitting, and their internal decision-making processes are extremely opaque, making mechanism-based attribution analysis exceptionally difficult and limiting their application in scientific research and policy-making that require high credibility.

[0008] To overcome these challenges, research trends are converging towards improving model performance and enhancing model interpretability. In terms of performance improvement, feature engineering and model ensemble are key. Traditional manual feature engineering is inefficient and relies heavily on expert knowledge. Deep Cross Network (DCN) models, by introducing explicit feature cross layers, can automatically learn high-order interactions between features, providing an efficient and automated solution for feature engineering. DCN V2, as the latest evolution, integrates linear and exponential cross layers, theoretically capturing interaction patterns of different orders more comprehensively. Meanwhile, ensemble learning (such as Stacking), by combining the advantages of multiple heterogeneous base models, can effectively reduce variance and improve the model's generalization ability and robustness.

[0009] In terms of interpretability, Explainable Artificial Intelligence (XAI) techniques, particularly the game theory-based SHAP method, provide a consistent and reliable ex post-hoc explanation framework for complex "black box" models. SHAP values, with a solid mathematical foundation (Shapley values), provide an additive and equitable allocation of the contribution of each feature to a single prediction, thus clearly demonstrating the driving or inhibiting effects of features.

[0010] However, existing technologies have not yet systematically integrated advanced DCN V2 automated feature enhancement technology, multi-model stacking ensemble learning framework, and SHAP interpretability analysis, and specifically applied them to the atmosphere. Publicly available schemes for analyzing concentration-driving factors generally suffer from low automation, low computational efficiency, and insufficient robustness. Therefore, this invention addresses these technical pain points by proposing an end-to-end method... The concentration analysis method integrates DCNV2 and ensemble learning to improve prediction accuracy, combines SHAP to identify physically interpretable driving factors, and adds multiple optimization strategies to provide a complete technical solution for air pollution prevention and control. Summary of the Invention

[0011] 1. Purpose of the invention

[0012] This invention addresses existing To address the shortcomings of traditional concentration analysis methods, such as insufficient prediction accuracy, weak nonlinear feature mining capabilities, poor model interpretability, low automation, and insufficient robustness, this paper proposes a method based on DCN V2 feature enhancement and ensemble learning. A method for analyzing concentration drivers. This method enhances the high-order nonlinearity of the original features using DCN V2, improves prediction accuracy by combining Stacking ensemble learning, and utilizes the SHAP method to achieve physical interpretability attribution of the original features. Simultaneously, it integrates optimization strategies such as GPU acceleration, automatic task detection, parallel training, and backup training to automate, optimize, robust, and reproducible the analysis process, providing a basis for atmospheric... It provides scientific and reliable technical support for pollution prevention and control.

[0013] 2. Technical Solution

[0014] This invention provides a feature enhancement and ensemble learning based on DCN V2. The concentration driver analysis method comprises five core steps: data standardization preprocessing, DCN V2 nonlinear feature enhancement, Stacking ensemble model construction and hyperparameter optimization, end-to-end model training and performance evaluation, and SHAP driver identification and mechanism analysis. It also includes strategies for automatic task detection, model training optimization, and SHAP analysis optimization. The overall technical solution is as follows:

[0015] (1) Collect atmospheric pollutant concentration and meteorological observation data of typical Chinese cities, perform data cleaning and quality control, and generate a multidimensional feature dataset through standardized preprocessing for training, validation and testing in specific city-season scenarios. Concentration prediction model;

[0016] (2) Construct a deep learning model based on DCN V2 to perform nonlinear feature enhancement, explicitly learn the high-order interaction between features, extract the enhanced feature vector and concatenate it with the original features to construct a high-dimensional feature space;

[0017] (3) Based on the Stacking ensemble learning framework, a two-layer prediction model containing multiple heterogeneous base models and linear meta-models is constructed, and the hyperparameters are dynamically tuned using the Bayesian optimization algorithm.

[0018] (4) Utilize a fully trained ensemble model to Concentration is predicted, and regression accuracy is used to evaluate model performance.

[0019] (5) The Shapley sum and interpretation (SHAP) method was applied to quantify the contribution of each driving factor to the model prediction, revealing the influence of... The key physicochemical mechanisms of concentration changes.

[0020] The specific technical solution is as follows:

[0021] (1) Used for atmospheric environmental data collection, preprocessing and standardization

[0022] Collect data from environmental monitoring stations in the target area. Target concentration, precursor pollutants ( (etc.) and meteorological parameters (temperature) relative humidity Wind speed The dataset consists of time-series observations (e.g., [various variables]). Based on prior knowledge and data-driven principles, secondary transformation indicators and interfering variables with direct transformation relationships to the target variable are removed. The resulting feature set is represented as follows:

[0023]

[0024] in This represents the original set of all features collected, where SOR is the sulfur oxidation rate and NOR is the nitrogen oxidation rate.

[0025] Missing values ​​are handled in the filtered dataset, and sample records containing null values ​​(NaN) are removed;

[0026] The Z-Score standardization method is used to normalize the input features and eliminate dimensional differences. The standardization formula for the j-th feature is:

[0027]

[0028] in For the first The sample at the th The original values ​​of each feature For the first The mean of each feature, For the first The standard deviation of each feature was used; a random partitioning method was employed, with random seeds set to ensure reproducibility, and the standardized dataset D was divided into training sets at a ratio of 80% and 20%. and test set :

[0029]

[0030] Where Y is the target variable. Concentration, X is the corresponding input feature matrix.

[0031] (2) Feature enhancement based on DCN V2 deep learning model

[0032] A parallel deep cross-network model is constructed, consisting of a cross-network and a deep network, to automatically learn and extract complex nonlinear feature interaction representations from preprocessed raw atmospheric features.

[0033] Cross-networks are used to explicitly learn finite-order interactions between features, including The first cross layer, the... Layer output Defined as:

[0034]

[0035] in It is the original input feature vector. It is the first The output of the layer, and They are the first Layer weights and bias parameters, This indicates element-wise multiplication; through Layer cross operations capture third-order explicit interaction information between features.

[0036] Deep networks are used to implicitly learn high-order nonlinear interactions between features. They consist of multiple fully connected neural networks, with each layer defined as follows:

[0037]

[0038] in For deep networks Layer weights and biases For batch normalization operations, For activation function, For random deactivation operations, the number of hidden layer nodes is set to [256, 128, 64].

[0039] Feature enhancement and concatenation: Enhancing the output of the final layer of the cross-network With the final layer output of the deep network The features are concatenated to form an enhanced feature vector. The final feature inputs used for ensemble learning are the original features X and the enhanced features. Combinations:

[0040]

[0041] This combined feature set retains the original physical meaning while incorporating higher-order nonlinear interaction information.

[0042] (3) Building a prediction model and optimizing hyperparameters based on Stacking ensemble learning

[0043] Construct a two-layer stacking ensemble learning model. The first layer contains multiple heterogeneous gradient boosting tree-based models, specifically the extreme gradient boosting (XGBoost) model, the lightweight gradient boosting machine (LightGBM) model, and the categorical gradient boosting (CatBoost) model.

[0044] A Bayesian optimization algorithm is used to optimize the hyperparameter space of each base model, with the objective function being to minimize the mean squared error (MSE) of K-fold cross-validation.

[0045]

[0046] in , For hyperparameter combinations (including learning rate) Tree depth Number of estimators wait), This is the model prediction function configured with specific parameters. Five-fold cross-validation is used to generate meta-features for each sample in the training set. Its corresponding meta-feature vector Composed of the predictions of each base model at the validation fold:

[0047]

[0048] Constructing a second-layer meta-model A linear regression model was used to fit the meta-features. With real labels Learn the optimal linear combination weights of each base model. :

[0049]

[0050] in, The intercept is... These are the weight coefficients for each base model.

[0051] (4) Interpreting the ensemble model to identify Concentration key drivers

[0052] Based on the Shapley and Interpretation (SHAP) method, the pair of features is quantified. Marginal contribution of concentration prediction.

[0053] The TreeExplainer interpreter was used to calculate the test set samples respectively. SHAP value matrix on each base model :

[0054]

[0055] in Indicates the first In the base model, the first The feature is related to the first Marginal contribution of each sample's predicted value;

[0056] Regression coefficients of the combined meta-model Calculate the weighted SHAP value of the ensemble model:

[0057]

[0058] in These are the normalized meta-model weight coefficients;

[0059] Calculate the importance of global features , defined as the average of the absolute values ​​of the feature's SHAP values ​​across all samples:

[0060]

[0061] according to Sort all features in descending order to identify influences. The key drivers of concentration are identified, and the positive or negative impact of features on predicted values ​​is demonstrated using SHAP bee colony diagrams.

[0062] (5) Evaluation of the accuracy of the prediction model

[0063] Use the test dataset reserved in step (1) Validate the trained Stacking ensemble model. Input the test set into the model to obtain predictions. :

[0064]

[0065] The model performance is evaluated using regression accuracy metrics, including the coefficient of determination (COP). The root mean square error (RMSE) and mean absolute error (MAE) are calculated using the following formulas:

[0066]

[0067]

[0068]

[0069] Furthermore, the goodness of fit of the model and the normality of the residuals were further examined by plotting scatter plots of predicted and observed values, residual distribution plots, and QQ plots.

[0070] (6) Automated task management and error handling mechanism

[0071] The method further includes a complete automated task management and error handling mechanism. Before starting the analysis, the system automatically checks whether a complete analysis result file already exists in the target output directory. This file includes at least the trained ensemble model file, performance evaluation graph, and SHAP analysis graph. If a completed analysis result is detected, the system automatically skips the analysis task for that city-season combination and directly proceeds to the next pending task, avoiding redundant calculations and wasted computing resources. Simultaneously, the method employs a multi-process parallel approach to train multiple heterogeneous base models simultaneously, significantly shortening the overall training time. When the Bayesian hyperparameter optimization process fails, the system automatically continues training the base model using preset default parameter configurations, ensuring the integrity and continuity of the analysis process.

[0072] (7) Optimization measures for DCN V2 model training

[0073] The method integrates several optimization measures in DCN V2 model training. The system automatically detects available graphics processing units (GPUs) and prioritizes their use for model training. If an GPU is unavailable, it automatically switches to the central processing unit (CPU) and performs cache cleanup. An early stopping mechanism for monitoring validation loss is implemented during training. Training automatically stops when the validation loss fails to improve within a preset number of training epochs (default 10) to prevent overfitting.

[0074] A dynamic learning rate adjustment strategy based on validation loss is adopted. When the validation loss stagnates, the learning rate is automatically reduced. The learning rate update formula is as follows: Where 0 < γ < 1 is the decay factor. The network weights are initialized using a Xavier normal distribution. For linear layer weight matrices... From a mean of 0 and a variance of Sampling is performed within a normal distribution to accelerate model convergence. A batch normalization layer is added after each fully connected layer of the deep network to standardize the data for each batch. The normalization formula is... Where μ6 and σ6 are the mean and standard deviation of the batch data, respectively, and ε is the numerical stability constant.

[0075] After training, the complete model state, feature normalizer, feature name list, and network structure parameters are saved to support subsequent feature extraction and model reuse.

[0076] (8) Optimization measures for ensemble learning model construction

[0077] For different types of base models, the optimal model saving format is adopted to improve compatibility and reusability: XGBoost models are saved in JSON format to meet the model serialization specifications of the XGBoost library; LightGBM models are saved in joblib serialization format to support the complete saving of model structure and parameters; CatBoost models are saved in CatBoost native format to ensure the integrity of model functionality.

[0078] All model hyperparameter configurations, training status information, and environment configurations are saved in a single configuration file, including the optimal hyperparameter set for each base model, the metamodel regression coefficient vector, training timestamps, training device information, and data preprocessing configuration.

[0079] A multi-process parallel training mechanism is adopted, and the maximum number of parallel processes is set to [value]. = min(4, ),in To accommodate the number of base models to be trained, a combination of delayed and parallel execution is employed to ensure that each model's training task is performed independently. An exception handling mechanism is added to each training task, ensuring that the failure of a single model does not affect the training of other models. When the Bayesian hyperparameter optimization process fails, the optimization anomaly is automatically captured, failure information is recorded, and training continues using the default parameter configurations of each base model, ensuring the robustness of the analysis process.

[0080] (9) SHAP analysis and targeted optimization measures

[0081] The proposed method features targeted optimizations for SHAP analysis. After DCN V2 feature enhancement, SHAP analysis is performed only on the original physical features, excluding enhanced features generated by deep cross-networks, thus ensuring the physical interpretability and environmental science significance of the driving factors.

[0082] The SHAP value of the ensemble model is calculated using a weighted average method based on the regression coefficients of the meta-model. When the meta-model is unavailable, an equal-weighted average method is used. To improve computational efficiency, when the number of samples exceeds a preset threshold (default 1000), a random sampling method is used to select representative samples for SHAP value calculation. A fixed random seed (default 42) is used during the sampling process to ensure reproducibility of the results, effectively balancing computational efficiency and analytical accuracy.

[0083] 3. Technological Advantages

[0084] Compared with the prior art, the present invention has the following significant advantages:

[0085] (1) Significantly improved prediction accuracy: By introducing the DCN V2 model, high-order cross features are explicitly constructed, compensating for the shortcomings of traditional feature engineering in nonlinear interaction mining and providing high-quality input to the model; at the same time, combined with the Stacking ensemble learning framework, the advantages of heterogeneous models such as XGBoost, LightGBM, and CatBoost are integrated, effectively reducing the variance and bias of a single model and significantly improving the accuracy. Accuracy of concentration prediction.

[0086] (2) The driving factor analysis has physical interpretability: Unlike traditional deep learning "black box" models, this invention integrates a SHAP interpretability analysis module. By calculating the weighted SHAP value of the integrated model, it is possible not only to identify the factors influencing the analysis but also to understand the underlying causes of the problem. The key drivers of concentration (such as specific meteorological conditions or precursors) can also be visually shown to demonstrate the direction of each factor's marginal contribution to concentration changes (promoting or inhibiting), providing a scientific basis at the physical and chemical levels for the analysis of the causes of air pollution.

[0087] (3) Automated and efficient analysis process: The integrated automatic task detection mechanism avoids redundant calculations, and the use of GPU acceleration and multi-process parallel training greatly shortens the training time, making it suitable for large-scale batch processing analysis in multiple cities and seasons; Bayesian optimization realizes automated hyperparameter optimization, avoiding the inefficiency and subjectivity of manual parameter tuning;

[0088] (4) The method is highly robust and reproducible: a fixed random seed is set to control all random operations, ensuring that the analysis results are consistent at different times and on different devices, meeting the reproducibility requirements of scientific research; the integrated backup training and anomaly handling mechanism ensures that the failure of a single model training or hyperparameter optimization does not affect the overall process, ensuring the integrity and continuity of the analysis process.

[0089] (5) Model traceability and strong reusability: The optimal saving format is adopted for different base models to improve model compatibility and loading and reuse efficiency; the hyperparameters, training status and environment configuration of all models are recorded through a unified configuration management file to achieve full life cycle traceability of the model;

[0090] (6) Strong generalization ability: The method design takes into account the heterogeneity of different cities and seasons, and has strong spatiotemporal generalization ability, making it suitable for the analysis of atmospheric pollution characteristics in different regions. Attached Figure Description

[0091] Figure 1 It is a technology roadmap.

[0092] Figure 2 It is the ensemble model on the test set A scatter plot of predicted and observed concentrations.

[0093] Figure 3 Is it the ensemble model in prediction? SHAP values ​​for interpreting and describing input features at concentration levels (bee colony diagram) Detailed Implementation

[0094] To make the technical solution and implementation effects of this invention clearer and more explicit, the specific implementation steps of this invention are described in detail below using seasonal atmospheric monitoring data from typical air-polluted cities in northern China (Beijing, Zhengzhou, Jinan, Shijiazhuang, Tianjin, and Taiyuan). This example is only used to explain this invention and is not intended to limit the scope of protection of this invention.

[0095] Example 1: Feature enhancement and ensemble learning based on DCN V2 Concentration driving factor analysis

[0096] Step 1. Data Preparation and Preprocessing

[0097] 1.1 Data Collection: Time-series observation data of air pollutants and meteorological parameters for the four seasons in the above-mentioned cities were collected.

[0098] 1.2 Feature Filtering: Exclusion and Secondary transformation indicators with direct transformation relationships (such as sulfur oxidation rate SOR and nitrogen oxidation rate NOR) and specific confounding variables (such as date and city name) ensure the independence of the driving factor analysis.

[0099] 1.3 Data cleaning: Remove sample records containing null values ​​to ensure the integrity of the dataset;

[0100] 1.4 Standardization Processing: Z-Score standardization is used to normalize all input features and eliminate differences in units.

[0101] 1.5 Data partitioning: Group the data by city-season dimension, set a fixed random seed, and divide each group of data into training set and test set at 80%:20% to ensure that the results are reproducible.

[0102] Step 2. DCN V2 Nonlinear Feature Enhancement

[0103] 2.1. Model Construction: Construct a parallel DCN V2 model, which includes 3 layers of cross-network and 3 layers of fully connected deep network (number of nodes [256, 128, 64]).

[0104] 2.2 Feature Extraction: GPU is used to accelerate model training, and an early stopping mechanism (Patience=10) is used to prevent overfitting.

[0105] 2.3 Feature Enhancement: The high-order cross-features (DCN Features) extracted by the model are concatenated with the original physical features to form a high-dimensional enhanced feature space, which serves as the input for subsequent ensemble learning.

[0106] Step 3. Stacking Ensemble Learning Model Training

[0107] 3.1 Base Model Training: The Bayesian optimization algorithm was used to automatically optimize the hyperparameters of XGBoost, LightGBM, and CatBoost under 3-fold cross-validation.

[0108] 3.2 Integration Construction: Five-fold cross-validation is used to generate meta-features. The second-level meta-model adopts a linear regression model to learn the optimal weight allocation of each base model, and finally, a Stacking ensemble prediction model is constructed.

[0109] 3.3 Task Management: The system automatically detects completed city-season tasks and skips them. For incomplete tasks, multi-process parallel training is used to improve efficiency.

[0110] Step 4. Model Evaluation and Performance Demonstration

[0111] 4.1 Calculation of indicators: Evaluate the model on the independent test set and calculate the coefficient of determination, root mean square error and mean absolute error.

[0112] 4.2 Visual Analysis: The goodness of fit of the model was verified by plotting scatter density plots and residual distribution plots of predicted and observed values. In this embodiment, the seasonal determination coefficients for each city showed high prediction accuracy.

[0113] Step 5. Identification of Driving Factors

[0114] 5.1 Contribution Quantification: The SHAP method is applied, and the weighted SHAP value of the ensemble model is calculated by combining the meta-model weights.

[0115] Focused Analysis: SHAP analysis focuses solely on the original physical characteristics, excluding DCN-enhanced features, to ensure the physical meaning of the environmental science interpretation.

[0116] 5.2 Result Identification: Influence factors were identified through global feature importance ranking and SHAP bee colony diagram. Identify the key drivers of pollution concentration and quantify their promoting or inhibiting effects to provide decision support for pollution prevention and control.

Claims

1. A method based on deep cross-network feature enhancement and ensemble learning The concentration-driven factor analysis method is characterized by, Includes the following steps: (1) Atmospheric environmental data collection and standardization preprocessing: Data containing sulfate were collected from multiple environmental monitoring stations in the target area. The dataset contains time-series observation data of target concentration, concentrations of various precursor pollutants, and meteorological parameters. Data cleaning, outlier detection and filtering, and feature selection operations are performed on the dataset to remove secondary transformation indicators and specific interference variables. The selected features are normalized by Z-Score, and the data are grouped according to city and season dimensions. Each group of data is divided into training subset and test subset by a fixed random seed. (2) DCN V2 nonlinear feature enhancement: Construct a DCN V2 model that integrates linear cross network LCN, exponential cross network ECN and multi-layer deep neural network to automatically learn and extract complex nonlinear feature interactions from the preprocessed original atmospheric features; After training, use the model to convert the original input features into high-dimensional enhanced features, and concatenate the enhanced features with the original features along the feature dimension to generate the final enhanced feature set for subsequent modeling; (3) Stacking ensemble learning model construction and hyperparameter optimization: A two-layer stacking ensemble learning model is constructed, in which the first layer contains multiple heterogeneous gradient boosting tree base models, specifically the extreme gradient boosting XGBoost model, the lightweight gradient boosting machine LightGBM model and the categorical gradient boosting CatBoost model; the second layer meta-model adopts a linear regression model; a Bayesian optimization framework is adopted, with the root mean square error of cross-validation as the optimization objective, to automatically and globally optimize the architecture hyperparameters of the DCN V2 model and the key hyperparameters of each base model of the ensemble model; (4) Integration model training and performance evaluation: Using the training subset generated in step (1) and the corresponding augmented features generated in step (2), the DCN V2 feature enhancer and Stacking integration learning model after hyperparameter optimization in step (3) are trained end-to-end; after training, the prediction performance of the trained integration model is comprehensively evaluated using the test subset reserved in step (1) and the corresponding augmented features. The evaluation indicators include at least the root mean square error RMSE, the coefficient of determination R² and the mean absolute error MAE. (5) Model interpretability analysis and driving factor identification based on SHAP: The Shapley and interpretable SHAP methods are applied to perform post-trained interpretability analysis on the fully trained Stacking ensemble learning model; the SHAP value of each original input feature on the model's predicted output is calculated and analyzed to quantify the impact of each feature on the model's prediction output. The marginal contribution and direction of concentration prediction are analyzed, and key driving factors are identified through global feature importance ranking, thereby identifying the influencing factors. The key drivers of concentration changes were identified, and their underlying physicochemical mechanisms were explained.

2. The method according to claim 1, characterized in that, The specific implementation of step (1) includes: Based on prior knowledge of atmospheric chemistry and data quality control principles, feature selection rules were defined to eliminate features that are not closely related to the target variable SO4. 2- The feature set after screening of secondary transformation indicators with direct transformation relationships and specific interference variables is represented as follows: ； in, This is the set of all features originally collected. For sulfur oxidation rate, For nitrogen oxidation rate, For cities, The mass concentration of particulate matter with an aerodynamic diameter ≤ 2.5 μm. The time marker itself is not a driving factor; Missing values ​​are handled in the filtered dataset, and sample records containing null values ​​NaN are removed; The input features are normalized using the Z-Score normalization method to eliminate dimensional differences. The standardization formula for each feature is: ； in, For the first The sample at the th The original values ​​of each feature For the first The mean of each feature, For the first The standard deviation of each feature; a random partitioning method is used, with a random seed set to ensure reproducibility, and the standardized dataset is divided into groups. The training set was divided into two groups of 80% and 20% respectively. and test set : 。 3. The method according to claim 1, characterized in that, The specific implementation of step (2) includes: A parallel deep cross-network model is constructed, consisting of two parts: a cross-network and a deep network. The cross-network is used to explicitly learn finite-order interactions between features, including... The first cross layer, the... Layer output Defined as: ； in, It is the original input feature vector. It is the first The output of the layer, and They are the first Layer weights and bias parameters, This indicates element-wise multiplication; through Layer crossover operations capture third-order explicit interaction information between features; deep networks are used to implicitly learn higher-order nonlinear interactions between features, and consist of multiple fully connected neural networks, with each layer defined as follows: ； in, and For deep networks Layer weights and biases For batch normalization operations, For activation function, For random deactivation operations, the number of hidden layer nodes is set to [256, 128, 64]; Feature enhancement and concatenation: Enhancing the output of the final layer of the cross-network With the final layer output of the deep network The features are concatenated to form an enhanced feature vector. The final feature inputs used for ensemble learning are the original features X and the enhanced features. Combinations: 。 4. As described in claim 1, characterized in that, The specific implementation of step (3) includes: Constructing a heterogeneous first-layer base model set The models are constructed based on histogram algorithms and symmetric tree structures, respectively. Bayesian optimization is used to optimize the hyperparameter space Λ of each base model, with the objective function being to minimize the mean squared error (MSE) of K-fold cross-validation. ； in , For hyperparameter combinations, including learning rate Tree depth Number of estimators , The model prediction function is configured with specific parameters; 5-fold cross-validation is used to generate meta-features for each sample in the training set. Its corresponding meta-feature vector Composed of the predictions from each base model at the validation fold: ； Constructing a second-layer meta-model A linear regression model was used to fit the meta-features. With real labels Learn the optimal linear combination weights of each base model. : 。 5. The method according to claim 1, characterized in that, The specific implementation of step (5) includes: The TreeExplainer interpreter was used to calculate the test set samples respectively. SHAP value matrix on each base model : ； in, Indicates the first In the base model, the first The feature is related to the first Marginal contribution of each sample's predicted value, The number of samples in the test set. This is the original physical feature dimension, without the cross features enhanced by DCN V2; Regression coefficients of the combined meta-model Calculate the weighted SHAP value of the ensemble model: ； in, These are the normalized meta-model weight coefficients; Calculate the importance of global features , defined as the average of the absolute values ​​of the feature's SHAP values ​​across all samples: ； according to Sort all features in descending order to identify influences. Key drivers of concentration.

6. The method according to claim 1, characterized in that, It also includes an automatic task detection and skipping mechanism, specifically implemented as follows: Before the analysis task starts, the system automatically checks whether a complete analysis result file already exists in the target output directory. This check is performed by verifying the existence and non-emptiness of the following key files: (1) Basic model files: including XGBoost model files, LightGBM model files, and CatBoost model files; (2) Model configuration file: A configuration file containing hyperparameter configuration and metamodel information; (3) Analysis results files: including performance evaluation graph files and SHAP analysis graph files; If all the above files are found to exist and be valid, the analysis task for the city-season combination is considered complete, and the task is automatically skipped, proceeding directly to the next pending task; otherwise, the complete analysis process for the city-season combination continues to be executed to avoid wasting computing resources and improve the execution efficiency of large-scale batch processing tasks.

7. The method according to claim 1, characterized in that, Step (2) of DCN V2 model training also includes the following specific technical details: (1) Graphics processor acceleration configuration: The system automatically detects available graphics processor devices, prioritizes the use of graphics processors for model training, and automatically switches to the central processing unit if unavailable. (2) Early stop mechanism: Set up validation loss monitoring, and automatically stop training when the validation loss has not improved in a preset number of consecutive training rounds; (3) Adaptive learning rate adjustment: A dynamic learning rate adjustment strategy based on validation loss is adopted, and the learning rate is automatically reduced when the validation loss stagnates; the adaptive learning rate adjustment uses the ReduceLROnPlateau scheduler and satisfies the following conditions: a. The monitoring metric is the verification loss L. val ; b. When the validation loss is continuous If there is no improvement after a certain number of training rounds, the learning rate is multiplied by a decay factor γ, where 0 < γ < 1; c. The learning rate update formula is: ; (4) Weight initialization strategy: The Xavier normal distribution initialization method is used to initialize the network weights to accelerate model convergence; for the linear layer weight matrix From a mean of 0 and a variance of Sampling from a normal distribution; (5) Batch normalization: Add a batch normalization layer after each fully connected layer of the deep network to improve training stability and convergence speed. ； Where μ6 and σ6 are the mean and standard deviation of the batch data, respectively, and ε is the numerical stability constant; (6) Model persistence: Save the complete model state, feature normalizer, feature name list and network structure parameters to support subsequent feature extraction and model reuse.

8. The method according to claim 1, characterized in that, The construction of the ensemble learning model in step (3) also includes the following specific technical details: (1) Parallel training mechanism: Multiple heterogeneous base models are trained simultaneously using a multi-process parallel approach, which significantly shortens the overall training time; the parallel training mechanism is implemented in the following ways: a. Set the maximum number of parallel processes = min(4, ),in The number of base models to be trained; b. Employ a combination of delayed execution and parallel execution to ensure that each model training task is performed independently; c. Add an exception handling mechanism to each training task so that the failure of a single model training does not affect the training of other models; (2) Backup training mechanism: When the Bayesian hyperparameter optimization process fails, the basic model is automatically trained using the preset default parameter configuration to ensure the integrity and continuity of the analysis process; The backup training mechanism is activated when Bayesian hyperparameter optimization fails, and includes the following steps: a. Capturing anomalies during the Bayesian optimization process ; b. Record optimization failure information; c. Use the default parameter configurations for each basic model. Continue training; (3) Heterogeneous model saving format: For different types of basic models, the optimal model saving format is adopted to improve compatibility and reusability; the heterogeneous model saving format adopts a specific format for different types of basic models: a. XGBoost models are saved in JSON format, which conforms to the XGBoost library's model serialization specification; b. LightGBM models are saved in joblib serialization format, which supports complete saving of model structure and parameters; c. Save the CatBoost model in the native CatBoost format (.cbm) to ensure the integrity of the model's functionality; (4) Unified configuration management: All model hyperparameter configurations, training status information and environment configurations are saved to a single configuration file; a. The optimal hyperparameter set Λ* = {λ1, λ2, λ3*} for each basic model; b. Metamodel regression coefficient vector β = [β0, β1, β2, β3]; c. Training timestamp and training equipment information ; d. List of feature names and data preprocessing configuration .

9. The method according to claim 1, characterized in that, Step (5) of the SHAP analysis also includes the following specific technical details: (1) Original feature-focused analysis mechanism: After DCN V2 feature enhancement, SHAP analysis is only performed on the original physical features, excluding the enhanced features generated by deep cross-networks, to ensure the physical interpretability and environmental science significance of the driving factors. (2) Weighted calculation method of ensemble SHAP value: The SHAP value of the ensemble model is calculated by the weighted average method based on the regression coefficients of the meta-model. When the meta-model is unavailable, the equal weighted average method is used. (3) Optimization of computational efficiency: When the number of samples exceeds a preset threshold, a random sampling method is used to select representative samples for SHAP value calculation, balancing computational efficiency and analytical accuracy. The following strategy is adopted for computational efficiency optimization: a. Set a sample size threshold = 1000; b. When the sample size > At that time, a random sampling method was used to select SHAP values ​​were calculated for each sample. c. The sampling process uses a fixed random seed. = 42 Ensures the results are reproducible.

Images