Near-surface atmospheric pollutant deep learning prediction method and system based on hyper-spectral satellite remote sensing and ground observation data

Abstract

The invention discloses a near-surface atmospheric pollutant deep learning prediction method based on hyper-spectral satellite remote sensing and ground observation data. The near-surface atmospheric pollutant deep learning prediction method comprises the following steps: preprocessing collected multi-source data; constructing an assimilation network adopting a convolutional neural process, ViT and ResNet combined architecture, training the assimilation network by using the preprocessed multi-source data, and generating an initial assimilation state at the current moment; and constructing a prediction network with the same architecture as the assimilation network, taking the initial assimilation state at the current moment as input, carrying out time extrapolation in combination with multi-source data at future moments, respectively constructing independent prediction networks at different future moments, carrying out independent training, and outputting pollutant concentration probability prediction results at different time periods in the future. Therefore, the probabilistic pollutant concentration prediction with high spatial resolution of 0.1 degree * 0.1 degree and 72-hour time efficiency is realized without depending on a chemical transmission model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of atmospheric environment monitoring and prediction, and particularly relates to a near-surface atmospheric pollutant deep learning prediction method and system based on hyperspectral satellite remote sensing and ground observation data. BACKGROUND

[0002] Near-surface atmospheric pollution poses a serious threat to public health, ecosystems and sustainable development. Air quality forecasting is particularly important.

[0003] Traditional air quality forecasting methods mainly rely on chemical transport models (CTMs), which simulate the generation, diffusion, transformation and deposition processes of pollutants through mathematical equations. However, CTMs have significant limitations: first, the spatial resolution is low, making it difficult to capture the fine pollution distribution at the regional scale; second, the computational complexity is high, requiring a large amount of computing resources, resulting in delayed forecasting; third, parameterization simplification and uncertainty quantification are difficult, such as insufficient simulation accuracy for O3 photochemical reactions, PM2.5 boundary layer diffusion, etc. Existing operational systems such as the Copernicus Atmospheric Monitoring Service (CAMS) have a correlation coefficient of less than 0.5 for NO2 and O3 prediction in China, and there is a serious overestimation problem (such as SO2 overestimation of 277%, NO2 overestimation of 54%).

[0004] In recent years, deep learning technology has shown potential in the field of meteorology and environment, but existing data-driven pollutant prediction models still have shortcomings: for example, the model disclosed in the literature "A Hybrid Data-Driven and Data Assimilation Method for Spatiotemporal Forecasting" relies on CTMs data as input or initial conditions, and cannot reason without traditional models; as pointed out in the literature "Accurate medium-range global weather forecasting with 3D neural networks", recurrent neural networks (RNN) and other architectures are prone to error accumulation, and their performance decreases sharply with increasing forecasting time; the receptive field of convolutional neural networks (CNN) and other models is limited, making it difficult to capture long-distance pollution transmission processes; at the same time, the heterogeneity and irregularity of multi-source data (ground stations, satellites, weather, etc.) also increase the difficulty of data fusion, making it difficult for existing models to achieve grid-based high-resolution forecasting. SUMMARY

[0005] In view of the above, the purpose of the present application is to provide a near-surface atmospheric pollutant deep learning prediction method and system based on hyperspectral satellite remote sensing and ground observation data, which does not need to rely on CTMs, and can realize high-precision, long-time, high-resolution atmospheric pollutant prediction through innovative network architecture and data fusion strategy, to meet the business needs of pollution prevention and control.

[0006] To achieve the above-mentioned purpose of the application, the embodiment provides a near-surface atmospheric pollutant deep learning prediction method based on hyperspectral satellite remote sensing and ground observation data, comprising: The collected multi-source data is preprocessed, wherein the multi-source data includes multi-source observation data and multi-source auxiliary data, and the multi-source observation data includes ground observation data and satellite remote sensing data. An assimilation network adopting a combined architecture of a convolutional neural process module, a ViT and a ResNet is constructed, the preprocessed multi-source data is input into the assimilation network, coordinate conversion is performed through the convolutional neural process module, spatial dependence capture is performed through the ViT, and dimension adjustment is performed through the ResNet, the assimilation network is trained and an initial assimilation state at the current time is generated; A prediction network with the same architecture as the assimilation network is constructed, the initial assimilation state at the current time is taken as input, time extrapolation is performed in combination with multi-source data at future times, independent prediction networks at different future times are respectively constructed and separate training is carried out, and deterministic prediction values of pollutant concentrations at different future times and uncertainty intervals of the 10th-90th percentile are output.

[0007] Preferably, the multi-source observation data includes ground station observation data and satellite remote sensing data, and the multi-source auxiliary data includes meteorological data, emission inventory data and static geographic data.

[0008] The ground station observation data is hourly pollutant concentration data of national control stations, including PM2.5, PM10, SO2, NO2, CO and O3 concentrations; and the satellite remote sensing data includes Himawari-8 aerosol optical depth (AOD) products and TROPOMI satellite-retrieved tropospheric O3 and NO2 tropospheric vertical column density (VCDs) data.

[0009] The meteorological data includes ERA5 reanalysis data used in the training phase and CAMS global atmospheric composition forecast data used in the business prediction phase, specifically including 10m wind field components (U10 / V10), 2m temperature (T2M) and relative humidity (RH); the relative humidity is calculated based on the Magnus formula from T2M and 2m dew point temperature.

[0010] The emission inventory data is an MEIC emission inventory; the static geographic data includes land use classification (LU), digital elevation model (DEM) (without convolutional neural process module), and population density distribution (POP) data, and the meteorological data, LU, and DEM data are resampled to 0.1°*0.1° grid during data preprocessing, which is consistent with the output resolution.

[0011] Preferably, the preprocessing of the multi-source observation data includes data standardization, time coding, grid resampling, and label one-hot coding.

[0012] Specifically, during the data preprocessing process, a differentiated standardization strategy is adopted according to the data distribution characteristics: ① for meteorological data (U10, V10, T2M, RH) conforming to normal distribution, Z-score standardization is adopted (Equation 1) ), which is calculated based on the mean (Equation 2) ) and standard deviation (Equation 3) ) of the training set; ② for emission inventory and population density distribution data (MEIC, POP) conforming to lognormal distribution, log transformation (Equation 4) ) is first performed, and then Z-score standardization is adopted; ③ for AOD and VCDs data of satellite remote sensing, maximum-minimum standardization (Equation 5) ) is directly adopted to map the numerical value to the interval [0, 1] and eliminate the dimensional difference.

[0013] During the data preprocessing process, continuous time is represented by sine-cosine coding (Equation 6) ), which converts day (day of the year) and hour (0-23) into periodic feature vectors, while generating one-hot coding of the day of the week (1-7) and normalized coding of the year (Equation 7) ) as time feature input to the network.

[0014] During the data preprocessing process, the pollutant concentration label is one-hot coded for 64 bits, which is used for probabilistic prediction training.

[0015] Preferably, the preprocessed multi-source observation data of the current time and auxiliary data are input into the assimilation network, wherein the meteorological data in the auxiliary data uses data of the current time and a previous period of time to predict the initial assimilation state of the current time, and a cross-entropy loss function is used to train the assimilation network: wherein, Convolutional Neural Process module, which realizes the coordinate conversion from irregular observation grid to regular grid and encodes the data sparsity through the density channel; ViT, which captures spatial dependencies through attention mechanism; ResNet, which adjusts the dimension to generate the probability output; the initial assimilation state output by the assimilation network, high-resolution pollution reanalysis data labels, denotes the cross-entropy loss, denotes the loss function.

[0016] Preferably, independent prediction networks for different future time points are constructed and trained separately, and the probability prediction results of pollutant concentration at different future time points are output, including: For each future time point, the initial assimilation state at the current time point, the preprocessed multi-source observation data at the current time point and auxiliary data are taken as the input of the prediction network, wherein the meteorological data in the auxiliary data uses data from the current time point to the future time point, and the prediction network predicts the probability density distribution of pollutant concentration at the future time point, avoiding the error accumulation problem caused by multiple iterations of extrapolation in traditional cycle models. During training, the cross-entropy loss function is also used to train the prediction network, and the prediction network is obtained: wherein, 、 and respectively represent the Convolutional Neural Process module, ViT and ResNet in the prediction network processing different input data; the initial assimilation state at the current time point output by the assimilation network, the probability density distribution of pollutant concentration at the future time point output by the prediction network, is the high-resolution pollution reanalysis data label corresponding to the time point .

[0017] Preferably, the Convolutional Neural Process module in the assimilation network and the prediction network realizes the conversion between non-grid observation data and regular grid observation data in the coordinate conversion, which specifically includes: Parameter initialization: determining the initial trainable parameters, data conversion mode, grid step, and whether to enable the density channel, wherein the data conversion mode includes the mode of scatter data to gridded data, the mode of gridded data to gridded data, and the mode of gridded data to scatter data; Distance and weight calculation: based on the spatial coordinates of the input data and the target data, first calculate the squared distance between them, then through the Gaussian kernel function, combined with trainable parameters, generate weight values reflecting the degree of association between data, the weight value is negatively related to the spatial distance of data, and ensure that the data with similar space contribute higher weight; Data conversion processing: if it is a scatter point data to grid data mode, first filter the valid data in the input non-grid data through the mask, remove the invalid value interference, then combine the calculated weight, and convert the non-grid data into regular grid data representation, output the data after density channel normalization; If it is a grid data to grid data mode, directly based on weight calculation, realize the conversion between different regular grid data, output the data after density channel normalization; If it is a grid data to scatter point data mode, first filter the valid output area in the target non-grid data, then convert the regular grid data into non-grid prediction data according to the weight, and output the data after density channel normalization; In the above process, a density channel is first added to mark the area where the data exists or not, providing an effectiveness identifier for subsequent data processing, and the identifier is input into the ViT together.

[0018] Preferably, the convolutional neural process module adopts different modes for different data: for ground station observation data, scatter point data to grid data mode is adopted, 6 pollutant channels (PM2.5, PM10, SO2, NO2, CO, O3) are set, and 6 density channels are added; for satellite remote sensing data, meteorological data, emission inventory data, and regular grid data containing missing values in static geographic data, grid data to grid data mode is adopted, and the original data is mapped to the standard grid; the remaining data (such as digital elevation model (DEM)) is directly inputted as it is grid data, and does not need mode conversion.

[0019] Preferably, the method further comprises performance evaluation of the prediction network, calculating the correlation coefficient, mean absolute error (MAE), root mean square error (RMSE) and pollution event identification accuracy (precision and recall) based on the national control site observation data, and completing the model performance evaluation.

[0020] To achieve the above-mentioned purposes, the embodiment of the present application also provides a near-surface atmospheric pollutant deep learning prediction system based on hyperspectral satellite remote sensing and ground observation data, comprising: A preprocessing module is used for preprocessing the collected multi-source data, wherein the multi-source data includes multi-source observation data and multi-source auxiliary data, and the multi-source observation data includes ground observation data and satellite remote sensing data; an assimilation module, configured to build an assimilation network adopting a convolutional neural process, a ViT and a ResNet combined architecture, input the preprocessed multi-source data into the assimilation network, perform coordinate conversion through the convolutional neural process module, capture spatial dependence through the ViT, and perform dimension adjustment through the ResNet, train the assimilation network and generate an initial assimilation state at the current time; a prediction module, configured to build a prediction network with the same architecture as the assimilation network, input the initial assimilation state at the current time, perform time extrapolation in combination with multi-source data at future times, build independent prediction networks for different future times respectively and carry out separate training, and output the pollutant concentration probability prediction results at different future times.

[0021] Compared with the prior art, the present application has at least the following beneficial effects: The method and system of the present application do not need to rely on chemical transmission models (CTMs), and by fusing multi-source data such as ground station observations, satellite remote sensing inversion, meteorological forecasts and emission inventories, a neural network architecture combined with a convolutional neural process, a Vision Transformer (ViT) and a ResNet is adopted to realize 0.1°x0.1° high spatial resolution and 72-hour time-effect probability pollutant concentration prediction. The system contains two core modules, namely an assimilation network and a prediction network. The assimilation network directly generates an initial assimilation state from multi-source observation data, and the prediction network is based on the initial assimilation state, adopts a meteorological data window at t=0 to t= =72 hours, builds independent prediction networks for different future times respectively and carries out separate training, outputs prediction results at each time through time extrapolation, and effectively avoids the error accumulation problem of the cycle model. According to the data verification of the Environmental Monitoring Center (CNEMC) in 2022, the prediction effect of the system on main pollutants is significantly better than that of the existing Copernicus Atmospheric Monitoring Service (CAMS) system, and it can accurately identify long-distance pollutant transmission and dust and other pollution events, providing efficient and reliable technical support for atmospheric pollution prevention and control decision-making. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0023] Figure 1 a flowchart of the method for predicting near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data according to the embodiments of the present application; Figure 2 ​A comparison chart of prediction performance of the USTC-AQM system and the CAMS system provided for the embodiment of the present application is provided; Figure 3 A comparison chart of prediction performance of the USTC-AQM system and the CAMS system provided for the embodiment of the present application is provided; Figure 4 A comparison chart of prediction performance of the USTC-AQM system and the CAMS system provided for the embodiment of the present application is provided; Figure 5 A comparison chart of prediction performance of the USTC-AQM system and the CAMS system provided for the embodiment of the present application is provided; DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer and more comprehensible, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0025] The present application provides a deep learning prediction method and system for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data. Taking the 72-hour prediction of six main atmospheric pollutants (PM2.5, PM10, SO2, NO2, CO, and O3) near the surface in China as an example, based on multi-source observation data from 2015 to 2022, a probabilistic forecast with a spatial resolution of 0.1°x0.1° is achieved. The entire method and system achieve pollutant prediction based on the collaborative work of assimilation networks and prediction networks. The assimilation network integrates observation data and auxiliary data to generate the initial assimilation state at t=0, and the prediction network performs time extrapolation to t=

[0026] As shown in Figure 1 , the deep learning prediction method for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data provided by the embodiment comprises the following steps: S1, pre-processing the collected multi-source data, wherein the multi-source data includes multi-source observation data and multi-source auxiliary data.

[0027] In the embodiment, the multi-source observation data includes ground station observation data and satellite remote sensing data, and the multi-source auxiliary data includes meteorological data, emission inventory data, and static geographic data.

[0028] ​Among them, the ground station observation data: 2015-2022 hourly PM2.5, PM10, SO2, NO2, CO, O3 concentration data of 2016 monitoring stations. Satellite remote sensing data: 2018-2022 Himawari-8 AOD products and USTC-TROPOMI O3, NO2 VCDs data.

[0029] Weather data: ERA5 reanalysis data (U10 / V10, T2M, dew point temperature) were downloaded from the Copernicus Climate Data Store during the training phase; CAMS global atmospheric composition forecast data were obtained from the Copernicus Atmosphere Data Store during the inference phase. Relative humidity (RH) was calculated by the Magnus formula.

[0030] Emission inventory and static geographic data: MEIC emission inventory (2015-2020, the latest MEIC inventory data available at the corresponding time is used during the operational run) was obtained from Tsinghua University, land use classification (LU) data (MCD12C1 Version 6.1) was obtained from the National Aeronautics and Space Administration Land Processes Distributed Active Archive Center, digital elevation model (DEM) data (GEBCO_2014 Grid) was obtained from the British Oceanographic Data Centre, and population density (POP) data (GPWv4 Revision 11) was obtained from the National Aeronautics and Space Administration Socioeconomic Data and Applications Center.

[0031] In the embodiment, the above-mentioned multi-source data is processed, and the processing content includes data standardization, time coding, grid resampling, and label one-hot coding, etc., specifically including: (1) Data division: the ground station observation data is divided into 2015-2020 training set, 2021 validation set, and 2022 test set according to time, to ensure that the test set does not participate in model parameter optimization.

[0032] (2) Grid resampling: a standard grid of 0.1°x0.1° (longitude 73°-135°, a total of 621 columns; latitude 18°-54°, a total of 361 rows) is constructed, and the LU, weather data, and DEM data are resampled to the standard grid using the bilinear interpolation method to be consistent with the output resolution; the ground station, satellite, emission inventory, and POP data remain in the original format, and are converted in coordinates after inputting the convolutional neural process module.

[0033] (3) Standardization processing: a differentiated standardization strategy is adopted according to the data distribution characteristics: ① for the meteorological data (U10, V10, T2M, RH) conforming to the normal distribution, Z-score standardization is adopted (Z = (X - μ) / σ), based on the mean (μ) and standard deviation (σ) of the training set. ​​) calculation; 2) for the emission inventory and population density data (MEIC, POP) conforming to the lognormal distribution, first logarithmic transformation (log(x+1)) and then Z-score standardization; 3) for satellite AOD and VCDs data, direct maximum-minimum standardization (x-min) / (max-min) to map the values to the interval [0, 1] and eliminate the dimensional difference.

[0034] (4) Time encoding: sinusoidal-cosine encoding is used to represent continuous time (t), and the day (day of the year) and hour (0-23) are converted into periodic feature vectors. At the same time, the one-hot encoding of the day of the week (1-7) and the normalized encoding of the year are generated as time feature inputs to the model.

[0035] (5) Label processing: high-resolution pollution reanalysis data from the Institute of Atmospheric Physics, Chinese Academy of Sciences is used as the label, and the pollutant concentration label is 64-bit one-hot encoded for probabilistic prediction training.

[0036] S2, construct an assimilation network using a combination of convolutional neural processes, ViT, and ResNet, input the preprocessed multi-source data into the assimilation network, perform coordinate conversion through the convolutional neural process, capture spatial dependence through ViT, and adjust the dimension through ResNet, train the assimilation network and generate the initial assimilation state at the current time.

[0037] As shown in Figure 1 , the assimilation network uses a combination of convolutional neural processes, ViT, and ResNet, and the core function is to convert multi-source irregular data into regular grid initial assimilation state. The convolutional neural process module solves the problem of data heterogeneity, realizes coordinate conversion and sparsity coding; the ViT module captures the spatial dependence relationship of horizontal scale through the self-attention mechanism, especially suitable for long-distance pollution transmission process; the ResNet module adjusts the feature dimension to generate the initial state that meets the prediction requirements. The assimilation process uses high-resolution pollution reanalysis data as the label, uses the cross-entropy loss function to optimize the model parameters, and ensures the accuracy of the initial assimilation state.

[0038] Specifically, the preprocessed multi-source observation data and auxiliary data at the current time (t=0) are input into the assimilation network, where the meteorological data in the auxiliary data uses the data at the current time and the previous period (e.g. 24 hours ago), and the assimilation network predicts the initial assimilation state at the current time, while using the cross-entropy loss function to train the assimilation network: ​​​​ wherein, is a convolutional neural process module, which realizes the coordinate conversion of the irregular observation grid to the regular grid and encodes the data sparsity through the density channel; is a ViT, which captures the spatial dependence relationship through the attention mechanism; is a ResNet, which adjusts the dimension to generate the probability output; is the initial assimilation state output by the assimilation network, is the high-resolution pollution reanalysis data label, represents the cross-entropy loss, represents the loss function.

[0039] In the embodiment, the convolutional neural process module solves the irregularity and heterogeneity of multi-source data and converts different formats of observation data into uniform regular grid features. The parameter configuration is as follows: ① the output grid is set to be the standard grid constructed in S1; ② for ground station observation data (irregular point data), the scatter data to grid data mode is adopted, 6 pollutant channels (PM2.5, PM10, SO2, NO2, CO, O3) are set, and 6 density channels (marking the coverage area of the station, 1 for data and 0 for no data) are added; ③ for satellite, emission inventory data and POP, etc. Regular grid data containing missing values, adopt grid data to grid data mode.

[0040] In the embodiment, the Vision Transformer (ViT) (spatial feature layer) captures the long-distance spatial dependence relationship of the pollutants (such as the cross-regional transmission of PM2.5), and the parameter configuration is as follows: ① the input feature map size is 361x621 (latitude x longitude); ② the patch division method of 10x10 is adopted to divide the feature map into 2331 patches (each patch contains 100 pixels); ③ 3 layers of encoders are set, each layer contains 1 Multi-Head Attention (MHA) module (attention head number 8) and 1 feedforward neural network, and the LayerNorm layer is normalized; ④ the output dimension is 64, and the patch features are reconstructed into a 361x621x64 grid feature map through the reshape operation, and the spatial position information is retained.

[0041] In the embodiment, the ResNet (prediction output layer) adjusts the feature dimension and outputs the pollutant concentration prediction probability density distribution, and the parameter configuration is as follows: ① contains 1 residual block composed of 2 3x3 convolution layers; ② the input channel number is 64 (ViT output channel), and the output channel number is 6x64 (corresponding to 64 one-hot encoded probability predictions of 6 types of pollutants), which realizes the probability prediction.

[0042] After the above network configuration, the assimilation network completes the training on 2 NVIDIA A100 40GB GPUs in about 3 days.

[0043] S3, a prediction network with the same architecture as the assimilation network is constructed, taking the initial assimilation state at the current time as input, and combining multi-source data at future time to perform time extrapolation. Independent prediction networks are constructed for different future times and trained separately, and the probability prediction results of pollutant concentration at different future times are output.

[0044] In the embodiment, the prediction network is used to predict multi-time pollutant concentration, wherein the prediction network architecture is consistent with the assimilation network architecture, and the initial assimilation state output by the assimilation network is used as the core input, and multi-source data in the corresponding period is combined to perform time extrapolation. Through the non-cyclic architecture design, the error accumulation problem of traditional RNN is avoided; at the same time, according to the prediction time, the meteorological data window (the assimilation network uses the previous 24 hours data, and the prediction network uses 0 to hour data) is adjusted to ensure the accuracy of prediction at different times. The prediction process also uses the cross-entropy loss function, and the reanalysis data is used as the label to optimize the model, and the pollutant concentration prediction results from 1 to 72 hours are output.

[0045] Specifically, for each future time ( , the value can be 1-72 hours). The initial assimilation state at the current time (t=0), the preprocessed multi-source observation data at the current time and auxiliary data are used as the input of the prediction network, wherein the meteorological data in the auxiliary data uses data from the current time to the future time, and the prediction network predicts the pollutant concentration prediction probability density distribution at the future time . During training, the cross-entropy loss function is also used to train the prediction network, and the prediction network is obtained: Among them, , and respectively represent the convolutional neural process, ViT and ResNet in the prediction network to process different input data; is the initial assimilation state at the current time output by the assimilation network, is the pollutant concentration prediction probability density distribution at the future time output by the prediction network, is the high-resolution pollutant reanalysis data label corresponding to the time .

[0046] The embodiment also provides a deep learning prediction system for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data, comprising a preprocessing module, an assimilation module, and a prediction module. The preprocessing module is configured to preprocess collected multi-source data, wherein the multi-source data comprises multi-source observation data and multi-source auxiliary data; the assimilation module is configured to construct an assimilation network adopting a combined architecture of a convolutional neural process, a ViT, and a ResNet, input the preprocessed multi-source data into the assimilation network, perform coordinate conversion through the convolutional neural process module, capture spatial dependence through the ViT, and perform dimension adjustment through the ResNet, train the assimilation network, and generate an initial assimilation state at a current time; and the prediction module is configured to construct a prediction network having the same architecture as the assimilation network, input the initial assimilation state at the current time as an input, combine multi-source data at a future time for time extrapolation, construct independent prediction networks for different future times respectively and perform separate training, and output pollutant concentration probability prediction results at different future times.

[0047] It should be noted that the deep learning prediction system for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data provided in the above embodiment is used for atmospheric pollutant prediction, and the division of the above functional modules is used for illustration. The above functions can be completed by different functional modules according to needs, that is, the internal structure of a terminal or a server is divided into different functional modules to complete all or part of the above-described functions. In addition, the deep learning prediction system for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data provided in the above embodiment and the deep learning prediction method for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data belong to the same concept, and the specific implementation process is described in detail in the deep learning prediction method for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data, which will not be described here.

[0048] The trained prediction network supports fast generation of prediction results on a single-GPU computer, and the specific deployment process is as follows: (1) Automatic data access process, real-time acquisition of multi-source data through API interface: ① ground station observation data: obtain the latest hourly concentration data from the environmental monitoring station website every hour; ② satellite remote sensing data: obtain Himawari-8 AOD and TROPOMI VCDs data from the University of Science and Technology of China every 30 minutes; ③ meteorological forecast data: obtain U10, V10, T2M, and RH data for the next 72 hours from the CAMS website every hour; ④ static geographic data (LU, DEM, POP) and inventory data: locally stored, no real-time update is required. All data are automatically preprocessed (standardized, interpolated, feature encoded, etc.) through Python scripts to trigger the prediction process.

[0049] (2) Achieving 0-72 hours China near-surface pollutant prediction. The trained prediction network can be deployed and run on a single NVIDIA GeForce RTX 4090 16GB GPU, and the prediction time of a single time step is less than 1 second, supporting uninterrupted operation for 24 hours a day.

[0050] In the embodiment, the trained prediction network is also evaluated for performance. Specifically, the hourly observation data of 2016 national control sites of the Environmental Monitoring General Station from January to December 2022 is selected, and compared with the system 72-hour prediction results, the correlation coefficient (R), mean absolute error (MAE), and root mean square error (RMSE) three core indicators are calculated, and the results are shown in Figure 2 The comparison of USTC-AQM prediction and CAMS system prediction and national control site observation values in different components and delivery cycles. USTC-AQM 0h prediction (lead time = 0) uses the output of the assimilation network. USTC-AQM generates hourly predictions, while CAMS provides global pollution predictions initialized at 00:00 UTC and 12:00 UTC, with PM2.5 and PM10 surface products available, and multi-level data available for NO2, SO2, CO and O3. The bottom layer concentration of NO2, SO2, CO and O3 in the CAMS prediction is converted to a unified unit using T2M and DEM data of ERA5 reanalysis data. The comparison and evaluation show that the daily average indicators are slightly better than the hourly results. Among all evaluation indicators, USTC-AQM prediction shows a clear advantage over CAMS prediction. USTC-AQM assimilation results show that the consistency with national control site observation results is significantly better than prediction, which is due to the direct inclusion of concurrent national control site data input. USTC-AQM achieves an average correlation coefficient of 0.824, 0.723, 0.712, 0.690 and 0.679 for delivery cycles of 0 hours, 12 hours, 24 hours, 48 hours and 72 hours, respectively. The relatively poor performance of SO2 and CO is mainly due to their inherent low concentration levels. It is worth noting that China's strict emission reduction policy requires high-polluting industries (such as thermal power, steel, and cement) to achieve ultra-low emissions, which has driven the nationwide SO2 concentration to continuously decrease from 26.04 in 2015 to 8.69 .

[0051] As Figure 3As shown, analysis of 2022 monitoring data (including over 600,000 samples) from national monitoring stations reveals that USTC-AQM achieves an accuracy of 80.4% and a recall of 62.3% when predicting light or more severe pollution (AQI>100) with a 12-hour lead time, outperforming CAMS's 32.2% and 51.8%, respectively. Even within a 72-hour lead time, the prediction network maintains reasonable performance, with an accuracy of 66.8% and a recall of 49.6%, superior to CAMS's 26.8% and 49.6%, respectively. Notably, the prediction network demonstrates particular effectiveness in predicting moderate and severe pollution events, despite their lower baseline rates. For 72-hour predictions, the accuracy for these higher threshold events remains high, at 59.6% and 45.5%, respectively, outperforming CAMS's 30.5% and 19.7%, respectively. This performance demonstrates the operational value of the system in advanced pollution early warning systems.

[0052] Based on the above prediction results, the examples demonstrate two typical prediction cases: (a) PM2.5 pollution events are frequent in winter and often move over a wide area with airflow. Due to the global sensing mechanism of the model, USTC-AQM can capture the long-distance transport of this pollutant. Figure 4 This section showcases a typical pollution transport event that occurred in February 2022. Pollutants were generated in Shaanxi and Henan provinces between 20:00 on the 23rd and 8:00 on the 24th, and were transported northeastward by southwesterly winds, reaching the Beijing-Tianjin-Hebei region, and then crossing the Bohai Sea to the Liaodong Peninsula, with a transport distance exceeding 1000 km.

[0053] (b) Cyclone activity on the Mongolian Plateau is typically most intense from March to May, generating strong winds that lift dust from the Gobi and deserts, which is then transported south or southeast into China. USTC-AQM is capable of predicting this dust transport process. Figure 5 The study presents a typical case of dust transport in April 2022. Accompanied by a strong southeasterly wind, the dust began to be transported into China at 20:00 on the 10th and ended at 11:00 on the 10th. The model predicted this dust transport event using data available at 20:00 on the 9th.

[0054] The application does not rely on chemical transmission model (CTM), captures spatial dependence through multi-source data fusion and ViT architecture, and provides more reliable data for air quality assessment. Seamless prediction of 0.1°*0.1° high spatial resolution and 72 hours long time is realized, which can capture the fine pollution distribution of urban scale and meet the demand of medium and long-term pollution prevention and control; at the same time, the non-circulation architecture design avoids the error accumulation of the circulation model, and the 72-hour prediction accuracy still maintains a high level. When deployed in business, a single GPU can support real-time prediction, and the generation time of a single time step is less than 1 second, which greatly shortens the prediction lag time compared with the CAMS system (8-12 hours delay), and meets the timeliness demand of emergency response. The probability prediction output of the 10th-90th percentile uncertainty interval helps users to quantify the prediction risk, and provides more comprehensive information support for decision-making in different scenarios (such as public travel and emergency measures).

[0055] The above specific embodiments have described the technical solutions and beneficial effects of the application in detail. It should be understood that the above description is only the most preferred embodiment of the application and is not intended to limit the application. Any modification, supplement and equivalent replacement within the principle range of the application shall be included in the protection scope of the application.

Claims

1. A deep learning prediction method for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data, characterized in that, The method comprises the following steps: Preprocessing the collected multi-source data, wherein the multi-source data comprises multi-source observation data and multi-source auxiliary data, and the multi-source observation data comprises ground observation data and satellite remote sensing data; An assimilation network adopting a combined architecture of a convolutional neural process, a ViT and a ResNet is constructed, the preprocessed multi-source data is input into the assimilation network, coordinate conversion is performed through a convolutional neural process module, spatial dependence capture is performed through the ViT, and dimension adjustment is performed through the ResNet, the assimilation network is trained, and an initial assimilation state at the current time is generated; A prediction network with the same architecture as the assimilation network is constructed, the initial assimilation state at the current time is taken as input, time extrapolation is performed in combination with multi-source data at future times, independent prediction networks at different future times are constructed and separately trained, and deterministic prediction values of pollutant concentrations at different future times and uncertainty intervals of the 10th-90th percentiles are output.

2. The method of claim 1, wherein the method is characterized by, The multi-source auxiliary data comprises meteorological data, emission inventory data and static geographic data. The preprocessing of the multi-source data comprises data standardization, time coding, grid resampling and label one-hot coding. 3.The method of claim 1, wherein, The training of the assimilation network and the generation of the initial assimilation state at the current time comprise: The preprocessed multi-source observation data at the current time and auxiliary data is input into a assimilation network, wherein the meteorological data in the auxiliary data uses data at the current time and a previous period of time, the assimilation network predicts an initial assimilation state at the current time, and a cross-entropy loss function is used to train the assimilation network: wherein, is a convolutional neural process module that implements the coordinate conversion from the irregular observation grid to the regular grid and encodes the data sparsity through the density channel; is a ViT that captures spatial dependencies through an attention mechanism; is a ResNet that performs dimension adjustment to generate a probability output; is the initial assimilation state output by the assimilation network, is the high-resolution pollution reanalysis data label, denotes the cross-entropy loss, denotes the loss function.

4. The method of claim 1, wherein the method is characterized by, The independent prediction networks at different future times are constructed and separately trained, and the probabilistic prediction results of the pollutant concentrations at different future times are output, comprising: For each future time, the initial assimilation state at the current time, the pre-processed multi-source observation data at the current time and auxiliary data as input of the prediction network, where the meteorological data in the auxiliary data adopt data from the current time to the future time, the prediction network predicts the pollutant concentration prediction probability density distribution at the future time, and during training, the prediction network is also trained using the cross-entropy loss function to obtain the prediction network: wherein, , and represent the convolutional neural process, ViT and ResNet processing different input data in the prediction network, respectively; is the initial assimilation state at the current time for the assimilation network output, is the future time pollutant concentration prediction probability density distribution output by the prediction network, is the high-resolution pollution reanalysis data label corresponding to time.

5. The method of claim 2, wherein the method is characterized by, The convolutional neural process module in the assimilation network and the prediction network realizes conversion between non-grid observation data and regular grid observation data in coordinate conversion, specifically comprising: Parameter initialization: determining initial trainable parameters, data conversion modes, grid steps and whether to enable a density channel, wherein the data conversion modes comprise a scattered point data to gridded data mode, a gridded data to gridded data mode and a gridded data to scattered point data mode; Distance and weight calculation: based on the spatial coordinates of the input data and the target data, the square distance between them is first calculated, and then the weight value reflecting the correlation degree between the data is generated through a Gaussian kernel function in combination with trainable parameters, the weight value is negatively correlated with the spatial distance of the data, and higher weight is ensured for spatially close data; Data conversion processing: if it is the scattered point data to gridded data mode, the valid data in the input non-grid data is first screened through a mask to remove invalid value interference, then the non-grid data is weighted and converted into regular grid data representation in combination with the calculated weight, and the data normalized through the density channel is output; If it is the gridded data to gridded data mode, the conversion between different regular grid data is realized based on the weight calculation, and the data normalized through the density channel is output; If it is the gridded data to scattered point data mode, the valid output area in the target non-grid data is first screened, then the regular grid data is converted into non-grid prediction data according to the weight, and the data normalized through the density channel is output; In the above process, a density channel is first added to mark the area where the data exists or not, and provide an effectiveness identifier for subsequent data processing, and the identifier is input into the ViT together.

6. The method of claim 5, wherein the method is characterized by, For ground station observation data, scatter data to gridded data mode is adopted; for satellite remote sensing data, meteorological data, emission inventory data, and static geographic data with missing values, gridded data to gridded data mode is adopted.

7. The method of claim 1, wherein the method is characterized by, Also includes: Performance evaluation of the prediction network, specifically based on national control site observation data to calculate correlation coefficient, mean absolute error, root mean square error, and pollution event identification accuracy, to complete the model performance evaluation.

8. A deep learning prediction system for near-surface atmospheric pollutants based on hyperspectral satellite remote sensing and ground observation data, characterized in that, Includes: A preprocessing module for preprocessing the collected multi-source data, wherein the multi-source data includes multi-source observation data and multi-source auxiliary data, and the multi-source observation data includes ground observation data and satellite remote sensing data; An assimilation module for building an assimilation network using a combination of convolutional neural process modules, ViT, and ResNet architecture, inputting the preprocessed multi-source data into the assimilation network, performing coordinate conversion through the convolutional neural process module, capturing spatial dependence through ViT, and adjusting the dimension through ResNet, training the assimilation network and generating the initial assimilation state at the current time; A prediction module for building a prediction network with the same architecture as the assimilation network, taking the initial assimilation state at the current time as input, combining future multi-source data for time extrapolation, building independent prediction networks for different future times and conducting separate training, and outputting pollution concentration probability prediction results for different future times.

Images