A method and system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data

Abstract

The present invention discloses a method and system for inverting high-resolution, three-dimensional, full-coverage aerosol spatial distribution data, relating to the field of atmospheric environment. The method comprises: collecting multi-source data and performing quality control and preprocessing; using the multi-source data as independent variables and the orbital layered aerosol extinction coefficients of cloud-aerosol lidar and infrared pathfinder satellites as dependent variables, and inputting these into the extreme gradient boosting model (XGBoost); generating prediction results, combining them with the original independent variables to form an enhanced feature set, which is then input into the lightweight gradient boosting machine learning model (LightGBM) to generate a process prediction set; and using a wavelet transform method to fuse the process prediction set with a global atmospheric reanalysis dataset, extracting the advantageous information, and obtaining an accurate prediction dataset of high-resolution, three-dimensional, full-coverage aerosol extinction coefficients. The present invention effectively improves the reliability, adaptability, and scalability of the inversion method, and the resulting dataset can comprehensively, accurately, and detailedly characterize the three-dimensional spatial distribution of aerosols.

Description

Technical Field

[0001] The present invention relates to the field of atmospheric environment technology, and more particularly to a method and system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data. Background Art

[0002] Aerosols not only reflect atmospheric pollution but also significantly influence Earth's weather and climate systems through direct and indirect effects. Accurately characterizing the spatial distribution and dynamics of aerosols, especially their vertical distribution, plays a key role in assessing atmospheric environmental evolution, radiative balance, and addressing climate change. However, current observational methods and related data products are unable to provide high-resolution, three-dimensional, and comprehensive spatial distribution information on aerosols. Ground-based radars can only continuously observe the vertical distribution of aerosols at a specific location and cannot provide information on a wide range of vertical distributions. Aircraft-borne radars can only observe the vertical distribution of aerosols in a specific area at a specific time period and cannot provide long-term, large-scale vertical distribution information. Traditional satellite-borne sensors can only observe aerosol or pollutant concentrations throughout the atmosphere or near the surface and cannot provide vertical stratification information. Satellite-borne lidars, while capable of continuously observing aerosol vertical distribution over a wide area, only cover the orbital strip over which the satellite passes, and cannot provide seamless, comprehensive vertical distribution information. Existing data products, whether remote sensing inversion or reanalysis data, mostly provide only surface or whole-layer aerosol concentrations, but lack vertical stratification information. While some products can provide full coverage of aerosol vertical stratification, their horizontal spatial resolution is extremely low (less than 55 km × 55 km), limiting their application prospects. Therefore, there is currently a lack of data that can comprehensively, accurately, and detailedly reflect the vertical and horizontal distribution of aerosols.

[0003] In recent years, a growing number of studies have used machine learning methods to generate high-quality remote sensing data products. Machine learning algorithms demonstrate significant advantages in processing large-scale, complex data and integrating data from multiple sources. These methods are not constrained by the temporal and spatial limitations of instruments, nor do they rely on the specific source and collection methods of the data. By using multi-source data to provide diverse perspectives, they can enhance the credibility of results and improve prediction accuracy. Consequently, many studies have attempted to use machine learning methods to invert the spatial distribution of atmospheric pollutants. However, the comprehensive, accurate, and detailed characterization of the three-dimensional spatial distribution of aerosols remains a pressing technical challenge for those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a method and system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, which solves the problems existing in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data includes the following steps:

[0007] Collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and construct preprocessed data sets;

[0008] The multi-source data in the preprocessed dataset are used as independent variables, and the orbital layered aerosol extinction coefficient of the cloud-aerosol lidar and infrared pathfinder satellite is used as the dependent variable. These are input into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient.

[0009] The XGBoost prediction results are combined with the original independent variables to form an enhanced feature set and input into the lightweight gradient boosting machine learning model LightGBM to generate a process prediction set of high-resolution three-dimensional full-coverage aerosol extinction coefficient;

[0010] The wavelet transform method is used to fuse the process prediction set with the global atmospheric reanalysis dataset, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

[0011] Optional multi-source data include: cloud-aerosol lidar and infrared pathfinder satellite data, China's multi-scale emission inventory model products, global climate and weather reanalysis datasets, China's high-resolution and high-quality near-surface air pollutant datasets, digital elevation model data, land use type data, and global atmospheric reanalysis datasets.

[0012] Optional quality control and pre-processing operations include the following steps:

[0013] Perform quality checks on multi-source data to remove missing values, outliers, and invalid data;

[0014] The bilinear interpolation method is used to unify the spatial resolution of the data, unifying the resolution of different data sets to the required high-resolution grid to ensure the spatial consistency of the data;

[0015] Time alignment is performed on data with inconsistent time scales to ensure that all data are compared and analyzed within the same time step;

[0016] The time-aligned data are standardized and normalized to construct a preprocessed dataset with consistent data dimensions.

[0017] Optional, independent variables include: PM2.5 from China Multi-Scale Emission Inventory Model2.5 , sulfur dioxide, nitrogen oxides and ammonia emissions, 10-meter wind speed, 2-meter air temperature, surface temperature and pressure, and precipitation from the Global Climate and Weather Reanalysis dataset, and PM2.5 from China's high-resolution and high-quality near-surface air pollutant dataset. 2.5 concentration, land use type data, and digital elevation model data.

[0018] Alternatively, the extreme gradient boosting model XGBoost uses a structure that integrates multiple decision trees, each of which is gradually optimized using a gradient boosting algorithm to minimize the loss function.

[0019] Alternatively, the lightweight gradient boosting machine learning model LightGBM uses a decision tree-based gradient boosting framework to fit complex data patterns by building multiple decision trees, and uses gradient clipping, sample bucketing, cross-validation, and early stopping strategies for model training.

[0020] Optionally, obtaining an accurate prediction dataset includes the following steps:

[0021] The process prediction dataset and the global atmospheric reanalysis dataset are decomposed by wavelet to obtain the sub-band coefficients of different frequency bands;

[0022] We perform weighted fusion on each frequency band and utilize the localization characteristics of wavelet transform to effectively fuse the spatial distribution characteristics of process prediction dataset and global atmospheric reanalysis dataset.

[0023] Through inverse wavelet transform, a high-resolution three-dimensional full-coverage aerosol extinction coefficient accurate prediction data set was reconstructed.

[0024] A system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, applying a method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data as described in any one of the above items, comprising:

[0025] Data acquisition and preprocessing module, used to collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and build preprocessed data sets;

[0026] The prediction module is used to input the multi-source data in the preprocessed dataset as independent variables and the orbital layered aerosol extinction coefficient of the cloud-aerosol lidar and infrared pathfinder satellite as dependent variables into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient;

[0027] The first generation module is used to combine the XGBoost prediction results with the original independent variables to form an enhanced feature set and input it into the lightweight gradient boosting machine learning model LightGBM to generate a process prediction set of high-resolution three-dimensional full-coverage aerosol extinction coefficient;

[0028] The second generation module is used to fuse the process prediction set with the global atmospheric reanalysis dataset through the wavelet transform method, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

[0029] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses a method and system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, which has the following beneficial effects: combining the two machine learning models of XGBoost and LightGBM can effectively improve the reliability, suitability and scalability of the inversion method. This XGBoost-LightGBM-Wavelet (XLW) method can be used to accurately and reliably produce high-resolution three-dimensional full-coverage aerosol extinction coefficient spatial distribution, so as to comprehensively, accurately and in detail characterize the three-dimensional spatial distribution of aerosols, providing a methodological basis for the subsequent batch production of related data products. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0031] Figure 1 A flow chart of a method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data provided by the present invention;

[0032] Figure 2 A flowchart for implementing the technical solution provided by the present invention;

[0033] Figure 3 A structural diagram of a system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data provided by the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] The embodiment of the present invention discloses a method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, such as Figure 1 、 Figure 2 As shown, the following steps are included:

[0036] Collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and construct preprocessed data sets;

[0037] The multi-source data in the preprocessed dataset are used as independent variables and the orbital layered aerosol extinction coefficient of the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite (CALIPSO) is used as the dependent variable. These are input into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient.

[0038] The XGBoost prediction results are combined with the original independent variables to form an enhanced feature set and input into the lightweight gradient boosting machine learning model LightGBM to generate a process prediction set of high-resolution three-dimensional full-coverage aerosol extinction coefficient;

[0039] The wavelet transform method is used to fuse the process prediction set with the global atmospheric reanalysis dataset, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

[0040] Furthermore, multi-source data include: Cloud-Aerosol Lidar and Infrared Pathfinder Satellite (CALIPSO) data, China Multiscale Emission Inventory Model (MEIC) products, Global Climate and Weather Reanalysis Dataset (ERA5), China High-Resolution and High-Quality Near-Surface Air Pollutant Dataset (CHAP), Digital Elevation Model data (DEM), Land Use Type Dataset (CLCD) and Global Atmospheric Reanalysis Dataset (MERRA2).

[0041] Furthermore, the quality control and pre-processing operations specifically include the following steps:

[0042] Perform quality checks on multi-source data to remove missing values, outliers, and invalid data;

[0043] The bilinear interpolation method is used to unify the spatial resolution of the data, unifying the resolution of different data sets to the required high-resolution grid to ensure the spatial consistency of the data;

[0044] Time alignment is performed on data with inconsistent time scales to ensure that all data are compared and analyzed within the same time step;

[0045] The time-aligned data are standardized and normalized to construct a preprocessed data set with consistent data dimensions for subsequent fusion and analysis.

[0046] Furthermore, the independent variables include: PM 2.5 , sulfur dioxide, nitrogen oxides and ammonia emissions, 10-meter wind speed, 2-meter air temperature, surface temperature and pressure, and precipitation from the Global Climate and Weather Reanalysis Dataset (ERA5), and PM2.5 from the China High-Resolution and High-Quality Near-Surface Air Pollutants Dataset (CHAP). 2.5 Concentration, land use type data (CLCD) and digital elevation model data (DEM) and other multi-source data.

[0047] Furthermore, the extreme gradient boosting model XGBoost adopts a structure that integrates multiple decision trees, and each decision tree is gradually optimized by the gradient boosting algorithm to minimize the loss function. In the specific design of this embodiment, aerosol-related multidimensional input variables (such as temperature, humidity, wind speed, aerosol extinction coefficient, etc.) are combined as features, and the nonlinear fitting ability of the model is used to make efficient predictions. Improvements are made on the basis of existing technologies, and model hyperparameters are optimized through feature selection and cross-validation to improve prediction accuracy and generalization ability. Compared with traditional methods, the XGBoost model can better capture the complex nonlinear relationship of aerosol distribution, especially in high-resolution data processing, showing higher accuracy. Before using the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient, the model is pre-trained with a historical data set to ensure its stability and accuracy, which has obvious advantages over traditional methods in prediction effect and computational efficiency.

[0048] Furthermore, the lightweight gradient boosting machine learning model LightGBM adopts a decision tree-based gradient boosting framework. It fits complex data patterns by constructing multiple decision trees, and uses strategies such as gradient clipping, sample bucketing, cross-validation, and early stopping for model training. Unlike traditional gradient boosting methods, LightGBM improves training speed and memory efficiency through histogram optimization and data splitting. In terms of model design, LightGBM adopts a leaf-first tree growth method, which can better capture complex feature relationships and improve the prediction effect of high-resolution aerosol spatial distribution. The model uses gradient clipping, sample bucketing and other strategies during training to effectively avoid overfitting and improve prediction accuracy. Compared with other machine learning methods, especially training on large-scale datasets, LightGBM has demonstrated significant computational efficiency advantages. Hyperparameter tuning is performed during model training, and cross-validation and early stopping strategies are used to ensure the efficiency and generalization ability of the model.

[0049] Furthermore, obtaining the accurate prediction data set specifically includes the following steps:

[0050] The process prediction dataset and the global atmospheric reanalysis dataset are decomposed by wavelet to obtain the sub-band coefficients of different frequency bands;

[0051] We perform weighted fusion on each frequency band and utilize the localization characteristics of wavelet transform to effectively fuse the spatial distribution characteristics of process prediction dataset and global atmospheric reanalysis dataset.

[0052] Through inverse wavelet transform, a high-resolution three-dimensional full-coverage aerosol extinction coefficient accurate prediction data set was reconstructed.

[0053] and Figure 1 Corresponding to the method described above, the embodiment of the present invention also provides a system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data for Figure 1 The specific implementation of the method in the embodiment of the present invention provides a system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, which can be applied to computer terminals or various mobile devices, such as Figure 3 As shown, specifically including:

[0054] Data acquisition and preprocessing module, used to collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and build preprocessed data sets;

[0055] The prediction module is used to input the multi-source data in the preprocessed dataset as independent variables and the orbital layered aerosol extinction coefficient of the cloud-aerosol lidar and infrared pathfinder satellite as dependent variables into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient;

[0056] The first generation module is used to combine the XGBoost prediction results with the original independent variables to form an enhanced feature set and input it into the lightweight gradient boosting machine learning model LightGBM to generate a process prediction set of high-resolution three-dimensional full-coverage aerosol extinction coefficient;

[0057] The second generation module is used to fuse the process prediction set with the global atmospheric reanalysis dataset through the wavelet transform method, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

[0058] In summary, this example combines the XGBoost and LightGBM machine learning models to effectively improve the reliability, adaptability, and scalability of the inversion method. This XGBoost-LightGBM-Wavelet (XLW) method produces accurate and effective predictions, with a correlation of up to 0.94 with ground-based observations. It can comprehensively, accurately, and detailedly characterize the three-dimensional spatial distribution of aerosols, providing a methodological foundation for the subsequent mass production of related data products.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0060] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, characterized in that: The following steps are involved: Collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and construct preprocessed data sets; The multi-source data in the preprocessed dataset are used as independent variables, and the orbital layered aerosol extinction coefficient of the cloud-aerosol lidar and infrared pathfinder satellite is used as the dependent variable. These are input into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient. The XGBoost prediction results are combined with the independent variables input into the extreme gradient boosting model XGBoost to form an enhanced feature set and input into the lightweight gradient boosting machine learning model LightGBM to generate a high-resolution three-dimensional full-coverage aerosol extinction coefficient process prediction set; The wavelet transform method is used to fuse the process prediction set with the global atmospheric reanalysis dataset, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

2. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: Multi-source data include: cloud-aerosol lidar and infrared pathfinder satellite data, China's multi-scale emission inventory model products, global climate and weather reanalysis datasets, China's high-resolution and high-quality near-surface air pollutant datasets, digital elevation model data, land use type data and global atmospheric reanalysis datasets.

3. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: The quality control and pre-processing operations include the following steps: Perform quality checks on multi-source data to remove missing values, outliers, and invalid data; The bilinear interpolation method is used to unify the spatial resolution of the data, unifying the resolution of different data sets to the required high-resolution grid to ensure the spatial consistency of the data; Time alignment is performed on data with inconsistent time scales to ensure that all data are compared and analyzed within the same time step; The time-aligned data are standardized and normalized to construct a preprocessed dataset with consistent data dimensions.

4. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: The independent variables include: PM2.5 in China's Multi-Scale Emission Inventory Model 2.5 , sulfur dioxide, nitrogen oxides and ammonia emissions, 10-meter wind speed, 2-meter air temperature, surface temperature and pressure, and precipitation from the Global Climate and Weather Reanalysis dataset, and PM2.5 from China's high-resolution and high-quality near-surface air pollutant dataset. 2.5 concentration, land use type data, and digital elevation model data.

5. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: The extreme gradient boosting model XGBoost adopts a structure that integrates multiple decision trees. Each decision tree is gradually optimized through the gradient boosting algorithm to minimize the loss function.

6. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: The lightweight gradient boosting machine learning model LightGBM adopts a gradient boosting framework based on decision trees. It fits complex data patterns by building multiple decision trees and uses gradient clipping, sample bucketing, cross-validation, and early stopping strategies for model training.

7. The method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to claim 1, characterized in that: The acquisition of accurate prediction data sets includes the following steps: The process prediction set and the global atmospheric reanalysis dataset are subjected to wavelet decomposition to obtain sub-band coefficients of different frequency bands; Weighted fusion is performed on each frequency band, and the localization characteristics of wavelet transform are used to effectively fuse the spatial distribution characteristics of the process prediction set and the global atmospheric reanalysis dataset; Through inverse wavelet transform, a high-resolution three-dimensional full-coverage aerosol extinction coefficient accurate prediction data set was reconstructed.

8. A system for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data, using the method for inverting high-resolution three-dimensional full-coverage aerosol spatial distribution data according to any one of claims 1 to 7, characterized in that: include: Data acquisition and preprocessing module, used to collect multi-source data and perform quality control and preprocessing, unify the horizontal and vertical resolutions of different types of data, and construct preprocessed data sets; The prediction module is used to input the multi-source data in the preprocessed dataset as independent variables and the orbital layered aerosol extinction coefficient of the cloud-aerosol lidar and infrared pathfinder satellite as dependent variables into the extreme gradient boosting model XGBoost to generate the XGBoost prediction results of the aerosol extinction coefficient; The first generation module is used to combine the XGBoost prediction results with the independent variables input into the extreme gradient boosting model XGBoost to form an enhanced feature set and input it into the lightweight gradient boosting machine learning model LightGBM to generate a process prediction set of high-resolution three-dimensional full-coverage aerosol extinction coefficient; The second generation module is used to fuse the process prediction set with the global atmospheric reanalysis dataset through the wavelet transform method, extract the advantageous information of the two datasets, and obtain an accurate prediction dataset of high-resolution three-dimensional full-coverage aerosol extinction coefficient.

Images