Method for inverting high-resolution sand and dust layer height by stationary satellite based on machine learning
By combining stationary satellites, reanalysis and onboard lidar data, machine learning algorithms and multivariate linear regression models are used to solve the inversion problem of high-temporal resolution dust layer height, achieving high-precision and widely applicable dust layer height inversion, suitable for sand and dust weather and perennial dust areas.
Patent Information
- Application Number
- CN202510437439.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-08-05
AI Technical Summary
The prior art is difficult to consistently invert the height of sand and dust layer with high temporal resolution in a larger space, and lacks verification and constraints for ground-based lidar network observations, resulting in greater uncertainty in the assessment of sand and dust climate environmental effects.
Combining stationary satellites, reanalyzing data and satellite-borne lidar data, machine learning algorithms are used to train models, use multiple machine learning models and multiple linear regression models, and combine ground-based radar observations for verification, inversion of the height of high-spatial-time resolution dust layer.
It improves the inversion accuracy and spatial and temporal resolution of the dust layer height, can finely track sand and dust weather processes, analyze the three-dimensional spatial and temporal evolution characteristics of sand and dust transmission, and is widely applicable.
Smart Images
Figure CN120428253A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning. Background Art
[0002] Dust is one of the most abundant aerosol types in the global aerosol load and has significant climatic, environmental, and health effects. Dust can directly or indirectly alter the radiative balance, thereby changing atmospheric and surface heating, further influencing weather and climate systems, which in turn influences cloud formation and precipitation. The vertical distribution characteristics of dust aerosols significantly influence the vertical profile of atmospheric radiative heat, thereby altering the stability of turbulent motion and cloud lifetime, making them a key factor in assessing the radiative effects of aerosols. However, dust aerosols exhibit high spatiotemporal variability, particularly in dust source regions and during long-range transport. The lack of information on their vertical distribution leads to significant uncertainty in accurately assessing their climatic and environmental effects. Therefore, obtaining information on the vertical distribution of dust aerosols is crucial for accurately quantifying their impact on the climate and environment and for a detailed understanding of their role in regulating the radiative balance, climate change, and air quality. Dust layer height (DLH) is a key parameter for characterizing the vertical distribution of dust under different atmospheric conditions. Information on dust layer height can not only be used to identify and track the long-range transport and dynamic evolution trends of dust, but is also crucial for understanding the life cycle of dust aerosols and monitoring air quality to predict severe pollution events.
[0003] In recent years, researchers both domestically and internationally have conducted extensive inversion work on the aerosol layer height and dust layer height. Inversion algorithms based on geostationary satellite observations are based on a radiation transfer model, resulting in complex computational processes and low efficiency. Furthermore, they lack the validation and constraints of ground-based lidar network observations. Therefore, consistently inverting more accurate DLHs at high temporal resolution over a larger area is essential for a comprehensive understanding of the continuous dynamic evolution of dust. Summary of the Invention
[0004] The present invention aims to provide a method for inverting high-resolution dust layer height from a geostationary satellite based on machine learning, which overcomes the above-mentioned problems or at least partially solves the above-mentioned problems.
[0005] To achieve the above object, the technical solution of the present invention is specifically implemented as follows:
[0006] The present invention provides a method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning, comprising:
[0007] Obtain geostationary satellite raw data, reanalysis data and spaceborne lidar raw data, and perform preprocessing;
[0008] Calculate the effective dust layer height using data from spaceborne lidar observations;
[0009] Get DEM terrain height data;
[0010] The geostationary satellite raw data, the reanalysis data and the DEM terrain height data are used as inputs of a machine learning model, and the effective dust layer height data is used as the target output of the machine learning model to train the machine learning model;
[0011] Using the machine learning model to predict the effective dust layer height at a preset time interval;
[0012] Collaborative verification using ground-based lidar observation network.
[0013] Optionally, calculating the effective dust layer height using data observed by a space-borne laser radar includes:
[0014] For a given atmospheric column, remove the profiles with clouds;
[0015] If the characteristic classification flag indicates dust or polluted dust and it is a continuous layer, dust is determined to be present;
[0016] Determine the vertical extent of the dust layer based on the top and bottom heights of a single or multiple dust layers;
[0017] All dust layer heights within a pixel are weighted by the AOT of the corresponding layer to obtain the effective dust layer height at a given location. The formula is as follows:
[0018]
[0019] where z(i) and AOT(i) are the average altitude and AOT of the i-th dust layer, respectively, N is the total number of dust layers, AOT(total) is the sum of the AOTs in N individual dust layers, and z dust is the effective dust layer height.
[0020] Optionally, training the machine learning model includes:
[0021] The machine learning model is trained using a stacked model training method.
[0022] Optionally, the adopting the stacked model training method to train the machine learning model includes:
[0023] Identify multiple machine learning models as base learners for training, and develop a meta-learner based on the output of the base learners.
[0024] Optionally, the multiple machine learning models include:
[0025] RF, XGBoost, LightGBM, GBDT and KNN.
[0026] Optionally, determining multiple machine learning models as basic learners for training includes:
[0027] Each base model is trained on the training set using Bayesian parameter tuning and 10-fold crossover.
[0028] The training set is divided into 10 copies, the model is trained on 9 copies, and prediction is performed on the remaining 1 copy. This is repeated 10 times until the prediction of the entire training set is obtained, and the first prediction result is obtained;
[0029] The predictions for the test set are generated simultaneously in 10 training iterations, and all predictions are averaged to obtain the predictions for the test set to obtain the second prediction result.
[0030] Optionally, developing a meta-learner based on the output of the base learner includes:
[0031] A meta-model is trained on a new training set consisting of the first prediction results and the second prediction results of different base models.
[0032] Optionally, the meta-model is a multiple linear regression model.
[0033] Optionally, the collaborative verification using ground-based lidar observation network includes: performing backscattering weighting on the effective dust layer height at the preset time interval to obtain the dust layer height.
[0034] Optionally, performing backscatter weighting on the effective dust layer height at the preset time interval to obtain the dust layer height includes:
[0035] The weighted backscatter height is estimated by the following formula:
[0036]
[0037] Among them, β i (z i ) represents the aerosol backscattering coefficient at the 532 nm channel at the height i (Mm -1 sr -1 ), z i is the height of the vertical profile signal of the aerosol layer i, DLH bsc is the layer height calculated from the backscatter profile.
[0038] It can be seen that the method for inverting high-resolution dust layer height from geostationary satellites based on machine learning provided by the present invention combines geostationary satellite observation data, reanalysis data, terrain height data with polar-orbiting satellite CALIPSO data, and utilizes the characteristics of fast calculation speed and high inversion accuracy of machine learning algorithms. Through a variety of machine learning models and multivariate linear regression models, the inversion results are further improved. Finally, the results are verified by ground-based radar observation data, and the high-temporal and spatial resolution of the dust layer height is inverted, the dust weather process is tracked in detail, and the three-dimensional spatiotemporal evolution characteristics of dust transmission are analyzed, thereby improving the spatiotemporal resolution and inversion accuracy, and having a wide range of applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0040] Figure 1 A flowchart of a method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning provided in an embodiment of the present invention;
[0041] Figure 2 A flowchart of a specific example of a method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0043] Figure 1 The flowchart of the method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning provided by an embodiment of the present invention is shown. Figure 2 A flowchart showing a specific example of a method for retrieving high-resolution dust layer height from a geostationary satellite based on machine learning provided by an embodiment of the present invention is shown. Figure 1 and Figure 2 The embodiment of the present invention provides a method for inverting high-resolution dust layer height using a geostationary satellite based on machine learning, including:
[0044] S1, obtains geostationary satellite raw data, reanalysis data and spaceborne lidar raw data, and performs preprocessing.
[0045] Specifically, the present invention can pre-process geostationary satellite raw data, reanalysis data from ERA5 or merrA2, and CALIPSO spaceborne lidar raw data. The satellites can be a combination of multiple satellites, and the present invention does not impose any specific restrictions on the satellites.
[0046] S2, calculate the effective dust layer height using data from spaceborne lidar observations.
[0047] Specifically, the present invention can utilize the secondary data VFM and ALAY products observed by the CALIPSO spaceborne lidar. For the CALIOP L2 product, each pixel may have several layers of aerosol. First, the influence of clouds is eliminated, and then the aerosol pixel feature classification mark information is used to identify dust and polluted dust layers. CALIOP dust height information is extracted by weighting the top and bottom heights of the dust / polluted dust layer by the corresponding layer's aerosol optical thickness (AOT). The purpose is to develop and verify the dust height inversion algorithm, which is defined as the effective dust layer height (DLH).
[0048] As an optional implementation of the embodiment of the present invention, calculating the effective dust layer height using data observed by a space-borne laser radar includes:
[0049] For a given atmospheric column, remove the profiles with clouds;
[0050] If the characteristic classification flag indicates dust or polluted dust and it is a continuous layer, dust is determined to be present;
[0051] Determine the vertical extent of the dust layer based on the top and bottom heights of a single or multiple dust layers;
[0052] All dust layer heights within a pixel are weighted by the AOT of the corresponding layer to obtain the effective dust layer height at a given location. The formula is as follows:
[0053]
[0054] where z(i) and AOT(i) are the average altitude and AOT of the i-th dust layer, respectively, N is the total number of dust layers, AOT(total) is the sum of the AOTs in N individual dust layers, and z dust is the effective dust layer height.
[0055] Specifically, the present invention calculates the effective dust layer height as follows: For a given atmospheric column, cloud profiles are removed using the secondary product VFM. If the CALIPSO feature classification flag indicates dust or contaminated dust and is a continuous layer, dust is determined to be present. The vertical extent of the dust layer is then determined based on the top and bottom heights of the single or multiple dust layers derived from CALIPSO. All dust layer heights within a CALIPSO pixel are weighted by the AOT of the corresponding layer to obtain the effective dust layer height at a given location using the following formula:
[0056]
[0057] Where z(i) and AOT(i) are the average altitude and AOT of the i-th dust layer, respectively, and N is the total number of dust layers in the CALIPSO product. Therefore, in formula (1), AOT(total) is the sum of the AOTs in N individual dust layers. Due to the weighted average, z dust It can be regarded as the “effective dust layer height” and for simplicity, it is collectively referred to as the “dust layer height (DLH)” in this study.
[0058] S3, obtain DEM terrain height data.
[0059] Specifically, the ETOPO global terrain model, developed by the National Geophysical Data Center of the National Oceanic and Atmospheric Administration (NOAA), integrates topography, bathymetry, and coastline data from regional and global datasets to achieve a comprehensive, high-resolution rendering of the geophysical features of the Earth's surface. The DEM terrain data uses a Digital Elevation Model (DEM), specifically selected from the ETOPO 2022 global dataset (60 arc-second resolution).
[0060] S4, uses the geostationary satellite raw data, reanalysis data and DEM terrain height data as the input of the machine learning model, and uses the effective dust layer height data as the target output of the machine learning model to train the machine learning model.
[0061] As an optional implementation of an embodiment of the present invention, training a machine learning model includes: training the machine learning model using a stacked model training method. Training the machine learning model using the stacked model training method includes: determining multiple machine learning models as base learners for training, and developing a meta-learner based on outputs of the base learners.
[0062] The multiple machine learning models include RF, XGBoost, LightGBM, GBDT, and KNN. The meta-model is a multiple linear regression model. Of course, the machine learning model of the present invention can also adopt other machine learning models, all of which should fall within the scope of protection of the present invention.
[0063] Identifying multiple machine learning models as base learners for training includes: training each base model on a training set using Bayesian parameter tuning and a 10-fold crossover method; dividing the training set into 10 equal copies, training the model on 9 copies, and making predictions on the remaining copy, repeating 10 times until a prediction for the entire training set is obtained, obtaining a first prediction result; simultaneously generating a prediction for the test set during the 10 training iterations, averaging all predictions to obtain a prediction for the test set, and obtaining a second prediction result. Developing a meta-learner based on the output of the base learner includes: training the meta-model on a new training set consisting of the first prediction results and the second prediction results of different base models.
[0064] In specific implementation, the present invention performs temporal and spatial matching on four data sets: effective dust layer height data, multi-channel brightness temperature observed by geostationary satellites, reanalysis data (specific humidity, temperature, ozone mass mixing ratio), and DEM terrain height data. Brightness temperature, reanalysis data, and terrain height data are used as inputs of the machine learning model, and effective dust layer height data is used as the target output of the machine learning model, and the data are divided into a training set and a validation set.
[0065] The stacking model training method of the present invention is as follows: Stacking, also known as stacking generalization, is an integrated learning strategy. In the stacking strategy, several machine learning models are trained as base learners, and a meta-learner is developed based on the output of the base learner. First, each base model (RF, XGBoost, LightGBM, GBDT, KNN) is trained on the training set using Bayesian parameter adjustment and 10-fold crossover method. The training set is divided into 10 copies on average, the model is trained on 9 copies, and predictions are made on the remaining 1 copy. This process is repeated 10 times until a prediction for the entire training set is obtained (denoted as prediction 1). Secondly, predictions for the test set are generated simultaneously in 10 training iterations. All predictions are averaged to obtain a prediction for the test set (denoted as prediction 2). Finally, a meta-model (multiple linear regression) is trained on a new training set consisting of predictions from different basic models. The inversion results of the five models are further optimized using a multiple linear regression (MLR) model to improve the inversion accuracy. This study combined the prediction results of five models, including RF, XGBoost, LightGBM, GBDT, and KNN, into new training data for the MLR model. Similarly, the predictions of the basic models on the test set were combined into new test data for the MLR model.
[0066] S5, using a machine learning model to predict the effective dust layer height at a preset time interval.
[0067] Specifically, the present invention spatially matches the hourly multi-channel brightness temperature data and reanalysis data observed by geostationary satellites with DEM terrain data, and inputs them into the optimized model to obtain the effective dust layer height on an hourly or even minute level.
[0068] S6, collaborative verification using ground-based lidar observation network.
[0069] Specifically, the present invention utilizes collaborative verification using a ground-based LiDAR observation network. The inverted effective dust layer height is further verified against ground-based observation data from the ground-based LiDAR observation network. The ground-based observation data has a vertical spatial resolution of 3.75 meters and a temporal resolution of 5 minutes. The ground-based LiDAR observation data can be preprocessed, including but not limited to background subtraction, range correction, polarization calibration, and overlap correction.
[0070] As an optional implementation of an embodiment of the present invention, collaborative verification using a ground-based lidar observation network includes: performing backscattering weighting on the effective dust layer height at a preset time interval to obtain the dust layer height.
[0071] As an optional implementation manner of the embodiment of the present invention, performing backscatter weighting on the effective dust layer height at a preset time interval to obtain the dust layer height includes:
[0072] The weighted backscatter height is estimated by the following formula:
[0073]
[0074] Among them, β i (z i ) represents the aerosol backscattering coefficient at the 532 nm channel at the height i (Mm -1 sr -1 ), z i is the height of the vertical profile signal of the aerosol layer i, DLH bsc is the layer height calculated from the backscatter profile.
[0075] Specifically, the present invention adopts the backscatter weighted height (DLH bsc ), the weighted backscatter height is estimated by the following formula:
[0076]
[0077] In formula (2), β i (z i) represents the aerosol backscattering coefficient at the 532 nm channel at the height i (Mm -1 sr -1 ), z i is the height of the vertical profile signal of the aerosol layer i (km). Based on the above equation, the layer height is calculated according to the backscatter profile, and the symbol is DLH bsc .
[0078] Of course, the present invention may also adopt other verification methods, which should all fall within the protection scope of the present invention.
[0079] It can be seen that the high-resolution dust layer height inversion method based on machine learning provided by the embodiment of the present invention adopts a multi-model fusion machine learning architecture to perform output fusion of the multivariate linear regression model, utilizes multi-source data collaborative processing technology, utilizes the spatiotemporal matching rules of CALIPSO and geostationary satellite observation brightness temperature and reanalysis data, combines the features of multi-channel brightness temperature data, reanalysis data and DEM terrain height data, and adopts collaborative verification of ground-based lidar observation networking, so that the spatiotemporal resolution of dust layer height inversion is high and the spatial coverage is wide, while the accuracy and reliability of dust layer height inversion are improved. The present invention has wide applicability and is not only suitable for sandstorm weather, but also for perennial sandstorm areas. The effective dust layer height can better reflect the true vertical distribution.
[0080] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.
Claims
1. A method for retrieving high-resolution dust layer height from a geostationary satellite based on machine learning, characterized in that: include: Obtain geostationary satellite raw data, reanalysis data and spaceborne lidar raw data, and perform preprocessing; Calculate the effective dust layer height using data from spaceborne lidar observations; Get DEM terrain height data; The geostationary satellite raw data, the reanalysis data, and the DEM terrain height data are used as inputs of a machine learning model, and the effective dust layer height data is used as the target output of the machine learning model to train the machine learning model; and the machine learning model is used to predict the effective dust layer height at a preset time interval. Collaborative verification using ground-based lidar observation network.
2. The method according to claim 1, characterized in that The calculation of the effective dust layer height using the data observed by the spaceborne lidar includes: For a given atmospheric column, remove the profiles with clouds; If the characteristic classification flag indicates dust or polluted dust and it is a continuous layer, dust is determined to be present; Determine the vertical extent of the dust layer based on the top and bottom heights of a single or multiple dust layers; All dust layer heights within a pixel are weighted by the AOT of the corresponding layer to obtain the effective dust layer height at a given location. The formula is as follows: where z(i) and AOT(i) are the average altitude and AOT of the i-th dust layer, respectively, N is the total number of dust layers, AOT(total) is the sum of the AOTs in N individual dust layers, and z dust is the effective dust layer height.
3. The method according to claim 1, characterized in that The training of the machine learning model includes: The machine learning model is trained using a stacked model training method.
4. The method according to claim 3, characterized in that The stacked model training method is used to train the machine learning model, including: Identify multiple machine learning models as base learners for training, and develop a meta-learner based on the output of the base learners.
5. The method according to claim 4, characterized in that The multiple machine learning models include: RF, XGBoost, LightGBM, GBDT and KNN.
6. The method according to claim 5, characterized in that Determining multiple machine learning models as basic learners for training includes: Each base model is trained on the training set using Bayesian parameter tuning and 10-fold crossover. The training set is divided into 10 copies, the model is trained on 9 copies, and prediction is performed on the remaining 1 copy. This is repeated 10 times until the prediction of the entire training set is obtained, and the first prediction result is obtained; The predictions for the test set are generated simultaneously in 10 training iterations, and all predictions are averaged to obtain the predictions for the test set to obtain the second prediction result.
7. The method according to claim 6, characterized in that The developing of a meta-learner based on the output of the base learner includes: A meta-model is trained on a new training set consisting of the first prediction results and the second prediction results of different base models.
8. The method according to claim 7, characterized in that The meta-model is a multiple linear regression model.
9. The method according to claim 8, characterized in that The collaborative verification using the ground-based lidar observation network includes: performing backscattering weighting on the effective dust layer height at the preset time interval to obtain the dust layer height.
10. The method according to claim 9, characterized in that The step of performing backscattering weighting on the effective dust layer height at the preset time interval to obtain the dust layer height comprises: The weighted backscatter height is estimated by the following formula: Among them, β i (z i ) represents the aerosol backscattering coefficient at the 532 nm channel at the height i (Mm -1 sr -1 ), z i is the height of the vertical profile signal of the aerosol layer i, DLH bsc is the layer height calculated from the backscatter profile.