Geostationary satellite remote sensing AOD image data missing filling method and system

Abstract

The invention relates to the field of satellite remote sensing image data processing and modeling, in particular to a geostationary satellite remote sensing AOD image data missing filling method and a geostationary satellite remote sensing AOD image data missing filling system. According to the method, a time interpolation technology is adopted to process reanalysis data, and data of a plurality of geostationary satellites such as Himawari-8 / 9, GOCI-II, GEMS and FY-4B are fused, so that the data of the geostationary satellites are obtained. According to the method, the generation of the full-coverage AOD product per hour day and night is realized, the space-time coverage and resolution of the AOD product are remarkably improved, and a finer data basis is provided for atmospheric pollution monitoring and climate research. By constructing the fusion model based on the error weight and the sample size weight, the precision of fusion data is remarkably improved, and the problem that inversion of a single satellite product is inaccurate under complex conditions is solved.

Description

Technical Field

[0001] The present invention relates to the field of satellite remote sensing image data processing and modeling, and in particular to a method and system for filling missing geostationary satellite remote sensing AOD image data. Background Art

[0002] In recent years, with the rapid development of remote sensing technology, satellite-derived aerosol optical depth (AOD) data has become an important means of studying regional and global aerosol distribution. Multiple satellites (such as MODIS, Himawari, FY series, etc.) provide high-resolution, long-time series AOD products through different inversion algorithms, which provide important data support for the study of atmospheric environmental pollution, climate change and related fields. However, there are still some technical problems and limitations in the practical application of satellite AOD data, mainly including the lack of satellite AOD data, which is usually caused by the following factors: Cloud cover: Aerosol inversion relies on surface observations, but clouds block surface signals, making it impossible to obtain observation data. For example, the MODIS satellite has a significantly increased data loss rate in areas with large cloud cover or seasonal cloudy conditions.

[0003] Interference from surface characteristics: Certain surface types (e.g., areas with high reflectivity such as deserts and snow) can interfere with the satellite's aerosol inversion algorithm, resulting in less data or complete lack of data in these areas.

[0004] Observation angle and time: The observation angle and observation time of the satellite will directly affect the coverage and quality of AOD data. For example, when the solar altitude angle is too low, the inversion result may be unreliable or unusable.

[0005] Algorithm limitations: Existing AOD inversion algorithms (such as the Deep Blue algorithm and the Dark Target algorithm) have limited aerosol inversion accuracy under complex conditions (such as mixed aerosol types and multi-layer aerosols), which may lead to missing data or increased errors.

[0006] Hardware problems or transmission interruptions: Satellite observation equipment may fail or data transmission may be interrupted, resulting in missing data records.

[0007] The lack of satellite AOD data will have a significant impact on scientific research and practical applications. Specifically: Insufficient temporal and spatial coverage: Missing data will lead to incomplete temporal and spatial distribution of AOD, affecting the accurate description of regional aerosol distribution characteristics and changing trends.

[0008] Increased uncertainty in results: During aerosol modeling and validation, missing data may lead to insufficient model input information, increasing the uncertainty of the results.

[0009] Difficulty in fusing multi-source data: When data from different satellites or sensors contain inconsistent or missing information, data fusion and comparison will face additional technical challenges.

[0010] At present, there are many solutions to the problem of missing satellite AOD data, including: Interpolation method: Use spatial interpolation (such as Kriging interpolation) or temporal interpolation (such as time series prediction model) to fill in the missing data. This method relies on existing data points, and its accuracy is greatly affected by the spatial and temporal distribution of observation data. Especially in areas where data is sparse or seriously missing, interpolation may introduce large errors. Multi-source data assimilation: Combine ground-based observation data (such as AERONET) and other satellite data to supplement the missing satellite AOD data through data assimilation technology. Although it can integrate information from different observation sources, the complexity of the assimilation algorithm makes it difficult to guarantee the reliability of the fusion results. Simulation method: Use chemical transport models (such as GEOS-Chem, WRF-Chem) to simulate and supplement the missing aerosol data, but the computational cost is high, and it is difficult to quickly generate reliable AOD data in large-scale high-resolution applications. Summary of the invention

[0011] The object of the present invention is to provide a method and system for filling missing geostationary satellite remote sensing AOD image data, so as to solve the problem of large-scale missing satellite remote sensing AOD data and low temporal resolution and lack of night data in the prior art.

[0012] In order to achieve the above object, the technical solution specifically adopted by the present invention is as follows: A method for filling missing data of geostationary satellite remote sensing AOD image comprises the following steps: S1. Data collection: Collect multi-source satellite remote sensing AOD data, including Himawari-8 / 9, GOCI-II, GEMS, FY-4B geostationary satellite AOD data, reanalysis AOD data (such as MERRA2 and GEOS-FP reanalysis AOD products) and other auxiliary data, to provide a multi-temporal and multi-type data foundation for subsequent fusion; S2. Data preprocessing: The collected satellite remote sensing AOD data are uniformly preprocessed, including format conversion, quality screening and spatial resampling, to ensure the consistency of data format and the reliability of quality, laying the foundation for subsequent analysis; S3. Interpolation of reanalysis AOD data: Temporal interpolation of reanalysis AOD data such as MERRA2 and GEOS-FP reanalysis AOD data is performed to generate fused reanalysis AOD data with high temporal and spatial resolution to compensate for the problem of insufficient satellite data coverage; S4. Accuracy verification: Use the AOD observation data of the ground station AERONET to verify the accuracy of the AOD data of four geostationary satellites (FY-4B, GOCI-Ⅱ, GEMS, Himawari-9), calculate the error index of the satellite data, and provide the error weight basis for model training; S5, fusion model construction: random forest is selected as the fusion model. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained separately to explore the best data fusion strategy to ensure the best generalization ability and accuracy of the model. S6. Model evaluation and selection: The input data are applied to the four fusion models in turn to generate AOD data with full coverage during the day and night. Subsequently, the prediction results of each combined model are compared with the AERONET data of the ground station. The performance of the model is verified by indicators such as RMSE and R², and the model with the best overall performance is selected as the final model.

[0013] S7. Result output: The full-coverage AOD data for day and night predicted by the optimal model is output in a gridded form, providing high-resolution and high-coverage data support for aerosol research and related scientific applications.

[0014] Furthermore, the step S5 specifically includes the following steps: Feature selection: reanalysis AOD data, satellite data and other auxiliary data are uniformly interpolated to the spatial resolution consistent with the AOD data of each geostationary satellite as input features; Data division: Geostationary satellite AOD data, reanalysis AOD data and auxiliary data are divided into training set and validation set in a ratio of 7:3; Training weight calculation: Calculate the error weight according to the error index in step S4, and calculate the sample size weight of each satellite data every day, and fuse them as the training weight of the model; the calculation formulas for the error weight, sample size weight and model training weight are: ; ; ; Among them, e is the error between the AOD data of each geostationary satellite and the AOD data of the ground-based AERONET monitoring station, N is the sample size of the AOD data of each geostationary satellite input to the model, ω is the final calculated comprehensive weight of each geostationary satellite input to the model training, i represents the index of a single satellite, j is used to traverse all satellites, and k represents the total number of all satellites involved in the calculation.

[0015] Model training: Different random forest models are constructed based on different combinations of data from four geostationary satellites. The model is trained using the training set data through machine learning methods, and the model parameters are optimized based on the validation set data to obtain the best data fusion strategy.

[0016] Furthermore, in step S6, after the training set data is input into the random forest model, it is compared with the ground station AERONET data, and the model training results are evaluated by RMSE, R², Bias and correlation coefficient (R), and the model accuracy is dynamically adjusted using the error weight. The calculation formula is as follows: ; ; ; ; ; ; Among them, ErrorWeight represents the error weight of each geostationary satellite. The sample size weight is not introduced when calculating each evaluation index. true,q represents the target variable of the qth input model, y pred,q represents the qth model prediction value, represents the mean value of the target variable input into the model, It represents the average value of the model prediction, q represents the number of samples used to traverse all the calculations of the model evaluation indicators, and n represents the total number of samples involved in the calculation.

[0017] Furthermore, the day and night full coverage AOD data predicted by the optimal model of the present invention has a time resolution of 1 hour and a spatial resolution of 0.025°×0.025°.

[0018] The present invention also provides a geostationary satellite remote sensing AOD image data missing filling system, which adopts the above-mentioned geostationary satellite remote sensing AOD image data missing filling method to achieve geostationary satellite remote sensing AOD image data missing filling, including: Data acquisition module, used to collect multi-source satellite remote sensing AOD data, including geostationary satellite AOD data, reanalysis AOD data and other auxiliary data; Data processing module, used for preprocessing of geostationary satellite AOD data, reanalysis AOD data and other auxiliary data and interpolation processing of reanalysis AOD data; among which, preprocessing includes format conversion, quality screening and spatial resampling; In the model training module, random forest is selected as the fusion model. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained to explore the best data fusion strategy. The data output module is used to output high-resolution AOD products with full coverage every hour during the day and night.

[0019] The method for filling missing data of geostationary satellite remote sensing AOD images of the present invention generates full coverage AOD products every hour of day and night by fusing multi-source data, and has the following beneficial effects: 1. Significantly improved coverage The present invention integrates data from multiple geostationary satellites such as Himawari-8 / 9, GOCI-II, GEMS, and FY-4B, and combines them with reanalysis data (such as MERRA2 and GEOS-FP) to achieve the generation of full-coverage AOD products every hour of the day and night, solving the coverage blind spot problem of traditional satellite AOD data in space and time, and providing more comprehensive data support for continuous observation of the spatiotemporal distribution of aerosols.

[0020] 2. High temporal and spatial resolution The present invention uses time interpolation technology to process reanalysis data and effectively improves the temporal resolution (from daily scale to hourly scale) and spatial resolution of AOD data through high-frequency fusion of multi-source satellite data, providing a more sophisticated data basis for atmospheric pollution monitoring and climate research.

[0021] 3. Improve data accuracy The satellite AOD products were error verified with the help of AERONET data from ground stations. By constructing a fusion model based on error weights and sample size weights, the accuracy of the fused data was significantly improved, and the problem of inaccurate inversion of single satellite products under complex conditions was reduced.

[0022] 4. The fusion method is flexible and efficient By training and verifying the model for the combination of different geostationary satellite data, the present invention can flexibly select the optimal fusion method, give full play to the advantages of different satellite products, and further improve the reliability and applicability of fused data.

[0023] 5. All-time data available The present invention makes up for the lack of night-time data in traditional satellite AOD products by integrating multi-source satellite and reanalysis data, realizes the availability of data at all times of day and night, and provides the possibility for all-weather observation of aerosols.

[0024] 6. Wide applicability The output high-resolution gridded AOD data can be directly applied to multiple fields such as air pollution monitoring, climate change research, environmental management, and health assessment, while providing more reliable input data for aerosol-based numerical simulations.

[0025] In summary, the present invention significantly improves the spatiotemporal coverage, resolution and accuracy of AOD products through multi-source data fusion and accuracy optimization, providing high-quality data support for studying the spatiotemporal distribution of aerosols and their environmental effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is an overall structural block diagram of a method for filling missing data of a geostationary satellite remote sensing AOD image according to an embodiment of the present invention; Figure 2 This is the accuracy verification diagram of MERRA2, GEOS-FP, and fused GEOS-FP in the embodiment of the present invention; Figure 3 This is the hourly accuracy verification diagram of MERRA2 and the fused GEOS-FP in the embodiment of the present invention; Figure 4 It is a diagram for verifying the accuracy of each satellite AOD product and the ground station AERONET in the embodiment of the present invention; Figure 5 Graphs showing the training accuracy of four combined models in the embodiments of the present invention; Figure 6 This is a comparison chart of the accuracy of the four combined model results and the ground station AERONET in the embodiment of the present invention; Figure 7 This is a verification diagram of the accuracy of the optimal combination model (FY-4B, GEMS, GOCI-Ⅱ, Himawari-9) during the day, night, all day and ground stations in the embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0028] Example 1 In order to solve the problems of insufficient coverage, low temporal resolution (mostly daily), lack of nighttime data, etc. of satellite remote sensing AOD (aerosol optical depth) products and improve their application accuracy in scientific research, the present invention proposes a method for filling missing data of geostationary satellite remote sensing AOD images that can generate full coverage AOD data every hour during the day and night, including the following steps: S1: Data Collection Collect multi-source satellite remote sensing AOD data, including: AOD data from Himawari-8 / 9, GOCI-II, GEMS, and FY-4B geostationary satellites, with high temporal resolution and wide coverage; MERRA2 and GEOS-FP reanalysis AOD data, which have stability and high temporal and spatial resolution; Other auxiliary data, including surface reflectivity data, meteorological data (such as temperature, humidity, wind speed, etc.), and surface height data, provide additional input features for the fusion model.

[0029] S2: Data preprocessing Preprocess the collected multi-source satellite remote sensing AOD data to ensure data consistency and quality reliability. Specifically include: Format conversion: convert data from different sources into a format that is easy for MATLAB to operate; Quality screening: Eliminate low-quality data based on the quality flag (QA Flag) of satellite products; Spatial resampling: resample all data to a spatial resolution consistent with that of each satellite.

[0030] S3: Interpolation of reanalysis AOD data The MERRA2 and GEOS-FP reanalysis AOD data are temporally interpolated. Specifically, the preprocessed MERRA2 and GEOS-FP reanalysis AOD data are interpolated from the original 3-hour or 1-hour resolution to 1-hour resolution. Linear interpolation or high-order interpolation methods are used to ensure the smoothness and accuracy of the interpolated data.

[0031] S4: Accuracy verification of satellite remote sensing AOD data Using RMSE, R², Bias and correlation coefficient (R) as evaluation indicators, the multi-source satellite remote sensing AOD data after preprocessing were compared with the ground station AERONET observation data, and the error indicators of satellite products were calculated. The results were used for weight calculation of subsequent model training.

[0032] S5: Fusion model construction Random forest is selected as the fusion model, and a fusion model is established. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained separately to explore the best data fusion strategy to ensure the best generalization ability and accuracy of the model. Specifically: Feature selection: Geostationary satellite AOD data, reanalysis AOD data, and auxiliary data are unified to the same spatial resolution as input features; Data division: Taking days as units, the geostationary satellite AOD data, reanalysis AOD data and auxiliary data are divided into training set and validation set in a ratio of 7:3; Weight design: According to the error index calculation results in step S4, the error weight is generated. At the same time, in order to avoid the influence of sample size differences on the model results, the sample size weight is calculated according to the sample size of each satellite data input into the model every day. Finally, the error weight is combined with the sample size weight as the training weight of the model to optimize the model performance; the calculation formulas of the error weight, sample size weight and model training weight are respectively:

[0033] Among them, e is the error between the AOD data of each geostationary satellite and the AOD data of the ground-based AERONET monitoring station, N is the sample size of the AOD data of each geostationary satellite input into the model, and ω is the comprehensive weight of each geostationary satellite input into the model training. i represents the index of a single satellite, j is used to traverse all satellites, and k represents the total number of satellites involved in the calculation.

[0034] Model training: Different random forest models are constructed based on different combinations of data from four geostationary satellites. The model is trained using the training set data through machine learning methods, and the model parameters are optimized based on the validation set data to obtain the best data fusion strategy.

[0035] S6: Model prediction and accuracy evaluation The input data is applied to the four fusion models in turn to predict and generate AOD data with full coverage of day and night. Then, the prediction results of each combined model are compared with the ground station AERONET data. The performance of the model is verified by indicators such as RMSE and R², and the model with the best comprehensive performance is selected as the final model. Specifically, after the training set data is input into the random forest model, it is compared with the ground station AERONET data, and the model training results are evaluated by RMSE, R², Bias and correlation coefficient (R), and the error weight is used to dynamically adjust the model accuracy. The calculation formula is as follows:

[0036] Among them, ErrorWeight represents the error weight of each geostationary satellite. The sample size weight is not introduced when calculating each evaluation index. true,q represents the target variable of the qth input model, y pred,q represents the qth model prediction value, represents the mean value of the target variable input into the model, It represents the average value of the model prediction, q represents the number of samples used to traverse all the calculations of the model evaluation indicators, and n represents the total number of samples involved in the calculation.

[0037] S7: Result output Finally, an AOD product with full coverage every hour of day and night is generated and output in the form of a grid, in this embodiment.

[0038] Through the above implementation mode, the present invention can generate AOD products with high temporal and spatial resolution and global day and night coverage. The full-coverage AOD data for day and night predicted by the optimal model has a temporal resolution of 1 hour and a spatial resolution of 0.025°×0.025°, providing reliable data support for studying the temporal and spatial distribution of aerosols and their environmental effects.

[0039] Example 2 A system for filling missing data of a geostationary satellite remote sensing AOD image, which uses the method for filling missing data of a geostationary satellite remote sensing AOD image described in Example 1 to fill missing data of a geostationary satellite remote sensing AOD image, comprising: The data acquisition module is used to collect multi-source satellite remote sensing AOD data, including geostationary satellite AOD data, reanalysis AOD data and other auxiliary data; among them, geostationary satellite AOD data include Himawari-8 / 9, GOCI-II, GEMS, FY-4B geostationary satellite AOD data, reanalysis AOD data include MERRA2 and GEOS-FP reanalysis AOD product data, other auxiliary data include surface reflectivity data, meteorological data (such as temperature, humidity, wind speed, etc.) and surface height data.

[0040] The data processing module is used for the preprocessing of geostationary satellite AOD data, reanalysis AOD data and other auxiliary data and the interpolation processing of reanalysis AOD data; among them, Preprocessing includes format conversion, quality screening and spatial resampling to ensure the consistency of data format and the reliability of quality, laying the foundation for subsequent analysis; the interpolation processing of reanalysis AOD data is used to perform temporal interpolation processing on MERRA2 and GEOS-FP reanalysis AOD data to generate fused reanalysis AOD data with high temporal and spatial resolution to make up for the problem of insufficient satellite data coverage; In the model training module, random forest is selected as the fusion model. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained to explore the best data fusion strategy. Specifically, the geostationary satellite AOD data, reanalysis AOD data and auxiliary data are first unified to the same spatial resolution as input features. Then, the geostationary satellite AOD data, reanalysis AOD data and auxiliary data are divided into training set and validation set in a ratio of 7:3 on a daily basis. According to the error index calculation result in step S4, the error weight is generated, and the sample size weight is calculated according to the sample size of each satellite data input into the model every day. The error weight and the sample size weight are combined as the training weight of the model. Finally, different random forest models are constructed based on different combinations of data from four geostationary satellites. After the training set data is input into the random forest model through machine learning methods, it is compared with the ground station AERONET data. The model training results are evaluated through indicators such as RMSE, R², Bias and correlation coefficient (R), and the model accuracy is dynamically adjusted using the error weight. The model with the best overall performance is selected as the final model.

[0041] The data output module is used to output high-resolution AOD products with full coverage every hour during the day and night.

[0042] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for filling missing data of geostationary satellite remote sensing AOD image, characterized in that: The steps include: S1. Data collection: Collect multi-source satellite remote sensing AOD data, including geostationary satellite AOD data, reanalysis AOD data and other auxiliary data; S2. Data preprocessing: uniform preprocessing of the collected satellite remote sensing AOD data, including format conversion, quality screening and spatial resampling; S3. Interpolation processing of reanalysis AOD data: Temporal interpolation processing is performed on the reanalysis AOD data to generate fused reanalysis AOD data with high temporal and spatial resolution; S4, accuracy verification: Use the AOD observation data of the ground station AERONET to verify the accuracy of the AOD data of the four geostationary satellites, calculate the error index of the satellite data, and provide the error weight basis for model training; S5, fusion model construction: Random forest is selected as the fusion model. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained to explore the best data fusion strategy. S6. Model evaluation and selection: The input data are applied to the four fusion models in turn to generate AOD data with full coverage during the day and night. Subsequently, the prediction results of each combined model are compared with the AERONET data of the ground station. The performance of the model is verified by RMSE and R², and the model with the best overall performance is selected as the final model. S7. Result output: The full-coverage AOD data for day and night predicted by the optimal model is output in a gridded form.

2. A method for filling missing data of geostationary satellite remote sensing AOD image according to claim 1, characterized in that: The multi-source satellite remote sensing AOD data include Himawari-8 / 9, GOCI-II, GEMS, FY-4B geostationary satellite AOD data, reanalysis AOD data and other auxiliary data.

3. A method for filling missing data of geostationary satellite remote sensing AOD image as claimed in claim 1, characterized in that: The step S5 specifically includes the following steps: Feature selection: reanalysis AOD data, satellite data and other auxiliary data are uniformly interpolated to the spatial resolution consistent with the AOD data of each geostationary satellite as input features; Data division: Geostationary satellite AOD data, reanalysis AOD data and auxiliary data are divided into training set and validation set in a ratio of 7:3; Training weight calculation: Calculate the error weight according to the error index in step S4, and calculate the sample size weight of each satellite data every day, and combine them as the training weight of the model; Model training: Different random forest models are constructed based on different combinations of data from four geostationary satellites. The model is trained using the training set data through machine learning methods, and the model parameters are optimized based on the validation set data to obtain the best data fusion strategy.

4. A method for filling missing data of geostationary satellite remote sensing AOD image as claimed in claim 3, characterized in that: In step S5, the calculation formulas for the error weight, sample size weight and model training weight are respectively: ； ； ； Among them, e is the error between the AOD data of each geostationary satellite and the AOD data of the ground-based AERONET monitoring station, N is the sample size of the AOD data of each geostationary satellite input to the model, ω is the final calculated comprehensive weight of each geostationary satellite input to the model training, i represents the index of a single satellite, j is used to traverse all satellites, and k represents the total number of all satellites involved in the calculation.

5. A method for filling missing data of geostationary satellite remote sensing AOD image as claimed in claim 3, characterized in that: In step S6, after the training set data is input into the random forest model, it is compared with the ground station AERONET data, and the model training results are evaluated by RMSE, R², Bias and correlation coefficient (R), and the model accuracy is dynamically adjusted using the error weight. The calculation formula is as follows: ； ； ； ； ； ； Among them, ErrorWeight represents the error weight of each geostationary satellite. The sample size weight is not introduced when calculating each evaluation index. represents the target variable of the qth input model, represents the qth model prediction value, represents the mean value of the target variable input into the model, It represents the average value of the model prediction, q represents the number of samples used to traverse all the calculations of the model evaluation indicators, and n represents the total number of samples involved in the calculation.

6. A method for filling missing data of geostationary satellite remote sensing AOD image according to claim 1, characterized in that: The full-coverage AOD data for day and night predicted by the optimal model has a temporal resolution of 1 hour and a spatial resolution of 0.025°×0.025°.

7. A geostationary satellite remote sensing AOD image data missing filling system, characterized by: The method for filling missing data of geostationary satellite remote sensing AOD image as described in any one of claims 1 to 6 is used to fill missing data of geostationary satellite remote sensing AOD image.

8. A geostationary satellite remote sensing AOD image data missing filling system as claimed in claim 7, characterized in that: include: Data acquisition module, used to collect multi-source satellite remote sensing AOD data, including geostationary satellite AOD data, reanalysis AOD data and other auxiliary data; Data processing module, used for preprocessing of geostationary satellite AOD data, reanalysis AOD data and other auxiliary data and interpolation processing of reanalysis AOD data; In the model training module, random forest is selected as the fusion model. Based on different combinations of data from four geostationary satellites, multiple fusion models are trained to explore the best data fusion strategy. The data output module is used to output high-resolution AOD products with full coverage every hour during the day and night.

Images