A Multi-Source Satellite Data Fusion Method and System for Carbon Dioxide Column Concentration

Abstract

The present invention provides a method and system for fusing multi-source satellite data of carbon dioxide column concentration. Among them, the method includes: constructing a calibration model between multi-source satellites through the random forest algorithm represented in ensemble learning, using the parameters and observation data of the satellites themselves as the input of the model, taking the high-precision satellite observation data as the benchmark, unifying the data accuracy and spatial resolution of multi-source satellites, and finally using the method of unit weight to perform fusion to obtain a global XCO2 fusion dataset with a larger observation coverage range, higher spatio-temporal resolution and accuracy than single-satellite data. The solution proposed by the present invention can greatly improve the accuracy of satellite data; starting from the satellite data itself, the calibration process does not need to specifically consider the satellite observation location and observation time; the original unqualified dataset is optimized, greatly increasing the spatial range of data observation and the overall utilization efficiency of the data; the problem of different spatial resolutions of different satellites is solved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of atmospheric environment remote sensing, standard quantitative products, multi-source data fusion, and XCO2, and particularly relates to a multi-source satellite data fusion method and system for carbon dioxide column concentration. Background Art

[0002] Carbon dioxide (CO2) is one of the main greenhouse gases that have a significant impact on human life.

[0003] Currently, the main methods for obtaining atmospheric CO2 concentration are ground measurement method, satellite detection method, and model simulation method. Ground measurement has the highest accuracy and time continuous observation ability, and the main methods include gas sampling measurement and spectrometer measurement. The Total Carbon Column Observing Network (TCCON) has stations distributed in many places around the world. Using Fourier transform spectrometers to measure atmospheric CO2 concentration, it is currently the most widely used and highly recognized ground CO2 monitoring network. However, although such station networks have a wide observation coverage and long-term observational data accumulation, their station distributions are not uniform, making it difficult to well capture the global change characteristics of CO2.

[0004] The carbon dioxide column concentration (XCO2) is defined as the average molar mixing ratio of the total column of the atmosphere (excluding water vapor molecules) from the Earth's surface to the top of the atmosphere. Combining the absorption characteristics of CO2 in the shortwave infrared range, the penetration ability of solar radiation, and signal characteristics, researchers have developed various algorithms for XCO2 inversion for different satellite payloads, mainly divided into full physical inversion algorithms based on differential optical absorption spectroscopy (DOAS) and methods based on statistical estimation. The former representatives include: WFM-DOAS, IMAP-DOA, and DOAS-BESD, and the latter representative methods include: NIES, ACOS, UOL_FP, IAPCAS, FOCAL, and Remotec, etc. The basic principle of XCO2 inversion is to use a forward radiative transfer model to simulate the backscattered signal received by the satellite sensor in the O2 and CO2 absorption bands when the known parameter vector (parameter vector) and the unknown parameter vector (state vector) are known; then, methods such as optimal estimation or probability theory are used to adjust the state vector to minimize the error between the simulated observation value and the true satellite observation value; finally, the retrieved state vector is considered to be the true atmospheric state. The differences between the above algorithms lie in aspects such as aerosol and cloud scattering effect processing, inversion band, initial state vector, etc.

[0005] Satellite remote sensing technology features wide coverage, high stability, long time series, and high precision. Based on the development of the above algorithms and the characteristics of remote sensing data, the products produced can well make up for the sparse distribution of ground monitoring stations and provide a convenient and globally consistent format of CO2 spatial distribution data for research and analysis. In March 2002, the Scanning Imaging Absorption spectrometer for Atmospheric CartograpHY (SCIAMACHY) carried by the European ENVISAT satellite; in January 2009, the Thermal And Near-infrared Sensor for carbon Observation (TANSO) carried by the Japanese GOSAT satellite; in July 2014, the hyperspectral resolution grating spectrometer carried by the US OCO-2 satellite; in December 2016, the ultra-high spectral resolution atmospheric CO2 grating spectrometer carried by the Chinese TanSat satellite, etc. By detecting the infrared and near-infrared spectra of the atmosphere, the CO2 concentration in the atmosphere can be obtained, so as to analyze the spatio-temporal distribution characteristics of atmospheric CO2. However, at present, single satellite sensors are often limited by the defects of their respective software and hardware supports, and it is often difficult to obtain continuous global CO2 concentration distribution results with high spatio-temporal resolution.

[0006] Defects of the prior art

[0007] An important issue in multi-source data fusion is to deeply understand the differences between multi-source satellite data and comprehensively consider the impacts of data quality, spatio-temporal scales, and fusion algorithms on the presentation effect of the dataset in the final practical application.

[0008] There have been many research bases on the consistency analysis of multi-source CO2 satellite data. Liu Liangyun et al. relatively comprehensively summarized the current situation and progress of global carbon monitoring satellites. Noel et al. retrieved XCO2 from GOSAT and GOSAT-2 through the Focal algorithm, and concluded that GOSAT-2 was slightly better than GOSAT by comparing with TCCON. Mustafa compared the CO2 data observed by OCO-2 and GOSAT with the CO2 data of the CarbonTracker (CT) model, and found that the correlation coefficients between CT and GOSAT, and CT and OCO-2 were 0.93 and 0.89 respectively, and the root mean square errors were 2.61 ppm and 2.16 ppm respectively. Wang et al. used the product data of GOSAT and SCIAMACHY for data fusion, and the mutual correction between different satellite observation platforms would affect the data quality. Hong et al. verified the accuracy of GOSAT through two observation stations in East Asia by TCCON. Hwang et al. evaluated the XCO2 product of OCO-2 with portable measurement equipment. Taylor et al. evaluated the consistency of the NASA ACOS version of OCO-2 and OCO-3 by TCCON, and found that OCO-3 was slightly better than OCO-2. Zhang et al. used a high-precision surface model to fuse the data of the atmospheric chemistry transport model GEOS-Chem and the ground-based TCCON station data, and evaluated the data quality of GOSAT and OCO2 based on this fused data set, and found that the accuracy of OCO2 was better than GOSAT.

[0009] Research shows that due to the parameter differences of satellite payloads and the influence of the atmosphere in the observation conditions among different carbon satellites, there are huge differences in the distortion degree of the data itself and the effects between different data sources. The current mainstream method is to consider the mechanism process and use physical algorithms to correct the inversion results. This method is complex to operate, requires researchers to be familiar with the satellite physical inversion model, and has little effect on correcting the invalid observation data of the satellite. At the same time, the differences in correction methods will also lead to relatively large differences in the final results of the satellite. Another method is to select ground-based observations as a reference for correction. However, the number of TCCON sites is scarce, and the correction results are doubtful. No researcher has successfully corrected the invalid observation data of the satellite using TCCON, resulting in a waste of a large amount of satellite observation data. Summary of the Invention

[0010] To solve the above technical problems, the present invention proposes a technical solution for a multi-source satellite data fusion method for carbon dioxide column concentration to solve the above technical problems.

[0011] The first aspect of the present invention discloses a multi-source satellite data fusion method for carbon dioxide column concentration, and the method includes:

[0012] Step S1: Obtain long - time - series global multi - source carbon satellite data and ground - based observation data with predefined credibility;

[0013] Step S2: Perform spatio - temporal matching on the satellite data and the ground - based observation data; select labeled data according to the annual average RMSE before correction of the satellite data and the ground - based observation data after matching;

[0014] Step S3: Use the annual average RMSE before correction as weights to fuse the selected labeled data to construct a labeled data set; perform spatio - temporal matching on the satellite data to be corrected and the labeled data set to construct a satellite sample set to be corrected;

[0015] Step S4: Use the training set and feature data of the satellite sample set to be corrected to train a random forest model; use the trained random forest model to correct the satellite data to be corrected;

[0016] Step S5: Compare the annual average RMSE of the satellite data after data correction and the ground - based observation data with the annual average RMSE before correction to perform accuracy verification of the corrected data;

[0017] Step S6: Merge the satellite data after data correction and the labeled data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and the ground - based observation data in the merged data set; calculate the fusion weights according to the data source RMSE; fuse the satellite data in the merged data set according to the fusion weights.

[0018] According to the method of the first aspect of the present invention, in the step S2, the conditions for performing spatio - temporal matching on the satellite data and the ground - based observation data include:

[0019] In terms of spatial position, select the data in the satellite data that covers the ground - based observation sites according to the spatial resolution of the satellite; in terms of time scale, select the ground - based observation data within two hours before and after the satellite passes.

[0020] According to the method of the first aspect of the present invention, in the step S3, using the annual average RMSE before correction as weights to fuse the selected labeled data includes:

[0021] Use the annual average RMSE before correction of each type of satellite data as weights to perform weighted average on multiple types of satellite data.

[0022] According to the method of the first aspect of the present invention, in the step S5, performing accuracy verification of the corrected data includes:

[0023] Compare the annual average RMSE of the satellite data after data correction and the ground - based observation data with the annual average RMSE before correction. If the comparison result is less than the first preset value, discard the satellite data after data correction, re - perform feature selection, and perform data correction.

[0024] According to the method of the first aspect of the present invention, in step S5, the re - feature selection includes:

[0025] Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; repeat the above operation multiple times for each feature, calculate the average value of the multiple RMSEs as the measurement standard of the feature importance; delete the features whose measurement standard is less than the second preset value from the input of the random forest model; the features must be data other than the prior data and the posterior data;

[0026] Set a change threshold; record the RMSE of a certain satellite data in the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", and then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0027] Compare the annual average RMSE of the satellite data after the second data correction and the ground - based observation data with the annual average RMSE before correction. If the comparison result is less than the first preset value, then abandon the fusion of the satellite data.

[0028] According to the method of the first aspect of the present invention, in step S6, calculating the fusion weight according to the data source RMSE includes:

[0029] Divide the mean square error of unit weight by the data source RMSE of a certain satellite data and the ground - based observation data in a certain year to obtain the fusion weight corresponding to the satellite data in that year.

[0030] According to the method of the first aspect of the present invention, in step S6, fusing the satellite data in the merged dataset according to the fusion weight includes:

[0031] Use the fusion weight of each satellite data in the merged dataset as the corresponding weight to perform weighted averaging on multiple satellite data in the merged dataset.

[0032] The second aspect of the present invention discloses a multi - source satellite data fusion system for carbon dioxide column concentration, and the system includes:

[0033] A first processing module, configured to obtain long - time - series global multi - source carbon satellite data and ground - based observation data with predefined credibility;

[0034] A second processing module, configured to perform spatio - temporal matching on the satellite data and the ground - based observation data; select labeled data according to the annual average RMSE before correction of the satellite data and the ground - based observation data after matching;

[0035] The third processing module is configured to fuse the selected labeled data with the pre-calibration annual average RMSE as the weight to construct a labeled data set; perform spatio-temporal matching between the satellite data to be calibrated and the labeled data set to construct a satellite sample set to be calibrated;

[0036] The fourth processing module is configured to train a random forest model using the training set and feature data of the satellite sample set to be calibrated; use the trained random forest model to calibrate the satellite data to be calibrated;

[0037] The fifth processing module is configured to compare the annual average RMSE of the satellite data after data calibration and the ground-based observation data with the pre-calibration annual average RMSE to perform calibration data accuracy inspection;

[0038] The sixth processing module is configured to merge the satellite data after data calibration and the labeled data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and the ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; fuse the satellite data in the merged data set according to the fusion weight.

[0039] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in a multi-source satellite data fusion method for carbon dioxide column concentration according to any one of the first aspects of the present disclosure are implemented.

[0040] A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps in a multi-source satellite data fusion method for carbon dioxide column concentration according to any one of the first aspects of the present disclosure are implemented.

[0041] In summary, the solution proposed by the present invention can greatly improve the accuracy of satellite data; starting from the satellite data itself, the calibration process does not need to specifically consider the satellite observation location and observation time, and can be regarded as a post-processing optimization method for satellite inversion algorithms; based on a high-precision standard satellite data set, the original unqualified data set is optimized, greatly increasing the spatial range of data observation and the overall utilization efficiency of the data; solving the problem of different satellite spatial resolutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 Flow chart of a multi-source satellite data fusion method for carbon dioxide column concentration according to an embodiment of the present invention;

[0044] Figure 2 Structure diagram of a multi-source satellite data fusion system for carbon dioxide column concentration according to an embodiment of the present invention;

[0045] Figure 3 Structure diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0046] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] The first aspect of the present invention discloses a multi-source satellite data fusion method for carbon dioxide column concentration. Figure 1 Flow chart of a multi-source satellite data fusion method for carbon dioxide column concentration according to an embodiment of the present invention, as Figure 1 shown, the method includes:

[0048] Step S1, obtaining long-time series global multi-source carbon satellite data and ground-based observation data with predefined credibility;

[0049] Step S2, performing spatio-temporal matching on the satellite data and the ground-based observation data; selecting labeled data according to the annual average RMSE before calibration of the matched satellite data and ground-based observation data;

[0050] Step S3, using the annual average RMSE before calibration as a weight, fusing the selected labeled data to construct a labeled data set; performing spatio-temporal matching on the satellite data to be calibrated and the labeled data set to construct a satellite sample set to be calibrated;

[0051] Step S4, training a random forest model using the training set and feature data of the satellite sample set to be calibrated; performing data calibration on the satellite data to be calibrated using the trained random forest model;

[0052] Step S5, comparing the annual average RMSE of the satellite data after data calibration and the ground-based observation data with the annual average RMSE before calibration, and performing calibration data accuracy test;

[0053] Step S6: Merge the satellite data after data correction with the label data to obtain a merged dataset; calculate the data source RMSE of each type of satellite data in the merged dataset and the ground-based observation data; calculate the fusion weights based on the data source RMSE; and perform fusion on the satellite data in the merged dataset according to the fusion weights.

[0054] In step S1, obtain long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility.

[0055] Specifically, obtain long-term global multi-source carbon satellite data, such as GOSAT, GOSAT-2, OCO-2, and OCO-3, and highly credible ground-based observation data, such as TCCON.

[0056] In step S2, perform spatio-temporal matching on the satellite data and the ground-based observation data; select label data according to the pre-correction annual mean RMSE of the matched satellite data and ground-based observation data.

[0057] In some embodiments, in step S2, the conditions for performing spatio-temporal matching on the satellite data and the ground-based observation data include:

[0058] In terms of spatial location, select the data in the satellite data that covers the ground-based observation sites according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes.

[0059] Specifically, mainly perform qualitative and quantitative analysis of the multi-source satellite data relative to the ground-based observations to determine the accuracy effect of the satellite data and design a correction scheme. The specific criteria and content of this step are described as follows:

[0060] Determine the spatio-temporal matching conditions between the satellite data and the ground-based observation data. In terms of spatial location, select the data in OCO-2 data that covers the ground-based TCCON sites according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes, and use its mean value as the true value reference for the corresponding time and range.

[0061] When calculating the satellite data accuracy, the data corrected by the satellite design team should be selected. Calculate the annual mean RMSE using the root mean square error RMSE as the measurement standard to ensure a sufficiently large sample size. The calculation method is shown in formula (1).

[0062]

[0063] where, y s represents the satellite observation value, y t is the TCCON observation value, and m is the number of matches.

[0064] Determine the satellite data to be corrected according to the RMSE results. Taking the four satellites GOSAT, GOSAT-2, OCO-2, and OCO-3 as examples, the data accuracy of GOSAT and GOSAT-2 is relatively poor and is regarded as the satellite to be corrected. For the OCO series of satellites, due to quality screening during their design, the data with label 0 is regarded as having better quality, and the data volume of the data with label 1, which has relatively poor quality, cannot be ignored compared with the former. Therefore, for the two satellites OCO-2 and OCO-3, the data with relatively poor quality also needs to be corrected, denoted as OCO2(1) and OCO3(1).

[0065] In step S3, using the annual average RMSE before correction as the weight, select and fuse the labeled data to construct a labeled data set; perform spatio-temporal matching between the satellite data to be corrected and the labeled data set to construct a sample set of the satellite to be corrected.

[0066] In some embodiments, in the step S3, the step of using the annual average RMSE before correction as the weight to select and fuse the labeled data includes:

[0067] Using the annual average RMSE before correction of each type of satellite data as the weight, perform weighted averaging on multiple types of satellite data.

[0068] Specifically, perform weighted fusion on the satellite data with label values to ensure the sample size of the label values as much as possible. The specific criteria and operations of this step are described as follows:

[0069] Taking the four satellites GOSAT, GOSAT-2, OCO-2, and OCO-3 as examples, use the data of OCO2(0) and OCO3(0) with higher quality as the label y value, and use the RMSE statistically calculated by the satellite as the weight standard to perform weighted fusion on the labeled data of the two satellites.

[0070] Perform spatio-temporal matching on the data of OCO2(0) and OCO3(0). Since the spatial resolutions of the two satellite data are the same, the unified spatial matching condition is set to 0.01°×0.02°, and the temporal matching condition is the 2-hour time window before and after during data screening. Perform weighted fusion on the data that meets the matching conditions according to the above principles, and the calculation method is shown in formula (2).

[0071]

[0072] Take the median of the time of OCO2(0) data and the time of OCO3(0) data as the time of the fused data. For the data that does not meet the matching conditions, retain it and combine it with the fused data to form the OCO(0) data set, which is used as the label value in the subsequent correction process.

[0073] Perform spatio-temporal matching for all data to be corrected based on the label, and construct a training sample set. The specific criteria and operations for this step are described as follows:

[0074] Use the OCO(0) dataset of the corrected OCO series satellites as the label value.

[0075] Perform spatio-temporal data matching for the two satellites. Keep the spatial matching benchmark determined by the satellite with the highest spatial resolution. The spatial resolution of GOSAT is 0.1°×0.1°, while the spatial resolution of OCO is 0.01°×0.02°. To ensure convenience for the final fusion after correction, taking the GOSAT data matching as an example, its spatial matching condition should be determined by the OCO satellite, that is, the matching condition is 0.01°×0.02°. The time matching condition is the 2-hour time window before and after used for data screening.

[0076] Construct and randomly divide the sample set, with 80% for training and 20% for testing. At this time, important information in the data of other satellites needs to be added to the sample set, such as the data uncertainty of GOSAT, the prior value of XCO2, or other indicators that may affect the data calculation involved. The above additional indicators will be used as the features for training in the random forest correction process. For each satellite, due to the differences in the transmission of the satellite and its payload and the inversion algorithm, the parameters involved in the inversion process of different satellites are different, so the additional indicators (features) can be different.

[0077] In step S4, use the training set and feature data of the sample set of the satellite to be corrected to train the random forest model; use the trained random forest model to correct the data of the satellite to be corrected.

[0078] Specifically, the random forest algorithm is widely used in regression prediction problems and can perform a certain degree of quality control through the parameter settings of the tree structure. In addition, considering that the characteristics of each satellite are different, separate models need to be built for the satellite data to be corrected to achieve prediction, and the model structure settings can be the same.

[0079] According to prior knowledge, more trees can improve the stability and accuracy of the model, but at the same time, it will increase the calculation cost. Therefore, it is set that the random forest model contains 100 decision trees. The number of layers of each tree is limited to less than 10 layers to prevent the occurrence of overfitting problems, that is, the situation where the model performs too well on the training data but poorly on the new data. At the same time, set a fixed random seed during the model training process to ensure the repeatability of the results, which is convenient for subsequent analysis of the features and further improvement of the model to improve the accuracy. Taking the OCO-2, OCO-3, GOSAT, and GOSAT-2 data as examples. The model composition is shown in formula (3):

[0080]

[0081] Among them, Features represents the features selected by the satellite.

[0082] Note: The features should at least include the prior data and posterior data provided inside the satellite to ensure the unity of the satellite data correction method. At the same time, the feature selection should not include specific time and location information, so as to ensure that the model optimizes the internal algorithm of the satellite and ensures that it has good correction effects at any position and time.

[0083] In step S5, compare the annual average RMSE of the satellite data after data correction and the ground-based observation data with the annual average RMSE before the correction to conduct a correction data accuracy test.

[0084] In some embodiments, in the step S5, the conduct of the correction data accuracy test includes:

[0085] Compare the annual average RMSE of the satellite data after data correction and the ground-based observation data with the annual average RMSE before the correction. If the comparison result is less than the first preset value, discard the satellite data after data correction, re-perform feature selection, and conduct data correction.

[0086] The re-performing of feature selection includes:

[0087] Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; repeat the above operation multiple times for each feature, calculate the average value of the multiple RMSEs as the measurement standard of the feature importance; delete the features whose measurement standard is less than the second preset value from the input of the random forest model; the features must be data other than the prior data and posterior data;

[0088] Set a change threshold; record the RMSE of a certain satellite data in the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", and then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before the correction;

[0089] Compare the annual average RMSE of the satellite data after re-data correction and the ground-based observation data with the annual average RMSE before the correction. If the comparison result is less than the first preset value, discard the fusion of the satellite data.

[0090] Specifically, ground-based verification is carried out for each satellite, keeping the spatio-temporal matching conditions of the verification the same as above. Taking GOSAT data as an example, during the above-mentioned correction process, the GOSAT series of satellites with an original resolution of 0.1°×0.2° are corrected to the OCO series of satellites to obtain data with a corrected spatial resolution of 0.01°×0.02°. Therefore, this type of data needs to be rematched with ground-based observation data.

[0091] Compare the ground-based verification accuracy of the corrected data with that of the data before correction. Using RMSE as a reference index, if there is no obvious improvement, then abandon the correction result, or re-perform feature selection to achieve improvement.

[0092] Re-perform feature selection to achieve improvement, including:

[0093] Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; repeat the above operation 10 times for each feature, calculate the average value of the 10 RMSEs as the measurement standard of the feature importance; delete the feature whose measurement standard is less than the second preset value from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0094] Set a change threshold of 0.01 ppm; record the RMSE of a certain satellite data in the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0095] Compare the annual average RMSE of the satellite data after the second data correction with the ground-based observation data and the annual average RMSE before the correction. If the comparison result is less than the first preset value, then abandon fusing the satellite data. For example, during the data correction process of OCO-2, OCO-3, GOSAT, and GOSAT-2, if it is found that the correction result for OCO2(1) is not good, then abandon fusing this part of the data.

[0096] In step S6, merge the satellite data after data correction with the label data to obtain a merged data set; calculate the data source RMSE of each satellite data in the merged data set and the ground-based observation data; calculate the fusion weight according to the data source RMSE; fuse the satellite data in the merged data set according to the fusion weight.

[0097] In some embodiments, in the step S6, the calculating the fusion weight according to the data source RMSE includes:

[0098] Divide the unit weight mean square error by the RMSE of the data source of a certain satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to the satellite data of a certain type in a certain year.

[0099] The fusing of the satellite data in the merged dataset according to the fusion weight includes:

[0100] Use the fusion weight of each type of satellite data in the merged dataset as the corresponding weight, and perform weighted averaging on multiple types of satellite data in the merged dataset.

[0101] Specifically, for multi-source satellite data fusion. According to years or months, verify the accuracy of the corrected satellite data, uncorrected satellite data and ground-based stations, calculate the RMSE as the accuracy index, take the mean of all RMSEs as the unit weight mean square error, and calculate the weights at each time point to simplify the fusion process.

[0102] Taking the fusion of OCO-2, OCO-3, GOSAT and GOSAT-2 satellite data as an example, the process is as follows:

[0103] Calculate the ground-based verification accuracy. Obtain the RMSE of the corrected satellite data (including GOSAT, GOSAT2 and OCO3(1)), uncorrected satellite data (including OCO2(0) and OCO3(0), and the fused OCO(0)) and ground-based verification for different years.

[0104] Calculate the corresponding weights. Calculate the mean as the unit weight mean square error, and then calculate the fusion weights of different data in different years.

[0105]

[0106] Among them, represents the weight corresponding to a certain satellite data in a certain year; represents the RMSE of the data source of a certain satellite data and ground-based observation data in a certain year; represents the unit weight mean square error.

[0107] Data fusion. For grids with a global spatial resolution of 0.01°×0.02° on an annual scale, the recorded data volume of available data sources may be different. If the data source is single, calculate its mean value; if there are more than 2 data sources, perform fusion calculation through formula (5).

[0108]

[0109] Among them, Y represents the final XCO2 value of the grid, P S is the weight of the data source, m S is the total number of XCO2 values of the data source in the grid, y SIt is the XCO2 value of this data source.

[0110] In summary, the solution proposed by the present invention can use the random forest algorithm to perform calibration based on multi-source satellite products themselves. Without considering the integration of ground-based data, compared with the existing calibration methods, the accuracy of satellite data is greatly improved. Starting from the satellite data itself, the calibration process does not need to particularly consider the satellite observation location and observation time, and can be regarded as a post-processing optimization method for satellite inversion algorithms. Based on the satellite data set with high-precision standards, the originally unqualified data set is optimized, greatly increasing the spatial range of data observation and the overall utilization efficiency of the data. The problem of different satellite spatial resolutions is solved. The fused data set is in the leading position in both time and spatial resolutions. In actual case operations, the spatial resolution can reach 0.01°×0.02°, and the time resolution can reach 1 day.

[0111] The second aspect of the present invention discloses a multi-source satellite data fusion system for carbon dioxide column concentration. Figure 2 It is a structural diagram of a multi-source satellite data fusion system for carbon dioxide column concentration according to an embodiment of the present invention; as Figure 2 shown, the system 100 includes:

[0112] The first processing module 101 is configured to obtain long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility;

[0113] The second processing module 102 is configured to perform spatio-temporal matching on the satellite data and the ground-based observation data; select labeled data according to the annual average RMSE before calibration of the matched satellite data and ground-based observation data;

[0114] The third processing module 103 is configured to use the annual average RMSE before calibration as the weight to fuse the selected labeled data to construct a labeled data set; perform spatio-temporal matching on the satellite data to be calibrated and the labeled data set to construct a satellite sample set to be calibrated;

[0115] The fourth processing module 104 is configured to train a random forest model using the training set and feature data of the satellite sample set to be calibrated; use the trained random forest model to perform data calibration on the satellite data to be calibrated;

[0116] The fifth processing module 105 is configured to compare the annual average RMSE of the satellite data after data calibration and the ground-based observation data with the annual average RMSE before calibration to perform calibration data accuracy verification;

[0117] The sixth processing module 106 is configured to merge the satellite data after data correction with the label data to obtain a merged data set; calculate the data source RMSE between each type of satellite data in the merged data set and the ground-based observation data; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

[0118] According to the system of the second aspect of the present invention, the first processing module 101 is specifically configured to obtain long-term global multi-source carbon satellite data, such as GOSAT, GOSAT-2, OCO-2, and OCO-3, and highly reliable ground-based observation data, such as TCCON.

[0119] According to the system of the second aspect of the present invention, the second processing module 102 is specifically configured that the conditions for spatio-temporal matching between the satellite data and the ground-based observation data include:

[0120] In terms of spatial location, select the data in the satellite data that covers the ground-based observation sites according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes by.

[0121] Specifically, mainly conduct qualitative and quantitative analysis of the multi-source satellite data relative to the ground-based observation to determine the accuracy effect of the satellite data and design a correction scheme. The specific standards and content of this step are described as follows:

[0122] Determine the spatio-temporal matching conditions between the satellite data and the ground-based observation data. In terms of spatial location, select the data in the OCO-2 data that covers the ground-based TCCON sites according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes by, and use its average value as the true value reference for the corresponding moment and range.

[0123] When calculating the satellite data accuracy, the data corrected by the satellite design team should be selected. Calculate the annual average RMSE using the root mean square error RMSE as the measurement standard to ensure that the sample size is large enough. The calculation method is shown in formula (1).

[0124]

[0125] where y s represents the satellite observation value, y t is the TCCON observation value, and m is the number of matches.

[0126] Determine the satellite data to be corrected according to the RMSE results. Taking four satellites, namely GOSAT, GOSAT-2, OCO-2, and OCO-3, as examples, the data accuracies of GOSAT and GOSAT-2 are relatively poor, and they are regarded as satellites to be corrected. For the OCO series of satellites, due to quality screening during their design, the data with label 0 is regarded as having better quality, and the data volume of the data with label 1, which has relatively poor quality, cannot be ignored compared with the former. Therefore, for the two satellites OCO-2 and OCO-3, the data with relatively poor quality also needs to be corrected, denoted as OCO2(1) and OCO3(1).

[0127] According to the system of the second aspect of the present invention, the third processing module 103 is specifically configured to, taking the annual average RMSE before correction as the weight, the fusion of the selected label data includes:

[0128] Taking the annual average RMSE before correction of each type of satellite data as the weight, perform weighted averaging on multiple types of satellite data.

[0129] Specifically, perform weighted fusion on the satellite data with label values to ensure the sample size of the label values as much as possible. The specific criteria and operations of this step are described as follows:

[0130] Taking four satellites, namely GOSAT, GOSAT-2, OCO-2, and OCO-3, as examples, use the data of OCO2(0) and OCO3(0) with higher quality as the label y value, and perform weighted fusion on the label data of the two satellites according to the RMSE statistically calculated for the satellites as the weight standard.

[0131] Match the data of OCO2(0) and OCO3(0) in terms of time and space. Since the spatial resolutions of the two satellite data are the same, the unified spatial matching condition is set to 0.01°×0.02°, and the time matching condition is the 2-hour time window before and after during data screening. Perform weighted fusion on the data that meets the matching conditions according to the above principles, and the calculation method is shown in formula (2).

[0132]

[0133] Take the median of the time of OCO2(0) data and the time of OCO3(0) data as the time of the fused data. For the data that does not meet the matching conditions, retain it and combine it with the fused data to form the OCO(0) data set, which is used as the label value in the subsequent correction process.

[0134] Taking the label as the benchmark, perform time-space matching on all the data to be corrected to construct a training sample set. The specific criteria and operations of this step are described as follows:

[0135] Use the OCO(0) data set for correcting the OCO series of satellites as the label value.

[0136] Perform spatio-temporal data matching for two satellites. Keep the spatial matching benchmark determined according to the satellite with the highest spatial resolution. The spatial resolution of GOSAT is 0.1°×0.1°, while the spatial resolution of OCO is 0.01°×0.02°. To ensure convenience for final fusion after calibration, taking the GOSAT data matching as an example, its spatial matching condition should be determined by the OCO satellite, that is, the matching condition is 0.01°×0.02°. The time matching condition is the 2-hour time window before and after used in data screening.

[0137] Construct and randomly divide the sample set, with 80% for training and 20% for testing. At this time, important information in other satellite data needs to be added to the sample set, such as the data uncertainty of GOSAT, the prior value of XCO2, or other indicators that may affect the data calculation involved. The above additional indicators will be used as the features for training in the random forest calibration process. For each satellite, due to the differences in the transmission of its satellite and payload and the inversion algorithm, the parameters involved in the inversion process of different satellites are different, so the additional indicators (features) can be different.

[0138] According to the system of the second aspect of the present invention, the fourth processing module 104 is specifically configured that the random forest algorithm is widely used in regression prediction problems and can perform a certain degree of quality control through the parameter settings of the tree structure. In addition, considering that the characteristics of each satellite are not the same, separate models need to be built for the satellite data to be corrected to achieve prediction, and the model structure settings can be the same.

[0139] According to prior knowledge, more trees can improve the stability and accuracy of the model, but at the same time increase the computational cost. Therefore, it is set that the random forest model contains 100 decision trees. Each tree is restricted to have less than 10 layers to prevent the occurrence of overfitting problems, that is, the situation where the model performs too well on the training data but poorly on the new data. At the same time, a fixed random seed is set during the model training process to ensure the repeatability of the results, which is convenient for subsequent analysis of the features and further improvement of the model to improve the accuracy. Taking the OCO-2, OCO-3, GOSAT, and GOSAT-2 data as examples. The model composition is shown in formula (3):

[0140]

[0141] Among them, Features represents the features selected by the satellite.

[0142] Note: The features should at least include the prior data and posterior data provided inside the satellite to ensure the unity of the satellite data correction method. At the same time, the feature selection should not include specific time and location information, so as to ensure that the model optimizes the internal algorithm of the satellite and ensures good correction effect at any position and time.

[0143] For the system according to the second aspect of the present invention, the fifth processing module 105 is specifically configured that the performing accuracy inspection of the correction data includes:

[0144] Compare the annual average RMSE of the satellite data after data correction with the ground-based observation data and the annual average RMSE before the correction. If the comparison result is less than the first preset value, discard the satellite data after data correction, re-perform feature selection, and perform data correction.

[0145] The re-performing feature selection includes:

[0146] Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; repeat the above operation for each feature multiple times, calculate the average value of the multiple RMSEs as the measurement standard of the feature importance; delete the feature whose measurement standard is less than the second preset value from the input of the random forest model; the feature must be data other than the prior data and posterior data;

[0147] Set a change threshold; record the RMSE of a certain satellite data in the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before the correction;

[0148] Compare the annual average RMSE of the satellite data after re-data correction with the ground-based observation data and the annual average RMSE before the correction. If the comparison result is less than the first preset value, discard the fusion of the satellite data.

[0149] Specifically, perform ground-based verification on each satellite, keeping the spatio-temporal matching conditions of the verification the same as above. Taking GOSAT data as an example, during the above correction process, the GOSAT series satellites with an original resolution of 0.1°×0.2° are corrected to the OCO series satellites to obtain data with a corrected spatial resolution of 0.01°×0.02°. Therefore, this type of data needs to be rematched with the ground-based observation data.

[0150] Compare the ground-based verification accuracy of the corrected data with the ground-based verification accuracy before the correction. Taking RMSE as the reference index, if there is no obvious improvement, discard the correction result, or re-perform feature selection to achieve improvement.

[0151] Re - perform feature selection to achieve improvement, including:

[0152] Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be calibrated; repeat the above operation 10 times for each feature, calculate the average value of the 10 RMSEs as the measurement standard of the feature importance; delete the features whose measurement standard is less than the second preset value from the input of the random forest model; the features must be data other than prior data and posterior data;

[0153] Set a change threshold of 0.01 ppm; record the RMSE of a certain satellite data in the test set of the satellite sample set to be calibrated in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before calibration;

[0154] Compare the annual average RMSE of the satellite data after re - data calibration and the ground - based observation data with the annual average RMSE before calibration. If the comparison result is less than the first preset value, then abandon fusing the satellite data. For example, during the data calibration of OCO - 2, OCO - 3, GOSAT, and GOSAT - 2, if it is found that the calibration result for OCO2(1) is not good, then abandon fusing this part of the data.

[0155] According to the system of the second aspect of the present invention, the sixth processing module 106 is specifically configured that calculating the fusion weight according to the data source RMSE includes:

[0156] Divide the mean square error of unit weight by the data source RMSE of a certain satellite data and the ground - based observation data in a certain year to obtain the fusion weight corresponding to the satellite data in that year.

[0157] Fusing the satellite data in the combined dataset according to the fusion weight includes:

[0158] Use the fusion weight of each satellite data in the combined dataset as the corresponding weight to perform weighted averaging on multiple satellite data in the combined dataset.

[0159] Specifically, for multi - source satellite data fusion. According to years or months, perform accuracy verification on the calibrated satellite data, uncalibrated satellite data, and ground - based sites, calculate the RMSE as the accuracy index, take the average value of all RMSEs as the mean square error of unit weight, and calculate the weights at each time point to simplify the fusion process.

[0160] Taking the fusion of OCO - 2, OCO - 3, GOSAT, and GOSAT - 2 satellite data as an example, the process is as follows:

[0161] Calculate the verification accuracy of the ground-based data. Obtain the RMSE between the calibrated satellite data (including GOSAT, GOSAT2, and OCO3(1)) and the uncalibrated satellite data (including OCO2(0) and OCO3(0), which are fused to obtain OCO(0)) and the ground-based verification for different years.

[0162] Calculate the corresponding weights. Calculate the mean value as the mean square error of unit weight, and then calculate the fusion weights of different data for different years.

[0163]

[0164] Among them, represents the weight corresponding to a certain type of satellite data in a certain year; represents the data source RMSE between a certain type of satellite data and the ground-based observation data in a certain year; represents the mean square error of unit weight.

[0165] Data fusion. For grids with a global spatial resolution of 0.01°×0.02° on an annual scale, the recorded data volumes of their available data sources may vary. If the data source is single, calculate its mean value; if there are more than two data sources, perform fusion calculation through formula (5).

[0166]

[0167] Among them, Y represents the final XCO2 value of the grid, P S is the weight of the data source, m S is the total number of XCO2 values of the data source within the grid, y S is the XCO2 value of the data source.

[0168] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. When the processor executes the computer program stored in the memory, it implements the steps in a multi-source satellite data fusion method for carbon dioxide column concentration according to any one of the first aspects disclosed in the present invention.

[0169] Figure 3 is a structural diagram of an electronic device according to an embodiment of the present invention, as shown in Figure 3As shown, the electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, near-field communication (NFC), or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0170] Those skilled in the art can understand that Figure 3 the structure shown in is only the structure diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0171] The fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in a method for fusing multi-source satellite data of carbon dioxide column concentration according to any one of the first aspects disclosed in the present invention are implemented.

[0172] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should be considered as within the scope described in this specification. The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for fusing multi-source satellite data of carbon dioxide column concentration, characterized in that The method includes: Step S1: Obtain long-time-series global multi-source carbon satellite data and ground-based observation data with predefined credibility. Step S2: Perform spatio-temporal matching on the satellite data and the ground-based observation data; select labeled data according to the annual average RMSE before correction of the satellite data and the ground-based observation data after matching. Step S3: Use the annual average RMSE before correction as a weight to fuse the selected labeled data to construct a labeled data set; perform spatio-temporal matching on the satellite data to be corrected and the labeled data set to construct a satellite sample set to be corrected. Step S4: Use the training set and feature data of the satellite sample set to be corrected to train a random forest model; use the trained random forest model to correct the satellite data to be corrected. Step S5: Compare the annual average RMSE of the satellite data after data correction and the ground-based observation data with the annual average RMSE before correction to perform accuracy test on the corrected data. Step S6: Merge the satellite data after data correction and the labeled data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and the ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; fuse the satellite data in the merged data set according to the fusion weight.

2. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, wherein In the step S2, the conditions for performing spatio-temporal matching on the satellite data and the ground-based observation data include: In terms of spatial position, select the data in the satellite data that covers the ground-based observation sites according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes.

3. A method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that In the step S3, using the annual average RMSE before correction as a weight to fuse the selected labeled data includes: Use the annual average RMSE before correction of each type of satellite data as a weight to perform weighted average on multiple types of satellite data.

4. A method for multi-source satellite data fusion of carbon dioxide column concentration according to claim 1, characterized in that, In the step S5, performing accuracy test on the corrected data includes: Compare the annual average RMSE of the satellite data after data correction and the ground-based observation data with the annual average RMSE before correction. If the comparison result is less than the first preset value, discard the satellite data after data correction, re-perform feature selection, and perform data correction.

5. A method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 4, characterized in that, In the step S5, the re-performing feature selection includes: Randomly shuffle one feature each time, input it into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; repeat the above operation multiple times for each feature, calculate the average value of the multiple RMSEs as the measurement standard of the feature importance; delete the feature whose measurement standard is less than the second preset value from the input of the random forest model; the feature must be data other than the prior data and the posterior data. Set a change threshold; record the RMSE of a certain type of satellite data in the test set of the satellite sample set to be corrected in the initial state. If the change in RMSE caused by a certain feature is lower than the change threshold, it indicates that the feature is "unimportant", and then delete the feature; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction. Compare the annual average RMSE of the satellite data after re-data correction with the ground-based observation data and the annual average RMSE before the correction. If the comparison result is less than the first preset value, discard the fusion of the satellite data.

6. A method for multi-source satellite data fusion of carbon dioxide column concentration according to claim 1, characterized in that, In the step S6, the calculating of the fusion weights according to the data source RMSE includes: Dividing the mean square error of unit weight by the data source RMSE of a certain type of satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to the certain type of satellite data in the certain year.

7. A method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that, In the step S6, the fusing of the satellite data in the merged dataset according to the fusion weights includes: Taking the fusion weights of each type of satellite data in the merged dataset as the corresponding weights, and performing weighted averaging on multiple types of satellite data in the merged dataset.

8. A multi-source satellite data fusion system for carbon dioxide column concentration, characterized in that, The system includes: A first processing module, configured to obtain long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility; A second processing module, configured to perform spatio-temporal matching on the satellite data and the ground-based observation data; select labeled data according to the annual average RMSE before correction of the matched satellite data and ground-based observation data; A third processing module, configured to use the annual average RMSE before correction as a weight to fuse the selected labeled data to construct a labeled dataset; perform spatio-temporal matching on the satellite data to be corrected and the labeled dataset to construct a satellite sample set to be corrected; A fourth processing module, configured to train a random forest model using the training set and feature data of the satellite sample set to be corrected; perform data correction on the satellite data to be corrected using the trained random forest model; A fifth processing module, configured to compare the annual average RMSE of the satellite data after data correction with the ground-based observation data and the annual average RMSE before the correction to perform accuracy verification of the corrected data; A sixth processing module, configured to merge the satellite data after data correction with the labeled data to obtain a merged dataset; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged dataset; calculate fusion weights according to the data source RMSE; fuse the satellite data in the merged dataset according to the fusion weights.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. When the processor executes the computer program stored in the memory, the steps in any one of claims 1 to 7 of a method for fusing multi-source satellite data of carbon dioxide column concentration are implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, the steps in any one of claims 1 to 7 of a method for fusing multi-source satellite data of carbon dioxide column concentration are implemented.

Images