Carbon dioxide column concentration multi-source satellite data fusion method and system

Abstract

The invention provides a carbon dioxide column concentration multi-source satellite data fusion method and system. The method comprises the following steps of: constructing a correction model among multi-source satellites through a random forest algorithm represented in ensemble learning, and unifying data precision and spatial resolution of the multi-source satellites by taking parameters and observation data of the satellites as input of the model and taking high-precision satellite observation data as a reference; and finally, performing fusion by using a unit weight method to obtain a global XCO2 fusion data set which is larger in observation coverage range and higher in temporal-spatial resolution and precision compared with single satellite data. According to the scheme provided by the invention, the precision of satellite data can be greatly improved; starting from satellite data, satellite observation sites and observation time do not need to be specially considered in the correction process; an original unqualified data set is optimized, and the spatial range of data observation and the overall utilization efficiency of data are greatly increased; the problem that the spatial resolutions of different satellites are different is solved.

Description

Technical Field

[0001] The invention belongs to the fields of atmospheric environment remote sensing, standard quantitative products, multi-source data fusion and XCO2 technology, and in particular relates to a method and system for fusing multi-source satellite data of carbon dioxide column concentration. Background Art

[0002] Carbon dioxide (CO2) is one of the main greenhouse gases that has a significant impact on human life.

[0003] At present, the main methods for obtaining atmospheric CO2 concentration are ground measurement, satellite detection and model simulation. Ground measurement has the highest accuracy and time-continuous observation capability, and the main methods include gas sampling measurement and spectrometer measurement. The Total Carbon Column Observation Network (TCCON) stations are distributed in many places around the world. It uses Fourier transform spectrometers to measure atmospheric CO2 concentration and is currently the most widely used and recognized ground CO2 monitoring network. However, although this type of station network has a wide observation coverage and has accumulated long-term observation data, its station distribution is not uniform, making it difficult to capture the global change characteristics of CO2.

[0004] The carbon dioxide column concentration (XCO2) is defined as the average molar mixing ratio of the total atmospheric column (excluding water vapor molecules) from the earth's surface to the top of the atmosphere. Combining the absorption characteristics of CO2 in the short-wave infrared range, the penetration ability of solar radiation and the signal characteristics, researchers have developed a variety of algorithms for the inversion of XCO2 for different satellite payloads. They are mainly divided into full-physics inversion algorithms based on differential optical absorption spectroscopy (DOAS) and methods based on statistical estimation. The former are represented by: WFM-DOAS, IMAP-DOA and DOAS-BESD, and the latter are represented by: NIES, ACOS, UOL_FP, IAPCAS, FOCAL and Remotec. The basic principle of XCO2 inversion is to use the forward radiation transfer model to simulate the backscattered signals received by the satellite sensor in the O2 and CO2 absorption bands under the condition of known parameter vectors (parameter vectors) and unknown parameter vectors (state vectors); then the state vector is adjusted by optimal estimation or probabilistic methods to minimize the error between the simulated observations and the actual satellite observations; finally, the retrieved state vector is considered to be the real atmospheric state. The differences between the above algorithms lie in the processing of aerosol and cloud scattering effects, inversion bands, initial state vectors, etc.

[0005] Satellite remote sensing technology has the characteristics of wide coverage, high stability, long time series and high accuracy. Based on the development of the above algorithm and the characteristics of remote sensing data, the products produced can well make up for the sparse distribution of ground monitoring stations, and provide a global CO2 spatial distribution data that is convenient for research and analysis and has a consistent format. In March 2002, the Scanning Imaging Absorption spectrometer for Atmospheric CartograpHY (SCIAMACHY) carried by the European ENVISAT satellite, in January 2009, the Thermal And Near-infrared Sensor for carbon Observation (TANSO) carried by the Japanese GOSAT satellite, in July 2014, the high spectral resolution grating spectrometer carried by the US OCO-2 satellite, and in December 2016, the ultra-high spectral atmospheric CO2 grating spectrometer carried by the Chinese TanSat satellite, etc., obtain the atmospheric CO2 concentration by detecting the infrared and near-infrared spectra of the atmosphere, and thus carry out the temporal and spatial distribution characteristics analysis of atmospheric CO2. However, currently single satellite sensors are limited by the defects of their own software and hardware support, and it is often difficult to obtain continuous global CO2 concentration distribution results with high temporal and spatial resolution.

[0006] Disadvantages of the prior art

[0007] An important issue in multi-source data fusion is to deeply understand the differences between multi-source satellite data and comprehensively consider the impact of data quality, spatiotemporal scale and fusion algorithm on the presentation effect of the data set in the final practical application.

[0008] There are many research bases for the consistency analysis of multi-source CO2 satellite data. Liu Liangyun et al. have summarized the current status and progress of global carbon monitoring satellites. Noel et al. used the Focal algorithm to invert XCO2 from GOSAT and GOSAT-2, and compared it with TCCON to conclude that GOSAT-2 is slightly better than GOSAT. Mustafa compared the CO2 data observed by OCO-2 and GOSAT with the CO2 data of CarbonTracker (CT) mode, and found that the correlation coefficients between CT and GOSAT, CT and OCO-2 were 0.93 and 0.89, respectively, and the root mean square errors were 2.61ppm and 2.16ppm, respectively. Wang et al. used GOSAT and SCIAMACHY product data for data fusion, and the mutual correction between satellite observation platforms would affect the data quality. Hong et al. verified the accuracy of GOSAT through two TCCON observation stations in East Asia. Hwang et al. evaluated the XCO2 product of OCO-2 using portable measurement equipment. Taylor et al. evaluated the consistency of NASA ACOS versions of OCO-2 and OCO-3 and found that OCO-3 was slightly better than OCO-2. Zhang et al. used a high-precision surface model to fuse the atmospheric chemical transport model GEOS-Chem data and ground-based TCCON station data, and evaluated the data quality of GOSAT and OCO2 based on this fused data set, and found that OCO2 was more accurate than GOSAT.

[0009] Studies have shown that due to differences in satellite payload parameters and the influence of the atmosphere in observation conditions, there are huge differences in the degree of data distortion and the effects of different data sources between different carbon satellites. The current mainstream method is to use physical algorithms to correct the inversion results based on the mechanism process. This method is complicated to operate and requires researchers to be familiar with the satellite physical inversion model. It has little effect on correcting invalid satellite observation data. At the same time, differences in correction methods will also lead to large differences in the final results of satellites. Another method is to use ground-based observations as a reference for correction, but the number of TCCON sites is scarce and the correction results are questionable. No researcher has successfully used TCCON to correct the invalid satellite observation data, resulting in a large amount of satellite observation data being wasted. Summary of the invention

[0010] In order to solve the above technical problems, the present invention proposes a technical solution of a method for fusing multi-source satellite data of carbon dioxide column concentration to solve the above technical problems.

[0011] The first aspect of the present invention discloses a method for fusing multi-source satellite data of carbon dioxide column concentration, the method comprising:

[0012] Step S1, obtaining long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility;

[0013] Step S2: performing temporal and spatial matching of satellite data and ground-based observation data; selecting label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data;

[0014] Step S3, using the annual average RMSE before correction as a weight, the selected label data are merged to construct a label data set; the satellite data to be corrected are matched with the label data set in time and space to construct a satellite sample set to be corrected;

[0015] Step S4, using the training set and feature data of the satellite sample set to be corrected to train the random forest model; using the trained random forest model to perform data correction on the satellite data to be corrected;

[0016] Step S5, comparing the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction, and performing a correction data accuracy test;

[0017] Step S6: Merge the corrected satellite data with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

[0018] According to the method of the first aspect of the present invention, in step S2, the conditions for temporal and spatial matching of the satellite data and the ground-based observation data include:

[0019] In terms of spatial position, data covering ground-based observation sites in satellite data are selected according to the spatial resolution of the satellite; in terms of time scale, ground-based observation data within two hours before and after the satellite transit are selected.

[0020] According to the method of the first aspect of the present invention, in step S3, the step of fusing the selected label data using the average annual RMSE before correction as a weight includes:

[0021] The weighted average of multiple satellite data is performed using the annual average RMSE of each satellite data before correction as the weight.

[0022] According to the method of the first aspect of the present invention, in step S5, the performing of the calibration data accuracy check includes:

[0023] The annual average RMSE of the satellite data and the ground-based observation data after data correction is compared with the annual average RMSE before correction. If the comparison result is less than a first preset value, the satellite data after data correction is abandoned, feature selection is performed again, and data correction is performed.

[0024] According to the method of the first aspect of the present invention, in step S5, re-selecting features includes:

[0025] Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated multiple times for each feature, and the average value of multiple RMSEs is calculated as a measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0026] Set a change threshold; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0027] The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned.

[0028] According to the method of the first aspect of the present invention, in step S6, calculating the fusion weight according to the data source RMSE includes:

[0029] The unit weighted mean error is divided by the data source RMSE of a certain satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to a certain satellite data in a certain year.

[0030] According to the method of the first aspect of the present invention, in step S6, fusing the satellite data in the merged data set according to the fusion weight includes:

[0031] The fusion weight of each satellite data in the merged data set is used as the corresponding weight to perform weighted averaging on the multiple satellite data in the merged data set.

[0032] The second aspect of the present invention discloses a multi-source satellite data fusion system for carbon dioxide column concentration, the system comprising:

[0033] The first processing module is configured to obtain long-term global multi-source carbon satellite data and ground-based observation data with a predefined credibility;

[0034] The second processing module is configured to perform temporal and spatial matching of satellite data and ground-based observation data; select label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data;

[0035] The third processing module is configured to fuse the selected label data with the annual average RMSE before correction as a weight to construct a label data set; perform time-space matching between the satellite data to be corrected and the label data set to construct a satellite sample set to be corrected;

[0036] The fourth processing module is configured to train a random forest model using the training set and feature data of the satellite sample set to be corrected; and perform data correction on the satellite data to be corrected using the trained random forest model;

[0037] A fifth processing module is configured to compare the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction to perform a correction data accuracy test;

[0038] The sixth processing module is configured to merge the satellite data after data correction with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

[0039] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any one of the multi-source satellite data fusion methods for carbon dioxide column concentration in the first aspect of the present disclosure are implemented.

[0040] The fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any one of the methods for fusing multi-source satellite data of carbon dioxide column concentration in the first aspect of the present disclosure.

[0041] In summary, the solution proposed in the present invention can greatly improve the accuracy of satellite data; starting from the satellite data itself, the correction process does not need to specially consider the satellite observation location and observation time, and can be regarded as a post-processing optimization method of the satellite inversion algorithm; based on the high-precision standard satellite data set, the original unqualified data set is optimized, which greatly increases the spatial range of data observation and the overall utilization efficiency of the data; and solves the problem of different spatial resolutions of different satellites. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 This is a flow chart of a method for fusing multi-source satellite data of carbon dioxide column concentration according to an embodiment of the present invention;

[0044] Figure 2 A structural diagram of a multi-source satellite data fusion system for carbon dioxide column concentration according to an embodiment of the present invention;

[0045] Figure 3 The figure is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] The first aspect of the present invention discloses a method for fusing multi-source satellite data of carbon dioxide column concentration. Figure 1 FIG. 4 is a flow chart of a method for fusing multi-source satellite data of carbon dioxide column concentration according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0048] Step S1, obtaining long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility;

[0049] Step S2: performing temporal and spatial matching of satellite data and ground-based observation data; selecting label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data;

[0050] Step S3, using the annual average RMSE before correction as a weight, the selected label data are merged to construct a label data set; the satellite data to be corrected are matched with the label data set in time and space to construct a satellite sample set to be corrected;

[0051] Step S4, using the training set and feature data of the satellite sample set to be corrected to train the random forest model; using the trained random forest model to perform data correction on the satellite data to be corrected;

[0052] Step S5, comparing the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction, and performing a correction data accuracy test;

[0053] Step S6: Merge the corrected satellite data with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

[0054] In step S1, long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility are obtained.

[0055] Specifically, obtain long-term global multi-source carbon satellite data, such as GOSAT, GOSAT-2, OCO-2 and OCO-3, as well as highly reliable ground-based observation data, such as TCCON.

[0056] In step S2, the satellite data and the ground-based observation data are matched in time and space; and the label data are selected according to the pre-correction annual average RMSE of the matched satellite data and the ground-based observation data.

[0057] In some embodiments, in step S2, the conditions for temporal and spatial matching of the satellite data and the ground-based observation data include:

[0058] In terms of spatial position, data covering ground-based observation sites in satellite data are selected according to the spatial resolution of the satellite; in terms of time scale, ground-based observation data within two hours before and after the satellite transit are selected.

[0059] Specifically, it mainly involves qualitative and quantitative analysis of multi-source satellite data relative to ground-based observations to determine the accuracy of satellite data and design a correction scheme. The specific standards and contents of this step are described as follows:

[0060] Determine the temporal and spatial matching conditions between satellite data and ground-based observation data. In terms of spatial location, select the data covering the ground-based TCCON station in the OCO-2 data according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes, and use its mean as the true value reference for the corresponding time and range.

[0061] When calculating the accuracy of satellite data, the data corrected by the satellite design team should be selected. The root mean square error (RMSE) is used as the measurement standard to calculate the annual average RMSE to ensure that the sample size is large enough. The calculation method is shown in formula (1).

[0062]

[0063] Among them, y s represents the satellite observation value, y t is the TCCON observation value, and m is the number of matches.

[0064] The satellite data to be corrected are determined based on the RMSE results. Taking GOSAT, GOSAT-2, OCO-2, and OCO-3 as examples, the data accuracy of GOSAT and GOSAT-2 is relatively poor and they are considered to be corrected satellites. For the OCO series of satellites, due to the quality screening in their design, the data with label 0 is considered to be of better quality, and the data volume of label 1 with poorer quality is not negligible compared to the former. Therefore, for the OCO-2 and OCO-3 satellites, the data with poorer quality also need to be corrected, denoted as OCO2(1) and OCO3(1).

[0065] In step S3, the selected label data are fused with the annual average RMSE before correction as a weight to construct a label data set; the satellite data to be corrected are matched with the label data set in time and space to construct a satellite sample set to be corrected.

[0066] In some embodiments, in step S3, the step of fusing the selected label data using the average annual RMSE before correction as a weight includes:

[0067] The weighted average of multiple satellite data is performed using the annual average RMSE of each satellite data before correction as the weight.

[0068] Specifically, weighted fusion is performed on the satellite data of the label value to ensure the sample volume of the label value as much as possible. The specific standards and operations of this step are described as follows:

[0069] Taking GOSAT, GOSAT-2, OCO-2 and OCO-3 as examples, the OCO2(0) and OCO3(0) data with higher quality are used as label y values, and the label data of the two satellites are weightedly fused according to the RMSE of satellite statistics.

[0070] The OCO2(0) and OCO3(0) data were matched in time and space. Since the spatial resolution of the two satellite data was the same, the unified spatial matching condition was set to 0.01°×0.02°, and the temporal matching condition was the 2-hour time window used in data screening. The data that met the matching conditions were weighted fused according to the above principles. The calculation method is shown in formula (2).

[0071]

[0072] The median of the OCO2(0) data time and the OCO3(0) data time is taken as the time of the fused data. For the data that does not meet the matching conditions, it is retained and combined with the fused data to form the OCO(0) data set as the label value in the subsequent correction process.

[0073] Based on the label, time-space matching is performed for all the data to be corrected to construct a training sample set. The specific standards and operations of this step are described as follows:

[0074] The OCO(0) dataset of the corrected OCO series satellites is used as the label value.

[0075] Match the time and space data of the two satellites. Keep the spatial matching benchmark determined by the satellite with the highest spatial resolution. The spatial resolution of GOSAT is 0.1°×0.1°, while the spatial resolution of OCO is 0.01°×0.02°. In order to facilitate the final fusion after correction, the spatial matching condition of GOSAT data matching should be determined by the OCO satellite, that is, the matching condition is 0.01°×0.02°. The time matching condition is the 2-hour time window before and after the data screening.

[0076] Construct and randomly divide the sample set, 80% for training and 20% for testing. At this time, it is necessary to add important information from other satellite data to the sample set, such as GOSAT data uncertainty, XCO2 prior values, or other indicators that may affect and participate in data calculation. The above additional indicators will be used as features for training in the random forest correction process. For each satellite, due to the different transmissions of its satellite and payload and the differences in the inversion algorithm, the parameters involved in the inversion process of different satellites are different, so the additional indicators (features) may be different.

[0077] In step S4, the training set and feature data of the satellite sample set to be corrected are used to train the random forest model; and the trained random forest model is used to perform data correction on the satellite data to be corrected.

[0078] Specifically, the random forest algorithm is widely used in regression prediction problems, and the quality control can be performed to a certain extent by setting the parameters of the tree structure. In addition, considering that the characteristics of each satellite are different, a separate model needs to be built for each satellite data that needs to be corrected to achieve prediction, and the structure settings of each model can be the same.

[0079] According to prior knowledge, more trees can improve the stability and accuracy of the model, but at the same time it will increase the computational cost, so the random forest model is set to contain 100 decision trees. The number of layers of each tree is limited to less than 10 to prevent the occurrence of overfitting problems, that is, the model performs too well on the training data but performs poorly on new data. At the same time, a fixed random seed is set during the model training process to ensure the repeatability of the results, facilitate subsequent feature analysis and further improve the model to improve accuracy. Take OCO-2, OCO-3, GOSAT and GOSAT-2 data as examples. The model composition is shown in formula (3):

[0080]

[0081] Among them, Features represents the features selected by the satellite.

[0082] Note: The features must contain at least the priori data and a posteriori data provided by the satellite to ensure the uniformity of the satellite's data correction method. At the same time, feature selection should not contain specific time and location information, which is needed to ensure that the model is optimizing the satellite's own internal algorithm and ensures that it has a good correction effect at any location and time.

[0083] In step S5, the annual average RMSE of the satellite data and the ground-based observation data after data correction is compared with the annual average RMSE before correction to perform a correction data accuracy test.

[0084] In some embodiments, in step S5, performing the calibration data accuracy check includes:

[0085] The annual average RMSE of the satellite data and the ground-based observation data after data correction is compared with the annual average RMSE before correction. If the comparison result is less than a first preset value, the satellite data after data correction is abandoned, feature selection is performed again, and data correction is performed.

[0086] The re-selection of features comprises:

[0087] Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated multiple times for each feature, and the average value of multiple RMSEs is calculated as a measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0088] Set a change threshold; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0089] The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned.

[0090] Specifically, ground-based verification is performed on each satellite, and the time-space matching conditions of the verification are kept the same as above. Taking GOSAT data as an example, in the above correction process, the GOSAT series satellites with an original resolution of 0.1°×0.2° are corrected to the OCO series satellites to obtain data with a corrected spatial resolution of 0.01°×0.02°, so this type of data needs to be re-matched with the ground-based observation data.

[0091] Compare the ground verification accuracy of the corrected data with that before correction, and use RMSE as a reference indicator. If there is no obvious improvement, abandon the correction result or perform feature selection again to achieve improvement.

[0092] Feature selection was reworked to achieve improvements, including:

[0093] Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated 10 times for each feature, and the average value of the 10 RMSEs is calculated as the measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0094] Set the change threshold to 0.01ppm; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0095] The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned. For example, during the correction process of OCO-2, OCO-3, GOSAT and GOSAT-2 data, if it is found that the correction result for OCO2(1) is not good, the fusion of this part of the data is abandoned.

[0096] In step S6, the satellite data after data correction is merged with the label data to obtain a merged data set; the data source RMSE of each type of satellite data and ground-based observation data in the merged data set is calculated; the fusion weight is calculated according to the data source RMSE; and the satellite data in the merged data set are fused according to the fusion weight.

[0097] In some embodiments, in step S6, calculating the fusion weight according to the data source RMSE includes:

[0098] The unit weighted mean error is divided by the data source RMSE of a certain satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to a certain satellite data in a certain year.

[0099] The fusing the satellite data in the merged data set according to the fusion weights comprises:

[0100] The fusion weight of each satellite data in the merged data set is used as the corresponding weight to perform weighted averaging on the multiple satellite data in the merged data set.

[0101] Specifically, multi-source satellite data fusion. By year or month, the corrected satellite data, the satellite data not involved in the correction and the ground-based stations are verified for accuracy, the RMSE is calculated as the accuracy index, the mean of all RMSE is taken as the unit weighted mean error, and the weight of each time point is calculated to simplify the fusion process.

[0102] Taking the fusion of OCO-2, OCO-3, GOSAT and GOSAT-2 satellite data as an example, the process is as follows:

[0103] Calculate the accuracy of ground-based verification. Obtain the RMSE of the corrected satellite data (including GOSAT, GOSAT2 and OCO3(1)) and the uncorrected satellite data (including OCO2(0) and OCO3(0), fused to obtain OCO(0)) and ground-based verification in different years.

[0104] Calculate the corresponding weights. Calculate the mean as the unit weighted error, and then calculate the fusion weights of different data in different years.

[0105]

[0106] in, Indicates the weight corresponding to a certain satellite data in a certain year; It represents the RMSE of the data source of a certain satellite data and ground-based observation data in a certain year; Represents the unit weight error.

[0107] Data fusion. For a grid with a global spatial resolution of 0.01°×0.02° at an annual scale, the amount of recorded data from the available data sources may be different. If there is only one data source, its mean is calculated; if there are more than two data sources, a fusion calculation is performed using formula (5).

[0108]

[0109] Among them, Y represents the final XCO2 value of the grid, P S is the weight of the data source, m S is the total number of XCO2 values ​​of the data source in the grid, y SThe XCO2 value of the data source.

[0110] In summary, the solution proposed in the present invention can use the random forest algorithm to perform corrections based on the multi-source satellite products themselves. Without considering the integration of ground-based data, the accuracy of satellite data is greatly improved compared to existing correction methods. Starting from the satellite data itself, the correction process does not need to specifically consider the satellite observation location and observation time, and can be regarded as a post-processing optimization method of the satellite inversion algorithm. Based on the high-precision standard satellite data set, the original unqualified data set is optimized, which greatly increases the spatial range of data observation and the overall utilization efficiency of the data. The problem of different spatial resolutions of different satellites is solved. The fused data set is in a leading position in both temporal and spatial resolution. In actual case operations, the spatial resolution can reach 0.01°×0.02°, and the temporal resolution can reach 1 day.

[0111] The second aspect of the present invention discloses a carbon dioxide column concentration multi-source satellite data fusion system. Figure 2 is a structural diagram of a multi-source satellite data fusion system for carbon dioxide column concentration according to an embodiment of the present invention; Figure 2 As shown, the system 100 includes:

[0112] The first processing module 101 is configured to obtain long-term global multi-source carbon satellite data and ground-based observation data with a predefined credibility;

[0113] The second processing module 102 is configured to perform temporal and spatial matching of satellite data and ground-based observation data; select label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data;

[0114] The third processing module 103 is configured to fuse the selected label data with the annual average RMSE before correction as a weight to construct a label data set; perform time-space matching between the satellite data to be corrected and the label data set to construct a satellite sample set to be corrected;

[0115] The fourth processing module 104 is configured to train a random forest model using the training set and feature data of the satellite sample set to be corrected; and perform data correction on the satellite data to be corrected using the trained random forest model;

[0116] The fifth processing module 105 is configured to compare the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction to perform a correction data accuracy test;

[0117] The sixth processing module 106 is configured to merge the satellite data after data correction with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

[0118] According to the system of the second aspect of the present invention, the first processing module 101 is specifically configured to obtain long-term global multi-source carbon satellite data, such as GOSAT, GOSAT-2, OCO-2 and OCO-3, and highly reliable ground-based observation data, such as TCCON.

[0119] According to the system of the second aspect of the present invention, the second processing module 102 is specifically configured such that the conditions for temporal and spatial matching of the satellite data and the ground-based observation data include:

[0120] In terms of spatial position, data covering ground-based observation sites in satellite data are selected according to the spatial resolution of the satellite; in terms of time scale, ground-based observation data within two hours before and after the satellite transit are selected.

[0121] Specifically, it mainly involves qualitative and quantitative analysis of multi-source satellite data relative to ground-based observations to determine the accuracy of satellite data and design a correction scheme. The specific standards and contents of this step are described as follows:

[0122] Determine the temporal and spatial matching conditions between satellite data and ground-based observation data. In terms of spatial location, select the data covering the ground-based TCCON station in the OCO-2 data according to the spatial resolution of the satellite; in terms of time scale, select the ground-based observation data within two hours before and after the satellite passes, and use its mean as the true value reference for the corresponding time and range.

[0123] When calculating the accuracy of satellite data, the data corrected by the satellite design team should be selected. The root mean square error (RMSE) is used as the measurement standard to calculate the annual average RMSE to ensure that the sample size is large enough. The calculation method is shown in formula (1).

[0124]

[0125] Among them, y s represents the satellite observation value, y t is the TCCON observation value, and m is the number of matches.

[0126] The satellite data to be corrected are determined based on the RMSE results. Taking GOSAT, GOSAT-2, OCO-2, and OCO-3 as examples, the data accuracy of GOSAT and GOSAT-2 is relatively poor and they are considered to be corrected satellites. For the OCO series of satellites, due to the quality screening in their design, the data with label 0 is considered to be of better quality, and the data volume of label 1 with poorer quality is not negligible compared to the former. Therefore, for the OCO-2 and OCO-3 satellites, the data with poorer quality also need to be corrected, denoted as OCO2(1) and OCO3(1).

[0127] According to the system of the second aspect of the present invention, the third processing module 103 is specifically configured to fuse the selected label data using the average annual RMSE before correction as a weight, including:

[0128] The weighted average of multiple satellite data is performed using the annual average RMSE of each satellite data before correction as the weight.

[0129] Specifically, weighted fusion is performed on the satellite data of the label value to ensure the sample volume of the label value as much as possible. The specific standards and operations of this step are described as follows:

[0130] Taking GOSAT, GOSAT-2, OCO-2 and OCO-3 as examples, the OCO2(0) and OCO3(0) data with higher quality are used as label y values, and the label data of the two satellites are weightedly fused according to the RMSE of satellite statistics.

[0131] The OCO2(0) and OCO3(0) data were matched in time and space. Since the spatial resolution of the two satellite data was the same, the unified spatial matching condition was set to 0.01°×0.02°, and the temporal matching condition was the 2-hour time window used in data screening. The data that met the matching conditions were weighted fused according to the above principles. The calculation method is shown in formula (2).

[0132]

[0133] The median of the OCO2(0) data time and the OCO3(0) data time is taken as the time of the fused data. For the data that does not meet the matching conditions, it is retained and combined with the fused data to form the OCO(0) data set as the label value in the subsequent correction process.

[0134] Based on the label, time-space matching is performed for all the data to be corrected to construct a training sample set. The specific standards and operations of this step are described as follows:

[0135] The OCO(0) dataset of the corrected OCO series satellites is used as the label value.

[0136] Match the time and space data of the two satellites. Keep the spatial matching benchmark determined by the satellite with the highest spatial resolution. The spatial resolution of GOSAT is 0.1°×0.1°, while the spatial resolution of OCO is 0.01°×0.02°. In order to facilitate the final fusion after correction, the spatial matching condition of GOSAT data matching should be determined by the OCO satellite, that is, the matching condition is 0.01°×0.02°. The time matching condition is the 2-hour time window before and after the data screening.

[0137] Construct and randomly divide the sample set, 80% for training and 20% for testing. At this time, it is necessary to add important information from other satellite data to the sample set, such as GOSAT data uncertainty, XCO2 prior values, or other indicators that may affect and participate in data calculation. The above additional indicators will be used as features for training in the random forest correction process. For each satellite, due to the different transmissions of its satellite and payload and the differences in the inversion algorithm, the parameters involved in the inversion process of different satellites are different, so the additional indicators (features) may be different.

[0138] According to the system of the second aspect of the present invention, the fourth processing module 104 is specifically configured as follows: the random forest algorithm is widely used in regression prediction problems, and the quality control can be performed to a certain extent by setting the parameters of the tree structure. In addition, considering that the characteristics of each satellite are different, a separate model needs to be built for each satellite data that needs to be corrected to achieve prediction, and the structure settings of each model can be the same.

[0139] According to prior knowledge, more trees can improve the stability and accuracy of the model, but at the same time it will increase the computational cost, so the random forest model is set to contain 100 decision trees. The number of layers of each tree is limited to less than 10 to prevent the occurrence of overfitting problems, that is, the model performs too well on the training data but performs poorly on new data. At the same time, a fixed random seed is set during the model training process to ensure the repeatability of the results, facilitate subsequent feature analysis and further improve the model to improve accuracy. Take OCO-2, OCO-3, GOSAT and GOSAT-2 data as examples. The model composition is shown in formula (3):

[0140]

[0141] Among them, Features represents the features selected by the satellite.

[0142] Note: The features must contain at least the priori data and a posteriori data provided by the satellite to ensure the uniformity of the satellite's data correction method. At the same time, feature selection should not contain specific time and location information, which is needed to ensure that the model is optimizing the satellite's own internal algorithm and ensures that it has a good correction effect at any location and time.

[0143] According to the system of the second aspect of the present invention, the fifth processing module 105 is specifically configured as follows: performing the calibration data accuracy check includes:

[0144] The annual average RMSE of the satellite data and the ground-based observation data after data correction is compared with the annual average RMSE before correction. If the comparison result is less than a first preset value, the satellite data after data correction is abandoned, feature selection is performed again, and data correction is performed.

[0145] The re-selection of features comprises:

[0146] Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated multiple times for each feature, and the average value of multiple RMSEs is calculated as a measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0147] Set a change threshold; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0148] The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned.

[0149] Specifically, ground-based verification is performed on each satellite, and the time-space matching conditions of the verification are kept the same as above. Taking GOSAT data as an example, in the above correction process, the GOSAT series satellites with an original resolution of 0.1°×0.2° are corrected to the OCO series satellites to obtain data with a corrected spatial resolution of 0.01°×0.02°, so this type of data needs to be re-matched with the ground-based observation data.

[0150] Compare the ground verification accuracy of the corrected data with that before correction, and use RMSE as a reference indicator. If there is no obvious improvement, abandon the correction result or perform feature selection again to achieve improvement.

[0151] Feature selection was reworked to achieve improvements, including:

[0152] Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated 10 times for each feature, and the average value of the 10 RMSEs is calculated as the measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data;

[0153] Set the change threshold to 0.01ppm; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction;

[0154] The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned. For example, during the correction process of OCO-2, OCO-3, GOSAT and GOSAT-2 data, if it is found that the correction result for OCO2(1) is not good, the fusion of this part of the data is abandoned.

[0155] According to the system of the second aspect of the present invention, the sixth processing module 106 is specifically configured as follows: the calculating of the fusion weight according to the data source RMSE includes:

[0156] The unit weighted mean error is divided by the data source RMSE of a certain satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to a certain satellite data in a certain year.

[0157] The fusing the satellite data in the merged data set according to the fusion weights comprises:

[0158] The fusion weight of each satellite data in the merged data set is used as the corresponding weight to perform weighted averaging on the multiple satellite data in the merged data set.

[0159] Specifically, multi-source satellite data fusion. By year or month, the corrected satellite data, the satellite data not involved in the correction and the ground-based stations are verified for accuracy, the RMSE is calculated as the accuracy index, the mean of all RMSE is taken as the unit weighted mean error, and the weight of each time point is calculated to simplify the fusion process.

[0160] Taking the fusion of OCO-2, OCO-3, GOSAT and GOSAT-2 satellite data as an example, the process is as follows:

[0161] Calculate the accuracy of ground-based verification. Obtain the RMSE of the corrected satellite data (including GOSAT, GOSAT2 and OCO3(1)) and the uncorrected satellite data (including OCO2(0) and OCO3(0), fused to obtain OCO(0)) and ground-based verification in different years.

[0162] Calculate the corresponding weights. Calculate the mean as the unit weighted error, and then calculate the fusion weights of different data in different years.

[0163]

[0164] in, Indicates the weight corresponding to a certain satellite data in a certain year; It represents the RMSE of the data source of a certain satellite data and ground-based observation data in a certain year; Represents the unit weight error.

[0165] Data fusion. For a grid with a global spatial resolution of 0.01°×0.02° at an annual scale, the amount of recorded data from the available data sources may be different. If there is only one data source, its mean is calculated; if there are more than two data sources, a fusion calculation is performed using formula (5).

[0166]

[0167] Among them, Y represents the final XCO2 value of the grid, P S is the weight of the data source, m S is the total number of XCO2 values ​​of the data source in the grid, y S The XCO2 value of the data source.

[0168] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in the method for fusing multi-source satellite data of carbon dioxide column concentration disclosed in any one of the first aspects of the present invention are implemented.

[0169] Figure 3 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 3As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0170] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] The fourth aspect of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for fusing multi-source satellite data of carbon dioxide column concentration disclosed in any one of the first aspects of the present invention are implemented.

[0172] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A method for fusing multi-source satellite data of carbon dioxide column concentration, characterized in that: The method comprises: Step S1, obtaining long-term global multi-source carbon satellite data and ground-based observation data with predefined credibility; Step S2: performing temporal and spatial matching of satellite data and ground-based observation data; selecting label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data; Step S3, using the annual average RMSE before correction as a weight, the selected label data are merged to construct a label data set; the satellite data to be corrected are matched with the label data set in time and space to construct a satellite sample set to be corrected; Step S4, using the training set and feature data of the satellite sample set to be corrected to train the random forest model; using the trained random forest model to perform data correction on the satellite data to be corrected; Step S5, comparing the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction, and performing a correction data accuracy test; Step S6: Merge the corrected satellite data with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

2. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that: In step S2, the conditions for temporal and spatial matching of the satellite data and the ground-based observation data include: In terms of spatial position, data covering ground-based observation sites in satellite data are selected according to the spatial resolution of the satellite; in terms of time scale, ground-based observation data within two hours before and after the satellite transit are selected.

3. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that: In step S3, the step of fusing the selected label data using the average annual RMSE before correction as a weight includes: The weighted average of multiple satellite data is performed using the annual average RMSE of each satellite data before correction as the weight.

4. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that: In step S5, the calibration data accuracy check includes: The annual average RMSE of the satellite data and the ground-based observation data after data correction is compared with the annual average RMSE before correction. If the comparison result is less than a first preset value, the satellite data after data correction is abandoned, feature selection is performed again, and data correction is performed.

5. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 4, characterized in that: In step S5, re-selecting features includes: Each time, a feature is randomly shuffled and input into the model to calculate the RMSE of the test set of the satellite sample set to be corrected; the above operation is repeated multiple times for each feature, and the average value of multiple RMSEs is calculated as a measure of the importance of the feature; the feature whose measure is less than the second preset value is deleted from the input of the random forest model; the feature must be data other than the prior data and the posterior data; Set a change threshold; record the RMSE of a certain satellite data of the test set of the satellite sample set to be corrected in the initial state. If the RMSE change caused by a certain feature is lower than the change threshold, it means that the feature is "unimportant", and the feature is deleted; repeat the above operation until the RMSE effect in the test set is better than the RMSE before correction; The annual average RMSE of the satellite data and the ground-based observation data after the data correction is compared with the annual average RMSE before the correction. If the comparison result is less than a first preset value, the fusion of the satellite data is abandoned.

6. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that: In step S6, calculating the fusion weight according to the data source RMSE includes: The unit weighted mean error is divided by the data source RMSE of a certain satellite data and ground-based observation data in a certain year to obtain the fusion weight corresponding to a certain satellite data in a certain year.

7. The method for fusing multi-source satellite data of carbon dioxide column concentration according to claim 1, characterized in that: In the step S6, fusing the satellite data in the merged data set according to the fusion weight includes: The fusion weight of each satellite data in the merged data set is used as the corresponding weight to perform weighted averaging on the multiple satellite data in the merged data set.

8. A multi-source satellite data fusion system for carbon dioxide column concentration, characterized in that: The system comprises: The first processing module is configured to obtain long-term global multi-source carbon satellite data and ground-based observation data with a predefined credibility; The second processing module is configured to perform temporal and spatial matching of satellite data and ground-based observation data; select label data according to the pre-correction annual average RMSE of the matched satellite data and ground-based observation data; The third processing module is configured to fuse the selected label data with the annual average RMSE before correction as a weight to construct a label data set; perform time-space matching between the satellite data to be corrected and the label data set to construct a satellite sample set to be corrected; The fourth processing module is configured to train a random forest model using the training set and feature data of the satellite sample set to be corrected; and perform data correction on the satellite data to be corrected using the trained random forest model; A fifth processing module is configured to compare the annual average RMSE of the satellite data and the ground-based observation data after data correction with the annual average RMSE before correction to perform a correction data accuracy test; The sixth processing module is configured to merge the satellite data after data correction with the label data to obtain a merged data set; calculate the data source RMSE of each type of satellite data and ground-based observation data in the merged data set; calculate the fusion weight according to the data source RMSE; and fuse the satellite data in the merged data set according to the fusion weight.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the method for fusing multi-source satellite data of carbon dioxide column concentration described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for fusing multi-source satellite data of carbon dioxide column concentration according to any one of claims 1 to 7 are implemented.

Images