A spatio-temporal data fusion method and system based on a DFSTVI model

Abstract

The application discloses a kind of spatio-temporal data fusion method and system based on DFSTVI model, it is related to remote sensing image processing field, this method is based on the " conversion " thought of low spatial resolution to high spatial resolution similarity and the " regression " thought between different time of continuous geographical space phenomenon, propose a kind of efficient data fusion method DFSTVI;With high temporal resolution, low spatial resolution MODIS and low temporal resolution, high spatial resolution Landsat-8 satellite NDVI data product as example is fused and handled;When data fusion, with Sentinel satellite high-resolution land cover type data as the search basis of similar pixel, the result shows that, high-resolution land cover type data is more convenient and accurate for the search of similar pixel, and after being combined with DFSTVI, the accuracy and smoothness of fusion result can be effectively improved.The application has wide application prospect in many fields, and brings new innovation and development for remote sensing data fusion field.

Description

(I) Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a remote sensing image fusion method and system based on weighting functions. (II) Background Technology

[0002] In recent years, with the rapid development of remote sensing technology and the widespread application of high-resolution satellite data, the fields of land surface dynamic monitoring and ecological environment research have ushered in new opportunities. In particular, high spatiotemporal resolution satellite imagery provides crucial information for land cover type classification, vegetation monitoring, and dynamic assessment of the ecological environment. While high spatial resolution data can yield rich details about the land surface, its long temporal resolution, coupled with the influence of ground meteorological factors (clouds, rain, and snow) on sensor imaging, results in numerous invalid values ​​in the monitoring data. This leads to "gaps" in the basic observational data for long-term observational studies, making it impossible to support continuous dynamic monitoring research. Against this backdrop, taking vegetation indices, the most widely used in remote sensing field, as an example, vegetation indices, as important indicators reflecting vegetation growth and ecosystem health, play a crucial role in research and decision support in agriculture, forestry, and the ecological environment. Traditional low spatial resolution normalized vegetation index products, while possessing high temporal resolution, are far from sufficient in representing land surface details. At this point, spatiotemporal fusion technology based on multi-source data can solve the problem of simultaneously achieving high spatiotemporal resolution, becoming one of the current research hotspots.

[0003] Spatiotemporal data fusion, as a method to obtain high-quality and high-resolution surface information by combining multi-source remote sensing data at different temporal and spatial scales, provides an effective approach to solving the aforementioned problems. Currently, spatiotemporal fusion methods for remote sensing data can be broadly classified into three categories: unmixing-based models, learning-based models, and weight-based models. Unmixing-based methods, based on linear spectral mixing theory, unmix coarse-resolution image pixels according to the classification categories of the fine-resolution image, and then redistribute the contribution value of each category within the mixed pixels to the fine-resolution image pixels. Learning-based methods use machine learning algorithms to simulate the relationship between observed coarse-fine image pairs and then predict unobserved fine images. Among weight-based methods, the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is the most representative. This method assumes that all pixels in the fine-resolution image are pure pixels, meaning that the reflectance changes are consistent under both coarse and fine resolution. In this case, a weighted strategy is used to incorporate the reflectance changes between two coarse-resolution images into the prior fine-resolution image, thereby achieving prediction. Although there are many spatiotemporal data fusion methods, each has its own advantages and disadvantages, and it is impossible to simultaneously guarantee the accuracy of heterogeneous regions and the overall smoothness of the image. (III) Summary of the Invention

[0004] The purpose of this invention is to provide an efficient remote sensing spatiotemporal data fusion method and system based on the DFSTVI (Data Fusion of Spatial Temporal Vegetation Index) model. Under the condition of ensuring corresponding input data types and consistent data dimensionality, it can improve the temporal resolution of high spatial resolution images. The method of this invention is integrated as an extension module into the remote sensing image processing software Envi Classic to form a spatiotemporal data fusion system.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A spatiotemporal data fusion method and system based on the DFSTVI model, characterized by comprising:

[0007] (1) Acquire MODIS and Landsat imagery data of the study area; as well as land cover data products from Sentinel satellite;

[0008] (2) Preprocess the above remote sensing image data;

[0009] (3) Input the preprocessed remote sensing image data into the spatiotemporal data fusion model to obtain the fused predicted image; the spatiotemporal data fusion model is a coupling model proposed based on the idea of ​​"conversion" from low spatial resolution to high spatial resolution and the idea of ​​"regression" between different consecutive time periods;

[0010] Preferably, acquiring the remote sensing image data specifically includes:

[0011] (1) Data of sur_refl_b01 red band and sur_refl_b02 near-infrared band in MOD09GQ product of MODIS satellite;

[0012] (2) SR_B4 red band and SR_B4 near-infrared band data from Landsat-8 satellite;

[0013] (3) Land cover type products with a spatial resolution of 10m in 2020 based on data from Sentinel-1 and Sentinel-2 satellites by the European Space Agency.

[0014] Preferably, the remote sensing image data is preprocessed, specifically including:

[0015] (1) Cloud removal processing: Based on the principle that the cloud cover is less than 10%, the MODIS and Landsat-8 satellite reflectivity data products are subjected to cloud removal processing;

[0016] (2) Data filtering: Based on the study area and date, images with more complete reflectance data from MODIS and Landsat-8 satellites were filtered;

[0017] (3) NDVI Calculation: NDVI is calculated based on the red and near-infrared bands of MODIS and Landsat-8 satellite reflectivity data products. The calculation formula is as follows:

[0018] Where NIR represents the near-infrared reflectance data and R represents the red reflectance data;

[0019] (4) Maximum value synthesis: The average value of the NDVI image for this month is synthesized based on the maximum value of the NDVI image for each month; and named M1, L1, M2, L2, M3, L3 respectively, where M and L represent MODIS low spatial resolution image and Landsat-8 high spatial resolution image respectively; 1, 2, 3 represent the first month, the second month and the third month of three consecutive months respectively;

[0020] (5) Resampling and cropping: Based on a minimum pixel size of 30m, spatial resampling was performed on NDVI and land cover data products respectively, and cropping was performed using the study area as a mask.

[0021] Preferably, a coupled model is proposed based on the idea of ​​a similar "transformation" from low spatial resolution to high spatial resolution and the idea of ​​"regression" between different consecutive moments of geospatial phenomena, specifically including:

[0022] (1) Regression: The regression coefficients R are calculated using images M1, M2, and M3. 21 and R 23 ;

[0023] Then, the mean NDVI of the MODIS satellite in the second month was predicted using regressions of M1 and M3, respectively, to obtain M. 21 and M 23 ; and perform a difference operation with the true image M2 to obtain errors EM respectively. 21 and EM 23 The regression coefficients are multiplied by the high spatial resolution L1 and L3 to obtain the Landsat satellite's predicted imagery for the second month, which is L... 21 and L 23 ;

[0024] (2) Conversion: The conversion coefficients of the M1, M3, L1 and L3 images are calculated to obtain the conversion coefficient V;

[0025] Regression coefficients and EM 21 and EM 23 The calculation is performed to convert the error from low spatial resolution to high spatial resolution, and the error EL is obtained. 21 and EL 23 ;

[0026] (3) Filtering: For L 21 L 23 EL 21 and EL 23 The imagery is predicted by searching for other pixels similar to the center pixel of the sliding window, using a sliding window as the range and land cover data products as a reference. A weight function for similar pixels is calculated based on the distance, and filtering is performed based on the weight function.

[0027] (4) Error compensation: The filtered L 21 and L 23 Error EL after filtering 21 and EL 23 The images are added together, and then averaged to obtain the final average NDVI predicted image from the Landsat-8 satellite for the second month.

[0028] Preferably, when using land cover data products as a reference and a sliding window as the search for similar pixels, the land cover type classification is set to 11 and the sliding window radius is set to 13.

[0029] Corresponding to the above-mentioned spatiotemporal data fusion method, this invention also provides a spatiotemporal data fusion extension system based on the remote sensing processing software EnviClassic, comprising:

[0030] (1) Data fusion model selection: This module encapsulates six algorithms for users to choose from: STARFM, ESTARFM, ESTARFM_LC, FIT-FC, FIT-FC_LC and DFSTVI; the suffix LC indicates that land cover data products are referenced in the search of similar pixels; the DFSTVI algorithm is the coupling model written in this patent.

[0031] (2) Data fusion processing module: Input the pre-processed data as prompted to obtain the final fused image;

[0032] The advantages of this invention compared to the prior art are:

[0033] In the field of remote sensing imagery, multi-source remote sensing images of the same scene observe the same ground objects, but differ in observation dimensions, resulting in variations in spatial, spectral, and temporal resolutions. The information provided is both redundant and complementary, exhibiting collaborative characteristics. This invention addresses NDVI parameters by utilizing the innovative DFSTVI algorithm for data fusion, achieving high temporal and spatial resolution NDVI imagery. Compared to other existing technologies, this invention achieves higher accuracy, better image smoothness, and more efficient image fusion calculations. (iv) Description of the attached drawings

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart of the DFSTVI spatiotemporal data fusion method provided by this invention;

[0036] Figure 2 A schematic diagram of preprocessed MODIS and Lansat-8 satellite NDVI data;

[0037] Figure 3 This is a diagram illustrating the NDVI forecast data for February 2019.

[0038] Figure 4 This is a density scatter plot of NDVI prediction data and original data provided in the embodiments of this method.

[0039] Figure 5This is a schematic diagram of the Envi Classic extended system interface provided in an embodiment of the present invention. (V) Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The purpose of this invention is to provide an efficient remote sensing spatiotemporal data fusion method based on the DFSTVI model and an extended Envi Classic system, which can improve the temporal resolution of high spatial resolution images while ensuring that the input data types are corresponding and the data dimensions are consistent.

[0042] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the spatiotemporal data fusion method provided by the present invention includes:

[0044] Step 1: Acquire remote sensing image data of the study area. The study demonstration area is located in the southern part of Pu'er City, Yunnan Province, China, including Simao District of Pu'er City and Ning'er Hani and Yi Autonomous County. It is bordered by the Wuliang Mountains to the north and a branch of the Lancang River to the west, covering an area of ​​approximately 2128.3 km². 2 (22°41'48”~23°5'49”, 100°41'5”~101°8'37”). Remote sensing image data includes high-resolution and low-resolution remote sensing data. For example, Landsat-8 image data is used as the source of high-resolution remote sensing data, and MOD09GQ data from MODIS satellite imagery is used as the source of low-resolution remote sensing data. The spatial resolution of Landsat-8 data is 30m, and the spatial resolution of MOD09GQ data is 250m. In this embodiment of the invention, red and near-infrared band data are selected. In addition, the 2020 10m land cover type product based on Sentinel-1 and Sentinel-2 satellite data from the European Space Agency (ESA) is also required.

[0045] Step 2: Preprocess the remote sensing image data, including cloud removal, data filtering, NDVI calculation, maximum value synthesis, resampling, and cropping. Cloud removal is performed on MODIS and Landsat-8 satellite reflectivity data products based on cloud cover less than 10%. Data filtering involves selecting images with relatively complete MODIS and Landsat-8 satellite reflectivity data based on the study area and date. NDVI calculation is performed on the infrared and near-infrared bands of the MODIS and Landsat-8 satellite reflectivity data products. Maximum value synthesis involves synthesizing the average NDVI value for each month based on the maximum NDVI values ​​of the images, and naming them M1, L1, M2, L2, M3, and L3, where M and L represent MODIS low-spatial-resolution images and Landsat-8 high-spatial-resolution images, respectively, and 1, 2, and 3 represent the first, second, and third months of a consecutive three-month period, respectively. Resampling and cropping were performed on NDVI and land cover data products, with a minimum pixel size of 30m. The data was then cropped using the study area as a mask. Figure 2 As shown. All preprocessing steps are performed on the Google Earth Engine (GEE) cloud computing platform.

[0046] Step 3: Input the preprocessed NDVI data into the spatiotemporal data fusion model to obtain the predicted image. In this embodiment, high and low spatial resolution data from the first and third months are used as input data, and the low spatial resolution data from the second month is used as a reference to predict the high spatial resolution data for the current month. Specifically, this includes four stages: regression, transformation, filtering, and error compensation. Regression involves calculating regression coefficients using the M1, M2, and M3 images to obtain the regression coefficient R0. 21 and R 23 Then, the mean NDVI of the MODIS satellite in the second month was predicted using regressions with M1 and M3 respectively, to obtain M. 21 and M 23 The difference between the image and the true image M2 is then calculated, yielding errors EM. 21 and EM 23 The regression coefficients are multiplied by the high spatial resolution L1 and L3 to obtain the Landsat satellite's predicted imagery for the second month, denoted as L... 21 and L 23 The transformation involves calculating transformation coefficients v from M1, M3, L1, and L3 imagery. The regression coefficients are then compared with the EM... 21 and EM 23 The calculation is performed to convert the error from low spatial resolution to high spatial resolution, and the error EL is obtained. 21 and EL 23 Filtering, for L 21 L23 EL 21 and EL 23 The predicted imagery uses a sliding window as its scope and land cover data products as a reference to search for other pixels similar to the center pixel of the sliding window. A weight function is calculated based on the distance between these similar pixels, and filtering is performed according to this weight function. Error compensation is then applied to the filtered L... 21 and L 23 Error EL after filtering 21 and EL 23 The values ​​are added together, and finally averaged to obtain the final Landsat-8 satellite mean NDVI predicted image for the second month, as shown below. Figure 3 As shown. In this embodiment, when using land cover data products as a reference and a sliding window as the process for searching similar pixels, the land cover type classification is set to 11, and the sliding window radius is set to 13;

[0047] The DFSTVI spatiotemporal data fusion method proposed in this invention will be tested below. As shown in Table 1, the statistical evaluation results of the root mean square error (RMSE) and correlation coefficient (R) between the prediction results obtained using the DFSTVI method provided by this invention and the actual image are obtained, and a comparative analysis is performed with the ESTARFM method and the FIT-FC method. Combined with... Figure 4 It can be seen that the DFSTVI spatiotemporal fusion method provided by this invention is superior in both statistical indicators, with reduced root mean square error and a correlation coefficient reaching 0.886. Figure 2 The comparison results between the second month's real images show that, from a visual perspective, the fused result is very close to the real image. This method can reconstruct the spatial details of heterogeneous regions, has a good smoothing effect in image fusion, high prediction accuracy, and relatively stable overall performance.

[0048] Table 1 shows the statistical evaluation results of the root mean square error (RMSE) and correlation coefficient (R) between the prediction results obtained by the DFSTVI method, ESTARFM method, and FIT-FC method provided in this invention and the actual images.

[0049]

[0050] In addition, corresponding to the spatiotemporal data fusion method provided above, the present invention also provides the following implementation structure:

[0051] A spatiotemporal data fusion extension system based on the remote sensing image processing software Envi Classic, such as Figure 5As shown, the system includes an algorithm selection module and a data fusion module. The algorithm selection module encapsulates six algorithms for users to choose from: STARFM, ESTARFM, ESTARFM_LC, FIT-FC, FIT-FC_LC, and DFSTVI. The suffix LC indicates that land cover data products were referenced in the search for similar pixels. The DFSTVI algorithm is a coupling model developed in this patent. In the data fusion module, the preprocessed data is input as prompted to obtain the final fused image.

[0052] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A spatiotemporal data fusion method based on the DFSTVI model, characterized in that... Includes the following steps: Step 1: Acquire low spatial resolution and high temporal resolution satellite imagery and high spatial resolution and low temporal resolution satellite imagery data of the study area; as well as land cover data products based on high-resolution satellites; acquire remote sensing imagery data, specifically including: (1) Red band and near-infrared band data of satellite data products with low spatial resolution and high temporal resolution; (2) Red band and near-infrared band data from satellites with high spatial resolution and low temporal resolution; (3) High spatial resolution land cover type data; Step 2: Preprocess the above remote sensing image data, specifically including: (1) Cloud removal processing: Based on the set cloud amount threshold principle, cloud removal is performed on satellite imagery with low spatial resolution and high temporal resolution and satellite data products with high spatial resolution and low temporal resolution. (2) Data filtering: Based on the study area and date, filter satellite images with low spatial resolution and high temporal resolution and images with relatively complete satellite data with high spatial resolution and low temporal resolution; (3) NDVI Calculation: NDVI is calculated based on the outer red and near-infrared bands of satellite imagery with low spatial resolution and high temporal resolution, and satellite reflectivity data products with high spatial resolution and low temporal resolution. The calculation formula is as follows: ; Where NIR is the near-infrared reflectance data, R is the red reflectance data, and NDVI is the normalized vegetation index. (4) Maximum value synthesis: The maximum value of the NDVI image for each month is synthesized into the average value of the NDVI image for that month; and named M1, L1, M2, L2, M3, L3 respectively, where M and L represent satellite images with low spatial resolution and high temporal resolution and satellite images with high spatial resolution and low temporal resolution respectively; 1, 2, 3 represent the first month, the second month, and the third month of the three consecutive months of time series data respectively; (5) Resampling and cropping: Based on the minimum pixel, spatial resampling is performed on NDVI and land cover data products respectively, and cropping is performed using the study area as a mask; Step 3: Input the preprocessed remote sensing image data into the spatiotemporal data fusion model DFSTVI to obtain the fused predicted image; the spatiotemporal data fusion DFSTVI model is a coupled model proposed based on the "conversion" idea from low spatial resolution to high spatial resolution and the "regression" idea between continuous different time periods, and specifically includes the following steps: (1) Regression: The regression coefficients R are calculated using images M1, M2, and M3. 21 and R 23 Then, the mean NDVI values ​​of low spatial resolution and high temporal resolution satellites for the second month were predicted using regressions with M1 and M3 respectively, to obtain M. 21 and M 23 ; and perform a difference operation with the true image M2 to obtain errors EM respectively. 21 and EM 23 The regression coefficients are multiplied by the high spatial resolution L1 and L3 to obtain the second-month predicted imagery from the satellite with high spatial resolution and low temporal resolution, denoted as L... 21 and L 23 The formula for calculating the regression coefficient is: M1, M2, and M3 represent the average NDVI values ​​of low spatial resolution and high temporal resolution satellites for the first, second, and third months of the three consecutive months, respectively. 21 and R 23 These are the regression coefficients for two consecutive months within a three-month period; (2) Transformation: The transformation coefficient V is calculated using the M1, M3, L1, and L3 images; the regression coefficient is then compared with the EM... 21 and EM 23 The calculation is performed to convert the error from low spatial resolution to high spatial resolution, and the error EL is obtained. 21 and EL 23 The formula for calculating the conversion factor is: Where k represents the number of endmembers, M1 and M3 represent the average NDVI values ​​of low spatial resolution and high temporal resolution satellites in the first and third months of the three consecutive months, respectively, and L 1k and L 3k These represent the mean NDVI values ​​of low spatial resolution and high temporal resolution satellites for the first and third months of a consecutive three-month period, respectively. The spatial transformation coefficient is the sum of the first and third months of a three-month period. (3) Filtering: For L 21 L 23 EL 21 and EL 23 For image prediction, a suitable sliding window is selected as the radius. Using the sliding window as the range and land cover data products as a reference, other pixels similar to the center pixel of the sliding window are searched. A weight function for similar pixels is calculated based on their distance, and filtering is performed according to the weight function. The formula for the weight function is: Where i, j, and k represent the pixel in row i and column j at time k, and w / 2 is the center position of the sliding window; (4) Error compensation: The filtered L 21 and L 23 Error EL after filtering 21 and EL 23 The images are added together, and then averaged to obtain the final average NDVI predicted image from the Landsat-8 satellite for the second month.

2. A spatiotemporal data fusion system based on the DFSTVI model, used to implement the spatiotemporal data fusion method based on the DFSTVI model as described in claim 1, characterized in that, include: (1) Data fusion model selection module: The algorithm module is integrated into the ENVI Classic software system. This module encapsulates six algorithms for users to choose from: STARFM, ESTARFM, ESTARFM_LC, FIT-FC, FIT-FC_LC and DFSTVI; the suffix LC indicates that land cover data products were referenced in the search of similar pixels. (2) Data fusion processing module: The pre-processed data is input with prompts, and the final fused image can be obtained through one-click processing.

Images