A cloud computing spatio-temporal fusion method considering the error correction of coarse-to-fine resolution scale conversion

Abstract

The application discloses a cloud computing space-time fusion method considering coarse and fine resolution scale conversion error correction, which comprises the following steps: synthesizing a cloud-free reference image pair based on the principles of time consistency and space similarity, and determining a coarse resolution remote sensing image and a fine resolution remote sensing image at a predicted time; expressing the spatial heterogeneity of the earth's surface; calculating a scale conversion error term; calculating a residual term; correcting the scale conversion error term and the residual term based on the result of superpixel segmentation, and compensating to the fine resolution remote sensing image in the cloud-free reference image pair, so as to obtain a final fine resolution remote sensing image fusion result. The application improves the fusion precision of heterogeneity and changing landscapes by introducing the scale conversion error term; stable fusion results can still be obtained even in the scene of thick cloud and continuous cloud pollution; the method is not only suitable for the reconstruction of NDVI time series, but also suitable for the fusion of multispectral reflectivity.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of spatio-temporal image fusion methods, specifically, a kind of cloud computing spatio-temporal fusion (STESTF) method considering coarse and fine resolution scale conversion error correction, belong to time series remote sensing image fusion technical field. BACKGROUND

[0002] Generally speaking, after more and more remote sensing satellites are launched into space, the number of remote sensing data sets of different characteristics (such as spectrum, space, time) obtained by different sensors will significantly increase. However, due to sensor hardware and weather factors, a single sensor only has high spatial resolution or high temporal resolution characteristics, and it is difficult to obtain remote sensing data with high temporal and spatial resolution at the same time.

[0003] Specifically, dense high spatio-temporal resolution data is crucial for studying dynamic changes of the earth's surface, and contains time-varying information of surface objects, which plays an irreplaceable role in identifying land cover types, detecting vegetation growth, crop yield estimation, and studying urban expansion. However, desktop processing software cannot meet the application requirements of fusion, processing and analysis of massive remote sensing data sets, and how to process and analyze these massive remote sensing data is also a challenging task.

[0004] Therefore, how to quickly and efficiently fuse images from multiple sensors to generate high-quality dense time series data has become an urgent task for fine observation of the earth. SUMMARY

[0005] The purpose of the present application is to address the fact that existing STF methods generally ignore the scale error from coarse resolution upsizing to fine resolution. Therefore, a spatio-temporal fusion computing method (STESTF) with coarse and fine resolution scale conversion error correction is proposed, based on the basic principles of spatio-temporal fusion and the characteristics of the cloud computing remote sensing platform, to solve the bottleneck problem of low fusion accuracy in the presence of continuous cloud pollution.

[0006] The present application achieves the above-mentioned purpose by the following technical solution: a cloud computing spatio-temporal fusion method considering coarse and fine resolution scale conversion error correction, comprising the following steps:

[0007] Step S1, based on the time consistency and spatial similarity principle, synthesize a pair of cloud-free reference images from the time series remote sensing images of the working area, and determine the coarse resolution remote sensing image and fine resolution remote sensing image at the predicted time,

[0008] Step S2, the fine resolution remote sensing image in the cloud-free reference image pair is superpixel segmented to express the spatial heterogeneity of the earth's surface;

[0009] Step S3, determining conversion coefficients according to the cloud-free reference image pair, and calculating a scale conversion error term according to the coarse resolution remote sensing image in the cloud-free reference image pair and the coarse resolution remote sensing image at the predicted time;

[0010] Step S4, calculating a residual term according to the conversion coefficients and the coarse resolution remote sensing image at the reference time and the coarse resolution remote sensing image at the predicted time and the fine resolution remote sensing image;

[0011] Step S5, correcting the scale conversion error term and the residual term based on the result of the superpixel segmentation, and compensating to the fine resolution remote sensing image in the cloud-free reference image pair, so as to obtain a final fine resolution remote sensing image fusion result.

[0012] As a further scheme of the present application: the step S1 specifically comprises:

[0013] S11 selecting a fine resolution remote sensing image to be predicted in a work area, and obtaining surface reflectivity of the time sequence fine resolution remote sensing image and the coarse resolution remote sensing image in the work area;

[0014] S12 synthesizing a cloud-free image by using the time consistency principle based on the surface reflectivity of the time sequence fine resolution remote sensing image;

[0015] S13 synthesizing a cloud-free image by using the time consistency principle based on the surface reflectivity of the time sequence coarse resolution remote sensing image;

[0016] S14 obtaining two pairs of coarse and fine resolution cloud-free remote sensing images before and after the predicted time by using the time consistency principle;

[0017] S15 calculating a structural similarity index between the coarse resolution remote sensing image at the predicted time and the coarse resolution remote sensing image in the cloud-free remote sensing image pair;

[0018] S16 selecting the cloud-free remote sensing image pair with a higher structural similarity index in S15 as the cloud-free reference image pair;

[0019] As a further scheme of the present application: the step S2 specifically comprises:

[0020] S21 dividing the fine resolution remote sensing image in the cloud-free reference image pair into a plurality of grids with equal size by using a superpixel segmentation method, and taking each grid center as an initial clustering center;

[0021] S22 calculating image color space distance and Euclidean distance of pixel position of 8 neighborhood pixels of each initial clustering center;

[0022] S23 each neighborhood pixel inputs a priority queue Q, each element in the priority queue Q is sorted by the distance from its initial clustering center, thereby popping out the neighborhood pixel with the minimum distance, and meanwhile assigning the neighborhood pixel with the minimum distance with label information and updating the feature mean value of the pixels with the same label;

[0023] S24 until the queue is empty and all the pixels are labeled, obtaining a label map of the superpixels, thereby expressing the spatial heterogeneity of the ground surface.

[0024] As a further scheme of the present application, the step S3 specifically comprises:

[0025] S31 the spatial resolution of the cloud-free coarse-resolution remote sensing image in the cloud-free reference image pair is resampled, and the spatial resolution is an integer multiple of the spatial resolution of the fine-resolution remote sensing image in the cloud-free reference image pair;

[0026] S32 the resampled cloud-free coarse-resolution remote sensing image is subjected to cubic convolution interpolation, so that the spatial resolution of the cloud-free coarse-resolution remote sensing image is consistent with the spatial resolution of the fine-resolution remote sensing image in the cloud-free reference image pair;

[0027] S33 the cloud-free coarse-resolution remote sensing image subjected to the cubic convolution interpolation is divided by the fine-resolution remote sensing image in the cloud-free reference image pair, so as to obtain the conversion coefficient between the coarse-resolution remote sensing image and the fine-resolution remote sensing image;

[0028] S34 the coarse-resolution remote sensing image in the cloud-free reference image pair is subjected to cubic convolution interpolation and then multiplied by the conversion coefficient;

[0029] S35 the coarse-resolution remote sensing image at the predicted time is subjected to cubic convolution interpolation and then multiplied by the conversion coefficient

[0030] S36 the result of S35 is subtracted from the result of S34, so as to obtain the error of the conversion coefficient between the reference time and the predicted time, i.e. the scale conversion error term.

[0031] As a further scheme of the present application, the step S4 specifically comprises:

[0032] S41 the conversion coefficient in step S3 is multiplied by the resampled cloud-free coarse-resolution remote sensing image at the predicted time,

[0033] S42 the fine-resolution remote sensing image in the cloud-free reference image pair is subtracted from S41, so as to obtain the change information of the predicted image and the reference image, and then the change information is down-scaled to the coarse resolution;

[0034] S43 the coarse-resolution remote sensing image at the predicted time is subtracted from the coarse-resolution remote sensing image in the cloud-free reference image pair, so as to obtain the real change information of the ground object at the predicted time and the reference time;

[0035] S44 subtracts the result of S42 from the result of S43 to obtain a residual term of the fusion result;

[0036] S45 resamples the residual term to a fine spatial resolution, i.e., a residual in time of the spatio-temporal fusion result.

[0037] As a further scheme of the present application, the step S5 specifically comprises:

[0038] S51 corrects the scale conversion error term and the residual term based on the superpixel segmentation result in step S2, since the pixel-based prediction inevitably has many uncertainties in the heterogeneous space;

[0039] S52 the conversion coefficient error determined in step S3 is caused by the change of the ground surface at the reference time and the prediction time, and the scale conversion error term is spatially weighted by using the clustering unit of the superpixel, assuming that the pixels in the same superpixel have consistent temporal changes;

[0040] S53 the residual term in step S4 is caused by the change of the ground surface at the reference time and the prediction time, and the residual term is spatially weighted by using the clustering unit of the superpixel, assuming that the pixels in the same superpixel have consistent temporal changes;

[0041] S54 adds the scale conversion error term and the residual term after spatial weighting to obtain the change of the fine spatial resolution image from the reference time to the prediction time;

[0042] S55 adds the change of the image from the reference time to the prediction time to the synthetic cloud-free coarse resolution remote sensing image at the reference time to finally obtain the fine resolution remote sensing image at the prediction time, i.e., the final fusion result.

[0043] The present application has the following beneficial effects:

[0044] 1) Based on the principles of temporal consistency and spatial similarity, a reference image pair synthesis strategy at the pixel level is proposed, which solves the input image selection problem under continuous missing values;

[0045] 2) The present application extracts the scale conversion error between the coarse and fine resolutions. Unlike the residual term, the scale conversion error reflects the error of the coarse resolution downscaling to the fine resolution, which includes the error generated by interpolation and the error of the conversion coefficient caused by the land cover type at the prediction time. The introduction of the scale conversion error improves the prediction accuracy of the heterogeneous surface;

[0046] 3) The surface heterogeneity expressed by superpixel segmentation results is used as a spatial weight to correct residual error terms and scale conversion error terms, generating high-quality dense time series data to provide technical support for surface observation tasks; because it can be implemented on a cloud platform, stable fusion results can still be obtained even in thick cloud and continuous cloud pollution scenarios, making it not only suitable for NDVI time series reconstruction, but also suitable for multispectral reflectance fusion. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the present application is shown in the figure.

[0048] Figure 2 The base time and prediction time of the present application are shown in the figure. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0050] As shown in the figure, C1 and F1 are a pair of coarse and fine resolution base images synthesized according to T1 time, wherein T1 is the base time; C2 is a coarse resolution remote sensing image corresponding to T2 time, wherein T2 is the prediction time, and F2 is the fine resolution remote sensing image to be predicted. Figure 2

[0051] Embodiment one

[0052] A cloud computing spatio-temporal fusion method considering coarse and fine resolution scale conversion error correction, comprising the following steps:

[0053] Step S1: Synthesize a pair of cloud-free base images based on the principles of time consistency and spatial similarity through the remote sensing images of the time series in the working area, and determine the coarse resolution remote sensing image and the fine resolution remote sensing image at the prediction time.

[0054] ​Specifically: S11 selects a subdivided resolution remote sensing image to be predicted in a work area, and obtains surface reflectivity of the subdivided resolution remote sensing image and a coarse resolution remote sensing image in a time sequence of the work area; S12 synthesizes a cloud-free image based on the surface reflectivity of the subdivided resolution remote sensing image in the time sequence of the work area by using a time consistency principle; S13 synthesizes a cloud-free image based on the surface reflectivity of the coarse resolution remote sensing image in the time sequence of the work area by using the time consistency principle; S14 obtains two pairs of cloud-free remote sensing images of coarse and subdivided resolutions before and after a prediction time by using the time consistency principle; S15 calculates a structural similarity index between the coarse resolution remote sensing image at the prediction time and a coarse resolution remote sensing image in the pair of cloud-free remote sensing images; and S16 selects the pair of cloud-free remote sensing images with a higher structural similarity index in S15 as a cloud-free reference image pair.

[0055] Step S2: performing superpixel segmentation on the subdivided resolution remote sensing image in the cloud-free reference image pair, so as to express spatial heterogeneity of the ground surface as spatial weights of different heterogeneous surfaces.

[0056] Specifically: S21 divides the subdivided resolution remote sensing image in the cloud-free reference image pair into a plurality of grids of equal size by using a superpixel segmentation method, and each grid center is an initial clustering center; S22 calculates image color space distance and Euclidean distance of pixel positions of eight neighborhood pixels of each initial clustering center; S23 inputs each neighborhood pixel into a priority queue Q, each element in the priority queue Q is sorted by distance from the initial clustering center, so that the neighborhood pixel with the minimum distance is popped out, and the neighborhood pixel with the minimum distance is assigned with label information and the feature mean value of the pixels with the same label is updated; and S24 until the queue is empty and all pixel points are labeled, a label map of the superpixel is obtained, so as to express spatial heterogeneity of the ground surface.

[0057] Step S3: determining a conversion coefficient according to the cloud-free reference image pair, and calculating a scale conversion error term between a reference time and the prediction time according to the coarse resolution remote sensing image in the cloud-free reference image pair and the coarse resolution remote sensing image at the prediction time.

[0058] Specifically comprising: S31 resampling the spatial resolution of the cloud-free coarse resolution remote sensing image in the cloud-free reference image pair, and the spatial resolution is an integer multiple of the resolution of the fine resolution remote sensing image in the cloud-free reference image pair (if the spatial resolution of the coarse resolution remote sensing image is 250 meters, and the spatial resolution of the fine resolution remote sensing image is 30 meters, then the resampling of the general coarse resolution remote sensing image is 240 meters); S32 performing cubic convolution interpolation on the resampled cloud-free coarse resolution remote sensing image to make the spatial resolution consistent with that of the fine resolution remote sensing image in the cloud-free reference image pair; S33 dividing the cloud-free coarse resolution remote sensing image after cubic convolution interpolation by the fine resolution remote sensing image in the cloud-free reference image pair to obtain the conversion coefficient between the coarse and fine resolution remote sensing images; S34 performing cubic convolution interpolation on the coarse resolution remote sensing image in the cloud-free reference image pair and multiplying the result by the conversion coefficient; S35 performing cubic convolution interpolation on the coarse resolution remote sensing image at the predicted time and multiplying the result by the conversion coefficient; S36 subtracting the result of S35 from the result of S35 to obtain the error of the conversion coefficient between the reference time and the predicted time, which is the scale conversion error term.

[0059] Step S4: calculating the residual term according to the conversion coefficient and the coarse and fine resolution remote sensing images at the reference time and the coarse resolution remote sensing image at the predicted time.

[0060] Specifically: S41 multiplying the conversion coefficient in step S3 by the resampled cloud-free coarse resolution remote sensing image at the predicted time, S42 subtracting the fine resolution remote sensing image in the cloud-free reference image pair from the result of S41 to obtain the change information of the predicted image and the reference image, and then downscaling the change information to coarse resolution; S43 subtracting the coarse resolution remote sensing image in the cloud-free reference image pair from the coarse resolution remote sensing image at the predicted time to obtain the real change information of the ground object at the predicted time and the reference time; S44 subtracting the result of S42 from the result of S43 to obtain the residual term of the fusion result; S45 resampling the residual term to fine spatial resolution, which is the residual in time of the spatio-temporal fusion result.

[0061] Step S5: correcting the scale conversion error term and the residual term based on the result of superpixel segmentation, and compensating to the fine resolution remote sensing image in the cloud-free reference image pair to obtain the final fine resolution remote sensing image fusion result.

[0062] Specifically: S51 has many uncertainties in the heterogeneous space based on pixel-by-pixel prediction, and the scale conversion error term and the residual term are corrected using the results of superpixel segmentation in step S2; S52 the error of the conversion coefficient determined in step S3 is caused by the change of the ground surface at the reference time and the prediction time, assuming that the pixels in the same superpixel have consistent time variation, the scale conversion error term is spatially weighted using the clustering unit of the superpixel; S53 the residual term in step S4 is caused by the change of the ground surface at the reference time and the prediction time, therefore, assuming that the pixels in the same superpixel have consistent time variation, the residual term is spatially weighted using the clustering unit of the superpixel; S54 the scale conversion error term and the residual term after spatial weighting are added to obtain the change of the image at the reference time to the prediction time at the fine spatial resolution; S55 the change of the image at the reference time and the prediction time is added to the cloud-free coarse resolution remote sensing image synthesized at the reference time, and finally the fine resolution remote sensing image at the prediction time is obtained, which is the final fusion result.

[0063] Embodiment three

[0064] A cloud computing spatio-temporal fusion method considering coarse and fine resolution scale conversion error correction, the fusion method comprising:

[0065] Step S1, collecting the fine resolution remote sensing image to be predicted in the operation area, obtaining the time sequence Landsat and MODIS surface reflectance image of the operation area; according to the principle of the nearest time to the prediction image time, two pairs of cloud-polluted remote sensing images are synthesized using the images containing cloud pollution before and after the prediction time; the structural similarity between the MOD09Q1 data in the two pairs of reference images and the MOD09Q1 data at the prediction time is calculated, and the remote sensing image with the maximum structural similarity is selected as the reference image; here, the Landsat-8 satellite is equipped with Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS). It covers different wavelength ranges from infrared to visible light; here, the MOD09Q1 product provides an estimate of the surface spectral reflectance of the near-infrared and red bands at a resolution of 250 meters, and is corrected according to atmospheric conditions such as gas, aerosol and Rayleigh scattering;

[0066] Step S2, based on the reference image, superpixel segmentation is performed using the synthesized cloud-free Landsat-8, and the spatial consistency of the ground object is classified into a class to express the spatial heterogeneity of the ground surface as a spatial weight;

[0067] Step S3, the spatial resolution of the synthesized cloud-free MOD09Q1 is resampled to an integer multiple of the Landsat-8 resolution, i.e. 240m*240m; then a conversion coefficient between the coarse and fine resolution remote sensing images is calculated based on the resampled cloud-free MOD09Q1 image and the synthesized cloud-free Landsat-8 image; the determined conversion coefficient is multiplied with the cubic convolution interpolated MOD09Q1 image at the prediction and reference time respectively, and then the difference is obtained to obtain the scale conversion error;

[0068] Step S4, from the time aspect, the difference image of the MOD09Q1 at the prediction and reference time can capture the real change of the ground object; multiplying the conversion coefficient determined in step S3 with the reference image at the prediction time, and then subtracting the synthesized cloud-free Landsat-8 image at the reference time can predict the virtual change of the ground object; the real change of the ground object is subtracted from the predicted virtual change of the ground object to obtain the residual error of the image fusion;

[0069] Step S5, since the scale conversion error term determined in step S3 and the residual error term in step S4 are both caused by the change of the ground surface at the reference time and the prediction time, the scale conversion error term and the residual error term are spatially weighted using the superpixel segmentation result in step S2 to correct the error of the spatio-temporal fusion; the above spatially weighted result is added to the synthesized cloud-free Landsat-8 image at the reference time to obtain the final image fusion result.

[0070] Based on the above example, the method step S1 includes:

[0071] 1. Select the fine resolution remote sensing image to be predicted in the operation area, and obtain the surface reflectance images of Landsat and MODIS in the time sequence of the operation area;

[0072] 2. Synthesize a cloud-free image based on the Landsat-8 using the principle of the closest time to the prediction image;

[0073] 3. Synthesize a cloud-free image based on the surface reflectance image of the MOD09Q1 using the principle of the closest time to the prediction image;

[0074] 4. Obtain two pairs of high and low resolution cloud-free remote sensing image pairs before and after the prediction time using the principle of the closest time to the prediction image;

[0075] 5. Calculate the structural similarity index between the MOD09Q1 image at the prediction time and the synthesized cloud-free MOD09Q1 before the prediction time;

[0076] 6. Calculate the structural similarity index between the MOD09Q1 image at the prediction time and the synthesized cloud-free MOD09Q1 after the prediction time;

[0077] 7. Compare the structure similarity index calculated in steps 5 and 6, and select the image with higher structure similarity as the reference image;

[0078] 8. Use the structure similarity index to determine the image pair that is most consistent with the spatial structure of the image at the prediction time, as the reference image pair for spatio-temporal fusion.

[0079] Based on the above example, the method step S2 includes:

[0080] 1. Taking the Landsat-8 image in the reference image pair in the work area as the input data of the superpixel segmentation method;

[0081] 2. The superpixel segmentation method first divides the Landsat-8 image into a plurality of grids of equal size, and each grid center is an initial cluster center;

[0082] 3. Then, the image color space distance and the Euclidean distance of the pixel position of the 8-neighborhood pixels of each cluster center are calculated;

[0083] 4. Each neighborhood pixel is input into a priority queue Q, and each element in the priority queue Q is sorted by the distance from the initial cluster center, so that the neighborhood pixel with the smallest distance is popped out, and the neighborhood pixel is assigned a label information and the feature mean value of the pixels with the same label is updated;

[0084] 5. Until the queue is empty and all the pixels are labeled, the label map of the superpixel can be obtained, thereby expressing the spatial heterogeneity of the ground surface.

[0085] Based on the above example, the method step S3 includes:

[0086] 1. Resample the spatial resolution of the synthesized cloud-free MOD09Q1 to an integer multiple of the resolution of Landsat-8, and the spatial resolution of MOD09Q1 is 250 meters. Since the spatial resolution of Landsat-8 is 30 meters, the resampling is generally 240 meters;

[0087] 2. Perform cubic convolution interpolation on the resampled coarse resolution remote sensing image to make its spatial resolution consistent with that of Landsat-8;

[0088] 3. Then, divide the cloud-free MOD09Q1 image after cubic convolution interpolation by the synthesized cloud-free Landsat-8 image to obtain the conversion coefficient between the coarse and fine resolution remote sensing images;

[0089] 4. Multiply the determined conversion coefficient by the MOD09Q1 image at the prediction time;

[0090] 5. Perform cubic convolution interpolation on the MOD09Q1 image at the reference time, and then multiply the MOD09Q1 image at the reference time by the conversion coefficient;

[0091] 6. Multiply the MOD09Q1 image at the prediction time by the conversion coefficient, and then subtract the MOD09Q1 image at the reference time multiplied by the conversion coefficient to finally obtain the scale conversion error between the coarse resolution and the fine resolution.

[0092] Based on the above example, the method step S4 includes:

[0093] 1. Multiply the conversion coefficient in step S3 by the resampled cloud-free MOD09Q1 at the prediction time, and then resample to 30 m, which is generally consistent with the spatial resolution of Landsat-8 at the reference time;

[0094] 2. Subtract the Landsat-8 at the reference time from the result of multiplying the resampled cloud-free MOD09Q1 at the prediction time by the conversion coefficient and then resampling to 30 m to obtain the change information of the fine resolution remote sensing image at the prediction time and the fusion result at the reference time, and then downscale to the coarse resolution size;

[0095] 3. Subtract the cloud-free MOD09Q1 synthesized at the reference time from the MOD09Q1 image at the prediction time to obtain the real change information of the coarse resolution remote sensing image at the prediction time and the reference time;

[0096] 4. Subtract the result of 2 in S4 from the result of 3 in S4 to obtain the residual term of the fusion result

[0097] 5. Resample the residual term to 30 m resolution, which is the residual in time of the spatio-temporal fusion result.

[0098] Based on the above example, the method step S5 includes:

[0099] 1. Since the pixel-based prediction inevitably has many uncertainties in heterogeneous space, the results of superpixel segmentation in step S2 are used to correct the scale conversion error term and the residual term;

[0100] 2. Since the scale conversion error term determined in step S3 is due to the change of the ground surface at the reference time and the prediction time, it is assumed that the pixels in the same superpixel have consistent temporal changes, and the scale conversion error term is spatially weighted using the clustering unit of the superpixel;

[0101] 3. Similarly, since the residual term in step S4 is due to the change of the ground surface at the reference time and the prediction time, it is assumed that the pixels in the same superpixel have consistent temporal changes, and the residual term is spatially weighted using the clustering unit of the superpixel;

[0102] 4. The high spatial resolution image from the reference time to the prediction time is obtained by adding the scale conversion error term and the residual term after spatial weighting;

[0103] 5. The image changed from the reference time to the prediction time is combined with the cloud-free Landsat-8 image at the reference time, and finally the subdivided resolution remote sensing image at the prediction time is obtained, which is the final fusion result.

[0104] Working principle: Obtain satellite remote sensing images of different time and spatial resolution of crops in the work area; input the satellite remote sensing image into the spatio-temporal fusion method based on cloud computing, and output the fused high spatio-temporal resolution remote sensing image. The spatio-temporal fusion model based on cloud computing uses the time-series coarse and fine resolution remote sensing images to synthesize the two pairs of cloud-free remote sensing images before and after the prediction time. The structural similarity index (SSIM) is used to determine the reference image pair input into the model. Then, based on the high spatial resolution image in the image pair, superpixel segmentation is performed to express the spatial heterogeneity of the ground surface. Then, the initial conversion coefficient is calculated based on the reference image pair, and the initial conversion coefficient is applied to the coarse resolution remote sensing images at the two time points to obtain the scale conversion error term. In addition, the prediction time residual term is calculated, and the scale conversion error term and the prediction residual term are compensated in combination with the superpixel segmentation result. The compensation result is added to the reference time subdivided resolution remote sensing image to obtain the final spatio-temporal fusion result.

[0105] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended to encompass all changes falling within the meaning and scope of the equivalent elements of the claims. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0106] In addition, it should be understood that although the present specification is described in terms of embodiments, each embodiment does not contain only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be properly combined to form other embodiments that those skilled in the art can understand.

Claims

1. A cloud computing spatio-temporal fusion method considering coarse-to-fine resolution scale conversion error correction, characterized in that: Includes the following steps, Step S1: Using time-series remote sensing images of the operational area, synthesize cloud-free reference image pairs based on the principles of temporal consistency and spatial similarity, and determine the coarse-resolution and fine-resolution remote sensing images for the prediction time. Step S2: Perform superpixel segmentation on the fine-resolution remote sensing images in the cloudless reference image pair to express the spatial heterogeneity of the land surface; Step S3: Determine the conversion coefficients based on the cloudless reference image pair, and calculate the scale conversion error term based on the coarse-resolution remote sensing image in the cloudless reference image pair and the coarse-resolution remote sensing image at the predicted time. Step S3 specifically includes: S31 resamples the spatial resolution of the cloudless coarse-resolution remote sensing image in the cloudless reference image pair, and the spatial resolution is an integer multiple of the resolution of the fine-resolution remote sensing image in the cloudless reference image pair. S32 performs cubic convolution interpolation on the resampled cloudless coarse-resolution remote sensing image to make its spatial resolution consistent with the fine-resolution remote sensing image in the cloudless reference image pair. S33 uses the coarse-resolution remote sensing image without clouds after cubic convolution interpolation to divide the fine-resolution remote sensing image in the cloudless reference image pair to obtain the conversion coefficient between the coarse and fine-resolution remote sensing images. S34 performs cubic convolution interpolation on the coarse-resolution remote sensing image in the cloudless reference image pair and then multiplies it by the conversion coefficient. S35 performs cubic convolution interpolation on the coarse-resolution remote sensing image at the predicted time and then multiplies it by the transformation coefficient; S36 uses the result of S35 to subtract the result of S34 to obtain the error of the transformation coefficient between the reference time and the prediction time, which is the scaling transformation error term; Step S4: Calculate the residual term based on the conversion coefficients, the coarse-resolution remote sensing image and fine-resolution remote sensing image at the reference time, and the coarse-resolution remote sensing image at the prediction time. Step S5: Based on the results of superpixel segmentation, the scale conversion error term and the residual term are corrected and compensated onto the fine-resolution remote sensing image in the cloudless reference image pair, thereby obtaining the final fine-resolution remote sensing image fusion result.

2. The cloud computing spatio-temporal fusion method of claim 1, wherein: Step S1 specifically includes: S11 selects the fine-resolution remote sensing image to be predicted for the work area, and obtains the surface reflectance of the time series fine-resolution and coarse-resolution remote sensing images of the work area. S12 Based on the surface reflectance of the fine-resolution remote sensing images of the work area over time series, a cloudless image is synthesized using the principle of time consistency. S13 Based on the surface reflectance of the coarse-resolution remote sensing images of the work area's time series, a cloudless image is synthesized using the principle of time consistency. S14 utilizes the principle of temporal consistency to obtain two pairs of cloudless remote sensing images with coarse and fine resolutions before and after the prediction time. S15 calculates the structural similarity index between the coarse-resolution remote sensing image at the predicted time and the coarse-resolution remote sensing image in the cloudless remote sensing image pair. S16 selects the cloudless remote sensing image pair with a higher structural similarity index from S15 as the cloudless baseline image pair.

3. The cloud computing space-time fusion method of claim 2, wherein: Step S2 specifically includes: S21 uses a superpixel segmentation method to divide the fine-resolution remote sensing image in the cloudless reference image pair into multiple grids of equal size, with the center of each grid serving as the initial cluster center; S22 calculates the image color space distance and the Euclidean distance of the eight neighboring pixels of each initial cluster center; S23 Each neighboring pixel is input into a priority queue Q. Each element in the priority queue Q is sorted by its distance from its initial cluster center, thereby popping the neighboring pixel with the smallest distance. At the same time, the neighboring pixel with the smallest distance is assigned label information and the feature mean of the pixels with the same label is updated. S24 continues until the queue is empty and all pixels are labeled, thus obtaining a superpixel label map to express the spatial heterogeneity of the Earth's surface.

4. The cloud computing space-time fusion method of claim 1, wherein: Step S4 specifically includes: S41 multiplies the transformation coefficients from step S3 by the cloudless coarse-resolution remote sensing image resampled at the predicted time. S42 uses S41 to subtract the fine-resolution remote sensing image in the cloudless reference image pair to obtain the change information between the predicted image and the reference image, and then downscales it to coarse resolution. Subtract the coarse-resolution remote sensing image in the cloudless reference image pair from the coarse-resolution remote sensing image at the prediction time S43 to obtain the true change information of ground features in the coarse-resolution remote sensing image between the prediction time and the reference time. S44 subtracts the result of S42 from the result of S43 to obtain the residual term of the fusion result; S45 resamples the residual term to a fine spatial resolution, which is the temporal residual of the spatiotemporal fusion result.

5. The cloud computing spatiotemporal fusion method according to claim 1, characterized in that: Step S5 specifically includes: S51 Pixel-by-pixel prediction inevitably has many uncertainties in heterogeneous space. The scale transformation error term and residual term are corrected by using the superpixel segmentation results in step S2. The error of the conversion coefficient determined in step S3 of S52 is due to the change of the surface at the reference time and the prediction time. Assuming that the pixels in the same superpixel have consistent temporal changes, the scale conversion error term is spatially weighted using the clustering unit of the superpixel. In step S4 of S53, the residual terms are all caused by the changes in the ground surface at the reference time and the prediction time. Therefore, it is assumed that the pixels in the same superpixel have consistent temporal changes, and the residual terms are spatially weighted using the clustering unit of the superpixel. S54 obtains the change in fine spatial resolution of the image from the reference time to the prediction time by adding the spatially weighted scale transformation error term and the residual term; S55 adds the image showing the changes between the reference time and the predicted time to the cloudless coarse-resolution remote sensing image synthesized at the reference time, and finally obtains the fine-resolution remote sensing image at the predicted time, which is the final fusion result.

Images