A method and related apparatus for generating high spatiotemporal resolution images based on spatiotemporal fusion algorithms

Abstract

This application discloses a method and related apparatus for generating high spatiotemporal resolution images based on a spatiotemporal fusion algorithm, relating to the field of remote sensing image fusion technology. The method includes: for any target raster pixel, initial screening is performed according to reflectance differences within a moving window centered on the position of the target raster pixel; then, a final set of similar raster pixels is obtained by screening based on spectral distance, spatial distance, temporal distance, and corresponding thresholds; when calculating the spectral distance, different combinations of coefficient constants are used according to the different bands of each similar raster pixel to obtain a more accurate spectral distance calculation result; then, based on the weight matrix calculated therefrom, remote sensing images with high temporal and low spatial resolution and high spatial and low temporal resolution are effectively fused, solving the shortcomings of existing remote sensing products in terms of spatiotemporal resolution, especially the limitations of low-cost remote sensing products, which often have high temporal resolution but low spatial resolution, or high spatial resolution but low temporal resolution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image fusion, in particular to a high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm and related device. BACKGROUND

[0002] In recent years, global climate warming, water resource shortage, land degradation, biodiversity reduction and frequent occurrence of sandstorms have posed a serious threat to the living environment of human beings, and have also become an important factor restricting the sustainable development of society and economy. Therefore, it is crucial to monitor the vegetation, water body and soil and other ground objects continuously in space and time for environmental protection and management. Although traditional manual field sampling can provide high-precision data, its time-consuming and labor-intensive characteristics limit its application range, while remote sensing monitoring can achieve high-precision monitoring at a lower cost. However, although the existing satellite images (such as Sentinel-3) have a high temporal resolution (1 day), their 300-meter spatial resolution is limited for monitoring small water bodies and land details, and it is difficult to accurately capture the changes of ground objects. In contrast, Sentinel-2 images (10 meters, 20 meters, 60 meters) can more accurately monitor the details of ground object changes, but their 5-day temporal resolution limits the real-time tracking of phenological changes. On the other hand, although the GF series images have high spatio-temporal resolution, they cannot be widely applied to long-term environmental monitoring due to high cost, so there is an urgent need for technical breakthroughs in this field.

[0003] At present, the spatio-temporal fusion model is an effective solution to the lack of high spatio-temporal resolution images. The spatio-temporal fusion model fuses high spatial resolution and low spatial resolution images, and generates high spatial resolution images using low spatial resolution images when high spatial resolution images are missing, thereby supplementing the missing data. In recent years, deep learning methods have also been applied to spatio-temporal fusion models, although such methods can improve prediction accuracy, but due to the need for a large amount of training data and time, their widespread application is limited. Therefore, how to combine the advantages of high temporal resolution and high spatial resolution images while reducing the dependence on data training is crucial for realizing spatio-temporally continuous monitoring. SUMMARY

[0004] The purpose of the present application is to provide a high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm and related device, which can reduce the computational difficulty and improve the computational efficiency while ensuring the accuracy of the fused remote sensing monitoring image.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm, comprising the following steps:

[0007] The initial high-spatial-resolution image and the initial low-spatial-resolution image close in date are sequentially preprocessed respectively to obtain a preprocessed high-spatial-resolution image and a preprocessed low-spatial-resolution image; the projection coordinate systems and the grid sizes of the preprocessed high-spatial-resolution image and the preprocessed low-spatial-resolution image are the same; the initial high-spatial-resolution image and the initial low-spatial-resolution image are remote sensing monitoring images including a plurality of bands.

[0008] For any target grid pixel of the preprocessed low-spatial-resolution image, a preliminary screening is performed according to reflectivity differences in a moving window at a corresponding position of the preprocessed high-spatial-resolution image to obtain a preliminary screening similar grid pixel set of the target grid pixel; the moving window at the corresponding position is a moving window of a preset size with the position of the target grid pixel as a center, and the moving window includes a plurality of grid pixels.

[0009] A final similar grid pixel set of the target grid pixel is screened according to spectral distances, spatial distances, time distances and corresponding threshold values between each similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel; when calculating the spectral distance between any similar grid pixel and the target grid pixel, different spectral distance calculation coefficients and spectral distance calculation constant terms are used according to different bands of the similar grid pixel.

[0010] A weight matrix of each similar grid pixel is calculated according to the spectral distances, the spatial distances and the time distances between each similar grid pixel in the final similar grid pixel set and the target grid pixel.

[0011] A surface reflectivity of the target grid pixel is calculated according to the surface reflectivity values of each similar grid pixel and the weight matrix of each similar grid pixel, high-spatial-resolution fusion of the preprocessed low-spatial-resolution image is completed, and a high-temporal-spatial-resolution image is obtained.

[0012] Optionally, the preprocessing performed on the initial high-spatial-resolution image and the initial low-spatial-resolution image close in date respectively and sequentially includes: atmospheric correction processing by using ACOLITE atmospheric correction preprocessing software provided by a DSF atmospheric correction algorithm, re-projection by selecting a WGS_1984_UTM_Zone_51N projection system, and resampling by using a cubic convolution interpolation method.

[0013] Optionally, when the initial high-spatial-resolution image and the initial low-spatial-resolution image close in date are subjected to atmospheric correction, the surface reflectivity is calculated according to the following formula:

[0014]

[0015] wherein, ρ s is the surface reflectivity, ρt is the atmospheric top-of-atmosphere reflectance, t g is the atmospheric gas transmittance, p path is the atmospheric path reflectance, p sky is the sky diffuse reflectance, t du is the bidirectional diffuse atmospheric transmittance, s a is the spherical albedo of the atmosphere.

[0016] Optionally, the preliminary screening according to the reflectance difference in the moving window of the corresponding position of the preprocessed high spatial resolution image is performed according to the following formula:

[0017]

[0018] wherein H(x i ,y j ,t k ) is the surface reflectance value of the grid cell at position (x k ,y i ) in the preprocessed high spatial resolution image at time t j , H(x k ,y ijk ,t i ) is the surface reflectance value of the central grid cell of the preprocessed high spatial resolution image at time t j , w is the row and column size of the moving window, s is the standard deviation value of the surface reflectance within the moving window of the corresponding position of the preprocessed high spatial resolution image, and m is the number of ground object categories.

[0019] Optionally, the spectral distance between any similar grid cell in the preliminary screening similar grid cell set and the target grid cell is calculated according to the following formula:

[0020] S ijk = |aH(x i ,y j ,t k )+b-L(x i ,y j ,t k )|.

[0021] wherein S ijk is the spectral distance between the similar grid cell at position (x k ,y i ) in the preliminary screening similar grid cell set and the target grid cell at time t j , a is the spectral distance calculation coefficient, b is the spectral distance calculation constant term, H(x i ,y j ,t k ) is the surface reflectance value of the grid cell at position (x k ,y i ) in the preprocessed high spatial resolution image at time t j .The surface reflectance value of the raster pixel, L(x) i ,y j ,t k ) for t k The position of time in the preprocessed low spatial resolution image is (x i ,y j The surface reflectance value of the raster cell.

[0022] The temporal distance between any similar raster cell in the initial screening set of similar raster cells and the target raster cell is calculated using the following formula:

[0023] T ijk =|L(x i ,y j ,t k )-L(x i ,y j ,t0)|.

[0024] Among them, T ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y j The temporal distance between similar raster cells and the target raster cell, L(x) i ,y j (x, t0) represents the position of the image at time t0 in the preprocessed low spatial resolution image. i ,y j The surface reflectance value of the raster cell.

[0025] The spatial distance between any similar raster cell in the initial screening set of similar raster cells and the target raster cell is calculated using the following formula:

[0026]

[0027] Among them, D ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y j The spatial distance between similar raster cells and the target raster cell is w, where w is the row and column size of the moving window.

[0028] Optionally, the weight matrix of any similar raster cell can be calculated according to the following formula:

[0029]

[0030] Among them, W ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y jThe weight matrix S of similar raster cells. ijk T ijk and D ijk t k The position of the initial screening of similar raster cell set at time (x) i ,y j The spectral distance, temporal distance, and spatial distance between similar raster pixels and the target raster pixel are given by w, where w is the row and column size of the moving window, and n is the number of remote sensing image pairs used in the calculation.

[0031] The surface reflectance of the target raster cell is calculated using the following formula:

[0032]

[0033] in, For t k Surface reflectance of target raster cell at time step, L(x) i ,y j (x, t0) represents the position of the image at time t0 in the preprocessed low spatial resolution image. i ,y j The surface reflectance value of the raster pixel, H(x) i ,y j ,t k ) for t k The position of time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel, L(x) i ,y j ,t k ) for t k The position of time in the preprocessed low spatial resolution image is (x i ,y j The surface reflectance value of the raster cell.

[0034] Secondly, this application provides a high spatiotemporal resolution image generation system based on a spatiotemporal fusion algorithm, comprising:

[0035] The remote sensing image preprocessing module is used to preprocess initial high spatial resolution images and initial low spatial resolution images with similar dates, respectively, to obtain preprocessed high spatial resolution images and preprocessed low spatial resolution images. The projection coordinate system and raster size of the preprocessed high spatial resolution images and preprocessed low spatial resolution images are the same. Both the initial high spatial resolution images and the initial low spatial resolution images are remote sensing images that include several bands.

[0036] The similar raster pixel initial screening module is used to perform preliminary screening based on reflectance differences for any target raster pixel in the preprocessed low spatial resolution image within a moving window at the corresponding position in the preprocessed high spatial resolution image, thereby obtaining a set of initially screened similar raster pixels for the target raster pixel; the moving window at the corresponding position is a preset-sized moving window centered on the position of the target raster pixel, and the moving window includes several raster pixels.

[0037] The similar raster pixel final screening module is used to screen the target raster pixel to obtain the final similar raster pixel set based on the spectral distance, spatial distance, temporal distance and corresponding threshold between each similar raster pixel in the initial screening similar raster pixel set and the target raster pixel. When calculating the spectral distance between any similar raster pixel and the target raster pixel, different spectral distance calculation coefficients and spectral distance calculation constants are used according to the different spectral bands of the similar raster pixels.

[0038] The pixel weight matrix technology module is used to calculate the weight matrix of each similar raster pixel based on the spectral distance, spatial distance, and temporal distance between each similar raster pixel in the final set of similar raster pixels and the target raster pixel.

[0039] The surface reflectance fusion calculation module is used to calculate the surface reflectance of the target raster pixel based on the surface reflectance values ​​of each similar raster pixel and the weight matrix of each similar raster pixel, and to complete the high spatial resolution fusion of the preprocessed low spatial resolution image to obtain a high spatiotemporal resolution image.

[0040] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the high spatiotemporal resolution image generation method based on the spatiotemporal fusion algorithm described above.

[0041] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the high spatiotemporal resolution image generation method based on the spatiotemporal fusion algorithm described above.

[0042] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the high spatiotemporal resolution image generation method based on the spatiotemporal fusion algorithm described above.

[0043] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0044] This application provides a method and related apparatus for generating high spatiotemporal resolution images based on a spatiotemporal fusion algorithm. In this method, initial high spatial resolution images and initial low spatial resolution images with similar dates are preprocessed sequentially to ensure that their projection coordinate systems and raster sizes are identical, facilitating subsequent processing. Then, for any target raster pixel, a final set of similar raster pixels is obtained by filtering within a moving window centered on the target raster pixel's position according to reflectance differences, spectral distance, spatial distance, temporal distance, and corresponding thresholds. When calculating the spectral distance, different combinations of coefficient constants are used based on the different spectral bands of each similar raster pixel to obtain more accurate calculation results. Next, based on the obtained spectral distance, spatial distance, and temporal distance, the corresponding weight matrix is ​​calculated. Finally, based on the surface reflectance values ​​of each similar raster pixel and its weight matrix, the high spatial resolution fusion of the preprocessed low spatial resolution image is completed, resulting in a high spatiotemporal resolution image. This application effectively integrates remote sensing images with high temporal and low spatial resolution and high spatial and low temporal resolution to obtain high-precision remote sensing images with high spatiotemporal resolution. While ensuring the accuracy of the predicted images, the computation is less difficult and more efficient. It solves the shortcomings of existing remote sensing products in terms of spatiotemporal resolution. In particular, low-cost remote sensing products usually have the limitation of high temporal resolution but low spatial resolution, or high spatial resolution but low temporal resolution. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm, provided as an embodiment of this application.

[0047] Figure 2 This is a schematic diagram showing the regional division of the Zhukuma Reservoir, the research area, according to a specific embodiment of this application.

[0048] Figure 3 This is a comparison diagram of the preprocessed remote sensing image of Zhukuma Reservoir and the initial remote sensing image in a specific embodiment of this application.

[0049] Figure 4 This is a true-color comparison image of the fusion result in a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm, provided in a specific embodiment of this application.

[0050] Figure 5This is a box plot showing the band-by-band comparison of the fusion results in a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm, provided in one embodiment of this application.

[0051] Figure 6 This image shows a comparison of the fusion results of a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm provided in an embodiment of this application with the fusion results of other methods.

[0052] Figure 7 A scatter plot comparing the inversion results in a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm provided in an embodiment of this application.

[0053] Figure 8 This is a schematic diagram of the time series of inversion results in a high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm provided in an embodiment of this application.

[0054] Figure 9 This is a schematic diagram of the functional modules of a high spatiotemporal resolution image generation system based on a spatiotemporal fusion algorithm, provided in an embodiment of this application.

[0055] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] The high spatiotemporal resolution image generation method based on spatiotemporal fusion algorithm provided in this application, in an exemplary embodiment, is as follows: Figure 1 As shown, it includes the following steps:

[0059] S1. Initial high-spatial-resolution (HSR) images and initial low-spatial-resolution (LSR) images from similar dates are preprocessed sequentially to obtain preprocessed HSR images and preprocessed LSR images. The preprocessed HSR images and preprocessed LSR images have the same projection coordinate system and raster size. Both the initial HSR images and initial LSR images are remote sensing monitoring images comprising several bands. Specifically, the initial HSR images are image data acquired by the Sentinel-2MSI multispectral imager, and the initial LSR images are image data acquired by the Sentinel-3OLCI ocean and land color instrument.

[0060] In a specific embodiment, the preprocessing performed sequentially on the initial high spatial resolution images and initial low spatial resolution images with similar dates in step S1 includes: screening for cloudless illumination conditions, performing atmospheric correction processing using the ACOLITE atmospheric correction preprocessing software provided by the DSF atmospheric correction algorithm, selecting the WGS_1984_UTM_Zone_51N projection system for reprojection, and performing resampling using the cubic convolution interpolation method.

[0061] Specifically, cloudless images within the study area were selected, and Level 1 images with poor lighting conditions were removed. Then, atmospheric correction was uniformly performed using the ACOLITE atmospheric correction preprocessing software provided by the Dark Spectrum Fitting (DSF) atmospheric correction algorithm. During the calculation process, certain values ​​(such as water vapor transmittance, Rayleigh optical thickness, and atmospheric path reflectance) need to be selected from the lookup table (LUT) generated using 6SV. The specific calculation method for surface reflectance is as follows:

[0062]

[0063] Where, ρ s ρ is the surface reflectivity. t t represents the atmospheric top reflectivity. g ρ is the atmospheric gas transmittance. path ρ is the atmospheric path reflectance. sky For the sky diffuse reflectance, t du For bidirectional diffuse atmospheric transmittance, s a This refers to the spherical albedo of the atmosphere. This method can eliminate the impact of flares on images.

[0064] To ensure a one-to-one correspondence between the raster pixels of the two remote sensing images, this embodiment further performs reprojection and resampling operations on the remote sensing images. Through comparative analysis, this embodiment first selects the WGS_1984_UTM_Zone_51N projection system to reproject the two remote sensing images, and then uses cubic convolution interpolation to complete the image resampling, ensuring that the two remote sensing images have the same projection coordinate system and a raster size of 10×10m, thereby guaranteeing the spatial consistency of the image data.

[0065] The above preprocessing steps provide a solid data foundation for the spatiotemporal fusion analysis of images and the extraction of remote sensing information, significantly improving the accuracy and reliability of the fusion results and meeting the needs of various remote sensing application scenarios.

[0066] Next, the improved STARFM model is used to fuse remote sensing image data. First, the input parameters of the STARFM model need to be determined, including the side length of the moving window, the spatial influence factor, and the number of land cover categories. After comparative analysis, this embodiment selects 3000m as the side length of the moving window, approximately equal to the size of 10×10 Sentinel-3 pixels, which can simultaneously ensure prediction accuracy and computational efficiency. The spatial influence factor is selected as 500 to balance the relationship between the strictness of pixel selection based on spatial distance and the number of pixels. Simultaneously, the number of land cover categories is set to 4 to ensure an appropriate balance between matching accuracy and the number of pixels.

[0067] S2. For any target raster pixel in the preprocessed low spatial resolution image, preliminary screening is performed according to reflectance differences within a moving window at the corresponding position in the preprocessed high spatial resolution image to obtain a preliminary set of similar raster pixels for the target raster pixel. The moving window at the corresponding position is a preset-sized moving window centered on the position of the target raster pixel, and includes several raster pixels. Specifically, in this embodiment, preliminary screening is performed according to reflectance differences within a moving window at the corresponding position in the preprocessed high spatial resolution image based on the following formula:

[0068]

[0069] Wherein, H(x) i ,y j ,t k ) for t k The position of time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel. For t kThe surface reflectance value of the center raster cell in the high spatial resolution image after time-based preprocessing, w is the row and column size of the moving window, σ is the standard deviation of surface reflectance within the moving window at the corresponding location in the preprocessed high spatial resolution image, and m is the number of land cover categories.

[0070] S3. Based on the spectral distance, spatial distance, temporal distance, and corresponding thresholds between each similar raster pixel in the initial screening set and the target raster pixel, a final set of similar raster pixels for the target raster pixel is obtained. When calculating the spectral distance between any similar raster pixel and the target raster pixel, different spectral distance calculation coefficients and constant terms are used according to the different spectral bands of the similar raster pixels. Specifically, in this embodiment, the spectral distance between any similar raster pixel in the initial screening set and the target raster pixel is calculated according to the following formula:

[0071] S ijk =|aH(x) i ,y j ,t k )+bL(x i ,y j ,t k )|.

[0072] Among them, S ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y j The spectral distance between similar raster pixels and the target raster pixel, where a is the spectral distance calculation coefficient, b is the spectral distance calculation constant, and H(x) is the spectral distance calculation constant. i ,y j ,t k ) for t k The position of time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel, L(x) i ,y j ,t k ) for t k The position of time in the preprocessed low spatial resolution image is (x i ,y j The surface reflectance value of the raster cell.

[0073] In this embodiment, the corresponding bands of the two remote sensing images are modeled, and the spectral distance calculation coefficients a and constant term b for each band are determined by linear regression calculation. The calculation results are shown in the table below.

[0074] Table 1. Calculation coefficients and constants for spectral distance in each band

[0075] Central wavelength (nm) a b 443 1.13 0.0002 490 0.98 -0.0001 560 0.85 0.0035 665 0.87 0.0019 705 0.87 0.0025 740 1.16 0.0011 783 1.04 0.0011 842 1.14 0.0006 865 1.15 0.0005

[0076] The temporal distance between any similar raster cell in the initial screening set of similar raster cells and the target raster cell is calculated using the following formula:

[0077] T ijk =|L(x i ,y j ,t k )-L(x i ,y j ,t0)|.

[0078] Among them, T ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y j The temporal distance between similar raster cells and the target raster cell, L(x) i ,y j (x, t0) represents the position of the image at time t0 in the preprocessed low spatial resolution image. i ,y j The surface reflectance value of the raster cell.

[0079] The spatial distance between any similar raster cell in the initial screening set of similar raster cells and the target raster cell is calculated using the following formula:

[0080]

[0081] Among them, D ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y j The spatial distance between similar raster cells and the target raster cell is w, where w is the row and column size of the moving window.

[0082] When the spectral distance, temporal distance, and spatial distance are all less than their respective thresholds, they are identified as similar pixels and participate in subsequent calculations.

[0083] S4. Based on the spectral distance, spatial distance, and temporal distance between each similar raster pixel in the final set of similar raster pixels and the target raster pixel, calculate the weight matrix of each similar raster pixel. In this embodiment, the weight matrix of any similar raster pixel is calculated according to the following formula:

[0084]

[0085] Among them, W ijk For t k The position of the initial screening of similar raster cell set at time (x) i ,y jThe weight matrix S of similar raster cells. ijk T ijk and D ijk t k The position of the initial screening of similar raster cell set at time (x) i ,y j The calculation considers the spectral, temporal, and spatial distances between similar raster pixels and the target raster pixel, where w is the row and column size of the moving window, and n is the number of remote sensing image pairs used in the calculation. Specifically, when fusing high- and low-spatial-resolution images, at least one pair of remote sensing image pairs (high spatial-low temporal and high temporal-low spatial) needs to be used for calculation, and n is the number of remote sensing image pairs used in the calculation. For example, the time of the first pair of remote sensing image pairs is t1, the time of the second pair is t2, and so on, with the last pair of image pairs being t... n .

[0086] S5. Based on the surface reflectance values ​​of each similar raster pixel and the weight matrix of each similar raster pixel, the surface reflectance of the target raster pixel is calculated, completing the high spatial resolution fusion of the preprocessed low spatial resolution image to obtain a high spatiotemporal resolution image. Specifically, the surface reflectance of the target raster pixel is calculated according to the following formula:

[0087]

[0088] in, For t k Surface reflectance of target raster cell at time step, L(x) i ,y j (x, t0) represents the position of the image at time t0 in the preprocessed low spatial resolution image. i ,y j The surface reflectance value of the raster pixel, H(x) i ,y j ,t k ) for t k The position of time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel, L(x) i ,y j ,t k ) for t k The position of time in the preprocessed low spatial resolution image is (x i ,y j The surface reflectance values ​​of the raster cells are calculated. By iterating through the positions of all raster cells, the surface reflectance of all raster cells can be calculated, ultimately yielding a complete high spatiotemporal resolution remote sensing image.

[0089] The method proposed in this application will now be applied to a specific example: the Zhuwei Reservoir (39°50'-39°52'N, 122°45'-122°53'E) is located on the western tributary of the Zhuanghe River in Taipingling Township, Zhuanghe City, Liaoning Province. Figure 2 As shown, its reservoir area is relatively small, approximately 17 km². 2 In recent years, the water quality in the reservoir has deteriorated significantly during the high-water season, with algal blooms becoming increasingly severe. This provides excellent conditions for water quality inversion and can verify the accuracy of the generated high spatiotemporal resolution Sentinel2-like image dataset. The measured data were obtained through field sampling and laboratory analysis, with sampling conducted on average once a month during the non-freezing period. After applying the above-mentioned scheme to the Zhukuma Reservoir, the comparison between the preprocessing results of step S1 and the original image is shown in the figure below. Figure 3 As shown.

[0090] The test results of Zhukuma Reservoir are as follows Figure 4 and Figure 5 As shown, the accuracy of the water remote sensing reflectance in the fusion results was verified using indicators such as root mean square error (RMSE), structural similarity (SSIM), peak signal-to-noise ratio (PSNR), and dimensionless global comprehensive relative error (ERGAS) for the spectral / spatial dimensions. The test results from 2016 to 2024 showed RMSE < 0.015sr. -1 PSNR > 37 dB, SSIM > 0.8, dimensionless global comprehensive relative error ERGAS in the spectral dimension spectral <3.8, Dimensionless Global Comprehensive Relative Error ERGAS in Spatial Dimensions spatial The value <3.3 indicates that the method in this embodiment has high fusion accuracy. By comparing it with the ESTARFM method, as... Figure 6 As shown, it was found that the high spatiotemporal resolution Sentinel2-like image generated by the improved STARFM method used in this embodiment has a color closer to that of the real Sentinel-2 image, and the fusion accuracy is higher.

[0091] The test results of inversion using the Sentinel2-like dataset generated at Zhukuma Reservoir using the method of this embodiment are as follows: Figure 7 As shown, the inversion model was applied to the Sentinel-2 and Sentinel2-like datasets, and the obtained correlation coefficient R0 was obtained. 2 The values ​​are 0.65 and 0.59 respectively, which meet the accuracy requirements for long-term inversion of chlorophyll a (Chla) concentration in water bodies. Through comparative analysis of the inversion results of Sentinel-2, Sentinel-3, and Sentinel2-like images, as shown... Figure 8As shown, Sentinel2-like images can accurately capture water body details, and the obtained Chla values ​​are close to Sentinel-2. They can be effectively applied to long-term water body monitoring and ecological assessment, and have high practical value.

[0092] Based on the same inventive concept, this application also provides a system for implementing the high spatiotemporal resolution image generation method based on spatiotemporal fusion algorithm described above. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more system embodiments provided below can be found in the limitations of the high spatiotemporal resolution image generation method based on spatiotemporal fusion algorithm described above, and will not be repeated here.

[0093] In one exemplary embodiment, such as Figure 9 As shown, a high spatiotemporal resolution image generation system based on a spatiotemporal fusion algorithm is provided, including the following functional modules:

[0094] The remote sensing image preprocessing module is used to preprocess initial high spatial resolution images and initial low spatial resolution images with similar dates, respectively, to obtain preprocessed high spatial resolution images and preprocessed low spatial resolution images. The projection coordinate system and raster size of the preprocessed high spatial resolution images and preprocessed low spatial resolution images are the same. Both the initial high spatial resolution images and the initial low spatial resolution images are remote sensing images that include several bands.

[0095] The similar raster pixel initial screening module is used to perform preliminary screening based on reflectance differences for any target raster pixel in the preprocessed low spatial resolution image within a moving window at the corresponding position in the preprocessed high spatial resolution image, thereby obtaining a set of initially screened similar raster pixels for the target raster pixel; the moving window at the corresponding position is a preset-sized moving window centered on the position of the target raster pixel, and the moving window includes several raster pixels.

[0096] The similar raster pixel final screening module is used to screen the target raster pixel to obtain the final similar raster pixel set based on the spectral distance, spatial distance, temporal distance and corresponding threshold between each similar raster pixel in the initial screening similar raster pixel set and the target raster pixel. When calculating the spectral distance between any similar raster pixel and the target raster pixel, different spectral distance calculation coefficients and spectral distance calculation constants are used according to the different spectral bands of the similar raster pixels.

[0097] The pixel weight matrix technology module is used to calculate the weight matrix of each similar raster pixel based on the spectral distance, spatial distance, and temporal distance between each similar raster pixel in the final set of similar raster pixels and the target raster pixel.

[0098] The surface reflectance fusion calculation module is used to calculate the surface reflectance of the target raster pixel based on the surface reflectance values ​​of each similar raster pixel and the weight matrix of each similar raster pixel, and to complete the high spatial resolution fusion of the preprocessed low spatial resolution image to obtain a high spatiotemporal resolution image.

[0099] certainly, Figure 9 The architecture shown is merely exemplary; it can be omitted as needed when implementing different functionalities. Figure 9 One or at least two components of the system shown.

[0100] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10 As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it can implement the high spatiotemporal resolution image generation method based on a spatiotemporal fusion algorithm provided in the above embodiment.

[0101] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0102] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0103] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0104] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0107] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0108] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm, characterized in that, The application relates to a high-spatial-resolution image fusion method and device. The initial high-spatial-resolution image and the initial low-spatial-resolution image are respectively sequentially preprocessed to obtain a preprocessed high-spatial-resolution image and a preprocessed low-spatial-resolution image; the projection coordinate systems and the grid sizes of the preprocessed high-spatial-resolution image and the preprocessed low-spatial-resolution image are the same; the initial high-spatial-resolution image and the initial low-spatial-resolution image are remote monitoring images including a plurality of bands; the preprocessing of the initial high-spatial-resolution image and the initial low-spatial-resolution image respectively sequentially includes atmospheric correction processing by using ACOLITE atmospheric correction preprocessing software provided by a DSF atmospheric correction algorithm, re-projection by selecting a WGS_1984_UTM_Zone_51N projection system, and resampling by using a cubic convolution interpolation method; A target grid pixel of the preprocessed low-spatial-resolution image is preliminarily screened in a moving window at a corresponding position of the preprocessed high-spatial-resolution image according to reflectivity difference to obtain a preliminary screening similar grid pixel set of the target grid pixel; the moving window at the corresponding position is a moving window of a preset size with the position of the target grid pixel as the center, and the moving window includes a plurality of grid pixels; A final similar grid pixel set of the target grid pixel is screened according to the spectral distance, the spatial distance, the time distance and the corresponding threshold of each similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel; when the spectral distance between any similar grid pixel and the target grid pixel is calculated, different spectral distance calculation coefficients and spectral distance calculation constant terms are used according to the bands of the similar grid pixel; The weight matrix of each similar grid pixel is calculated according to the spectral distance, the spatial distance and the time distance of each similar grid pixel and the target grid pixel in the final similar grid pixel set; The surface reflectivity of the target grid pixel is calculated according to the surface reflectivity values of the similar grid pixels and the weight matrix of each similar grid pixel, high-spatial-resolution fusion of the preprocessed low-spatial-resolution image is completed, and a high-temporal-spatial-resolution image is obtained. The spatial distance between any similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel is calculated according to the following formula: Among them, D ijk For t k The position of the initially screened similar raster pixel set at time (x) i ,y j The spatial distance between similar raster pixels and the target raster pixel is w, where w is the row and column size of the moving window; the side length of the moving window is 3000m, and the spatial influence factor is set to 500 and the number of land cover categories is 4 during the initial screening process. 2.The high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm according to claim 1, characterized in that, When the initial high-spatial-resolution image and the initial low-spatial-resolution image are subjected to atmospheric correction, the surface reflectivity is calculated according to the following formula: where p s is the surface reflectance, p t is the top-of-atmosphere reflectance, t g is the atmospheric gas transmittance, p path is the atmospheric path reflectance, p sky is the sky diffuse reflectance, t du is the bidirectional diffuse atmospheric transmittance, s a is the spherical albedo of the atmosphere. 3.The high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm of claim 1, wherein, The preliminary screening in the moving window at the corresponding position of the preprocessed high-spatial-resolution image is carried out according to the reflectivity difference according to the following formula: Wherein, H(x) i ,y j ,t k ) for t k The position of the time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel. For t k The surface reflectance value of the center raster cell of the preprocessed high spatial resolution image at time t, w is the row and column size of the moving window, σ is the standard deviation of surface reflectance within the moving window at the corresponding position of the preprocessed high spatial resolution image, and m is the number of land cover categories. 4.The high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm of claim 1, wherein, The spectral distance between any similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel is calculated according to the following formula: S ijk = |aH(x i ,y j ,t k ) - bL(x i ,y j ,t k )|; Among them, S ijk For t k The position of the initially screened similar raster pixel set at time (x) i ,y j The spectral distance between similar raster pixels and the target raster pixel, where a is the spectral distance calculation coefficient, b is the spectral distance calculation constant, and H(x) is the spectral distance calculation constant. i ,y j ,t k ) for t k The position of the time in the preprocessed high spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel, L(x) i ,y j ,t k ) for t k The position of the time in the preprocessed low spatial resolution image is (x i ,y j The surface reflectance value of the raster pixel; The time distance between any similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel is calculated according to the following formula: T ijk = |L(x i ,y j ,t k )-L(x i ,y j ,t0)|; wherein T(x ijk ,y k ,t0) is the time distance between the similar grid cell located at (x i ,y j ) in the preliminary screening similar grid cell set and the target grid cell at the time t0, and L(x i ,y j ,t0) is the surface reflectivity value of the grid cell located at (x i ,y j ) in the pre-processed low spatial resolution image at the time t0.

5. The high spatio-temporal resolution image generation method based on a spatio-temporal fusion algorithm according to claim 1, characterized in that, The weight matrix of any similar grid pixel is calculated according to the following formula: wherein, W ijk is the weight matrix of the similar grid pixel at position (x k ,y i ) in the preliminary screening similar grid pixel set at time t j , S ijk , T ijk and D ijk are the spectral distance, time distance and spatial distance between the similar grid pixel at position (x k ,y i ) in the preliminary screening similar grid pixel set and the target grid pixel at time t j , w is the row and column size of the moving window, and n is the number of pairs of remote sensing monitoring images selected for calculation. The surface reflectivity of the target grid pixel is calculated according to the following formula: wherein, is the surface reflectance of the target grid cell at time t k is the surface reflectance of the target grid cell at time t i 0, L(x j ,y i ,t0) is the surface reflectance value of the grid cell in the pre-processed low spatial resolution image at position (x i ,y j ) at time t0, H(x i ,y j ,t k ) is the surface reflectance value of the grid cell in the pre-processed high spatial resolution image at position (x k ,y i ) at time t j , L(x i ,y j ,t k ) is the surface reflectance value of the grid cell in the pre-processed low spatial resolution image at position (x k ,y i ) at time t j .

6. A high spatio-temporal resolution image generation system based on a spatio-temporal fusion algorithm, characterized in that, The application relates to a high-spatial-resolution image fusion method and device. The remote sensing monitoring image preprocessing module is used for sequentially preprocessing initial high spatial resolution images and initial low spatial resolution images with similar dates respectively to obtain preprocessed high spatial resolution images and preprocessed low spatial resolution images; the projection coordinate systems and the grid sizes of the preprocessed high spatial resolution images and the preprocessed low spatial resolution images are the same; the initial high spatial resolution images and the initial low spatial resolution images are remote sensing monitoring images including a plurality of bands; the preprocessing of the initial high spatial resolution images and the initial low spatial resolution images with similar dates respectively sequentially includes: atmospheric correction processing by using ACOLITE atmospheric correction preprocessing software provided by a DSF atmospheric correction algorithm, re-projection by selecting a WGS_1984_UTM_Zone_51N projection system, and resampling by using a cubic convolution interpolation method; The similar grid pixel preliminary screening module is used for performing preliminary screening according to reflectivity differences in a moving window at a corresponding position of the preprocessed high spatial resolution images for any target grid pixel of the preprocessed low spatial resolution images to obtain a preliminary screening similar grid pixel set of the target grid pixel; the moving window at the corresponding position is a moving window with a preset size and taking the position of the target grid pixel as a center, and the moving window includes a plurality of grid pixels; The similar grid pixel final screening module is used for screening a final similar grid pixel set of the target grid pixel according to spectral distances, spatial distances, time distances and corresponding threshold values of each similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel; when calculating the spectral distance of any similar grid pixel and the target grid pixel, different spectral distance calculation coefficients and spectral distance calculation constant terms are used according to different bands of the similar grid pixel; The spatial distance of any similar grid pixel in the preliminary screening similar grid pixel set and the target grid pixel is calculated according to the following formula: wherein D ijk is the spatial distance between the similar grid cell at position (x k ,y j ) in the preliminary screening similar grid cell set and the target grid cell at time t i , and w is the row and column size of the moving window. The pixel weight matrix technology module is used for calculating a weight matrix of each similar grid pixel according to the spectral distance, the spatial distance and the time distance of each similar grid pixel in the final similar grid pixel set and the target grid pixel; The surface reflectivity fusion calculation module is used for calculating the surface reflectivity of the target grid pixel according to the surface reflectivity values of each similar grid pixel and the weight matrix of each similar grid pixel to complete high spatial resolution fusion of the preprocessed low spatial resolution images and obtain high spatio-temporal resolution images; the side length of the moving window is 3000 m, and the spatial influence factor is set to 500 and the number of ground object categories is set to 4 in the preliminary screening process.

7. A computer device comprising: A memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the high spatio-temporal resolution image generation method based on the spatio-temporal fusion algorithm in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the high spatio-temporal resolution image generation method based on the spatio-temporal fusion algorithm in any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the high spatio-temporal resolution image generation method based on the spatio-temporal fusion algorithm in any one of claims 1-5.

Images