A method for object-level spatiotemporal fusion of remote sensing image data

Abstract

The invention discloses a method for object-level spatiotemporal fusion of remote sensing image data. The method comprises: obtaining a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of the second time phase, and performing interpolation processing on the second remote sensing image and the third remote sensing image respectively based on a preset algorithm to obtain two interpolated images; segmenting the first remote sensing image by a preset segmentation tool based on a preset rule to obtain a plurality of segmented areas; constructing a local regression model for each segmented area in each segmented area according to the two interpolated images, and obtaining a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model; obtaining a residual value of each pixel in each segmented area according to the local regression model and the two interpolated images, and obtaining a target spatiotemporal fusion result corresponding to each segmented area according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area.

Description

Technical Field

[0001] The invention relates to the field of remote sensing technology, and in particular to a method for object-level spatiotemporal fusion of remote sensing image data. Background Art

[0002] With the rapid development of remote sensing technology, it plays an increasingly important role in monitoring land cover changes, ecosystem dynamics, natural disasters and other applications. If high spatiotemporal resolution images can be obtained, dense time series ground monitoring can be achieved, greatly enhancing the value of remote sensing images. However, due to the contradiction between satellite spatiotemporal resolution, it is impossible to obtain high spatiotemporal resolution images using a single satellite. In addition, problems such as clouds, stripes, and noise further increase the difficulty of obtaining high-quality time series images. Spatiotemporal fusion technology is a feasible solution to the spatiotemporal contradiction. Without changing the existing observation conditions, this technology can make full use of the complementarity of multi-source remote sensing images in spatiotemporal resolution to reconstruct high spatiotemporal resolution images and achieve continuous ground observation.

[0003] Although spatiotemporal fusion technology is developing rapidly and a large number of new algorithms have been developed, most of them focus on the accuracy of fusion rather than the efficiency of fusion. In addition, almost all spatiotemporal fusion methods are pixel-level fusion, especially the widely used weighted function method, which means that fusing large-scale images requires huge time costs, resulting in low fusion efficiency.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for object-level spatiotemporal fusion of remote sensing image data in view of the above-mentioned defects of the prior art, aiming to solve the problem of low efficiency of remote sensing image data fusion in the prior art.

[0006] The technical solution adopted by the present invention to solve the problem is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a method for object-level spatiotemporal fusion of remote sensing image data, wherein the method comprises:

[0008] Acquire a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and perform interpolation processing on the second remote sensing image and the third remote sensing image respectively based on a preset algorithm to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images;

[0009] Based on preset rules, the first remote sensing image is segmented at multiple scales by a preset segmentation tool to obtain a plurality of segmented regions;

[0010] For each segmented area in each segmented area, a local regression model is constructed according to the first interpolated image and the second interpolated image, and a preliminary spatiotemporal fusion result corresponding to each segmented area is obtained according to the first remote sensing image and the local regression model;

[0011] According to the local regression model, the first interpolated image and the second interpolated image, the residual value of each pixel in each segmented area is obtained, and according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area, the target spatiotemporal fusion result corresponding to each segmented area is obtained.

[0012] In one implementation, the time corresponding to the first time is different from the time corresponding to the second time.

[0013] In one implementation, the interpolation processing is performed on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain the first interpolation image and the second interpolation image, respectively, including:

[0014] Performing image preprocessing on the second remote sensing image and the third remote sensing image respectively to obtain a first preprocessed image and a second preprocessed image;

[0015] Based on a nearest neighbor interpolation algorithm, the first preprocessed image and the second preprocessed image are interpolated respectively to obtain a first interpolated image and a second interpolated image.

[0016] In one implementation, the spectral difference of the ground objects in the segmented area is less than a set value.

[0017] In one implementation, the first remote sensing image is segmented at multiple scales by a preset segmentation tool based on a preset rule to obtain a plurality of segmented regions including:

[0018] Obtaining preset segmentation parameters based on preset rules;

[0019] Based on the segmentation parameters, the multi-scale segmentation tool of Yikang software is called to perform multi-scale segmentation on the first remote sensing image to obtain a number of segmented areas.

[0020] In one implementation, for each segmented area in each segmented area, constructing a local regression model according to the first interpolated image and the second interpolated image, and obtaining a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model includes:

[0021] For each segmented area among the segmented areas, using the first interpolated image as a guide image, using the second interpolated image as an input image, performing guided filtering on the guide image and the input image, and obtaining a local regression model for each segmented area;

[0022] Each pixel in the first remote sensing image is substituted into the local regression model of each segmented area to obtain a preliminary spatiotemporal fusion result corresponding to each segmented area.

[0023] In one implementation, obtaining the residual value of each pixel in each segmented area according to the local regression model, the first interpolated image, and the second interpolated image, and obtaining the target space-time fusion result corresponding to each segmented area according to the residual value and the preliminary space-time fusion result corresponding to each segmented area includes:

[0024] Substituting the first interpolation image into the local regression model and performing linear regression calculation to obtain a linear regression calculation result;

[0025] Subtracting the linear regression calculation result from the second interpolation image to obtain a residual value for each pixel;

[0026] According to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented region, a target spatiotemporal fusion result corresponding to each segmented region is obtained.

[0027] In one implementation, obtaining a target spatiotemporal fusion result corresponding to each segmented region according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented region includes:

[0028] Averaging the residual values ​​to obtain a residual average value for each segmented area;

[0029] The residual mean value is added to the preliminary spatiotemporal fusion result corresponding to each segmented region to obtain the target spatiotemporal fusion result corresponding to each segmented region.

[0030] In a second aspect, an embodiment of the present invention further provides a device for object-level spatiotemporal fusion of remote sensing image data, wherein the device comprises:

[0031] Two interpolation image acquisition modules, used to acquire a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and to interpolate the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolation image and a second interpolation image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images;

[0032] Several segmentation region acquisition modules, used for performing multi-scale segmentation on the first remote sensing image based on preset rules and using preset segmentation tools to obtain several segmentation regions;

[0033] a preliminary spatiotemporal fusion result determination module, configured to construct a local regression model for each segmented area in each segmented area according to the first interpolated image and the second interpolated image, and obtain a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model;

[0034] The target spatiotemporal fusion result determination module is used to obtain the residual value of each pixel in each segmented area according to the local regression model, the first interpolated image and the second interpolated image, and to obtain the target spatiotemporal fusion result corresponding to each segmented area according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area.

[0035] In the third aspect, an embodiment of the present invention also provides an intelligent terminal, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include a method for executing object-level spatiotemporal fusion of remote sensing image data as described in any one of the above items.

[0036] In a fourth aspect, an embodiment of the present invention further provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute a method for object-level spatiotemporal fusion of remote sensing image data as described in any one of the above.

[0037] Beneficial effects of the present invention: The embodiment of the present invention first obtains a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and based on a preset algorithm, interpolates the second remote sensing image and the third remote sensing image respectively to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images; then based on preset rules, the first remote sensing image is multi-scale segmented by a preset segmentation tool to obtain a plurality of segmented regions; then, for each segmented region in each segmented region, a local regression model is constructed according to the first interpolated image and the second interpolated image, and a local regression model is constructed according to the first remote sensing image and the third interpolated image. A local regression model is used to obtain a preliminary spatiotemporal fusion result corresponding to each segmented area; finally, the residual value of each pixel in each segmented area is obtained according to the local regression model, the first interpolated image and the second interpolated image, and the target spatiotemporal fusion result corresponding to each segmented area is obtained according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area; it can be seen that in the embodiment of the present invention, the image is split into multiple segmented areas by using multi-scale segmentation technology, and the area is used as the basic fusion unit to achieve object-level spatiotemporal fusion. Compared with the traditional pixel-level fusion method, it can effectively capture the changes of the objects in two phases and realize fast spatiotemporal data fusion, thereby significantly reducing the amount of calculation, improving the fusion efficiency, and effectively restoring the phenological changes of the objects in two phases. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A schematic flow chart of a method for object-level spatiotemporal fusion of remote sensing image data provided by an embodiment of the present invention.

[0040] Figure 2 A principle block diagram of a device for object-level spatiotemporal fusion of remote sensing image data provided by an embodiment of the present invention.

[0041] Figure 3 This is a block diagram of the internal structure principle of the intelligent terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention discloses a method for object-level spatiotemporal fusion of remote sensing image data. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0044] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0045] In the existing technology, in recent years, more and more remote sensing satellites have been launched, but due to the limitation of satellite optical sensor technology, the improvement of the spatial resolution of optical remote sensing images can often only be achieved by sacrificing the advantages of other aspects, such as temporal resolution, spectral resolution, etc. To address this problem, it can be solved by launching more satellites to form a constellation group. With the development of commercial satellites in recent years, it has become possible to achieve dense time series earth observation. For example, the constellation group formed by Planet using cube satellites can obtain global daily 5-meter spatial resolution images, but due to the huge investment cost, these commercial satellite images cannot be obtained for free. In addition, for studies that require long-term historical images, this solution cannot fill the gap in historical data. And obtaining high-quality dense time series data by integrating multi-source images through spatiotemporal data fusion is a feasible solution to the above problems. Spatiotemporal fusion technology makes full use of the spatiotemporal complementarity of multi-source remote sensing images, and realizes high spatiotemporal information reconstruction by fusing images with frequent revisits but low spatial resolution with images with sparse revisits but high spatial resolution. Compared with a single image, fused images can mine more information.

[0046] Although spatiotemporal fusion technology is developing rapidly and a large number of new algorithms have been developed, most of them focus on the accuracy of fusion rather than the efficiency of fusion. In addition, almost all spatiotemporal fusion methods are pixel-level fusion, especially the widely used weighted function method, which means that fusing large-scale images requires huge time costs. This problem greatly limits the application of spatiotemporal fusion.

[0047] In order to solve the problems of the prior art, the present embodiment provides a method for object-level spatiotemporal fusion of remote sensing image data. The present invention utilizes multi-scale segmentation technology to split the image into multiple segmented regions, and uses the region as the basic fusion unit to achieve object-level spatiotemporal fusion. Compared with the traditional pixel-level fusion method, it can significantly reduce the amount of calculation, improve the fusion efficiency, and effectively restore the phenological changes of the objects in two time phases. In the specific implementation, the first remote sensing image and the second remote sensing image of the first phase and the third remote sensing image of the second phase are first obtained, and based on a preset algorithm, the second remote sensing image and the third remote sensing image are interpolated respectively to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images; then, based on a preset rule, the first remote sensing image is multi-scale segmented by a preset segmentation tool to obtain a number of segmented areas; then, for each segmented area in each segmented area, a local regression model is constructed according to the first interpolated image and the second interpolated image, and a preliminary spatiotemporal fusion result corresponding to each segmented area is obtained according to the first remote sensing image and the local regression model; finally, the residual value of each pixel in each segmented area is obtained according to the local regression model, the first interpolated image and the second interpolated image, and the target spatiotemporal fusion result corresponding to each segmented area is obtained according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area.

[0048] Exemplary Methods

[0049] This embodiment provides a method for object-level spatiotemporal fusion of remote sensing image data, which can be applied to remote sensing intelligent terminals. Figure 1 As shown, the method includes:

[0050] Step S100: acquiring a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and performing interpolation processing on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images;

[0051] Specifically, the time corresponding to the first time phase T1 is different from the time corresponding to the second time phase T2; the resolutions (also known as spatiotemporal resolutions) of the first remote sensing image, the second remote sensing image and the third remote sensing image are different, wherein the resolution of the first remote sensing image is higher than that of the second remote sensing image and the third remote sensing image (such as 25 meters), and the resolutions of the second remote sensing image and the third remote sensing image are at the same level, which is low resolution (such as 500 meters). After obtaining the first remote sensing image, the second remote sensing image and the third remote sensing image, interpolation processing is performed on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolation image and a second interpolation image; correspondingly, the interpolation processing is performed on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolation image and a second interpolation image, which includes the following steps: performing image preprocessing on the second remote sensing image and the third remote sensing image to obtain a first preprocessed image and a second preprocessed image; interpolating the first preprocessed image and the second preprocessed image based on a nearest neighbor interpolation algorithm to obtain a first interpolation image and a second interpolation image.

[0052] In practice, the acquired first remote sensing image, the second remote sensing image and the third remote sensing image will be preprocessed, and the preprocessing operations include but are not limited to registration and radiation correction. The first remote sensing image is preprocessed to obtain the third preprocessed image. Since this step is a routine step after the remote sensing image is acquired, this step is not described in detail. In addition to using the nearest neighbor interpolation algorithm, the interpolation step can also use the difference algorithm that appears in the prior art, which is not repeated here. In this embodiment, based on the nearest neighbor interpolation algorithm, the first preprocessed image and the second preprocessed image are interpolated respectively, and the first preprocessed image and the second preprocessed image are downscaled to the same resolution as the first remote sensing image, that is, the first preprocessed image and the second preprocessed image are downscaled to the same resolution as the third preprocessed image, and the first interpolated image C1 and the second interpolated image C2 are obtained, and the first remote sensing image is recorded as F1.

[0053] After obtaining the first interpolation image and the second interpolation image, you can perform the following Figure 1 The following steps are shown: S200, based on a preset rule, the first remote sensing image is segmented at multiple scales by a preset segmentation tool to obtain a plurality of segmented areas;

[0054] Specifically, the preset rule may be to classify the spectral difference of the objects less than the set value into one area, or to classify the spectral density of the objects less than the set value into one area. The preset tool may be the ESP optimal segmentation scale tool, or the Yikang multi-scale segmentation tool, without specific limitation. In this way, the first remote sensing image may be subjected to multi-scale segmentation by the ESP optimal segmentation scale tool to obtain a number of segmented areas; or the first remote sensing image may be subjected to multi-scale segmentation by the Yikang multi-scale segmentation tool to obtain a number of segmented areas. In one implementation, the spectral difference of the objects in the segmented areas is less than the set value.

[0055] Step S200 includes the following steps: obtaining preset segmentation parameters based on preset rules; based on the segmentation parameters, calling the Yikang software multi-scale segmentation tool to perform multi-scale segmentation on the first remote sensing image to obtain a number of segmented areas.

[0056] Specifically, the preset rule is to classify the spectral difference of the objects less than the set value into one area, and set the corresponding segmentation parameters according to the preset rules. The segmentation parameters include but are not limited to scale parameters, spectral weights and shape weights; such as: the scale parameter is a key parameter that determines the size of the segmentation area. It is set smaller when the surface heterogeneity is high (the surface is complex), and is set larger when the heterogeneity scale is low (the surface is relatively uniform), generally set in the range of 100-200. The spectral weight should be set larger than the shape weight because the remote sensing image contains rich spectral information, and is generally set to 0.6 and 0.5 respectively. The focus is on setting the scale parameter. Generally, after setting the spectral weight and shape weight to 0.6 and 0.5 respectively, the scale parameter is gradually increased from 100 (such as by 25) and determined by visual segmentation results. After obtaining these preset segmentation parameters, the system calls the multi-scale segmentation tool of Yikang software to perform multi-scale segmentation on the first remote sensing image to obtain several segmented areas.

[0057] After obtaining several segmented regions, you can perform the following operations: Figure 1 The following steps are shown: S300, for each segmented area in each segmented area, a local regression model is constructed according to the first interpolation image and the second interpolation image, and a preliminary spatiotemporal fusion result corresponding to each segmented area is obtained according to the first remote sensing image and the local regression model; accordingly, step S300 includes the following steps:

[0058] S301, for each segmented area, using the first interpolated image as a guide image, using the second interpolated image as an input image, performing guided filtering on the guide image and the input image, and obtaining a local regression model for each segmented area;

[0059] S302: Substitute each pixel in the first remote sensing image into the local regression model of each segmented region to obtain a preliminary spatiotemporal fusion result corresponding to each segmented region.

[0060] Specifically, for each segmented area, guided filtering is used to perform local regression on the first interpolation image and the second interpolation image, the first interpolation image is used as the guide image, the second interpolation image is used as the input image, and the guide image and the input image are subjected to guided filtering. After the guided filtering is completed, a local regression model of each segmented area is obtained, that is, each segmented area will correspond to two linear regression coefficients a(s, B) and b(s, B), where s is the segmentation block number and B is the band number. Then, each pixel in the first remote sensing image F1 is substituted into the local regression model of each segmented area to obtain the preliminary spatiotemporal fusion result corresponding to each segmented area. The specific calculation formula is as follows:

[0061]

[0062] Among them, F1(s,B) is each pixel in the first remote sensing image F1, The preliminary spatiotemporal fusion results corresponding to each segmented region.

[0063] After obtaining the preliminary spatiotemporal fusion results corresponding to each segmented area, the following operations can be performed: Figure 1 The following steps are shown:

[0064] S400. Obtain a residual value of each pixel in each segmented area according to the local regression model, the first interpolated image and the second interpolated image, and obtain a target space-time fusion result corresponding to each segmented area according to the residual value and the preliminary space-time fusion result corresponding to each segmented area.

[0065] Step S400 includes the following steps: substituting the first interpolation image into the local regression model and performing linear regression calculation to obtain a linear regression calculation result; subtracting the linear regression calculation result from the second interpolation image to obtain a residual value for each pixel; averaging the residual values ​​to obtain a residual average value for each segmented area; adding the residual average value to the preliminary spatiotemporal fusion result corresponding to each segmented area to obtain a target spatiotemporal fusion result corresponding to each segmented area.

[0066] Specifically, each pixel C1(s, B) of the first interpolation image is substituted into the local regression model and a linear regression calculation is performed to obtain a linear regression calculation result [a(s, B)×C1(s, B)+b(s, B)], and then each pixel C2(s, B) of the second interpolation image is subtracted from the linear regression calculation result to obtain a residual value C2(s, B)-[a(s, B)×C1(s, B)+b(s, B)] of each pixel, and then the residual values ​​are averaged to obtain the residual average value of each segmented area:

[0067] r(s,B)=E[C2(s,B)-[a(s,B)×C1(s,B)+b(s,B)]]

[0068] Among them, r is the average residual value of each segmented area, and E represents the average value. Finally, the residual average value is added to the preliminary spatiotemporal fusion result corresponding to each segmented area to obtain the target spatiotemporal fusion result corresponding to each segmented area. The calculation formula is as follows:

[0069]

[0070] in, is the preliminary spatiotemporal fusion result corresponding to each segmented region. The target spatiotemporal fusion result corresponding to each segmented region and the target spatiotemporal fusion results of all segmented regions constitute the final spatiotemporal fusion result.

[0071] In order to illustrate the fusion effect of the remote sensing image spatiotemporal data fusion method provided by the embodiment of the present invention, a comparative experiment is conducted between the spatiotemporal fusion method proposed by the present invention and the spatiotemporal adaptive reflection fusion model. The changes produced by the two images used for experimental comparison are the phenological growth changes of farmland.

[0072] Table 1 gives the quantitative analysis indicators of the above three spatiotemporal fusion methods. Method A is a spatiotemporal adaptive reflection fusion model, and method B is a spatiotemporal fusion method for remote sensing images provided by an embodiment of the present invention; three accuracy indicators, root mean square error (RMSE), structural similarity (SSIM), and correlation coefficient (r), are selected to reflect the spectral difference, structural similarity, and correlation between the fusion result and the real image, respectively. The closer the former is to 0 and the closer the latter two are to 1, the better the fusion effect. The time required for the fusion operation is selected in seconds.

[0073] From the comparison in Table 1, it can be seen that the spatiotemporal fusion result provided by the embodiment of the present invention is better than the fusion result of the spatiotemporal adaptive reflection fusion model: the embodiment of the present invention has the smallest root mean square error (RMSE), the largest structural similarity (SSIM) and correlation coefficient (r), and the shortest running time in all bands. Therefore, the remote sensing image spatiotemporal fusion method provided by the embodiment of the present invention can achieve fast spatiotemporal fusion and obtain better spatiotemporal fusion results.

[0074] Table 1

[0075]

[0076] Exemplary Devices

[0077] like Figure 2 As shown in , an embodiment of the present invention provides a device for object-level spatiotemporal fusion of remote sensing image data, the device comprising two interpolation image acquisition modules 501, a plurality of segmentation region acquisition modules 502, a preliminary spatiotemporal fusion result determination module 503 and a target spatiotemporal fusion result determination module 504, wherein:

[0078] Two interpolation image acquisition modules 501 are used to acquire a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and perform interpolation processing on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolation image and a second interpolation image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images;

[0079] A plurality of segmented region acquisition modules 502, configured to perform multi-scale segmentation on the first remote sensing image based on a preset rule and using a preset segmentation tool to obtain a plurality of segmented regions;

[0080] The preliminary spatiotemporal fusion result determination module 503 is used to construct a local regression model for each segmented area according to the first interpolated image and the second interpolated image, and obtain a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model;

[0081] The target spatiotemporal fusion result determination module 504 is used to obtain the residual value of each pixel in each segmented area according to the local regression model, the first interpolated image and the second interpolated image, and to obtain the target spatiotemporal fusion result corresponding to each segmented area according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area.

[0082] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 3As shown. The intelligent terminal includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for object-level spatiotemporal fusion of remote sensing image data is implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor of the intelligent terminal is pre-set inside the intelligent terminal to detect the operating temperature of the internal device.

[0083] Those skilled in the art will understand that Figure 3 The schematic diagram in the figure is only a block diagram of a part of the structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0084] In one embodiment, a smart terminal is provided, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: acquiring a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and interpolating the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images;

[0085] Based on preset rules, the first remote sensing image is segmented at multiple scales by a preset segmentation tool to obtain a plurality of segmented regions;

[0086] For each segmented area in each segmented area, a local regression model is constructed according to the first interpolated image and the second interpolated image, and a preliminary spatiotemporal fusion result corresponding to each segmented area is obtained according to the first remote sensing image and the local regression model;

[0087] According to the local regression model, the first interpolated image and the second interpolated image, the residual value of each pixel in each segmented area is obtained, and according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area, the target spatiotemporal fusion result corresponding to each segmented area is obtained.

[0088] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0089] In summary, the present invention discloses a method for object-level spatiotemporal fusion of remote sensing image data, the method comprising: obtaining a first remote sensing image and a second remote sensing image of a first phase, and a third remote sensing image of a second phase, and interpolating the second remote sensing image and the third remote sensing image respectively based on a preset algorithm to obtain two interpolated images; based on a preset rule, segmenting the first remote sensing image by a preset segmentation tool to obtain a plurality of segmented areas; for each segmented area in each segmented area, constructing a local regression model according to the two interpolated images, and obtaining a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model; obtaining a residual value of each pixel in each segmented area according to the local regression model and the two interpolated images, and obtaining a target spatiotemporal fusion result corresponding to each segmented area according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented area. In the embodiment of the present invention, multi-scale segmentation technology is used to split the image into multiple segmented regions, and the region is used as the basic fusion unit to achieve object-level spatiotemporal fusion. Compared with the traditional pixel-level fusion method, it can significantly reduce the amount of calculation, improve the fusion efficiency, and effectively restore the phenological changes of the objects in two time phases.

[0090] Based on the above embodiments, the present invention discloses a method for object-level spatiotemporal fusion of remote sensing image data. It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, it can be improved or transformed according to the above description. All these improvements and transformations should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for object-level spatiotemporal fusion of remote sensing image data, characterized in that: The method comprises: Acquire a first remote sensing image and a second remote sensing image of a first time phase, and a third remote sensing image of a second time phase, and perform interpolation processing on the second remote sensing image and the third remote sensing image respectively based on a preset algorithm to obtain a first interpolated image and a second interpolated image; wherein the first remote sensing image is a high-resolution image, and the second remote sensing image and the third remote sensing image are both low-resolution images; Based on preset rules, the first remote sensing image is segmented at multiple scales by a preset segmentation tool to obtain a plurality of segmented regions; For each segmented area in each segmented area, a local regression model is constructed according to the first interpolated image and the second interpolated image, and a preliminary spatiotemporal fusion result corresponding to each segmented area is obtained according to the first remote sensing image and the local regression model; Obtaining a residual value of each pixel in each segmented area according to the local regression model, the first interpolated image, and the second interpolated image, and obtaining a target space-time fusion result corresponding to each segmented area according to the residual value and the preliminary space-time fusion result corresponding to each segmented area; The calculation formula of the target spatiotemporal fusion result corresponding to each segmented area is: ； ； ； in, For each segmented area, the target spatiotemporal fusion result is: is the preliminary spatiotemporal fusion result corresponding to each segmented region. is the residual mean value for each segmented region, and There are two linear regression coefficients for each segmentation area. is the segment block number, is the band number, The first remote sensing image Each pixel in It means to find the average value, represents each pixel of the second interpolated image, are the pixels of the first interpolated image.

2. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The time corresponding to the first time is different from the time corresponding to the second time.

3. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The interpolation processing is performed on the second remote sensing image and the third remote sensing image based on a preset algorithm to obtain a first interpolation image and a second interpolation image, respectively, including: Performing image preprocessing on the second remote sensing image and the third remote sensing image respectively to obtain a first preprocessed image and a second preprocessed image; Based on a nearest neighbor interpolation algorithm, the first preprocessed image and the second preprocessed image are interpolated respectively to obtain a first interpolated image and a second interpolated image.

4. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The spectral difference of the ground objects in the segmented area is less than a set value.

5. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The first remote sensing image is segmented at multiple scales based on the preset rules by using a preset segmentation tool to obtain a number of segmented areas including: Obtaining preset segmentation parameters based on preset rules; Based on the segmentation parameters, the multi-scale segmentation tool of Yikang software is called to perform multi-scale segmentation on the first remote sensing image to obtain a number of segmented areas.

6. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The step of constructing a local regression model for each segmented area according to the first interpolated image and the second interpolated image, and obtaining a preliminary spatiotemporal fusion result corresponding to each segmented area according to the first remote sensing image and the local regression model includes: For each segmented area among the segmented areas, using the first interpolated image as a guide image, using the second interpolated image as an input image, performing guided filtering on the guide image and the input image, and obtaining a local regression model for each segmented area; Each pixel in the first remote sensing image is substituted into the local regression model of each segmented area to obtain a preliminary spatiotemporal fusion result corresponding to each segmented area.

7. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 1, characterized in that: The step of obtaining a residual value of each pixel in each segmented area according to the local regression model, the first interpolated image, and the second interpolated image, and obtaining a target space-time fusion result corresponding to each segmented area according to the residual value and the preliminary space-time fusion result corresponding to each segmented area includes: Substituting the first interpolation image into the local regression model and performing linear regression calculation to obtain a linear regression calculation result; Subtracting the linear regression calculation result from the second interpolation image to obtain a residual value for each pixel; According to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented region, a target spatiotemporal fusion result corresponding to each segmented region is obtained.

8. The method for object-level spatiotemporal fusion of remote sensing image data according to claim 7, characterized in that: The obtaining of the target spatiotemporal fusion result corresponding to each segmented region according to the residual value and the preliminary spatiotemporal fusion result corresponding to each segmented region comprises: Averaging the residual values ​​to obtain a residual average value for each segmented area; The residual mean value is added to the preliminary spatiotemporal fusion result corresponding to each segmented region to obtain the target spatiotemporal fusion result corresponding to each segmented region.

9. An intelligent terminal, characterized in that: The device comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs include being used to execute the method according to any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as claimed in any one of claims 1 to 8.

Images