A remote sensing image filling method, system, electronic device and storage medium

Abstract

The present invention discloses a remote sensing image filling method, system, electronic device and storage medium, the method comprising: S1 obtaining image data to be filled and a remote sensing image database, wherein the remote sensing image database comprises a plurality of remote sensing images; S2 obtaining the missing area of ​​the image to be filled based on the image to be filled, and sequentially using a first filling strategy, a second filling strategy, a third filling strategy and a fourth filling strategy to fill the missing area of ​​the image to be filled, until the missing area is filled and a target image is obtained. By adopting a step-by-step filling strategy, different retrieval and filling strategies are sequentially used to fill the missing area of ​​the image according to the image filling situation, so that the missing image can be filled and repaired as much as possible while ensuring the authenticity of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing and analysis, and in particular to a remote sensing image filling method and system. Background Art

[0002] Remote sensing satellite images are widely used in environmental monitoring, resource management, urban planning and other fields. However, due to factors such as cloud cover and sensor failure, the acquired image data often has missing problems, which directly affects the accuracy and reliability of applications such as surface change monitoring, environmental analysis, and disaster assessment. In the existing technology, the filling is mainly carried out through time expansion, interpolation, image fusion and other methods, but the existing image filling methods are relatively simple and still cannot meet the needs in some cases. Therefore, a more comprehensive and efficient image filling method and system is needed. Summary of the invention

[0003] The purpose of the present invention is to provide a remote sensing image filling method and system, which can achieve accurate filling of missing images and improve the integrity and application value of remote sensing data by comprehensively applying multiple strategies.

[0004] According to a first aspect of the present invention, a remote sensing image filling method is provided, the method comprising:

[0005] S1: obtaining image data to be filled and a remote sensing image database, wherein the remote sensing image database includes a plurality of remote sensing image data;

[0006] The remote sensing image database includes remote sensing image data of various types, various resolutions and various times.

[0007] The remote sensing image data includes remote sensing images and image-related metadata.

[0008] The metadata includes image acquisition time, image resolution, sensor model, etc.

[0009] The image to be filled and the remote sensing image to be filled are both preprocessed, and the preprocessing includes: radiation calibration, atmospheric correction and orthorectification.

[0010] S2 obtains the missing area of ​​the image to be filled based on the image to be filled, and sequentially uses the first filling strategy, the second filling strategy, the third filling strategy and the fourth filling strategy to fill the missing area of ​​the image to be filled, until the image to be filled is completely filled to obtain the target image.

[0011] The first filling strategy includes: obtaining a first search condition based on the metadata of the image to be filled and the location information of the missing area of ​​the image to be filled, using the first search condition to retrieve a first filling image set in the remote sensing image database, and using the first filling image set to fill the missing area of ​​the image to be filled.

[0012] Furthermore, the first search condition includes location information, acquisition time, resolution, sensor model, image type, etc.

[0013] The first filling image set consists of remote sensing images of the same type that include the missing area of ​​the image to be filled and have the same acquisition time and resolution as the image to be filled.

[0014] The second filling strategy includes: obtaining a second search condition based on the metadata of the image to be filled and the location information of the first missing area, using the second search condition to search and screen in the remote sensing image database to obtain a second filling image set, and using the second filling image set to fill the first missing area; the first missing area is the area that has not been filled by the first filling strategy.

[0015] The second search condition is: expanding the acquisition time in the first search condition, and also expanding the resolution of the image to a higher resolution than the image to be filled;

[0016] The acquisition time in the search condition is limited to a set interval; the set interval represents a time range in which there is no difference in land feature characteristics caused by seasonal climate.

[0017] The process of screening and obtaining the second filling image set is: searching the remote sensing image database according to the second search condition to obtain multiple candidate second filling images, respectively obtaining the feature difference value between each candidate second filling image and the image to be filled, and selecting the image with the feature difference value less than the threshold as the second filling image set.

[0018] The characteristic difference value is the area ratio of the changed region of the candidate second filling image compared to the image to be filled.

[0019] The second filled image set consists of images that contain the first missing area and meet the screening condition.

[0020] The third filling strategy includes: setting a third retrieval condition, retrieving a low-resolution image set from a remote sensing image database, using a super-resolution reconstruction method to increase the image resolution in the low-resolution image set to the resolution level of the image to be filled, obtaining a third filling image set, and using the third filling image set to fill the second missing area; the second missing area is the area that has not been filled by the second filling strategy.

[0021] The third search condition includes: the same acquisition time as the image to be filled, lower resolution than the image to be filled, and covering the position information of the second missing area.

[0022] The third filling image set is composed of super-resolution images that are acquired at the same time as the image to be filled and contain the second missing area; the super-resolution images are obtained by super-resolving and reconstructing the low-resolution images.

[0023] The fourth filling strategy includes: constructing an image prediction model, using the image prediction model to predict and generate the third missing area to obtain a target image with the filling completed; the third missing area is the area that has not been filled by the third filling strategy.

[0024] The process of using the image prediction model to predict and generate the third missing area includes at least one of the three prediction methods: using the image information of the image to be filled itself for prediction, using the information of the time series image group of the image to be filled for prediction, or using the information of remote sensing images of different modalities in the same area as the image to be filled for prediction. Different image prediction models can be selected according to different prediction methods.

[0025] According to a second aspect of the present invention, a remote sensing image filling system is provided, which specifically comprises:

[0026] An image acquisition module is configured to acquire remote sensing image data and pre-process the remote sensing image, wherein the remote sensing image data includes images to be filled and remote sensing images in a remote sensing image database;

[0027] An image database is configured to store various remote sensing image data, including remote sensing images and metadata corresponding to the images;

[0028] An image retrieval module is configured to set retrieval conditions according to metadata information of the image to be filled, search the image database, and obtain each filled image set;

[0029] An image filling module is configured to fill the image to be filled using various filling strategies, and finally obtain a completed filled image;

[0030] The image output module is configured to output the filled image.

[0031] The image filling module includes a first image filling submodule, a second image filling submodule, a third image filling submodule and a fourth image filling submodule;

[0032] The first image filling submodule is used to adopt a first filling strategy to fill the missing area of ​​the image to be filled, so as to obtain a first image;

[0033] The second image filling submodule is used to fill the missing area of ​​the first image with a second filling strategy to obtain a second image;

[0034] The third image filling submodule is used to adopt a third filling strategy to fill the missing area of ​​the second image to obtain a third image;

[0035] The fourth image filling submodule is used to adopt a fourth filling strategy to fill the missing area of ​​the third image to obtain a filled image.

[0036] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in a remote sensing image filling method as described in the first aspect of the present invention are implemented.

[0037] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, they are used to implement the steps in a remote sensing image filling method as described in the first aspect of the present invention.

[0038] Technical effects of the present invention:

[0039] 1. The present invention adopts four step-by-step filling strategies. According to the filling and repairing conditions of the missing areas of the image to be filled, different retrieval and screening strategies are used in turn to screen different remote sensing images from the remote sensing image database to fill and repair the missing areas of the image to be filled. This can achieve the filling and repair of various images to be filled, and ensure the authenticity of the filled images as much as possible;

[0040] 2. The second filling strategy of the present invention limits the time range in the second search condition and screens the images retrieved, ensuring that there is no obvious difference in ground feature between the images in the second filling image set and the images to be filled, thereby ensuring the authenticity of the images filled by the second filling strategy;

[0041] 3. The fourth filling strategy of the present invention uses multimodal images of the same time and area as the image to be filled as the structural prediction prior information of the missing area, thereby ensuring the authenticity of the predicted image of the missing area and improving the accuracy of image filling. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a flow chart of a remote sensing image filling method provided by the present invention;

[0044] Figure 2 A schematic diagram of a process of a remote sensing image filling method provided by the present invention;

[0045] Figure 3 A schematic diagram of the structure of a remote sensing image filling system provided by the present invention;

[0046] Figure 4 A schematic diagram of the structure of a computer system used to implement an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0048] The present invention provides a remote sensing image filling method, such as Figure 1 As shown, the method specifically comprises the following steps:

[0049] S1: Obtain image data to be filled and a remote sensing image database, wherein the remote sensing image database includes a plurality of remote sensing image data.

[0050] The image to be filled and the remote sensing image to be filled are both preprocessed, and the preprocessing includes: radiation calibration, atmospheric correction and orthorectification.

[0051] The remote sensing image database includes remote sensing image data of various types, various resolutions and various times.

[0052] The remote sensing image data includes remote sensing images and image-related metadata.

[0053] The metadata includes the spatial location of the image, acquisition time, image resolution, sensor type, etc.

[0054] S2 obtains the missing area of ​​the image to be filled based on the image to be filled, and sequentially uses the first filling strategy, the second filling strategy, the third filling strategy and the fourth filling strategy to fill the missing area of ​​the image to be filled, until the image to be filled is completely filled to obtain the target image.

[0055] Specifically, the first filling strategy includes: obtaining a first search condition based on the metadata of the image to be filled and the location information of the missing area of ​​the image to be filled, using the first search condition to retrieve a first filling image set in the remote sensing image database, and using the first filling image set to fill the missing area of ​​the image to be filled.

[0056] If the first filling image set can cover the missing area of ​​the image to be filled, the target image with the filling completed is obtained; if the first filling image set cannot cover the missing area of ​​the image to be filled, the image obtained after filling with the first filling strategy is taken as the first image, and the missing area of ​​the first image is obtained as the first missing area.

[0057] Furthermore, the first search condition includes location information, acquisition time, resolution, sensor model, image type, etc.

[0058] The first filling image set consists of remote sensing images of the same type that include the missing area of ​​the image to be filled and have the same acquisition time and resolution as the image to be filled.

[0059] Specifically, the second filling strategy includes: obtaining a second search condition based on the metadata of the image to be filled and the location information of the first missing area, searching and screening in the remote sensing image database according to the second search condition to obtain a second filling image set, and filling the first missing area with the second filling image set.

[0060] If the second filled image set can cover the first missing area, the target image with the filling completed is obtained; if the second filled image set cannot cover the first missing area, the image obtained after filling with the second filling strategy is used as the second image, and the missing area of ​​the second image is obtained as the second missing area.

[0061] The second search condition is: expanding the acquisition time in the first search condition, and also expanding the resolution of the image to a higher resolution than the image to be filled.

[0062] The process of resolution expansion is as follows: if the resolution of the image to be filled is 10m, the resolution of the search condition is expanded to ≥10m, and a higher resolution such as 5m, 3m, 1m, etc. can be selected as the search condition to retrieve an image with a higher resolution.

[0063] Since the search conditions are expanded, there is a gap in the acquisition time of the retrieved images and the images to be filled. If the time difference is too large, the image's geographical features may have seasonal differences, affecting the temporal uniformity of the image content. Therefore, it is necessary to limit the time for the expansion of the search conditions to a set interval. Within the set interval, there are no differences in geographical features caused by seasonal climate. The specific range is set according to the time of the image to be filled.

[0064] Although the scope of time expansion is limited to avoid differences in ground feature characteristics due to seasonal climate differences, ground changes may still occur due to different acquisition times, resulting in the filling content in the retrieved image being inconsistent with the actual content of the missing area of ​​the image to be filled. Therefore, it is necessary to screen the retrieved images to reduce the possibility of the filling content being inconsistent with the actual content of the missing area due to ground changes.

[0065] The process of retrieving and screening to obtain the second filling image set is:

[0066] 1. Use the second search condition to search the remote sensing image database to obtain candidate second filling images with higher resolution within a wider acquisition time window.

[0067] 2. Obtain the feature difference value between each candidate second filling image and the image to be filled, and select the image whose feature difference value is less than the threshold as the second filling image set.

[0068] The process of obtaining the feature difference value between the candidate second filling image and the image to be filled is as follows:

[0069] (1) Perform preprocessing operations such as radiation correction and geometric correction on the candidate second infill image and the image to be infilled to eliminate the influence of factors such as atmosphere, surface reflection and sunlight conditions on the image;

[0070] (2) Use image registration technology to register the candidate second filling image with the image to be filled, and crop the second filling image and the image to be filled according to the registration results to ensure that the changes in the surface can be accurately reflected during comparison;

[0071] (3) A change detection method is used to perform change detection on the registered candidate image and the image to be filled, and the change area of ​​the candidate image relative to the image to be filled is obtained. The feature difference value is calculated based on the change area.

[0072] The change detection methods include: image interpolation method, image ratio method, principal component analysis method, pixel-based direct comparison method, post-classification comparison method, machine learning method, deep learning method, etc.

[0073] Specifically in this embodiment, a deep learning method is used to perform change detection on the candidate second filling image and the image to be filled. The image features of the candidate second filling image and the image to be filled are respectively extracted through a deep convolutional network, and the feature differences are analyzed to obtain the changed area of ​​the candidate second filling image compared with the image to be filled, so as to calculate the feature difference value of the candidate second filling image relative to the image to be filled.

[0074] The characteristic difference value is the area ratio of the candidate second filling image relative to the image to be filled, and the specific calculation method is as follows:

[0075] Feature difference value = the area of ​​the changed region of the candidate second filling image / the area of ​​the image to be filled after registration and cropping × 100%.

[0076] The screening condition is that the difference value of the image feature is <10%.

[0077] The second filled image set consists of images that contain the first missing area and meet the screening condition.

[0078] Specifically, the third filling strategy includes: setting a third retrieval condition, retrieving a low-resolution image set in the remote sensing image database, using a super-resolution reconstruction method to increase the resolution of the images in the low-resolution image set to the resolution level of the image to be filled, obtaining a third filling image set, and using the third filling image set to fill the second missing area.

[0079] If the third filled image set can cover the second missing area, the target image with completed filling is obtained; if the third filled image set cannot cover the second missing area, the image obtained by the third filling strategy is used as the third image, and the missing area of ​​the third image is obtained as the third missing area.

[0080] The third search condition includes: the same acquisition time as the image to be filled, a lower resolution than the image to be filled (for example, if the resolution of the image to be filled is 0.5m, the resolution in the search condition can be set to 1m, 2m, 5m, etc.), and includes the location information of the second missing area.

[0081] The specific process of obtaining the third infill image set by the super-resolution reconstruction method is as follows: the low-resolution images obtained by retrieval and screening are input into the remote sensing image super-resolution model for super-resolution reconstruction to obtain super-resolution images with clear detail features, thereby obtaining the third infill image set.

[0082] Specifically in this embodiment, the remote sensing image super-resolution model is based on the transformer network structure and introduces a variety of attention mechanisms, which can activate more relevant pixels and better generate pixel and spectral information in high-resolution images, making the high-resolution image obtained after super-resolution reconstruction more realistic and reliable, and ensuring the authenticity and accuracy of the filled image.

[0083] The specific structure of the remote sensing image super-resolution model includes: a shallow feature extraction module, a deep feature extraction module and an image reconstruction module. In the shallow feature extraction module, a separate convolutional layer is used to learn and extract the shallow features of the image. In the deep feature extraction module, an enhanced transformer submodule is used to integrate the channel attention mechanism and the overlapping cross attention block to extract the deep features of the image, and a residual mechanism is used after the enhanced transformer submodule to increase the perception field. In the image reconstruction module, a jump connection is used to fuse shallow features and deep features, and image reconstruction is performed through upsampling and convolution layer structures.

[0084] The third filling image set is composed of super-resolution images that are acquired at the same time as the image to be filled and contain the second missing area.

[0085] Specifically, the fourth filling strategy includes: constructing an image prediction model, using the image prediction model to predict and generate the third missing area, and obtaining a target image that has been filled.

[0086] The process of using the image prediction model to predict and generate the third missing area includes at least one of the three prediction methods: using the image information of the image to be filled itself for prediction, using the information of the time series image group of the image to be filled for prediction, or using the information of remote sensing images of different modalities in the same area as the image to be filled for prediction. Different image prediction models can be selected according to different prediction methods.

[0087] The image prediction model includes: statistical interpolation methods (such as linear interpolation, Kriging interpolation, etc.), machine learning-based regression models (such as random forest, support vector machine, etc.), and deep learning models (such as convolutional neural network CNN, recurrent neural network RNN ​​and its variants LSTM, GRU, etc.).

[0088] Specifically in this embodiment, a deep learning-based image prediction model is used to predict and generate the third missing area using information of remote sensing images of different modalities in the same area of ​​the third image, thereby obtaining a completed image.

[0089] The third image is a remote sensing image of the same area but in a different mode and contains image information of the third missing area.

[0090] The deep learning-based image prediction model mainly includes two structures: a structure generator and an image generator.

[0091] The process of predicting and generating the filled image by the image prediction model mainly includes two stages: structure prediction and image generation. The specific process is: the structure generator is used to predict the image structure information of the missing area by generating an edge map, and a complete image structure map is generated. Then the predicted image structure map is used as prior knowledge, combined with the image information of the known area in the image to guide the image generator to predict and generate the image of the missing area to obtain the filled image.

[0092] The specific process of using the structure generator to generate an edge map to predict the image structure information of the missing area is as follows:

[0093] 1. Use the Canny edge detector to perform edge detection on the third image and remote sensing images of different modes in the same area of ​​the third image, and generate an edge map for each.

[0094] 2. Input the two edge maps into the structure generator to predict and generate the complete image structure map of the third image.

[0095] The specific process of the image generator predicting and generating the image of the missing area is as follows:

[0096] 1. The complete image structure map generated in the structure generator is used as the structural prior information of the third image and is input into the image generator together with the third image.

[0097] 2. According to the structural information of the third missing area in the image structure diagram, the image information of the relevant structure is obtained in the known area of ​​the third image. According to the image information, the spectrum, texture and other information on the corresponding structure of the third missing area are predicted and supplemented, so as to generate an image with complete information, that is, to obtain the target image.

[0098] The structure generator and image generator both include a convolutional encoder and decoder structure, wherein the image generator also includes a global transformer and a local transformer, the global transformer aggregates high-level image context from the encoder from a global perspective and propagates the encoded global representation to the decoder in a multi-scale manner, and the local transformer extracts low-level image details within a local neighborhood; through the global transformer and the local transformer, the structure generator can simultaneously perform global relationship modeling and local detail encoding, significantly improving the realism of image generation in missing areas.

[0099] To implement the above method, the present invention also provides a remote sensing image filling system, which specifically includes:

[0100] The image acquisition module is configured to acquire remote sensing image data and pre-process the remote sensing image, wherein the remote sensing image data includes the image to be filled and the remote sensing image in the remote sensing image database.

[0101] The image database is configured to store various remote sensing image data, including remote sensing images and metadata corresponding to the images.

[0102] The image retrieval module is configured to set retrieval conditions according to metadata information of the image to be filled, search the image database, and obtain each filled image set.

[0103] The image filling module is configured to fill the image to be filled using various filling strategies, and finally obtain a completed filled image.

[0104] The image output module is configured to output the filled image.

[0105] The data acquisition module performs preprocessing on the remote sensing image, specifically including radiation calibration, atmospheric correction and orthorectification.

[0106] The image filling module includes a first image filling submodule, a second image filling submodule, a third image filling submodule and a fourth image filling submodule.

[0107] The first image filling submodule is used to adopt a first filling strategy to fill the missing area of ​​the image to be filled, so as to obtain a first image.

[0108] The second image filling submodule is used to adopt a second filling strategy to fill the missing area of ​​the first image to obtain a second image.

[0109] The third image filling submodule is used to adopt a third filling strategy to fill the missing area of ​​the second image to obtain a third image.

[0110] The fourth image filling submodule is used to adopt a fourth filling strategy to fill the missing area of ​​the third image to obtain a filled image.

[0111] The present invention discloses an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a remote sensing image filling method in any one of the embodiments 1 disclosed in the present invention are implemented.

[0112] Figure 4 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 4As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0113] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] The embodiment of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the remote sensing image filling method of any one of the embodiments of the present invention are implemented.

[0115] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0117] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention.

Claims

1. A remote sensing image filling method, characterized in that: include: S1: obtaining image data to be filled and a remote sensing image database, wherein the remote sensing image database includes a plurality of remote sensing image data; S2 obtains the missing area of ​​the image to be filled based on the image to be filled, and sequentially fills the missing area of ​​the image to be filled using the first filling strategy, the second filling strategy, the third filling strategy and the fourth filling strategy until the missing area is filled and the target image is obtained; The first filling strategy includes: obtaining a first search condition based on metadata of the image to be filled and location information of the missing area of ​​the image to be filled, using the first search condition to retrieve a first filling image set in a remote sensing image database, and filling the missing area of ​​the image to be filled with the first filling image set; The second filling strategy includes: obtaining a second search condition based on the metadata of the image to be filled and the location information of the first missing area, searching and screening the remote sensing image database using the second search condition to obtain a second filling image set, and filling the first missing area with the second filling image set; the first missing area is the area that has not been filled by the first filling strategy; The third filling strategy includes: setting a third search condition, retrieving a low-resolution image set in a remote sensing image database, using a super-resolution reconstruction method to improve the images in the low-resolution image set to the resolution level of the image to be filled, obtaining a third filling image set, and filling the second missing area with the third filling image set; the second missing area is the area that has not been filled by the second filling strategy; The fourth filling strategy includes: constructing an image prediction model, using the image prediction model to predict and generate the third missing area to obtain a target image with the filling completed; the third missing area is the area that has not been filled by the third filling strategy.

2. A remote sensing image filling method according to claim 1, characterized in that: The second search condition is: expanding the acquisition time in the first search condition, and also expanding the resolution of the image to a higher resolution than the image to be filled; The acquisition time is extended within a set interval; the set interval represents a time range where there is no difference in ground feature due to seasonal climate; The process of screening and obtaining the second filling image set is: searching the remote sensing image database according to the second search condition to obtain multiple candidate second filling images, respectively obtaining the feature difference value between each candidate second filling image and the image to be filled, and selecting the image whose feature difference value is less than the threshold as the second filling image set; The characteristic difference value is the area ratio of the changed region of the candidate second filling image compared with the image to be filled.

3. The remote sensing image filling method according to claim 1, characterized in that: The third search condition includes: the same acquisition time as the image to be filled, lower resolution than the image to be filled, and covering the position information of the second missing area.

4. The remote sensing image filling method according to claim 1, characterized in that: The fourth filling strategy uses an image prediction model to predict and generate the third missing area, including at least one of the three methods of using the image information of the image to be filled itself for prediction, using the information of the phase image group of the image to be filled for prediction, or using the information of remote sensing images of different modalities in the same area as the image to be filled for prediction. Different image prediction models can be selected according to different prediction methods.

5. The remote sensing image filling method according to claim 1, characterized in that: The method further includes: The first filling image set is composed of remote sensing images of the same type that contain the missing area of ​​the image to be filled and have the same acquisition time and resolution as the image to be filled; The second filled image set is composed of images that contain the first missing area and meet the screening condition; The third filling image set is composed of super-resolution images that are acquired at the same time as the image to be filled and contain the second missing area; the super-resolution images are obtained by super-resolving and reconstructing the low-resolution images.

6. A remote sensing image filling system, characterized in that: The system is used to implement a remote sensing image filling method according to any one of claims 1 to 5, specifically comprising: An image acquisition module is configured to acquire remote sensing image data and pre-process the remote sensing image, wherein the remote sensing image data includes images to be filled and remote sensing images in a remote sensing image database; An image database is configured to store various remote sensing image data, including remote sensing images and metadata corresponding to the images; An image retrieval module is configured to set retrieval conditions according to metadata information of the image to be filled, search the image database, and obtain each filled image set; An image filling module is configured to fill the image to be filled using various filling strategies, and finally obtain a completed filled image; The image output module is configured to output the filled image.

7. A remote sensing image filling system according to claim 6, characterized in that: The image filling module includes a first image filling submodule, a second image filling submodule, a third image filling submodule and a fourth image filling submodule; The first image filling submodule is used to adopt a first filling strategy to fill the missing area of ​​the image to be filled, so as to obtain a first image; The second image filling submodule is used to fill the missing area of ​​the first image with a second filling strategy to obtain a second image; The third image filling submodule is used to fill the missing area of ​​the second image with a third filling strategy to obtain a third image; The fourth image filling submodule is used to adopt a fourth filling strategy to fill the missing area of ​​the third image to obtain a filled image.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the remote sensing image filling method described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the remote sensing image filling method described in any one of claims 1 to 5 are implemented.

Images