An object-oriented filling method for satellite time series images

Abstract

The present application discloses an object-oriented filling method for satellite time series images, including: performing pixel-level quality control on the satellite time series images to mark the pixels contaminated by clouds and cloud shadows; segmenting and classifying the two clearest images with the shortest time interval from the target image to generate a segmentation map and a classification map to form independent objects with different category labels; finding similar pixels corresponding to the missing pixels of the target image and selecting reference images; adaptively filling the missing pixels with a single reference image or two reference images according to the comparison result between the correlation between the similar pixels in the target image and the reference image and a preset threshold; performing image post-processing on the filled target image. This can quickly and accurately identify available similar pixels in the time series that are not interfered by clouds and cloud shadows for data filling, reduce the dependence on reference images, and improve the prediction accuracy for areas where surface cover changes occur.

Description

Technical Field

[0001] The present invention relates to the field of data filling, and particularly to an object-oriented filling method for satellite time series images. Background Art

[0002] The PlanetScope satellite is currently the largest microsatellite constellation in the world. Since January 2014, more than 180 microsatellites have been launched in a cluster by the PlanetScope satellite, forming a global earth observation network, achieving nearly daily global data updates. The imaging spatial resolution of this data reaches 3m, including four spectral bands: blue, green, red, and near-infrared, providing unprecedented opportunities for monitoring fine-scale changes on the earth's surface. However, like other traditional optical satellites, the PlanetScope satellite also faces a key problem, that is, frequently occurring clouds and cloud shadows will interfere with visible light signals, resulting in data loss under clouds and cloud shadows, greatly limiting the wide application of PlanetScope satellite remote sensing observations. Especially for applications such as monitoring surface cover changes and phenology monitoring that require continuous time series observations, it is extremely disadvantageous. Therefore, it is necessary to fill in the missing data. However, there is currently no data filling method for the PlanetScope satellite.

[0003] The existing data filling methods mainly fill in medium-resolution or coarse-resolution satellite data and are not applicable to the emerging PlanetScope satellite data, mainly for the following two reasons: (1) As the number of missing pixels increases, the accuracy and efficiency of the existing data filling methods will be greatly reduced. Due to the ultra-high spatial resolution of microsatellites compared to traditional satellites, the chance of large areas of clouds or cloud shadows appearing is greater, resulting in larger data losses. (2) The accuracy of the existing data filling methods depends on the selection of adjacent reference images, and the accuracy is higher for areas where the surface cover type has not changed, while the uncertainty is greater for areas where rapid surface cover type changes have occurred. Due to the ultra-high spatial resolution of microsatellites, large-area data losses and landscape heterogeneity make it difficult to select appropriate reference images for data filling.

[0004] Therefore, how to solve the problem of data loss caused by cloud shadow pollution of images for microsatellites (such as PlanetScope satellites) is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an object-oriented filling method for satellite time series images, which can quickly and accurately identify spatially similar pixels that are available in the time series and are not interfered by clouds and cloud shadows for data filling, reducing the dependence on reference images. The specific scheme is as follows:

[0006] An object - oriented filling method for satellite time - series images, comprising:

[0007] Performing pixel - level quality control on the satellite time - series images, and marking the pixels contaminated by clouds and cloud shadows;

[0008] Segmenting and classifying the two clearest images with the shortest time interval from the target image to generate a segmentation map and a classification map, so as to form independent objects with different class labels;

[0009] According to the independent objects with different class labels, finding similar pixels corresponding to the missing pixels of the target image, and selecting a reference image;

[0010] According to the comparison result between the correlation between the similar pixels in the target image and the reference image and a preset threshold, adaptively filling the missing pixels with a single reference image or two reference images;

[0011] Performing image post - processing on the filled target image.

[0012] Preferably, in the above - mentioned object - oriented filling method for satellite time - series images provided by the embodiments of the present invention, before performing pixel - level quality control on the satellite time - series images, it further includes:

[0013] Performing initial cloud and cloud shadow marking on the satellite time - series images through an automatic cloud and cloud shadow detection algorithm or a manual cloud and cloud shadow marking method.

[0014] Preferably, in the above - mentioned object - oriented filling method for satellite time - series images provided by the embodiments of the present invention, the performing pixel - level quality control on the satellite time - series images and marking the pixels contaminated by clouds and cloud shadows includes:

[0015] Using morphological dilation operation to mark the thin clouds or thin cloud shadows around the initially marked clouds and cloud shadows, and the number of marks depends on the size and shape of the custom structural element for processing the image;

[0016] Marking the pixels that exceed the preset range in the pixel time series.

[0017] Preferably, in the above - mentioned object - oriented filling method for satellite time - series images provided by the embodiments of the present invention, the segmenting and classifying the two clearest images with the shortest time interval from the target image to generate a segmentation map and a classification map includes:

[0018] Segmenting the two clearest images with the shortest time interval from the target image into independent objects according to spectral and spatial features to generate a segmentation map;

[0019] An unsupervised classifier is used to automatically divide the two clear images into multiple categories according to the spectral similarity of pixels, generating a classification map.

[0020] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the step of dividing the two clear images with the shortest time interval from the target image into independent objects according to spectral and spatial features to generate a segmentation map includes:

[0021] In the entire time series, one clear image with the shortest time interval from the target image is selected before and after the target image respectively to form an image group.

[0022] A Laplace filter is used to sharpen the image group.

[0023] A Canny edge detection operator is used to extract the edges of the image group.

[0024] A watershed segmentation algorithm is used to divide the image group into independent single objects, generating a segmentation map.

[0025] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the step of finding similar pixels corresponding to the missing pixels of the target image and selecting reference images according to the independent objects with different category labels includes:

[0026] If some pixels are missing, search the entire time series to find clear pixels that belong to the same object and category and are not contaminated by clouds and cloud shadows, and mark them as the similar pixels.

[0027] If all pixels are missing, find clear pixels that belong to the same category but are the spatially closest objects and are not contaminated by clouds and cloud shadows, and mark them as the similar pixels.

[0028] Select the first reference image that is closest to the target image before the similar pixels, and the second reference image that is closest to the target image after the similar pixels.

[0029] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the step of adaptively filling the missing pixels with a single reference image or two reference images according to the comparison result of the correlation between the similar pixels in the target image and the reference images and a preset threshold includes:

[0030] Select the single reference image with the shortest absolute distance from the two reference images.

[0031] Compare the average value of the correlation between the target image of the similar pixels in different bands and the selected single reference image with the preset threshold;

[0032] If the average value of the correlation is greater than the preset threshold, it is determined that no surface cover change has occurred, and the selected single reference image is used to fill the missing pixels;

[0033] If the average value of the correlation is less than the preset threshold, it is determined that a surface cover change has occurred, and two reference images are used to fill the missing pixels.

[0034] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the step of using the selected single reference image to fill the missing pixels includes:

[0035] For a specific object and category, establish a first linear model; the first linear model is:

[0036] I t0，b，i = a hk，b × I tn，b，i + ε hk，b

[0037] where I t0，b，i and I tn，b，i are the spectral reflectance values of the similar pixel i in the band b of the target image I t0 and the selected single reference image I tn respectively; a hk，b and ε hk，b are the parameters of the first linear model, obtained by linear regression;

[0038] Fill the missing pixel j using the first linear model obtained by solving:

[0039]

[0040] where is the predicted reflectance value of the missing pixel j in the band b of the target image I t0 and I tn，b，j is the reflectance value of the missing pixel j in the reference image I tn in the band b.

[0041] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the step of using two reference images to fill the missing pixels includes:

[0042] For a specific object and category, establish a second linear model; the second linear model is:

[0043] I t0，b，i -I t0+1，b，i = a′ hk，b (I t0-1，b，i -I t0+1，b，i ) + ε hk，b

[0044] where I t0，b，i , I t0-1，b，i and I t0+1，b，i are the spectral reflectance values of similar pixel i in the target image I t0 and two selected reference images I t0-1 , I t0+1 in band b, and a′ hk，b , a″ hk，b and ε hk，b are the parameters of the second linear model, obtained by linear regression;

[0045] The missing pixel j is filled by the second linear model obtained by solving:

[0046]

[0047] where is the predicted reflectance value of the missing pixel j in the target image I t0 in band b, I t0-1，b，j is the reflectance value of the missing pixel j in the reference image I t0-1 in band b, and I t0+1，b，j is the reflectance value of the missing pixel j in the reference image I t0+1 in band b.

[0048] Preferably, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, the post-processing of the filled target image includes:

[0049] Denoising the filled target image using a guided filter.

[0050] As can be seen from the above technical solutions, an object-oriented filling method for satellite time series images provided by the present invention includes: performing pixel-level quality control on satellite time series images, and marking the pixels contaminated by clouds and cloud shadows; segmenting and classifying the two clear images with the shortest time interval from the target image to generate a segmentation map and a classification map, so as to form independent objects with different class labels; according to the independent objects with different class labels, finding similar pixels corresponding to the missing pixels of the target image, and selecting a reference image; adaptively filling the missing pixels with a single reference image or two reference images according to the comparison result of the correlation between the similar pixels in the target image and the reference image and a preset threshold; performing image post-processing on the filled target image.

[0051] By combining segmentation and classification algorithms to generate a segmentation map and a classification map respectively, the present invention can quickly and accurately identify available spatially similar pixels in the time series that are not interfered by clouds and cloud shadows for data filling, especially in the case of large-area data loss. In addition, according to the comparison result of the correlation between the similar pixels in the target image and the reference image and a preset threshold, different reference image scenarios are adaptively used to fill the missing pixels, which can reduce the dependence on the reference image and the influence of landscape heterogeneity, and improve the prediction accuracy of the areas where land cover changes occur. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0053] Figure 1 It is a flowchart of the object-oriented filling method for satellite time series images provided by the embodiments of the present invention;

[0054] Figure 2 It is a schematic structural diagram of the object-oriented filling method for satellite time series images provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0056] The present invention provides an object - oriented filling method for satellite time - series images, as Figure 1 shown, which includes the following steps:

[0057] S101. Perform pixel - level quality control on the satellite time - series images, and mark the pixels contaminated by clouds and cloud shadows;

[0058] It should be noted that the present invention can be applied to small satellites such as PlanetScope satellites, and can also be applied to other high - spatio - temporal - resolution optical satellites, such as: Gaofen - 1 satellite, SkySat satellite, SPOT - 7 satellite, etc. The above - mentioned time - series images can be entirely or partially clear and cloud - free. The above - mentioned pixel - level quality control refers to evaluating the quality of each pixel in the image pixel - by - pixel, excluding pixels with low quality, so as to minimize the influence of cloud and cloud - shadow contamination, marking the pixels contaminated by clouds and cloud shadows, and excluding them in subsequent operations.

[0059] S102. Segment and classify the two clearest images with the shortest time interval from the target image to generate a segmentation map and a classification map, so as to form independent objects with different category labels;

[0060] In practical applications, the above - mentioned target image is an image contaminated by clouds and cloud shadows in a certain satellite time - series image, and the present invention needs to fill the missing pixels of this image.

[0061] S103. According to the independent objects with different category labels, find the similar pixels corresponding to the missing pixels of the target image, and select the reference images;

[0062] Specifically, similar pixels (i.e., spatially adjacent and spectrally similar pixels) are the pixels with values in the neighborhood of the missing pixels, which are used to fill the missing pixels. In the present invention, the missing pixels and the similar pixels have similar time - change patterns.

[0063] S104. According to the comparison result between the correlation between the similar pixels in the target image and the reference image and a preset threshold, adaptively use a single reference image or two reference images to fill the missing pixels;

[0064] It should be noted that for each object with missing pixels in the present invention, it is divided into two filling scenarios, namely the single - reference - image scenario and the two - reference - image scenario, according to whether the correlation between the similar pixels in the target and reference images is greater than the preset threshold, and data filling is performed.

[0065] S105. Perform image post - processing on the filled target image.

[0066] The above step of performing image post - processing on the filled image can reduce random noise while not losing detail information.

[0067] In the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, by combining segmentation and classification algorithms to generate a segmentation map and a classification map respectively, it is possible to quickly and accurately identify spatially similar pixels that are available in the time series and not affected by clouds and cloud shadows for data filling, especially in the case of large-area data missing. In addition, according to the comparison result between the correlation between similar pixels in the target image and the reference image and a preset threshold, different reference image scenarios are adaptively used to fill the missing pixels, which can reduce the dependence on the reference image and the influence of landscape heterogeneity, and improve the prediction accuracy of the areas where land cover changes occur.

[0068] Furthermore, in specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, before performing step S101 for pixel-level quality control of the satellite time series images, it may further include: initially marking clouds and cloud shadows in the satellite time series images by an automatic cloud and cloud shadow detection algorithm or a manual cloud and cloud shadow marking method.

[0069] Specifically, the present invention may first use the automatic cloud and cloud shadow detection algorithm STI-ACSS method to initially mark clouds and cloud shadows in the satellite time series images, and then further mark the pixels contaminated by clouds and cloud shadows that are not marked by the STI-ACSS method through pixel-level quality control. In practical applications, the present invention may also not use the STI-ACSS method for initial cloud and cloud shadow marking, but use the cloud and cloud shadow marking carried by satellite products, other cloud and cloud shadow detection algorithms, or manual cloud and cloud shadow marking methods, etc., which are not limited herein.

[0070] In specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, in step S101, pixel-level quality control is performed on the satellite time series images, and the pixels contaminated by clouds and cloud shadows are marked. Specifically, it may include: First, morphological dilation operation is used to mark the thin clouds or thin cloud shadows around the initially marked clouds and cloud shadows. The number of marked areas depends on the size and shape of the custom structural element used to process the image (for example, using a 5×5 disk-shaped structural element with the center point as the center of the circle). Here, the structural element refers to the most basic component of the morphological dilation operation, which is used to perform operations on the image to be processed and is usually much smaller than the image to be processed. The two-dimensional planar structural element consists of a matrix with values of 0 or 1. The origin of the structural element specifies the pixel range to be processed in the image, and the points with a value of 1 in the structural element determine whether the neighboring pixels of the structural element need to participate in the calculation during the dilation operation. Then, the pixels exceeding the preset range in the pixel time series are marked, that is, the outliers in the pixel time series, namely the pixels greater than the user-defined upper limit (such as 99th percentile) or less than the lower limit (such as 1st percentile), are marked. The pixels marked once or twice through the above two steps are considered to be the pixels contaminated by clouds and cloud shadows.

[0071] It should be noted that the existing methods basically only mark the cloud and cloud shadow pixels based on the existing cloud and cloud shadow detection algorithms, without further performing pixel-level quality control. However, since no cloud and cloud shadow detection algorithm is perfect, the present invention uses user-predefined structural elements and upper and lower limits, and further quality control is performed through morphological dilation operation and time series outlier detection, so as to minimize the impact of cloud and cloud shadow contamination.

[0072] In specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, in step S102, the two clear images with the shortest time interval from the target image are segmented and classified to generate a segmentation map and a classification map, as Figure 2 shown. Specifically, it may include: First, the two clear images with the shortest time interval from the target image are segmented into independent objects according to spectral and spatial features to generate a segmentation map S ts , where the independent objects can have irregular sizes; then, an unsupervised classifier (such as a k-means classifier) is used to automatically divide the two clear images into multiple categories according to the spectral similarity of the pixels to generate a classification map C ts . That is to say, the present invention combines object-oriented segmentation and unsupervised classification algorithms to define the object attributes and category attributes of pixels for marking similar pixels, which can minimize the uncertainty during the filling of large-area data missing and improve the efficiency of marking similar pixels.

[0073] Further, in specific implementation, in the above steps, the two clear images with the shortest time interval from the target image are segmented into independent objects according to spectral and spatial features to generate a segmentation map, which may specifically include the following steps:

[0074] Step 1: In the entire time series, select one clear image with the shortest time interval from the target image before the target image and one clear image with the shortest time interval from the target image after the target image respectively to form image group I ts ;

[0075] Step 2: Use a Laplace filter to sharpen the image group I ts to enhance the spatial details of the image;

[0076] Step 3: Use a Canny edge detection operator to extract the edges of the image group I ts ;

[0077] Step 4: Use a watershed segmentation algorithm to divide the image group I ts into independent single objects to generate a segmentation map S ts 。

[0078] In specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, in the process of performing step S103 to find similar pixels corresponding to the missing pixels of the target image according to independent objects with different category labels and select a reference image, finding similar pixels can be specifically divided into two cases: the first case is that if some pixels are missing, search the entire time series to find clear pixels that belong to the same object (obtained from the segmentation map) and category (obtained from the classification map) and are not contaminated by clouds and cloud shadows, and mark them as similar pixels; the second case is that if all pixels are missing, find clear pixels that belong to the same category but are spatially closest to the object and are not contaminated by clouds and cloud shadows, and mark them as similar pixels. In this way, pixels belonging to the same category and the same object or adjacent objects of the same category are quickly and accurately marked as similar pixels for data filling, especially in the case of large-area data loss. In addition, the method for selecting a reference image may include: selecting the first reference image I t0 nearest to the target image before the similar pixels, and the second reference image I t0-1 nearest to the target image after the similar pixels t0 。 t0+1 。

[0079] In specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, in step S104, according to the comparison result between the correlation between the similar pixels in the target image and the reference image and a preset threshold, adaptively use a single reference image or two reference images to fill the missing pixels, such asFigure 2 As shown, it includes: selecting a single reference image (I t0-1 , I t0+1 ) with the shortest absolute distance from two reference images; tn , which may be I t0-1 or I t0+1 ); comparing the average value of the correlation between similar pixels in different bands of the target image I t0 and the selected single reference image I tn with the preset threshold r T ; if the average value of the correlation is greater than the preset threshold r T , it is determined that no surface cover change has occurred, and the selected single reference image I tn is used to fill in the missing pixels; if the average value of the correlation is less than the preset threshold r T , it is determined that a surface cover change has occurred, and two reference images (I t0-1 , I t0+1 ) are used to fill in the missing pixels. It should be noted that the preset threshold r T can be 0.75 - 0.80. The present invention selects the reference image with the shortest time interval and the highest similarity for data filling for each specific object and category, and adopts different filling scenarios according to whether surface cover change has occurred, which can minimize the uncertainty of filling in the area with rapid surface cover change, greatly reduce the dependence on reference images, and the impact of landscape heterogeneity.

[0080] Further, in specific implementation, using the selected single reference image I tn to fill in the missing pixels in the above steps specifically includes:

[0081] For a specific object h and category k, establish a first linear model; the first linear model is:

[0082] I t0，b，i = a hk，b × I tn，b，i + ε hk，b (1)

[0083] Wherein, I t0，b，i and I tn，b，i are the spectral reflectance values of the similar pixel i in the band b of the target image I t0 and the selected single reference image I tn respectively; a hk，b and ε hk，b are the parameters of the first linear model, which can be obtained by linear regression;

[0084] Using the first linear model obtained by solving to the target image I t0Fill in the missing pixel j:

[0085]

[0086] wherein, is the reflectance value of the predicted missing pixel j in the target image I t0 in band b, and I tn，b，j is the reflectance value of the missing pixel j in the reference image I tn in band b, while and are parameters obtained by formula (1).

[0087] Furthermore, in specific implementation, in the above steps, two reference images (I t0-1 , I t0+1 ) are used to fill in the missing pixels, which may specifically include:

[0088] Establish a second linear model for a specific object h and category k; the second linear model is:

[0089] I t0，b，i = a' hk，b × I t0-1，b，i + a'' hk，b × I t0+1，b，i + ε hk，b (3)

[0090] a' hk，b + a'' hk，b = 1 (4)

[0091] Substitute formula (4) into formula (3) to obtain:

[0092] I t0，b，i - I t0+1，b，i = a' hk，b (I t0-1 , b, i - I t0+1，b，i ) + ε hk，b (5)

[0093] wherein, I t0，b，i , I t0-1，b，i and I t0+1，b，i are the spectral reflectance values of the similar pixel i in band b of the target image I t0 and the two selected reference images I t0-1 , I t0+1 respectively, and a' hk，b , a'' hk，b and ε hk，b are the parameters of the second linear model and can be obtained by linear regression;

[0094] Fill in the missing pixel j with the second linear model obtained by solving:

[0095]

[0096] Among them, is the reflectance value of the predicted missing pixel j in the target image I t0 in band b, and I t0-1，b，j is the reflectance value of the missing pixel j in the reference image I t0-1 in band b, and I t0+1，b，j is the reflectance value of the missing pixel j in the reference image I t0+1 in band b, while and are parameters obtained by formula (5).

[0097] In the above steps, it can be seen that based on the assumption that the missing pixel and spatially adjacent and spectrally similar pixels (abbreviated as similar pixels) have similar temporal variation patterns, the present invention can establish a model to capture this temporal variation pattern and then apply the model to the missing pixel to achieve data filling.

[0098] It should be noted that most of the existing methods select reference images for data filling for the entire image or the entire data missing block. However, the present invention selects reference images for specific objects and categories, fully considering the influence of landscape heterogeneity on the selection of reference images. In addition, most of the existing methods use single-scene or multi-scene reference images for data filling without considering the situation of land cover change. However, the present invention adaptively uses single-scene or two-scene reference image filling scenarios according to the presence or absence of land cover change, thereby improving the prediction accuracy of the areas where land cover changes occur.

[0099] In specific implementation, in the above object-oriented filling method for satellite time series images provided by the embodiments of the present invention, step S105 performs image post-processing on the filled target image, which may specifically include: denoising the filled target image using a guided filter. The guided filter is a filter that preserves edge characteristics. By filtering the input image through a guidance map, the finally output image is generally similar to the initial image, but the texture part is similar to the guidance map. The present invention selects a clear and cloud-free image in the time series images as the guidance image to filter the filled target image, which can effectively reduce the interference of random noise and at the same time preserve the detail information of the filled image.

[0100] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0101] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0102] The steps of the methods or algorithms described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0103] In summary, an object-oriented filling method for satellite time series images provided by an embodiment of the present invention includes: performing pixel-level quality control on satellite time series images, and marking the pixels contaminated by clouds and cloud shadows; segmenting and classifying the two clear images with the shortest time interval from the target image to generate a segmentation map and a classification map, so as to form independent objects with different category marks; according to the independent objects with different category marks, finding similar pixels corresponding to the missing pixels of the target image, and selecting a reference image; adaptively filling the missing pixels with a single reference image or two reference images according to the comparison result of the correlation between the similar pixels in the target image and the reference image and a preset threshold; performing image post-processing on the filled target image. In this way, by combining segmentation and classification algorithms to generate a segmentation map and a classification map respectively, it is possible to quickly and accurately identify available spatial similar pixels in the time series that are not disturbed by clouds and cloud shadows for data filling, especially in the case of large-area data loss. In addition, according to the comparison result of the correlation between the similar pixels in the target image and the reference image and a preset threshold, different reference image scenarios are adaptively used to fill the missing pixels, which can reduce the dependence on the reference image and the influence of landscape heterogeneity, and improve the prediction accuracy of the areas where land cover changes occur.

[0104] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0105] The above has introduced in detail the object-oriented filling method for satellite time series images provided by the present invention. Specific examples are used in this text to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An object-oriented filling method for satellite time-series images, characterized in that, Including: Performing pixel-level quality control on satellite time-series images, and marking the pixels contaminated by clouds and cloud shadows; The pixel-level quality control refers to evaluating the quality of image pixels pixel by pixel, using morphological dilation operation to mark the thin clouds or thin cloud shadows around the initially marked clouds and cloud shadows. The number of marked pixels depends on the size and shape of the custom structural element used to process the image, and marking the pixels in the pixel time series that exceed the preset range, so as to mark the pixels contaminated by clouds and cloud shadows; Segmenting and classifying the two clear images with the shortest time interval from the target image to generate a segmentation map and a classification map, so as to form independent objects with different category markings; According to the independent objects with different category markings, searching for similar pixels corresponding to the missing pixels of the target image, and selecting a reference image; if some pixels are missing, searching the entire time series to find clear pixels that belong to the same object and category but are not contaminated by clouds and cloud shadows, and marking them as the similar pixels; If all pixels are missing, searching for clear pixels that belong to the same category but are the spatially closest objects and are not contaminated by clouds and cloud shadows, and marking them as the similar pixels; According to the comparison result between the correlation between the similar pixels in the target image and the reference image and a preset threshold, adaptively filling the missing pixels with a single reference image or two reference images; Performing image post-processing on the filled target image.

2. The object-oriented filling method for satellite time series images according to claim 1, characterized in that Before performing pixel-level quality control on the satellite time-series images, it further includes: Performing initial cloud and cloud shadow marking on the satellite time-series images through an automatic cloud and cloud shadow detection algorithm or a manual cloud and cloud shadow marking method.

3. The object-oriented filling method for satellite time series images according to claim 1, wherein The segmenting and classifying the two clear images with the shortest time interval from the target image to generate a segmentation map and a classification map includes: Segmenting the two clear images with the shortest time interval from the target image into independent objects according to spectral and spatial features to generate a segmentation map; Using an unsupervised classifier to automatically divide the two clear images into multiple categories according to the spectral similarity of the pixels to generate a classification map.

4. The object-oriented filling method for satellite time series images according to claim 3, characterized in that The segmenting the two clear images with the shortest time interval from the target image into independent objects according to spectral and spatial features to generate a segmentation map includes: In the entire time series, respectively selecting one clear image with the shortest time interval from the target image before the target image and one clear image with the shortest time interval from the target image after the target image to form an image group; Performing image sharpening on the image group using a Laplace filter; Performing edge extraction on the image group using a Canny edge detection operator; Using a watershed segmentation algorithm to divide the image group into independent single objects to generate a segmentation map.

5. The object-oriented filling method for satellite time series images according to claim 4, wherein The searching for similar pixels corresponding to the missing pixels of the target image and selecting a reference image according to the independent objects with different category markings includes: Selecting the first reference image closest to the target image before the similar pixels, and the second reference image closest to the target image after the similar pixels.

6. The object-oriented filling method for satellite time series images according to claim 5, wherein: Adapting to use a single reference image or two reference images to fill the missing pixels according to the comparison result between the correlation of the similar pixels between the target image and the reference image and a preset threshold, includes: Selecting a single reference image with the closest absolute distance from two reference images; Comparing the average value of the correlation between the similar pixels in the target image in different bands and the selected single reference image with the preset threshold; If the average value of the correlation is greater than the preset threshold, it is determined that no land cover change has occurred, and the selected single reference image is used to fill the missing pixels; If the average value of the correlation is less than the preset threshold, it is determined that a land cover change has occurred, and two reference images are used to fill the missing pixels.

7. The object-oriented filling method for satellite time series images according to claim 6, characterized in that The filling of the missing pixels with the selected single reference image includes: For specific objects and categories, establishing a first linear model; the first linear model is: I t0，b，i = a hk，b × I tn，b，i + ε hk，b Among them, I t0,b，i and I tn，b，i are the values of the spectral reflectance of the similar pixel i in the band b of the target image I t0 and the selected single reference image I tn ; a hk，b and ε hk，b are the parameters of the first linear model, obtained by solving through linear regression; Filling the missing pixel j through the first linear model obtained by solving; Among them, is the reflectance value of the missing pixel j in the target image I t0 in band b, and I tn，b，j is the reflectance value of the missing pixel j in the reference image I tn in band b, and are parameters obtained from the formula of the first linear model.

8. The object-oriented filling method for satellite time series images according to claim 6, characterized in that, The filling of the missing pixels with two reference images includes: For specific objects and categories, establishing a second linear model; the second linear model is: I t0,b,i -I t0+1，b，i = a' hk，b (I t0-1，b，i -I t0+1，b,i ) + ε hk，b Among them, I t0，b，i , I t0-1，b,i and I t0+1，b，i are the values of the spectral reflectance of the similar pixel i in the target image I t0 and the two selected reference images I t0-1 , I t0+1 in the band b, a′ hk，b and ε hk，b are the parameters of the second linear model, obtained by solving through linear regression; Filling the missing pixels through the second linear model obtained by solving; Among them, is the reflectance value of the missing pixel j predicted in the target image I t0 in band b, and I t0-1，b，j is the reflectance value of the missing pixel j in the reference image I t0-1 in band b, and I t0+1，b，j is the reflectance value of the missing pixel j in the reference image I t0+1 in band b, and are parameters obtained from the formula of the second linear model.

9. The object-oriented filling method for satellite time series images according to claim 1, characterized in that The post-processing of the target image after filling includes: Using a guided filter to perform denoising processing on the filled target image.

Images