A cloud-free remote sensing image synthesis method and system considering vegetation interference events

Abstract

The application relates to a cloud-free remote sensing image synthesis method and system considering vegetation interference events. For each input image in multi-temporal input remote sensing images, a target day weight value W1 of a pixel at a target position on the input image is determined, a target distance weight value W2 of the pixel at the target position on the input image is determined, a cloud and fog influence weight value W3 of the pixel at the target position on the input image is determined, and a pixel value of the pixel at the target position is determined according to the pixel value, the target day weight value W1, the target distance weight value W2 and the cloud and fog influence weight value W3 of the pixel at the target position of each input image. Since the best pixel selection criterion in the application identifies the best pixel as the weighted average value of all input pixels and pixel score weight values, the 'patch' problem caused by radiation differences of multi-temporal data in the synthesized image is effectively solved, the synthesized image has no obvious filling traces, and the consistency of the image picture is good.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a cloud-free remote sensing image synthesis method and system considering vegetation interference events. BACKGROUND

[0002] Forest resources are important material basis for national sustainable development, and are also renewable resources which are rare and renewable in economic construction and ecological construction. Monitoring and evaluation of forests help to understand the dynamic situation of forest resources in detail. With the continuous updating and development of global remote sensing technology, more and more medium and high resolution optical remote sensing images are obtained. Optical remote sensing images provide strong information support for earth resource exploration, aerial photography, surveying and mapping, and agricultural, forestry, water conservancy, and land resource investigation. Because the imaging quality of optical satellite sensors is easily affected by weather factors such as clouds and fog, the ground objects in the image are easily disturbed or blocked by a large amount of cloud / cloud shadow and fog, so it is difficult to obtain clear, complete and high-quality images. Cloud-free image synthesis technology emerges as the times require, and has become one of the rapidly developing remote sensing image processing technologies in recent years. Cloud-free image synthesis refers to using multi-period remote sensing images, selecting the best pixel that meets the synthesis standard by formulating the best pixel synthesis criterion, and finally obtaining cloud-free and clear high-quality remote sensing images.

[0003] Among the commonly used remote sensing image synthesis algorithms, the best available pixel (BAP) algorithm is relatively more flexible. Different evaluation rules can be set according to user requirements to determine the best pixel. The technical idea of BAP is to score all pixels of multi-temporal images by setting weight rules. For the same position pixel, the highest scorer is identified as the best pixel for pixel-level image synthesis. Users can set different pixel evaluation rules according to application requirements. However, due to the differences in imaging time, observation-sun geometry, ground object type change, and vegetation physiological state of multi-temporal input images, BAP uses the pixel with the highest score as the "best" pixel for synthesis, which is easy to make the filling area (original cloud / cloud shadow invalid area) show obvious "patch" phenomenon compared with other areas. SUMMARY

[0004] The purpose of the present application is to provide a cloud-free synthesized image with no obvious patches, a cloud-free remote sensing image synthesis method and system considering vegetation interference events.

[0005] To achieve the above purpose, the present application provides the following scheme:

[0006] A cloud-free remote sensing image synthesis method considering vegetation interference events, comprising the following steps:

[0007] For each input image in the multi-temporal input images:

[0008] determining a target day weight value W1 of a pixel at the target position on the input image according to an influence degree of a time length between a day of imaging of the input image and the target day on a quality of the pixel;

[0009] determining a target distance weight value W2 of the pixel at the target position on the input image according to an influence degree of a distance between the pixel at the target position on the input image and a cloud / cloud shadow on a quality of the pixel;

[0010] determining a cloud / fog influence weight value W3 of the pixel at the target position on the input image according to an influence degree of an interference of a cloud / fog on the pixel at the target position on the input image on a quality of the pixel;

[0011] determining a pixel value of the pixel at the target position according to the pixel value, the target day weight value W1, the target distance weight value W2 and the cloud / fog influence weight value W3 of the pixel at the target position on each of the input images.

[0012] Optionally, the specific steps of calculating the target day weight value W1 are as follows:

[0013] acquiring longitude and latitude information of the target position;

[0014] determining a growing season of vegetation in the target position according to the longitude and latitude information and a vegetation phenology change rule;

[0015] calculating the target day weight value according to the target day, imaging times of the multi-temporal input remote sensing images and a growing season time window; the growing season time window is determined according to the growing season of the vegetation in the target position.

[0016] Optionally, if the target day is a vegetation reduction moment, the method further comprises:

[0017] acquiring a vegetation growth condition model;

[0018] determining a vegetation reflectivity change curve of the target position with time according to the vegetation growth condition model;

[0019] determining the vegetation reduction moment of the target position according to the vegetation reflectivity change curve.

[0020] Optionally, the vegetation reduction moment is a breakpoint of the vegetation reflectivity change curve.

[0021] Optionally, the specific steps of calculating the target distance weight value W2 are as follows:

[0022] acquiring meteorological information of the target position;

[0023] judging whether there is a cloud / cloud shadow at the target position;

[0024] If yes, the target distance weight is 0;

[0025] If no, the distance between the target position and the nearest cloud / cloud shadow is calculated, denoted as adjacent distance;

[0026] If the adjacent distance is greater than a set threshold distance, the target distance weight is 1;

[0027] If the adjacent distance is less than the set threshold distance, the target distance weight W2 is calculated according to the following formula: Wherein, D req is the threshold distance, D i is the distance between the target pixel and the nearest adjacent cloud / cloud shadow pixel, and D min is the minimum value of D i .

[0028] Optionally, the specific steps of calculating the cloud / fog influence weight W3 are as follows:

[0029] According to the reflectivity of the input image spectral band, the cloud / fog influence weight W3 is calculated.

[0030] Optionally, the pixel value of the pixel at the target position is determined, specifically including:

[0031] Obtaining the multi-temporal input image weight parameter;

[0032] After the multi-temporal input image weight parameter is normalized, it is weighted and combined, specifically including:

[0033] Wherein, n is the number of input multi-temporal images, P i is the target pixel value of the i-th input image, W T is the total score of P i , P final is the synthesized cloud-free pixel value.

[0034] The application also provides a cloud-free remote sensing image synthesis system considering vegetation interference events, comprising a memory and a processor, the memory stores a computer program, and the processor realizes the method in the cloud-free remote sensing image synthesis method considering vegetation interference events when executing the computer program.

[0035] According to the specific embodiments of the present application, the present application discloses the following technical effects: the present application provides a cloud-free remote sensing image synthesis method considering vegetation interference events. For each input image in the multi-temporal input images, first, a target day weight value W1 of a pixel at a target position on the input image is determined according to an influence degree of a time length from an imaging day of the input image to a synthesis target day on the quality of the pixel, then a target distance weight value W2 of the pixel at the target position on the input image is determined according to an influence degree of a distance between the pixel at the target position on the input image and a cloud / cloud shadow on the quality of the pixel, and then a cloud / fog influence weight value W3 of the pixel at the target position on the input image is determined according to an influence degree of an interference of the cloud / fog on the input image on the quality of the pixel at the target position. Finally, a pixel value of the pixel at the target position is determined according to the pixel value of the pixel at the target position on each input image, the target day weight value W1, the target distance weight value W2 and the cloud / fog influence weight value W3. Since the pixel value of the pixel at the target position in the present application is a weighted average of all input pixels and pixel score weight values, the synthesized image has no obvious filling traces, and the consistency of the image picture is good. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description only constitute some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0037] Figure 1 A flow chart of a cloud-free remote sensing image synthesis method provided by the present application;

[0038] Figure 2 A vegetation reflectivity change curve of the present application;

[0039] Figure 3 A cloud-free synthesized remote sensing image (108.03°E-108.68°E, 22.78°N-23.15°N), wherein Figure 3 a is a BAP cloud-free synthesized remote sensing image, Figure 3 b is a WAP cloud-free synthesized remote sensing image;

[0040] Figure 4 A synthesized image and a local scene spectral profile curve, wherein Figure 4 a, Figure 4 e is a BAP synthesized image, Figure 4 c, Figure 4 g is a BAP local scene spectral profile curve; Figure 4 b, Figure 4 f is a WAP synthesized image of the present application;Figure 4 d、 Figure 4 h is the WAP local scene spectral profile curve;

[0041] Figure 5 The present application provides a cloud-free synthetic image considering forest change (forest reduction and forest increase), wherein Figure 5 a is a forest reduction case, Figure 5 b is a forest increase case. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0043] The present application aims to provide a cloud-free synthetic image considering vegetation interference events.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0045] The present application provides a cloud-free synthetic image considering vegetation interference events, comprising the following steps:

[0046] For each input image in the multi-temporal input images:

[0047] According to the influence degree of the time length from the imaging day to the synthetic target day on the quality of the pixel, a target day weight value W1 of the pixel at the target position on the input image is determined;

[0048] According to the influence degree of the distance between the pixel at the target position on the input image and the cloud / cloud shadow on the quality of the pixel, a target distance weight value W2 of the pixel at the target position on the input image is determined;

[0049] According to the influence degree of the interference of the cloud / fog on the quality of the pixel at the target position on the input image, a cloud / fog influence weight value W3 of the pixel at the target position on the input image is determined;

[0050] According to the pixel value, the target day weight value W1, the target distance weight value W2 and the cloud / fog influence weight value W3 of the pixel at the target position of each input image, the pixel value of the pixel at the target position is determined.

[0051] In the embodiment, the specific steps of calculating the target day weight value W1 can be as follows:

[0052] Obtain the latitude and longitude information of the target location.

[0053] Based on the latitude and longitude information, the growing season of vegetation within the target location is determined according to the phenological change patterns of vegetation.

[0054] The target day weight is calculated based on the time of vegetation reduction, the imaging time of the multi-temporal input image, and the growing season time window; the growing season time window is determined based on the growing season of the vegetation within the target location.

[0055] Specifically, the latitude and longitude information of the target location can be determined through satellite imagery.

[0056] Specifically, determining the growing season of vegetation within the target location based on the latitude and longitude information can be done as follows:

[0057] like Figure 2 As shown, the target location is within the range of 108.03°E-108.68°E and 22.78°N-23.15°N. Based on the latitude, longitude, and vegetation phenology information, the growing season of the target location can be determined to be approximately from March 1st to October 31st.

[0058] Specifically, the target day weight is calculated based on the vegetation reduction time, the imaging time of the multi-temporal input images, and the growing season time window. The specific formula is as follows:

[0059]

[0060] Where, x i To input the imaging time of the multi-temporal image, μ is the set target date for the composite image (i.e. the time of vegetation reduction), and σ is the input time window of the multi-temporal image (i.e. the number of growing season days at the target location). If the image coverage area is a subtropical or tropical region, the growing season time window σ is set to 120-150; if the image coverage area is a temperate or cold temperate region, σ is set to 60-90.

[0061] In some embodiments, if the target date for the synthesis is a time of vegetation reduction, the method further includes:

[0062] Obtain a model of vegetation growth status.

[0063] Based on the vegetation growth model, determine the vegetation reflectance change curve of the target location over time. Based on the vegetation reflectance change curve, determine the time of vegetation reduction at the target location.

[0064] The obtained vegetation growth status model is as follows:

[0065] According to seasonal changes of long time series satellite remote sensing images of vegetation regions and change rules of vegetation growth conditions with seasons, a vegetation reflectivity change curve is established.

[0066] According to the vegetation reflectivity change curve, a vegetation reduction time of the target position is determined, and the determination can be specifically as follows:

[0067] When the vegetation does not change, the vegetation reflectivity curve is in a sine or cosine state within a given growth season time frame; when the vegetation changes, that is, the vegetation reduction is detected, the vegetation reflectivity curve will have a breakpoint, and a new curve is generated to fit the state after the change event occurs.

[0068] In some embodiments, for cloud-free remote sensing image synthesis considering vegetation interference events, the input data of image synthesis is an image of a vegetation growth season, and when the vegetation interference event is monitored, the time when the vegetation changes is marked, and the image synthesis is performed by using the changed image. Since the vegetation will change the land cover type when it is affected by natural factors including wildfires, typhoons / hurricanes, hail, diseases and insect pests, and climate change, and human factors including fires, deforestation, land reclamation, and forest management, the continuous change detection and classification (CCDC) algorithm can be used to better monitor the change event of the cover type.

[0069] The CCDC method is used to detect the time when the change event occurs, and the specific calculation formula is as follows:

[0070]

[0071] The SWIR2 band is used for fitting, where is a predicted value at time t; T is the average number of days per year (usually 365.2425); c 0，i and c 1，i are the intercept value and slope coefficient of the fitting curve, respectively; a 1,i and b 1,i are the harmonic coefficients of the cosine / sine curve.

[0072] As shown in Figure 2 According to the vegetation reflectivity change curve, a vegetation reduction time of the target position is determined, and the determination can be specifically as follows:

[0073] According to the breakpoint of the curve in the vegetation reflectivity change curve, the vegetation reduction time of the target position is determined.

[0074] In this embodiment, the specific steps of calculating the target distance weight W2 can be as follows:

[0075] acquiring weather information of the target position;

[0076] judging whether the target position has cloud / cloud shadow or not;

[0077] if yes, the target distance weight value is 0;

[0078] if no, judging the distance between the target position and the nearest cloud / cloud shadow, denoted as adjacent distance;

[0079] if the adjacent distance is greater than a set threshold distance, the target distance weight value is 1;

[0080] if the adjacent distance is less than the set threshold distance, the target distance weight value W2 is calculated according to the following formula: wherein, D req is the threshold distance, D i is the distance between the target pixel and the nearest adjacent cloud / cloud shadow pixel, D min is the minimum value of D i , D min = 0, the threshold distance D req is related to the image resolution, if the input is medium resolution image (> 10m), D req is taken as 20-50; if the input is high resolution image (< 10m), D req is taken as 50-150.

[0081] wherein, the weather information of the target position is the weather information of the target position from the time of the vegetation reduction to the imaging time of the input multi-temporal image.

[0082] wherein, the target distance is the distance between the pixel of the target position and the cloud / cloud shadow.

[0083] In the embodiment, the specific steps of calculating the cloud / fog influence weight value W3 can be as follows:

[0084] according to the weather information, calculating the reflectivity of the spectral band of the target position;

[0085] according to the reflectivity of the spectral band, calculating the cloud / fog influence weight value W3.

[0086] wherein, the cloud / fog influence weight value W3 is evaluated by using the haze optimized transformation (HOT) to evaluate the degree of influence of the pixels by haze and thin cloud, and the specific calculation formula can be as follows:

[0087] wherein, p blue and p red represent the reflectivity of the blue band and the red band respectively.

[0088] In some embodiments, step 3 specifically includes:

[0089] Obtain the weight parameters of the multi-temporal input images;

[0090] The weights of the multi-temporal input images are normalized and then weighted and combined, specifically including:

[0091]

[0092] Where n is the number of scenes in the input multi-temporal image, P i W is the target pixel value of the input image for the i-th scene. T For P i The overall score of the image, P final This represents the synthesized cloudless pixel value.

[0093] The normalization of the weights of the multi-temporal input images is to normalize the weight parameters of the multi-temporal input images to the interval (0,1).

[0094] Among them, P final The best pixel is determined by the weighted average of all input pixels and pixel rating weights (Weighted Available Pixel, WAP), which is then used to synthesize the cloudless pixel value.

[0095] The present invention also provides a cloudless remote sensing image synthesis system that takes into account vegetation disturbance events, specifically including: a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, is used to implement the above-mentioned cloudless remote sensing image synthesis method that takes into account vegetation disturbance events.

[0096] This invention compares images and data based on existing BAP-synthesized images and the WAP-synthesized images of this invention, specifically including:

[0097] like Figure 2 As shown, assuming the vegetation growing season in this region is from March 1 to October 31, and forest loss occurred on March 12, 2020 (Julian Day 72), to obtain cloudless images representing forest loss, μ is set to 72, and the time phase of the input images is from March 12 to October 31. To obtain cloudless images representing forest gain, consider that the maximum NDVI time of vegetation recovery during this growing season is September 2, 2020 (Julian Day 246), in which case μ should be set to 246, and the time phase of the input images is from March 12 to October 31.

[0098] like Figure 3 As shown, Figure 3 In contrast, the BAP composite image in image a shows a more obvious "patchwork" effect in cloud filling.Figure 3 In b, the WAP composite image showed good radiometric consistency with no obvious filler marks.

[0099] like Figure 4 The magnified scene shown is based on Figure 4 a, Figure 4 c. Figure 4 e Figure 4 As can be seen from the BAP composite image and the spectral profile curves of the local scene in g, the filled area differs significantly from the surrounding area. Figure 4 b、 Figure 4 d、 Figure 4 f、 Figure 4 As can be seen from the WAP composite image and the spectral profile curve of the local scene in h, the filled area has good overall uniformity with the surrounding area.

[0100] like Figure 5 The cloudless composite image shown, which takes into account forest disturbance events (forest loss and forest gain), demonstrates that even for the same year (e.g., 2020), when considering forest loss, for example... Figure 5 a) In images synthesized using the method of this invention, patches of forest reduction are more pronounced, highlighting information about deforestation; when considering forest increase, for example... Figure 5 b. Images synthesized using the method of this invention present more information about forest restoration, showcasing the optimal state of vegetation restoration, and achieving an image synthesis effect that takes into account both forest disturbance events and user needs.

[0101] In summary, this invention has the following advantages: To achieve cloudless image synthesis from multi-temporal optical images that take into account vegetation disturbance events, the timing of vegetation reduction at the target location is first determined. Based on this timing, the weighting parameters of pixels at the target location in the multi-temporal input images are determined. The multi-temporal input images are then scored according to pixel evaluation rules, and the weighted composite value of all pixels is used for cloud filling. This results in cloudless images with better radiometric consistency. Furthermore, this invention is a cloudless image synthesis method for multi-temporal optical remote sensing data. Through experiments on large-area multi-scene Landsat-8OLI, Sentinel-2, and domestic satellite GF-1 / 6 images, the optimal pixel selection criterion in the existing BAP algorithm is optimized. The optimal pixel is identified as the weighted average of all input pixels and their pixel scoring weights (Weighted Available Pixel, WAP), effectively solving the "patchwork" problem caused by radiometric differences in multi-temporal data in the synthesized image.

[0102] The various embodiments described in this specification are presented for the purpose of illustrating the principles of the application and its best mode of operation. Each of the embodiments described in this specification has been provided for the purpose of illustration and is not intended to limit the application. The same or similar reference numerals in different drawings represent the same or similar elements.

[0103] The principles and operation of the present application have been explained so far with the help of specific examples. The examples have been presented for the purpose of illustration and are not intended to limit the application. The application as described in this specification can be implemented in hardware, software, or a combination thereof. The embodiments disclosed in this specification are not intended to limit the scope of the application.

Claims

1. A method for synthesizing cloudless remote sensing images that takes into account vegetation disturbance events, characterized in that, Includes the following steps: For each input image in a multi-temporal input remote sensing imagery: The target date weight W1 of the pixel at the target location on the input image is determined based on the degree of influence of the time length between the imaging date of the input image and the synthetic target date on the pixel quality. Based on the degree of influence of the distance between the pixel at the target location on the input image and the cloud / cloud shadow on the pixel quality, the target distance weight W2 of the pixel at the target location on the input image is determined; Based on the degree of influence of cloud and fog interference on the pixel quality of the pixel at the target location in the input image, the cloud and fog influence weight W3 of the pixel at the target location in the input image is determined. The pixel value of the pixel at the target location is determined based on the pixel value of the pixel at the target location of each input image, the target day weight W1, the target distance weight W2, and the cloud and fog influence weight W3. The specific steps for calculating the target day weight W1 are as follows: Obtain the latitude and longitude information of the target location; Based on the latitude and longitude information, the growing season of vegetation within the target location is determined according to the vegetation phenological change pattern; The target day weight is calculated based on the synthetic target day, the imaging time of the multi-temporal input remote sensing image, and the growing season time window; the growing season time window is determined based on the growing season of the vegetation within the target location. If the target date for the synthesis is a time of vegetation reduction, it also includes: Obtain a model of vegetation growth status; Based on the vegetation growth model, determine the vegetation reflectance change curve of the target location over time; Based on the vegetation reflectance change curve, determine the time of vegetation reduction at the target location.

2. The cloudless remote sensing image synthesis method considering vegetation disturbance events according to claim 1, characterized in that, The point at which vegetation decreases is the break point in the vegetation reflectance change curve.

3. The method for synthesizing cloudless remote sensing images considering vegetation disturbance events according to claim 1, characterized in that, The specific steps for calculating the target distance weight W2 are as follows: Obtain meteorological information for the target location; Determine whether there are clouds / cloud shadows at the target location; If so, the target distance weight is 0; If not, determine the distance between the target location and the nearest cloud / cloud shadow, and record it as the proximity distance; If the nearest neighbor distance is greater than the set threshold distance, then the target distance weight is 1; If the nearest neighbor distance is less than the set threshold distance, the target distance weight W2 is calculated according to the following formula: ,in, D req For threshold distance, D i The distance between the target pixel and its nearest neighbor cloud / cloud shadow pixel. D min for D i The minimum value.

4. The method for synthesizing cloudless remote sensing images considering vegetation disturbance events according to claim 1, characterized in that, The specific steps for calculating the cloud and fog impact weight W3 are as follows: The cloud and fog influence weight W3 is calculated based on the reflectance of the spectral bands of the input image.

5. The method for synthesizing cloudless remote sensing images considering vegetation disturbance events according to claim 1, characterized in that, Determining the pixel value of the pixel at the target location specifically includes: Obtain the weight parameters of the multi-temporal input remote sensing images; The weight parameters of the multi-temporal input images are normalized and then weighted and combined, specifically including: W T W1+W2+W3 ； Where n is the number of scenes in the input multi-temporal imagery. P i The target pixel value of the input image for the i-th scene. W T for P i The overall score of the image. P final This represents the synthesized cloudless pixel value.

6. A cloudless remote sensing image synthesis system that takes into account vegetation disturbance events, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method according to any one of claims 1-5.

Images