Cloud removal method based on generative model and synthetic aperture radar image

Abstract

The present disclosure relates to a cloud removal method based on a generative model and a synthetic aperture radar image. The method comprises: obtaining a synthetic aperture radar image corresponding to a geographical position of a to-be-cloud-removed optical image, and generating an optical image through the synthetic aperture radar image; obtaining a cloud contour image based on the to-be-cloud-removed optical image, determining a cloud region on the to-be-cloud-removed optical image through the cloud contour image, and performing transparent processing on the cloud region of the to-be-cloud-removed optical image to obtain a first optical image, determining a cloud replacement region on the generated optical image through the cloud contour image, and performing transparent processing on a region outside the cloud replacement region of the generated optical image to obtain a second optical image, and performing superimposition processing on the first optical image and the second optical image to obtain a synthesized optical image; and performing color uniformization processing on the synthesized optical image to obtain a cloud-removed optical image of the to-be-cloud-removed optical image. In this way, the cloud removal effect of the to-be-cloud-removed optical image can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and more particularly, to a cloud removal and restoration method based on a generative model and synthetic aperture radar images. Background Art

[0002] Due to the limitations of optical remote sensing, light waves cannot penetrate clouds, so optical images are inevitably contaminated by clouds. This can lead to information loss and hinder the subsequent interpretation and application of optical remote sensing images, especially for continuous monitoring tasks. Therefore, cloud removal technology for optical remote sensing images is becoming increasingly important.

[0003] Fusion of synthetic aperture radar (SAR) and optical images is an ideal method for removing clouds from optical remote sensing images. Prior art methods for removing clouds by fusing SAR and optical images generally involve first generating a simulated cloud-free optical image based on the SAR image. Then, the clouded areas of the cloud-filled optical image are replaced with the cloud-free optical image to produce the declouded optical image.

[0004] However, due to the significant difference between the cloud-free optical image generated by synthetic aperture radar images and the actual cloud-free optical image, and the low accuracy of obtaining the cloud area outline on the cloud-filled optical image, the final cloud-removed optical image is still significantly different from the actual cloud-free optical image, resulting in unsatisfactory cloud removal effect. Summary of the Invention

[0005] In a first aspect of an embodiment of the present disclosure, a cloud removal and restoration method based on a generative model and synthetic aperture radar images is provided. The method comprises:

[0006] Acquire a synthetic aperture radar image of the geographical location corresponding to the optical image to be declouded, and generate an optical image through the synthetic aperture radar image acquisition;

[0007] acquiring a cloud outline image based on the optical image to be declouded, determining a cloud region on the optical image to be declouded using the cloud outline image, and performing transparency processing on the cloud region of the optical image to be declouded to obtain a first optical image, determining a cloud replacement region on the subsequent optical image using the cloud outline image, and performing transparency processing on regions other than the cloud replacement region of the subsequent optical image to obtain a second optical image, and superimposing the first optical image and the second optical image to obtain a synthesized optical image;

[0008] The synthesized optical image is subjected to color uniformity processing to obtain a declouded optical image of the optical image to be declouded.

[0009] In a second aspect of the embodiments of the present disclosure, a cloud removal and restoration device based on a generative model and synthetic aperture radar images is provided. The device includes:

[0010] a post-generation optical image acquisition module configured to acquire a synthetic aperture radar image of a geographical location corresponding to the optical image to be declouded, and to generate an optical image through the synthetic aperture radar image acquisition;

[0011] a post-synthesized optical image acquisition module configured to acquire a cloud outline image based on the optical image to be de-clouded, determine a cloud region on the optical image to be de-clouded using the cloud outline image, and perform transparency processing on the cloud region of the optical image to be de-clouded to obtain a first optical image, determine a cloud replacement region on the post-generated optical image using the cloud outline image, and perform transparency processing on the region other than the cloud replacement region of the post-generated optical image to obtain a second optical image, and superimpose the first optical image and the second optical image to obtain a post-synthesized optical image;

[0012] The cloud-removed optical image acquisition module is configured to perform color uniformity processing on the synthesized optical image to obtain a cloud-removed optical image of the optical image to be clouded.

[0013] In a third aspect of the disclosed embodiment, a computer program product is provided, comprising a computer program, which implements the method provided according to the first aspect when executed by a processor.

[0014] In a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, including one or more processors, and:

[0015] A memory associated with one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the method provided according to the first aspect is executed.

[0016] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description.

[0017] Beneficial effects:

[0018] The cloud removal and restoration method and apparatus of the embodiments of the present disclosure first generate an optical image by acquiring a synthetic aperture radar image, then obtain a cloud outline image based on the optical image to be clouded, and then determine the cloud area of ​​the optical image to be clouded using the cloud outline image. Then, the cloud area of ​​the optical image to be clouded is combined with the post-generated optical image to obtain a synthesized optical image. Finally, the synthesized optical image is subjected to color uniformity processing. The cloud-removed optical image obtained through the color uniformity processing is more similar to a real cloud-free optical image, thereby improving the cloud removal effect of the optical image to be clouded. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0020] Figure 1 A flowchart of a cloud removal and restoration method based on a generative model and synthetic aperture radar images according to some embodiments of the present disclosure is shown;

[0021] Figure 2 A block diagram of a cloud removal and restoration device based on a generative model and synthetic aperture radar images according to some embodiments of the present disclosure is shown;

[0022] Figure 3 A block diagram of an electronic device illustrating some embodiments of the present disclosure is shown;

[0023] Figure 4 An optical image to be declouded of a geographic location according to some embodiments of the present disclosure is shown;

[0024] Figure 5 Shown Figure 4 The synthetic aperture radar image corresponding to the optical image to be declouded;

[0025] Figure 6 A schematic diagram showing an image pair according to some embodiments of the present disclosure is provided;

[0026] Figure 7 A schematic diagram illustrating another image pair according to some embodiments of the present disclosure is shown;

[0027] Figure 8 Shown by Figure 4 The cloud outline image obtained from the optical image to be declouded;

[0028] Figure 9 Shown by Figure 5 Post-generated optical image obtained from synthetic aperture radar image;

[0029] Figure 10 Shown by Figure 4 Optical image to be declouded Figure 9 A synthesized optical image obtained by generating an optical image;

[0030] Figure 11 Shown Figure 10 The declouded optical image is obtained after the color uniformity processing of the synthesized optical image. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0032] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0033] Figure 1 A flow chart of a cloud removal and restoration method 100 based on a generative model and synthetic aperture radar images according to some embodiments of the present disclosure is shown. The method 100 includes:

[0034] Step 102: Acquire a synthetic aperture radar image of the geographical location corresponding to the optical image to be declouded, and generate an optical image through the synthetic aperture radar image acquisition.

[0035] Synthetic Aperture Radar (SAR) images are high-resolution images generated using radar technology. For the same geographic location, both optical and SAR images can be acquired. However, optical images may contain clouds, so these clouds need to be removed. In this example, optical images containing clouds are used as the images to be declouded.

[0036] When the acquired optical image is an optical image to be declouded, it is necessary to acquire a synthetic aperture radar image of the geographical location corresponding to the optical image to be declouded. The synthetic aperture radar image is specifically acquired by directly using existing technology. Figure 4 shows an optical image of a certain geographical location to be declouded. Figure 5 Shown Figure 4 Synthetic aperture radar image of the optical image to be declouded.

[0037] After the synthetic aperture radar image of the optical image to be declouded is obtained, it is necessary to generate an optical image after obtaining the synthetic aperture radar image. In this embodiment, generating an optical image after obtaining the synthetic aperture radar image is specifically as follows:

[0038] The synthetic aperture radar image is input into the CycleGAN model and an optical image is generated after being output by the CycleGAN model, and the CycleGAN model is trained by a plurality of image pairs, each image pair including an optical image and a synthetic aperture radar image corresponding to the optical image. That is, only by inputting the synthetic aperture radar image into the CycleGAN model, the "post-generated optical image" can be obtained. The "to-be-removed cloud optical image" can be understood as a real optical image of a certain geographical location, but the optical image has clouds; the "post-generated optical image" can be understood as a simulated optical image formed based on a synthetic aperture radar image of a certain geographical location, and the two are different.

[0039] The CycleGAN (Cycle Generative Adversarial Network) model is a kind of cycle generative adversarial network model. The CycleGAN model needs to be trained before being used specifically, and the training data of the CycleGAN model is a large number of image pairs. An image pair includes an optical image and a synthetic aperture radar image of the same size, the optical image is a real optical image of a certain geographical location, and the synthetic aperture radar image is a synthetic aperture radar image of the same geographical location. Figure 6 A schematic diagram of a certain image pair is shown (the left is a synthetic aperture radar image, and the right is an optical image). Figure 7 A schematic diagram of another image pair is shown (the left is a synthetic aperture radar image, and the right is an optical image).

[0040] The CycleGAN model includes a generator G_A, a discriminator D_A, a generator G_B and a discriminator D_B. In specific training, the synthetic aperture radar image in the image pair is input into the generator G_A to obtain a generated optical image, and the generated optical image and the original optical image are input into the discriminator D_A; at the same time, the optical image in the image pair is input into the generator G_B to obtain a generated synthetic aperture radar image, and the generated synthetic aperture radar image and the original synthetic aperture radar image are input into the discriminator D_B. That is, the CycleGAN model is bidirectionally trained during training. In specific application, the synthetic aperture radar image can be input into the CycleGAN model to make the CycleGAN model output the optical image; or the optical image can be input into the CycleGAN model to make the CycleGAN model output the synthetic aperture radar image. In this embodiment, the synthetic aperture radar image is input into the CycleGAN model, and the CycleGAN model outputs the optical image, and the output optical image is the "post-generated optical image".

[0041] Before training the CycleGAN model, you also need to set relevant parameters for the generator G_A, generator G_B, discriminator D_A, and discriminator D_B.

[0042] Generator G_A's encoder uses resnet_9blocks (a deep convolutional neural network with 9 residual blocks). Its batch_size (the number of samples used in each iteration of model training) is set to 2. Generator G_A's total training epochs (the number of times the model traverses the entire training dataset and updates weights) is 100, with weights saved every 5 epochs. Generator G_A's optimizer uses Adam (an optimization algorithm widely used in machine learning and deep learning). Generator G_A's learn-rate (the learning rate) is set to 0.0002, and its learning rate decay strategy is linear. Generator G_B's parameter settings are the same as those of Generator G_A.

[0043] The loss function of the discriminator D_A is set to LSGAN, and the optimizer of the discriminator D_A uses Adam. The setting parameters of the discriminator D_B are the same as those of the discriminator D_A.

[0044] In summary, this step allows us to obtain the "optical image to be declouded," "synthetic aperture radar image," and "post-generated optical image" of a given geographic location. Furthermore, the "post-generated optical image" obtained in this embodiment using the synthetic aperture radar image and the CycleGAN model is as similar as possible to the actual optical image, thus facilitating the subsequent declouding of the optical image to be declouded. Figure 9 Shown by Figure 5 The post-generated optical image is obtained from the synthetic aperture radar image.

[0045] Further, Figure 1 A flow chart of a cloud removal and restoration method 100 based on a generative model and a synthetic aperture radar image according to some embodiments of the present disclosure is shown. The method 100 further includes:

[0046] Step 104: Acquire a cloud outline image based on the optical image to be declouded, determine the cloud region on the optical image to be declouded using the cloud outline image, and perform transparency processing on the cloud region of the optical image to be declouded to obtain a first optical image, determine the cloud replacement region on the post-generated optical image using the cloud outline image, and perform transparency processing on the region other than the cloud replacement region of the post-generated optical image to obtain a second optical image, and superimpose the first optical image and the second optical image to obtain a synthesized optical image.

[0047] The process of obtaining a cloud contour image based on the optical image to be declouded specifically includes:

[0048] Step 202, performing superpixel segmentation on the to-be-cloud-removed optical image to obtain a pixel segmented image, the pixel segmented image having a plurality of regions.

[0049] Superpixel segmentation is an existing image processing technology, which divides an image into a plurality of small and relatively consistent regions, which are usually referred to as “superpixels”. Each superpixel contains a plurality of pixels, and the plurality of pixels have similarities in color, texture or brightness, etc. In summary, superpixels can be regarded as a collection of pixels, which collectively represent a local region in the image. Compared with using pixels alone as the basic unit, superpixels can provide a more compact and meaningful image representation while reducing computational complexity.

[0050] The “superpixel segmentation” technology itself is existing, and the focus of this step is to set related parameters. Specifically, in this step, n_segments (i.e. the number of superpixels that the image is expected to be divided into in the image segmentation task) is set to 50000; compactness (i.e. the compactness factor) is set to 5.0; and sigma (i.e. the Gaussian filtering parameter) is set to 5.0. Such setting parameters can ensure that subtle changes in the cloud edge can also be deployed, and the superpixels obtained by segmentation are more consistent with the complex image edge of the cloud layer, and can avoid over-segmentation.

[0051] In summary, this step can divide the to-be-cloud-removed optical image into region blocks, and each region block is a “superpixel”. The pixel segmented image can be understood as a to-be-cloud-removed optical image having a plurality of “superpixels” (i.e. regions).

[0052] Step 204, performing region merging on the pixel segmented image to obtain a region merged image.

[0053] Specifically, the region merging on the pixel segmented image to obtain the region merged image is as follows:

[0054] determining whether the current pixel segmented image has two adjacent regions for which the color difference degree value has not been calculated, if yes, calculating the color difference degree value for the two adjacent regions for which the color difference degree value has not been calculated, and when the color difference degree value is less than the color difference degree threshold, merging the two adjacent regions; if not, taking the current pixel segmented image as the region merged image.

[0055] For example: Assume that the initial pixel segmented image has superpixel region A, superpixel region B, superpixel region C, and superpixel region D (assuming that superpixel region A, superpixel region B, superpixel region C, and superpixel region D are arranged in an up-down, left-right, and right-left order). Then, there must be two adjacent regions in the current pixel segmented image whose color difference values ​​have not been calculated. At this time, the color difference value between superpixel region A and superpixel region B can be calculated (assuming that the color difference value a is calculated), and the color difference value between superpixel region C and superpixel region D can be calculated at the same time (assuming that the color difference value b is calculated). Assuming that the color difference value a is greater than the color difference threshold and the color difference value b is less than the color difference threshold, then superpixel region A and superpixel region B are not merged, and superpixel region C and superpixel region D are merged to obtain region CD.

[0056] At this point, the current pixel-segmented image has superpixel region A, superpixel region B, and region CD. Since the color difference between superpixel region A and region CD has not yet been calculated, the color difference between superpixel region A and region CD is calculated (assuming the calculated color difference value c). Furthermore, assuming that color difference value c is less than the color difference threshold, superpixel region A and region CD are merged to obtain region ACD.

[0057] At this point, the current pixel-segmented image has region ACD and superpixel region B. Since the color difference between region ACD and superpixel region B has not yet been calculated, the color difference between region ACD and superpixel region B is calculated (assuming the calculated color difference value d). Furthermore, assuming that the color difference value d is greater than the color difference threshold, region ACD and superpixel region B are not merged.

[0058] Finally, the current segmented image still has region ACD and superpixel region B. At this point, there are no adjacent regions in the current segmented image whose color difference values ​​have not been calculated. Therefore, the current segmented image is used as the region-merged image. That is, the region-merged image is the segmented image that has region ACD and superpixel region B.

[0059] In this embodiment, the color difference value between two adjacent superpixel regions is calculated as follows:

[0060] First, obtain the average pixel value R1 (for example, 90) on the red channel, the average pixel value G1 (for example, 100) on the green channel, and the average pixel value B1 (for example, 110) on the blue channel of the first area of ​​the two adjacent areas, and at the same time obtain the average pixel value R2 (for example, 95) on the red channel, the average pixel value G2 (for example, 105) on the green channel, and the average pixel value B2 (for example, 115) on the blue channel of the second area of ​​the two adjacent areas.

[0061] Then the difference between the average pixel value R1 (i.e. 90) and the average pixel value R2 (i.e. 95) is calculated to obtain a first pixel difference value (5), while the difference between the average pixel value G1 (i.e. 100) and the average pixel value G2 (i.e. 105) is calculated to obtain a second pixel difference value (5), while the difference between the average pixel value B1 (i.e. 110) and the average pixel value B2 (i.e. 115) is calculated to obtain a third pixel difference value (5).

[0062] Finally, the sum value (75) of the square of the first pixel difference value (i.e. 5), the second pixel difference value (i.e. 5) and the third pixel difference value (i.e. 5) is calculated, and the square root of the sum value is taken to obtain the color difference value (8.66).

[0063] In summary, this step can merge all the regions that can be merged on the segmented image. The region-merged image can be understood as a cloud-removed optical image having a plurality of "regions", each "region" being obtained by merging a plurality of "superpixels".

[0064] Step 206, obtaining a classification brightness threshold of the region-merged image and classifying the regions on the region-merged image into cloud regions and non-cloud regions based on the classification brightness threshold to obtain a cloud contour image. There are many "regions" on the region-merged image, and this step needs to classify the "regions". Either the "regions" are classified into cloud regions (when the brightness value of the region is greater than the classification brightness threshold), or the "regions" are classified into non-cloud regions (when the brightness value of the region is less than or equal to the classification brightness threshold).

[0065] Among them, obtaining a classification brightness threshold of the region-merged image and classifying the regions on the region-merged image into cloud regions and non-cloud regions based on the classification brightness threshold to obtain a cloud contour image is specifically:

[0066] The classification brightness threshold of the region-merged image is obtained based on the Otsu algorithm.

[0067] The Otsu algorithm is an algorithm for automatically determining the threshold of image binarization, which finds an optimal threshold by maximizing the inter-class variance, which can divide the image into two classes. The Otsu algorithm itself is prior art, and this step directly uses the Otsu algorithm to obtain the classification brightness threshold of the region-merged image.

[0068] The brightness value of each region on the region-merged image is calculated. When the brightness value of the region is greater than the classification brightness threshold, the pixel value of the region is marked as a first pixel value (e.g. 255); otherwise, the pixel value of the region is marked as a second pixel value (e.g. 0). The specific values of the first pixel value and the second pixel value are different.

[0069] The brightness value of each area can be directly calculated using the brightness value calculation method in the prior art.

[0070] The area with the first pixel value (i.e., 255) is regarded as the cloud area, and the area with the second pixel value (i.e., 0) is regarded as the non-cloud area to obtain the cloud outline image. Figure 4 The cloud contour image obtained from the optical image to be declouded is as follows: Figure 8 shown.

[0071] In summary, this step allows the acquisition of a cloud outline image for the "optical image to be de-clouded" at a specific geographic location. Furthermore, the "cloud area" of the cloud outline image acquired through steps 202, 204, and 206 in this embodiment has a high degree of accuracy (i.e., the "cloud area" of the cloud outline image will be similar to the cloud area in the "optical image to be de-clouded"). A high degree of accuracy in the "cloud area" of the acquired cloud outline image leads to better subsequent cloud removal results.

[0072] After acquiring the cloud outline image in step 104, it is also necessary to determine the cloud area on the optical image to be declouded using the cloud outline image and perform transparency processing on the cloud area of ​​the optical image to be declouded to obtain a first optical image. The cloud outline image is used to determine the cloud replacement area on the post-generated optical image and perform transparency processing on the area outside the cloud replacement area of ​​the post-generated optical image to obtain a second optical image. The first optical image and the second optical image are then superimposed to obtain a synthesized optical image.

[0073] First, the cloud removal and restoration system of this embodiment can determine the outline position information of "cloud regions" from the cloud contour image. This outline position information of these "cloud regions" can be represented by SHP graphics files. The cloud removal and restoration system can obtain as many SHP graphics files as there are "cloud regions" in the cloud contour image. The cloud removal and restoration system then combines the SHP graphics files with the optical image to be declouded. Based on the outline position information provided by the SHP graphics files, the cloud removal and restoration system can automatically mark the "cloud regions" in the optical image to be declouded. Simultaneously, the cloud removal and restoration system combines the SHP graphics files with the post-production optical image. Based on the outline position information provided by the SHP graphics files, the cloud removal and restoration system can automatically mark the "cloud regions" in the post-production optical image.

[0074] Next, the cloud removal and restoration system converts the optical image to be declouded, marked with "cloudy areas," into RGBA mode (i.e., grayscale mode) and makes the "cloudy areas" of the optical image to be declouded transparent to obtain the first optical image. Simultaneously, the cloud removal and restoration system converts the post-generation optical image to be declouded, marked with "cloudy areas," into RGBA mode (i.e., grayscale mode) and makes the "non-cloudy areas" of the post-generation optical image transparent to obtain the second optical image.

[0075] The size of the optical image to be declouded is the same as the size of the subsequently generated optical image. Therefore, the size of the first optical image is also the same as the size of the second optical image. The "cloudy area" on the first optical image completely overlaps with the "cloudy area" on the second optical image. In this case, simply overlaying the first and second optical images will yield the desired composite optical image.

[0076] The synthesized optical image can be understood as being the result of stitching together the "non-cloudy areas" of the optical image to be declouded and the "cloudy areas" of the post-generated optical image. As long as the acquisition location of the "cloudy areas" is sufficiently accurate and the "post-generated optical image" generated from the synthetic aperture radar image closely resembles the actual optical image, the more similar the synthesized optical image will be to the actual, cloud-free optical image, and the better the declouding effect. In this embodiment, the "post-generated optical image" obtained through the CycleGAN model is as similar as possible to the actual optical image. Furthermore, through superpixel segmentation, region merging, and cloud area classification based on classification brightness thresholds, the acquisition location of the "cloudy areas" is sufficiently accurate, resulting in a superior declouding effect for the acquired "synthesized optical image." Figure 10 Shown by Figure 4 Optical image to be declouded Figure 9 The synthesized optical image is obtained by generating the optical image in the middle and later stages.

[0077] Further, Figure 1 A flow chart of a cloud removal and restoration method 100 based on a generative model and a synthetic aperture radar image according to some embodiments of the present disclosure is shown. The method 100 further includes:

[0078] Step 106 : performing color uniformity processing on the synthesized optical image to obtain a declouded optical image of the optical image to be declouded.

[0079] In order to improve the cloud removal effect, this embodiment also requires performing color uniformity processing on the synthesized optical image.

[0080] The step of performing color uniformity processing on the synthesized optical image to obtain the declouded optical image of the optical image to be declouded specifically includes:

[0081] Step 302: Obtain the histogram of the synthesized optical image. Figure 1 and obtaining the histogram of the first optical image Figure 2 , based on histogram Figure 1 Get the cumulative distribution function based on the histogram Figure 2A second cumulative distribution function is obtained, a pixel mapping relationship is obtained based on the first cumulative distribution function and the second cumulative distribution function, and pixel values ​​of the synthesized optical image at the cloud region position are adjusted based on the pixel mapping relationship to obtain a first uniform color processed image.

[0082] The histogram of an image is obtained by counting the number of pixels at each gray level in the image. The horizontal axis of the histogram represents the gray value, from 0 to 255; the vertical axis represents the number of pixels at each gray level. The synthesized optical image has three color channels, and each color channel needs to obtain the corresponding histogram. Figure 1 , so the synthesized optical image can obtain 3 histograms Figure 1 The first optical image also has three color channels, and each color channel needs to obtain the corresponding histogram. Figure 2 , so the first optical image can obtain 3 histograms Figure 2 . In the specific acquisition of histogram Figure 1 or histogram Figure 2 When the synthesized optical image is input into the histogram acquisition function, 3 histograms can be output. Figure 1 ; Input the first optical image into the histogram acquisition function to output 3 histograms Figure 2 .

[0083] When the three histograms of the optical image are synthesized Figure 1 and the three histograms of the first optical image Figure 2 After all are obtained, it is necessary to base on 3 histograms Figure 1 Get the corresponding three cumulative distribution functions based on three histograms Figure 2 Obtain the corresponding three cumulative distribution functions. In short, one histogram corresponds to one cumulative distribution function. The cumulative distribution function is obtained by counting the proportion of pixels less than or equal to each grayscale level in the histogram.

[0084] For example, suppose a histogram contains four grayscale levels: 0, 2, 5, and 7. The number of occurrences of grayscale level "0" is 1; the number of occurrences of grayscale level "2" is 3; the number of occurrences of grayscale level "5" is 4; and the number of occurrences of grayscale level "7" is 2. After normalization, the probability of grayscale level "0" occurring is 0.1; the number of occurrences of grayscale level "2" is 0.3; the number of occurrences of grayscale level "5" is 0.4; and the number of occurrences of grayscale level "7" is 0.2. Therefore, the cumulative distribution function is: grayscale level "0" is 0.1; grayscale level "2" is 0.4; grayscale level "5" is 0.8; and grayscale level "7" is 1.

[0085] In short, the cumulative distribution function is obtained based on the histogram. When obtaining the cumulative distribution function, you only need to call the relevant cumulative distribution function acquisition function (such as np.cumsum()). That is, you only need to add the three histograms. Figure 1 Input the cumulative distribution function to get the function and you can get 3 cumulative distribution functions. Figure 2 Input the cumulative distribution function to obtain the function and you will get 3 cumulative distribution functions 2 as output.

[0086] Once the three cumulative distribution functions (CDFs) of the synthesized optical image and the three cumulative distribution functions of the first optical image have been obtained, it is necessary to determine the pixel mapping relationship based on the CDFs of the synthesized optical image and the CDFs of the first optical image (three pixel mapping relationships can be determined for the three color channels). To determine the pixel mapping relationship, simply call the relevant pixel mapping relationship acquisition function (e.g., np.interp()). Specifically, simply input the CDFs of the red channel (CDFs) and CDFs of the red channel (CDFs) into the pixel mapping relationship acquisition function to output the pixel mapping relationship for the red channel; input the CDFs of the green channel (CDFs) and CDFs of the green channel (CDFs) into the pixel mapping relationship acquisition function to output the pixel mapping relationship for the green channel; and input the CDFs of the blue channel (CDFs) and CDFs of the blue channel (CDFs) into the pixel mapping relationship acquisition function to output the pixel mapping relationship for the blue channel.

[0087] Once the pixel mapping relationship is determined, the cloud removal and restoration system of this embodiment adjusts the pixel values ​​in the cloud region of the synthesized optical image based on the pixel mapping relationship to produce a first color-leveled image. Compared to the synthesized optical image, the first color-leveled image brings the colors of the original cloud region and the original non-cloud region closer together, thereby improving the cloud removal effect of the optical image to be declouded.

[0088] Furthermore, performing color uniformity processing on the synthesized optical image to obtain the declouded optical image to be declouded further includes:

[0089] Step 304: Perform multi-level pixel blending processing on the first uniform color processed image to obtain a second uniform color processed image.

[0090] The step of performing multi-level pixel blending on the first uniform color processed image to obtain the second uniform color processed image specifically includes:

[0091] Performing a primary pixel blending process on the first uniform color processed image: setting a first blur radius to determine a first transition area, and blending pixels in the first transition area according to a first transparency to obtain a first blended processed image;

[0092] Performing secondary pixel blending processing on the first blended image: setting a second blur radius to determine a second transition area, and blending pixels in the second transition area according to a second transparency to obtain a second blended image;

[0093] The second mixed image is subjected to a three-level pixel mixing process: a third blur radius is set to determine a third transition area, and pixels in the third transition area are mixed according to a third transparency to obtain a second uniform color image.

[0094] In this embodiment, when performing primary pixel blending on the first uniformly colored image, the first blur radius is set to 15 pixels, and the first transparency is set to 20%. Setting the first blur radius to 15 pixels defines the first transition region as an area extending 15 pixels from the cloud outline boundary toward the original cloud region and 15 pixels toward the original non-cloud region. Setting the first transparency to 20% blends the pixel values ​​of the first uniformly colored image in the first transition region (at a 20% ratio) with the pixel values ​​of the synthesized optical image in the first transition region (at an 80% ratio), and replaces the pixel values ​​of the first uniformly colored image in the first transition region with the resulting blended pixel values, ultimately resulting in the first blended image. Taking a pixel position as an example: assuming that the pixel value of a certain pixel position in the first transition area of ​​the first uniform color processed image is S1, and assuming that the pixel value of the corresponding pixel position in the first transition area of ​​the synthesized optical image is S2, then the mixed pixel value S3 of the corresponding pixel position in the first transition area is S1*20%+S2*80%. Finally, the mixed pixel value S3 replaces the pixel value S1 of the corresponding pixel position in the first transition area of ​​the first uniform color processed image.

[0095] In summary, once all pixel values ​​in the first transition region of the first uniformly colored image are replaced by the mixed pixel values, a first mixed image is obtained. This primary pixel mixing process eliminates the unnatural transition at the original cloud outline boundary in the first uniformly colored image, making the transition at the original cloud outline boundary in the first mixed image more natural, further improving the declouding effect of the optical image to be declouded.

[0096] In this embodiment, when performing secondary pixel blending on the first blended image, the second blur radius is set to 7 pixels and the second transparency is set to 50%. Setting the second blur radius to 7 pixels means that the area extending 7 pixels from the cloud outline boundary toward the original cloud area and 7 pixels toward the original non-cloud area is used as the second transition area. Setting the second transparency to 50% means that the pixel values ​​of the first blended image in the second transition area (at a 50% ratio) are blended with the pixel values ​​of the first uniform color image in the second transition area (at a 50% ratio), and the resulting pixel values ​​replace the pixel values ​​of the first blended image in the second transition area, ultimately resulting in the second blended image. Take a pixel position as an example: assuming that the pixel value of a certain pixel position of the first mixed processing image in the second transition area is P1, and assuming that the pixel value of the corresponding pixel position of the first uniform color processing image in the second transition area is P2, then the mixed pixel value P3 of the corresponding pixel position in the second transition area is P1*50%+P2*50%, and finally the mixed pixel value P3 replaces the pixel value P1 of the corresponding pixel position of the first mixed processing image in the second transition area.

[0097] In summary, once all the pixels in the second transition region of the first blended image are replaced by the blended pixel values, the second blended image is obtained. This secondary pixel blending process increases the contrast of the original cloud region in the first blended image, thereby improving the declouding effect of the optical image to be declouded.

[0098] In this embodiment, when performing three-level pixel blending on the second blended image, the third blur radius is set to 3 pixels, and the third transparency is set to 80%. Setting the third blur radius to 3 pixels means that the area extending 3 pixels from the cloud outline boundary toward the original cloud area and 3 pixels toward the original non-cloud area serves as the third transition area. Setting the third transparency to 80% means that the pixel values ​​of the second blended image in the third transition area (at a ratio of 80%) are blended with the pixel values ​​of the first blended image in the third transition area (at a ratio of 20%), and the resulting blended pixel values ​​replace the pixel values ​​of the second blended image in the third transition area, ultimately resulting in the second uniformly colored image. Take a pixel position as an example: assuming that the pixel value of a pixel position in the third transition area of ​​the second mixed processing image is Q1, and assuming that the pixel value of the corresponding pixel position in the third transition area of ​​the first mixed processing image is Q2, then the mixed pixel value Q3 of the corresponding pixel position in the third transition area is Q1*80%+Q2*20%, and finally the mixed pixel value Q3 replaces the pixel value Q1 of the corresponding pixel position in the third transition area of ​​the second mixed processing image.

[0099] In summary, once all the pixel values ​​in the third transition region of the second blended image are replaced by the blended pixel values, the second uniformly colored image is obtained. This three-level pixel blending process enhances the edge effect of the original cloud contour boundary in the second blended image, further improving the declouding effect of the optical image to be declouded.

[0100] In summary, compared with the first uniform color processed image, the second uniform color processed image obtained by multi-level pixel blending processing has a better cloud removal effect.

[0101] Furthermore, performing color uniformity processing on the synthesized optical image to obtain the declouded optical image to be declouded further includes:

[0102] Step 306: Gaussian blur the second uniformly colored image to obtain a blurred image, obtain a difference image 1 based on the second uniformly colored image and the blurred image, multiply the difference image 1 by a scaling factor to obtain a difference image 2, and add the difference image 2 to the second uniformly colored image to obtain a declouded optical image.

[0103] Gaussian blur is an existing image processing technology. In this step, the existing Gaussian blur function can be directly called to perform Gaussian blur processing on the second uniform color processed image. In this embodiment, the radius of the Gaussian blur is set to 2.

[0104] The first difference image is obtained based on the second uniform color image and the blurred image by subtracting the pixel value of each pixel position in the blurred image from the pixel value of the corresponding pixel position in the second uniform color image. The first difference image mainly retains edge information and some detail information.

[0105] Multiplying the difference image 1 by the scaling factor to obtain the difference image 2 specifically involves multiplying the pixel value of each pixel position in the difference image 1 by the scaling factor. In order to ensure the enhancement effect of image details, the scaling factor is set to 150 in this embodiment.

[0106] The method of adding the second difference image and the second uniform color processed image to obtain the declouded optical image is as follows: adding the pixel value of each pixel position point in the second difference image to the pixel value of the corresponding pixel position point in the second uniform color processed image.

[0107] In short, this step can enhance the edges and details in the second uniform color processed image, so that the final declouded optical image has a good declouding effect. Figure 11 Shown Figure 10 The declouded optical image is obtained after the color uniformity processing of the synthesized optical image.

[0108] Figure 2A block diagram of a cloud removal and restoration apparatus 200 based on a generative model and synthetic aperture radar images according to some embodiments of the present disclosure is shown. The apparatus 200 includes:

[0109] The post-generation optical image acquisition module 202 is configured to acquire a synthetic aperture radar image of a geographical location corresponding to the optical image to be declouded, and generate an optical image through the synthetic aperture radar image acquisition.

[0110] The synthesized optical image acquisition module 204 is configured to acquire a cloud outline image based on the optical image to be declouded, determine the cloud area on the optical image to be declouded using the cloud outline image, and perform transparency processing on the cloud area of ​​the optical image to be declouded to obtain a first optical image, determine the cloud replacement area on the post-generated optical image using the cloud outline image, and perform transparency processing on the area other than the cloud replacement area of ​​the post-generated optical image to obtain a second optical image, and superimpose the first optical image and the second optical image to obtain a synthesized optical image.

[0111] The declouded optical image acquisition module 206 is configured to perform color uniformity processing on the synthesized optical image to obtain a declouded optical image of the optical image to be declouded.

[0112] Furthermore, the post-generation optical image acquisition module 202 includes:

[0113] The post-generation optical image acquisition unit is configured to input the synthetic aperture radar image into the CycleGAN model and generate an optical image through the output of the CycleGAN model, wherein the CycleGAN model is trained by a plurality of image pairs, each image pair including an optical image and a synthetic aperture radar image corresponding to the optical image.

[0114] Furthermore, the synthesized optical image acquisition module 204 includes:

[0115] The cloud outline image acquisition unit is configured to perform superpixel segmentation on the optical image to be declouded to obtain a pixel-segmented image, wherein the pixel-segmented image has a plurality of regions; perform region merging on the pixel-segmented image to obtain a region-merged image; obtain a classification brightness threshold of the region-merged image and divide the region on the region-merged image into a cloud region and a non-cloud region based on the classification brightness threshold to obtain a cloud outline image.

[0116] Furthermore, the cloud contour image acquisition unit includes:

[0117] The region-merged image acquisition subunit is configured to determine whether there are two adjacent regions in the current pixel-segmented image whose color difference values ​​have not been calculated. If so, the color difference values ​​of the two adjacent regions whose color difference values ​​have not been calculated are calculated. When the color difference value is less than the color difference threshold, the two adjacent regions are merged; if not, the current pixel-segmented image is used as the region-merged image.

[0118] The cloud outline image acquisition subunit is configured to obtain a classification brightness threshold of the region-merged image based on the Otsu algorithm; calculate the brightness value of each region on the region-merged image, and when the brightness value of the region is greater than the classification brightness threshold, mark the pixel value of the region as a first pixel value; otherwise, mark the pixel value of the region as a second pixel value; the first pixel value is different from the second pixel value; and regard the region with the pixel value of the first pixel value as a cloud region and the region with the pixel value of the second pixel value as a non-cloud region to obtain a cloud outline image.

[0119] Furthermore, the cloud-removed optical image acquisition module 206 includes:

[0120] The first uniform color processing image acquisition unit is configured to acquire the histogram of the synthesized optical image. Figure 1 and obtaining the histogram of the first optical image Figure 2 , based on histogram Figure 1 Get the cumulative distribution function based on the histogram Figure 2 A second cumulative distribution function is obtained, a pixel mapping relationship is obtained based on the first cumulative distribution function and the second cumulative distribution function, and pixel values ​​of the synthesized optical image at the cloud region position are adjusted based on the pixel mapping relationship to obtain a first uniform color processed image.

[0121] The second uniform color processed image acquisition unit is configured to perform multi-level pixel mixing processing on the first uniform color processed image to obtain a second uniform color processed image.

[0122] The declouding optical image acquisition unit is configured to perform Gaussian blur on the second uniform color processed image to obtain a blurred image, obtain a difference image 1 based on the second uniform color processed image and the blurred image, multiply the difference image 1 by a proportional coefficient to obtain a difference image 2, and add the difference image 2 to the second uniform color processed image to obtain the declouding optical image.

[0123] Furthermore, the second uniform color processing image acquisition unit includes:

[0124] The first mixed processing image acquisition subunit is configured to perform a first-level pixel mixing process on the first uniform color processing image: set a first blur radius to determine a first transition area, and mix the pixels in the first transition area according to a first transparency to obtain a first mixed processing image.

[0125] The second mixed processing image acquisition subunit is configured to perform secondary pixel mixing processing on the first mixed processing image: set a second blur radius to determine a second transition area, and mix the pixels in the second transition area according to a second transparency to obtain a second mixed processing image.

[0126] The second uniform color processed image acquisition subunit is configured to perform three-level pixel mixing processing on the second mixed processed image: set a third blur radius to determine a third transition area, and mix the pixels in the third transition area according to a third transparency to obtain a second uniform color processed image.

[0127] Figure 3 A block diagram of an electronic device 300 according to some embodiments of the present disclosure is shown. The device 300 includes a processor 301, which can perform various appropriate actions and processes based on computer program instructions stored in a read-only memory (ROM) 302 and loaded into a random access memory (RAM) 303. RAM 303 may also store various programs and data required for the operation of the device 300. The processor 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0128] The various processes and procedures described above, such as method 100, may be executed by processor 301. For example, in some embodiments, method 100 may be implemented as a software program tangibly embodied on a machine-readable medium. In some embodiments, part or all of the software program may be loaded and / or installed onto device 300 via ROM 302. When the software program is loaded into RAM 303 and executed by processor 301, one or more actions of method 100 described above may be performed.

[0129] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chip systems (SOCs), programmable logic devices (CPLDs), and the like.

[0130] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] The present disclosure may be a method, apparatus, system and / or program product. The program product may include a machine-readable storage medium on which are loaded machine-readable program instructions for executing various aspects of the present disclosure. The machine-readable program instructions described herein may be downloaded from the machine-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. The network adapter card or network interface in each computing / processing device receives the machine-readable program instructions from the network and forwards the machine-readable program instructions for storage in the machine-readable storage medium in each computing / processing device.

[0132] The machine program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The machine-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the machine-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the machine-readable program instructions, thereby implementing various aspects of the present disclosure.

[0133] In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this disclosure. Certain features described in the context of a separate embodiment can also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0134] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A cloud removal and restoration method based on a generative model and synthetic aperture radar images, characterized in that: include: Acquire a synthetic aperture radar image of a geographical location corresponding to the optical image to be declouded, input the synthetic aperture radar image into a generation model, and generate an optical image after acquisition through the generation model; acquiring a cloud outline image based on the optical image to be declouded, determining a cloud region on the optical image to be declouded using the cloud outline image, and performing transparency processing on the cloud region of the optical image to be declouded to obtain a first optical image, determining a cloud replacement region on the post-generated optical image using the cloud outline image, and performing transparency processing on regions other than the cloud replacement region of the post-generated optical image to obtain a second optical image, and superimposing the first optical image and the second optical image to obtain a synthesized optical image; performing color uniformity processing on the synthesized optical image to obtain a declouded optical image of the optical image to be declouded; The step of performing color uniformity processing on the synthesized optical image to obtain the declouded optical image of the optical image to be declouded specifically includes: Obtaining a first histogram of the synthesized optical image and a second histogram of the first optical image, obtaining a first cumulative distribution function based on the first histogram and a second cumulative distribution function based on the second histogram, obtaining a pixel mapping relationship based on the first and second cumulative distribution functions, and adjusting pixel values ​​of the synthesized optical image at a cloud region position based on the pixel mapping relationship to obtain a first uniformly colored image; performing a multi-level pixel blending process on the first uniform color processed image to obtain a second uniform color processed image; Gaussian blur is performed on the second uniform color processed image to obtain a blurred image, a difference image 1 is obtained based on the second uniform color processed image and the blurred image, the difference image 1 is multiplied by a proportional coefficient to obtain a difference image 2, and the difference image 2 is added to the second uniform color processed image to obtain the declouded optical image.

2. The method according to claim 1, characterized in that Inputting the synthetic aperture radar image into a generation model and generating an optical image after acquiring the image through the generation model specifically includes: The synthetic aperture radar image is input into a CycleGAN model and the generated optical image is outputted by the CycleGAN model, wherein the CycleGAN model is trained by a plurality of image pairs, each of which includes an optical image and a synthetic aperture radar image corresponding to the optical image.

3. The method according to claim 1, characterized in that Acquiring a cloud contour image based on the optical image to be declouded specifically includes: Performing superpixel segmentation on the optical image to be declouded to obtain a pixel-segmented image, wherein the pixel-segmented image has a plurality of regions; Performing region merging on the pixel segmented image to obtain a region merged image; A classification brightness threshold of the region-merged image is obtained, and based on the classification brightness threshold, the region on the region-merged image is divided into a cloud region and a non-cloud region to obtain the cloud outline image.

4. The method according to claim 3, characterized in that Performing region merging on the pixel segmented image to obtain a region merged image specifically includes: Determine whether there are two adjacent areas in the current pixel segmentation image whose color difference values ​​have not been calculated. If so, calculate the color difference values ​​for the two adjacent areas whose color difference values ​​have not been calculated. When the color difference value is less than a color difference threshold, merge the two adjacent areas. If not, use the current pixel segmentation image as the area-merged image.

5. The method according to claim 3, characterized in that Acquiring a classification brightness threshold of the region-merged image and dividing the region on the region-merged image into a cloud region and a non-cloud region based on the classification brightness threshold to obtain the cloud outline image specifically includes: Obtaining a classification brightness threshold of the image after the region is merged based on the Otsu algorithm; Calculating the brightness value of each region on the image after the region merging, and when the brightness value of the region is greater than the classification brightness threshold, marking the pixel value of the region as a first pixel value; otherwise, marking the pixel value of the region as a second pixel value; the first pixel value and the second pixel value are different; The cloud outline image is obtained by taking an area with a first pixel value as a cloud area and an area with a second pixel value as a non-cloud area.

6. The method according to claim 1, characterized in that Performing multi-level pixel blending processing on the first color-evening processed image to obtain a second color-evening processed image specifically includes: Performing a primary pixel blending process on the first uniformly colored image: setting a first blur radius to determine a first transition region, and blending pixels in the first transition region according to a first transparency to obtain a first blended image; Performing secondary pixel blending processing on the first blended image: setting a second blur radius to determine a second transition area, and blending pixels in the second transition area according to a second transparency to obtain a second blended image; The second mixed-processed image is subjected to a three-level pixel mixing process: a third blur radius is set to determine a third transition area, and pixels in the third transition area are mixed according to a third transparency to obtain a second uniform color processed image.

7. A cloud removal and restoration device based on a generative model and synthetic aperture radar images, characterized in that: include: a post-generation optical image acquisition module configured to acquire a synthetic aperture radar image of a geographical location corresponding to the optical image to be declouded, input the synthetic aperture radar image into a generation model, and acquire a post-generation optical image through the generation model; a post-synthesized optical image acquisition module configured to acquire a cloud outline image based on the optical image to be de-clouded, determine a cloud region on the optical image to be de-clouded using the cloud outline image, and perform transparency processing on the cloud region of the optical image to be de-clouded to obtain a first optical image, determine a cloud replacement region on the post-generated optical image using the cloud outline image, and perform transparency processing on the region other than the cloud replacement region of the post-generated optical image to obtain a second optical image, and superimpose the first optical image and the second optical image to obtain a post-synthesized optical image; a post-cloud removal optical image acquisition module, configured to perform color uniformity processing on the synthesized optical image to obtain a post-cloud removal optical image of the optical image to be clouded; The step of performing color uniformity processing on the synthesized optical image to obtain the declouded optical image of the optical image to be declouded specifically includes: Obtaining a first histogram of the synthesized optical image and a second histogram of the first optical image, obtaining a first cumulative distribution function based on the first histogram and a second cumulative distribution function based on the second histogram, obtaining a pixel mapping relationship based on the first and second cumulative distribution functions, and adjusting pixel values ​​of the synthesized optical image at a cloud region position based on the pixel mapping relationship to obtain a first uniformly colored image; performing a multi-level pixel blending process on the first uniform color processed image to obtain a second uniform color processed image; Gaussian blur is performed on the second uniform color processed image to obtain a blurred image, a difference image 1 is obtained based on the second uniform color processed image and the blurred image, the difference image 1 is multiplied by a proportional coefficient to obtain a difference image 2, and the difference image 2 is added to the second uniform color processed image to obtain the declouded optical image.

8. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

9. An electronic device, characterized in that: include: One or more processors, and: A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the steps of the method according to any one of claims 1 to 6 are executed.

Images