General image restoration method and device for lens pollution remote sensing image

Abstract

The invention belongs to the technical field of aviation / space remote sensing, and particularly relates to a general image restoration method and device for a lens pollution remote sensing image. Acquiring or constructing a satellite image with a pure background with a pollution-free background signal; according to a set threshold value, obtaining a pollution spot position and a DN value of the pure background satellite image; establishing a pollution spot signal model through the pollution spots, the DN values of the pixels in the set range of the pollution spots and the background signals; subtracting a spot signal DN value obtained according to the pollution spot signal model from a remote sensing image to be restored, and removing pollution spots; and matching the image from which the pollution spots are removed to an original to-be-restored remote sensing image to complete restoration. The influence of spot pollution on remote sensing image application such as target detection and recognition can be effectively avoided.

Description

Technical Field

[0001] The invention belongs to the field of aviation / aerospace remote sensing technology, and in particular relates to a universal image restoration method and device for lens-contaminated remote sensing images. Background Art

[0002] Since light outside the designed optical path reaches the detector, for example, the side panels, top panels and circuit boards of the optical cabin enter the field of view through reflection, scattering and diffraction in the camera, speckle signals appear on the image, reducing the camera imaging contrast and signal-to-noise ratio, interfering with the extraction and recognition of imaging signals, and even causing damage to the detector, limiting the application of remote sensing images in meteorology, environmental monitoring and military target reconnaissance, causing huge economic losses and waste of resources. Therefore, the detection, correction and repair of speckle pollution in remote sensing images have high economic and social benefits for giving full play to the application value of remote sensing payloads with lens pollution.

[0003] The existing remote sensing image lens contamination removal method is generally carried out before the payload is produced, that is, during the payload development stage, the stray light is eliminated through optical path analysis to prevent it from converging on the image plane. The basic idea is to make the stray light path converge at a position far enough away from the image plane, and make its energy weak enough so that the remote sensing payload cannot detect it, so as to achieve the purpose of eliminating lens contamination.

[0004] Existing technologies are only applicable before the development of remote sensing payloads, and cannot solve the lens contamination problem of remote sensing payloads on orbit. Even if sufficient simulation analysis is carried out before the payload is developed, many optical remote sensing payloads will still encounter lens contamination problems on orbit due to changes in the on-orbit working environment, such as impact during the launch process, stress release, payload heating, and changes in solar radiation that are inconsistent with laboratory simulations. Summary of the invention

[0005] The purpose of the present invention is to solve the lens contamination problem of on-orbit remote sensing payloads and realize spot contamination detection, correction and repair in remote sensing images.

[0006] On one hand, the present invention provides a universal image restoration method for lens contaminated remote sensing images, the steps of which include:

[0007] Acquire or construct satellite images with pure background without contaminating background signals;

[0008] According to the set threshold, the location of the contaminated spot and its DN value of the satellite image of the pure background are obtained;

[0009] A pollution spot signal model is established by using the pollution spot and the DN value of the pixel within the set range thereof and the background signal;

[0010] Subtracting the speckle signal DN value obtained according to the contamination speckle signal model from the remote sensing image to be restored to remove the contamination speckles;

[0011] The image after removing the contaminated spots is matched to the original remote sensing image to be restored to complete the restoration.

[0012] Furthermore, the method for constructing the satellite image with a pure background without contamination background signals is as follows:

[0013] Acquire an observation image; the observation image is an image whose cloud coverage is less than a set value;

[0014] Determining the absence of contamination background signals through the observed image;

[0015] The cloud-covered area in the observed image is eliminated by the pollution-free background signal to obtain a satellite image with a pure background.

[0016] Further, the method for determining the pollution-free background signal is as follows:

[0017] Using a sliding window of a set size to intercept the observed image by a set step length to obtain multiple frames of small window area images;

[0018] Calculate the mean and standard deviation of the images in each small window area to obtain a mean set and a standard deviation set of the observed images;

[0019] The mean of the minimum sub-region of the mean value set and the standard deviation set is taken as the pollution-free background signal.

[0020] Furthermore, the method for removing the cloud covered area in the observed image is as follows:

[0021] Determine the cloud determination threshold of each small window area image based on the mean and standard deviation of the small window area image;

[0022] When a pixel value in a small window area image exceeds the cloud determination threshold, the pixel value is assigned the mean value of the small window area image;

[0023] The images of each small window area are stitched together to obtain the satellite image of the pure background.

[0024] Furthermore, the threshold is smaller than the pollution-free background signal.

[0025] Furthermore, the setting range may adopt 4 neighborhoods, 8 neighborhoods and 24 neighborhoods.

[0026] Furthermore, the contamination speckle signal model is obtained by polynomial least squares fitting.

[0027] Furthermore, the image after the pollution spots are removed is histogram matched to the original remote sensing image to be restored.

[0028] The present invention also provides an electronic device for lens contamination remote sensing images, comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing any of the above methods.

[0029] The present invention also provides a storage medium storing a computer program, wherein when the computer program is executed by a computer, any of the above-mentioned methods is implemented.

[0030] The beneficial effects of the present invention are as follows:

[0031] The present invention proposes a new and reliable universal image restoration method and device for remote sensing images contaminated by lens. The method for removing contaminated spots from remote sensing images by mathematically modeling the contaminated spot signals is simple and convenient, and can effectively avoid the influence of spot contamination on remote sensing image applications such as target detection and recognition.

[0032] The present invention constructs a pure background satellite image based on an observation image with less cloud cover, and uses the established mathematical model of background signal and pollution spot signal to remove pollution spots in the remote sensing image. It does not require improvement of the optical system design, nor does it require special acquisition of remote sensing images with specific backgrounds. It has few technical constraints and strong operability. At the same time, the simple, convenient and highly generalized algorithm in the present invention is also suitable for removing pollution spots from images taken by aerospace, aviation and commercial cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of a specific implementation mode of the present invention.

[0034] Figure 2 Schematic diagram of the neighborhood pixels of the polluted spots in a specific implementation manner of the present invention.

[0035] Figure 3 It is a schematic diagram of mathematical modeling of contamination spot signals in a specific embodiment of the present invention.

[0036] Figure 4 This is a comparison diagram before and after remote sensing image restoration in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the purposes, features and advantages of the present invention more obvious and easy to understand, the technical core of the present invention is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0038] The embodiment of the present invention provides a universal image restoration method for lens contaminated remote sensing images. The method constructs a pure background satellite image and establishes a mathematical model of contamination spot signals to remove contamination spots from remote sensing images. The specific process is as follows: Figure 1 shown.

[0039] S1 calculates the background signal DN when there is no pollution bkg .

[0040] Select observation images with less cloud coverage. Generally speaking, observation images with cloud coverage less than 5% are selected as source images to minimize the impact of cloud coverage on spot removal.

[0041] According to the actual required size, for example, in this embodiment, a 20*20 sliding window is used to continuously intercept the observed image with a step size of 1 to obtain each small window area image. Calculate the mean DN of each cloud-free small window area image bkg and standard deviation σ bkg , after integration, we get the mean set {DN bkg} N and the standard deviation set {σ bkg} N The average value set {DN bkg} N and the standard deviation set {σ bkg} N The smallest sub-area, based on the DN of the sub-area bkg As a non-contaminating background signal.

[0042] S2 constructs a satellite image with a pure background.

[0043] Generally speaking, cloud reflectivity is higher than sea surface reflectivity, and spot pollution will reduce image reflectivity, so DN cloud >DN sea >DN spot In S1, the mean DN of each small window area image is calculated bkg and standard deviation σ bkg Finally, 3σ is set as the threshold for removing cloud pollution based on cloud reflectivity, and as many cloud-contaminated pixels as possible are removed, and the pixel values ​​exceeding the threshold are assigned as DN bkg Remove the cloud-covered area of ​​each small window area image. By splicing multiple frames of small window area images without cloud cover, all pollution spots are included in the spliced ​​image to obtain a pure background satellite image.

[0044] S3 determines the location of the contamination spot and its DN value.

[0045] For the pure background satellite image acquired, according to the background signal DN bkg and the characteristics of the contamination spots to determine a suitable threshold DNTh , the threshold is lower than the background signal DN bkg , so that background and contamination spots can be distinguished.

[0046] Traverse each pixel of the satellite image and compare its DN value with the threshold DN Th If the pixel's DN value is lower than DN Th , it is considered that the pixel may be affected by the pollution spot and is regarded as a pollution spot. For the selected pollution spots, their positions and DN values ​​in the image are recorded.

[0047] S4 establishes a mathematical model of the contamination spot signal.

[0048] Select a certain number of polluted pixels on each polluted spot, record the DN values ​​of the polluted pixels, and generate the polluted pixel DN set {DN}. Figure 2 As shown, 4, 8 or 24 pixels around the spot can be taken as the center, which is specifically determined by the minimum spot size of the remote sensing image. In this embodiment, the 8-neighborhood solution is adopted.

[0049] Generate a background signal set corresponding to the polluted pixel {DN bkg}, using the background signal set {DN bkg} and the polluted pixel DN set {DN} are fitted with polynomial least squares to determine the polluted speckle signal model, such as Figure 3 shown.

[0050] S5 speckle signal removal and image histogram matching.

[0051] The decontaminated image can be obtained by subtracting the speckle signal DN value obtained by the established contamination speckle signal model from the remote sensing image. Finally, in order to reduce the impact of removing contamination spots on the remote sensing image, the input contaminated remote sensing image is used as a reference, and the decontaminated image is matched (such as histogram matching) to the original remote sensing image to finally obtain the decontaminated image.

[0052] Furthermore, the process of spot signal removal and histogram matching processing for remote sensing images is as follows:

[0053] Read the remote sensing image to be restored, substitute the DN of each pixel into the mathematical model of the pollution spot signal to obtain the pollution signal. Subtract the calculated pollution signal from the remote sensing image to be restored to obtain the initial image after decontamination. After subtracting the pollution signal from the remote sensing image to be restored, the grayscale level of the initial image after decontamination will decrease as a whole compared with the original image. In order to minimize the impact of decontamination on the remote sensing image, the initial image after decontamination is processed with histogram matching based on the original image, and finally the decontaminated image is obtained. The comparison before and after restoration is as follows: Figure 4 shown.

[0054] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A general image restoration method for lens contaminated remote sensing images, the steps comprising: Acquire or construct satellite images with pure background without contaminating background signals; According to the set threshold, the location of the contaminated spot and its DN value of the satellite image of the pure background are obtained; A pollution spot signal model is established by using the pollution spot and the DN value of the pixel within the set range thereof and the background signal; Subtracting the speckle signal DN value obtained according to the contamination speckle signal model from the remote sensing image to be restored to remove the contamination speckles; The image after removing the contaminated spots is matched to the original remote sensing image to be restored to complete the restoration.

2. The method according to claim 1, characterized in that The method for constructing the satellite image with pure background without pollution background signal is as follows: Acquire an observation image; the observation image is an image whose cloud coverage is less than a set value; Determining the absence of contamination background signals through the observed image; The cloud-covered area in the observed image is eliminated by the pollution-free background signal to obtain a satellite image with a pure background.

3. The method according to claim 2, characterized in that The method for determining the contamination-free background signal is as follows: Using a sliding window of a set size to intercept the observed image by a set step length to obtain multiple frames of small window area images; Calculate the mean and standard deviation of the images in each small window area to obtain a mean set and a standard deviation set of the observed images; The mean of the minimum sub-region of the mean value set and the standard deviation set is taken as the pollution-free background signal.

4. The method according to claim 2, characterized in that: The method for removing the cloud covered area in the observed image is as follows: Determine the cloud determination threshold of each small window area image based on the mean and standard deviation of the small window area image; When a pixel value in a small window area image exceeds the cloud determination threshold, the pixel value is assigned the mean value of the small window area image; The images of each small window area are stitched together to obtain the satellite image of the pure background.

5. The method according to claim 1, characterized in that The threshold is less than the non-contamination background signal.

6. The method according to claim 1, characterized in that The setting range may adopt 4 neighborhoods, 8 neighborhoods and 24 neighborhoods.

7. The method according to claim 1, characterized in that The contamination speckle signal model is obtained by polynomial least squares fitting.

8. The method according to claim 1, characterized in that The image after removing the contamination spots is histogram matched to the original remote sensing image to be restored.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A storage medium storing a computer program, wherein when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images