Multi-scale remote sensing image stripe noise removing method, device, equipment and medium

Abstract

The invention provides a multi-scale remote sensing image strip noise removal method, device, equipment and medium, and the method comprises the steps: obtaining an original image which comprises strip noise; performing T-layer Laplacian pyramid decomposition on the original image to obtain T sub-images with different scales; respectively carrying out filtering processing on the T sub-images to obtain corresponding T clean sub-images; and fusing the T clean sub-images to obtain a stripe-removed image corresponding to the original image. According to the method, the remote sensing image is decomposed into the sub-images with different scales through the Laplacian pyramid to remove the stripe noise, so that the stripe noise is removed more thoroughly; the sub-images are decomposed in the frequency domain, and the part containing the stripe noise is filtered, so that the detail weakening of the image can be avoided on the basis of accurately removing the stripe noise.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a method, device, equipment and medium for removing multi-scale stripe noise from remote sensing images. Background Art

[0002] In the process of remote sensing image acquisition, due to the different sensitivity of sensors to radiation, high-resolution land observation satellite images are often accompanied by stripe noise. The existence of stripe noise seriously affects subsequent image segmentation, target detection, etc.

[0003] There are three main traditional image denoising methods: spatial domain matching method, variational model method and digital filtering method. The spatial domain matching method mainly includes moment matching and histogram matching methods. Among them, the moment matching method can suppress the grayscale distortion caused by the standard moment matching process by processing satellite images using standard moment matching. However, this type of method is based on the premise that each sensor has the same grayscale statistical characteristics, and cannot be used to process images of a small range and containing multiple objects. The variational model method introduces regularization technology to solve the image as a whole, but it requires prior knowledge of image noise and multi-channel information, and it is necessary to establish a complex mathematical model, which requires a large amount of calculation.

[0004] The digital filtering method mainly uses Fourier transform, wavelet transform and principal component transform technologies to suppress the noise components in the image information by designing specific filters. However, the noise components are often difficult to distinguish clearly, and the frequency of the stripe noise is usually not unique, which makes it impossible to completely remove the stripe noise. In addition, the image detail information with a similar frequency to the stripe noise is also easily filtered out, resulting in image blur. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and medium for multi-scale removal of stripe noise in remote sensing images, aiming to solve the technical problem that stripe noise removal is not thorough and image detail information is easily removed. To achieve the above purpose, the technical solution adopted in the embodiments of this application is as follows:

[0006] In a first aspect, the present application provides a method for removing multi-scale stripe noise from remote sensing images, the method comprising:

[0007] S1: Acquire an original image, wherein the original image contains stripe noise;

[0008] S2: Decomposing the original image into T-layer Laplacian pyramids to obtain T sub-images of different scales;

[0009] S3: performing filtering processing on the T sub-images respectively to obtain corresponding T clean sub-images;

[0010] S4: Fusing the T clean sub-images to obtain a strip-free image corresponding to the original image.

[0011] Furthermore, the step S3 comprises:

[0012] S31: Perform Fourier transform on the T sub-images respectively to obtain corresponding T frequency domain sub-images;

[0013] S32: performing a first transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain first components, and performing a second transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain second components;

[0014] S33: performing inverse Fourier transform on the T frequency-domain first components respectively to obtain corresponding T time-domain first original components, and using a second Gaussian convolution kernel to perform filtering processing on the T time-domain first original components respectively to obtain corresponding T time-domain first components;

[0015] S34: performing inverse Fourier transform on the T frequency-domain second components respectively to obtain corresponding T time-domain second components;

[0016] S35: respectively fuse the T time-domain first components and the corresponding time-domain second components to obtain corresponding T clean sub-images.

[0017] Furthermore, the first transformation is to make the first row of the frequency domain sub-image remain unchanged and the other rows remain zero; the second transformation is to make the first row of the frequency domain sub-image remain zero and the other rows remain unchanged.

[0018] Optionally, the variance of the second Gaussian convolution kernel is 3.

[0019] Furthermore, the size of the sub-image of the topmost pyramid in the T-layer Laplacian pyramid is one pixel.

[0020] Optionally, a first Gaussian convolution kernel is used for filtering in the Laplacian pyramid decomposition, and a size of the first Gaussian convolution kernel is 5×5.

[0021] In a second aspect, the present application provides a device for removing multi-scale stripe noise from remote sensing images, the device comprising:

[0022] An image acquisition module, used for acquiring an original image, wherein the original image contains stripe noise;

[0023] A multi-scale decomposition module, used for performing T-layer Laplace pyramid decomposition on the original image to obtain T sub-images of different scales;

[0024] A multi-scale filtering module, used for filtering the T sub-images respectively to obtain corresponding T clean sub-images;

[0025] The fusion module is used to fuse the T clean sub-images to obtain a strip-free image corresponding to the original image.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it can execute the method of the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method of the first aspect can be executed.

[0028] The beneficial effects of this application include at least:

[0029] 1. By decomposing the original image into multi-scale sub-images using the Laplace pyramid, the stripe noise is removed at different scales, making the stripe noise removal more thorough, and there is no residual stripe noise when observed at multiple scales.

[0030] 2. By transforming sub-images at different scales into the frequency domain, decomposing them into a part containing stripe noise and a part not containing stripe noise in the frequency domain, and filtering the part containing stripe noise, it is possible to avoid weakening of image details while accurately removing the stripe noise.

[0031] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A schematic diagram of a flow chart of a method for removing multi-scale stripe noise from remote sensing images according to an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a process of filtering a sub-image according to an embodiment of the present invention;

[0035] Figure 3 The present invention is a block diagram of a device for removing multi-scale stripe noise from remote sensing images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0037] Figure 1 FIG. 1 is a flow chart of a method for removing multi-scale stripe noise from remote sensing images according to an embodiment of the present invention. Figure 1 As shown, the above method includes:

[0038] S1: Acquire an original image, wherein the original image contains stripe noise.

[0039] Specifically, firstly, an original image I is obtained. The original image I is an image collected by a remote sensing satellite and contains stripe noise. It should be noted that the stripe noise in this embodiment is a vertical stripe.

[0040] Optionally, the original image I may be an image block obtained by cropping an image collected by a remote sensing satellite.

[0041] S2: Decomposing the original image into T-layer Laplacian pyramids to obtain T sub-images of different scales.

[0042] Specifically, the original image I is decomposed into T layers by Laplacian pyramid decomposition to obtain T sub-images L corresponding to the T layers. 0 , L 1 , L 2 , ..., L T-1 , then:

[0043]

[0044] Where i∈[0,T-1].

[0045] The Laplacian pyramid is generated based on the Gaussian pyramid. The Laplacian image of the i-th layer is actually the difference between the Gaussian image of the i-th layer and the Gaussian image of the i+1-th layer after upsampling. The Gaussian pyramid refers to the process of continuously reducing the size of an image through Gaussian blur filtering and downsampling, thereby obtaining a set of images with multiple resolutions in the pyramid. Usually, the bottom layer of the Gaussian pyramid is the original image, and each time a layer is added upward, a Gaussian blur filtering and downsampling are performed, thereby reducing the size of the image.

[0046] In this embodiment, first, the original image I is decomposed into T-layer Gaussian pyramids to obtain the corresponding Gaussian sub-images G from the first layer to the Tth layer. 0 , G 1 , G 2 , ..., G T-1 Among them, the Gaussian sub-image G of the next layer i+1 is the Gaussian sub-image G of the previous layer. i Through the first Gaussian convolution kernel G a After Gaussian filtering and downsampling, it can be expressed as:

[0047] G i+1 =Dn(G i *G a ) (2)

[0048] Among them, G i is the i+1th layer Gaussian sub-image, i∈[0,T-1], Dn(·) represents downsampling, G a is the first Gaussian convolution kernel.

[0049] Next, the Gaussian sub-image of the i-th layer is subtracted from the up-sampled Gaussian sub-image of the i+1th layer to obtain the corresponding sub-image L of the i-th layer. i . Sub-image L i The corresponding Gaussian sub-image G can be used i It is expressed as:

[0050] L i =G i -Up(G i+1 ) (3)

[0051] Combining formula (2) and formula (3), we get

[0052] L i =G i -Up(Dn(G i *G a )) (4)

[0053] Where Up(·) represents upsampling. Optionally, the first Gaussian convolution kernel G a The size of is 5 × 5. In this embodiment, two times down sampling and two times up sampling are adopted.

[0054] The Gaussian sub-images G from the first layer to the Tth layer are 0 , G 1 , G 2 , ..., G T-1 Substituting into formula (4), we can obtain the sub-images L from the first layer to the Tth layer after the Laplace pyramid decomposition. 0 , L 1 , L2 , ..., L T-1 .

[0055] Optionally, the topmost sub-image L after Laplacian pyramid decomposition T-1 The size of is one pixel. It should be understood that the number of layers T of the Laplacian pyramid decomposition can be determined according to the size of the original image and the size of the topmost sub-image.

[0056] This application decomposes the original image into sub-images of different scales and resolutions, and removes the stripes at each different scale, so that the stripe noise removal is more accurate and thorough, especially for high-resolution remote sensing images, there is no residual stripe when observed at multiple scales.

[0057] S3: Perform filtering processing on the T sub-images respectively to obtain corresponding T clean sub-images.

[0058] Figure 2 FIG. 1 is a schematic diagram of a process of filtering a sub-image according to an embodiment of the present invention. Figure 2 As shown, step S3 includes:

[0059] S31: Perform Fourier transform on the T sub-images respectively to obtain corresponding T frequency domain sub-images.

[0060] For each of the T sub-images L 0 , L 1 , L 2 , ..., L T-1 Perform Fourier transform to obtain the corresponding T frequency domain sub-images

[0061] S32: Perform a first transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain first components, and perform a second transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain second components.

[0062] Specifically, the first transformation is to make the first row of the frequency domain sub-image remain unchanged and the other rows zero; the second transformation is to make the first row of the frequency domain sub-image zero and the other rows remain unchanged. For example, for the frequency domain sub-image Perform a first transformation so that the frequency domain sub-image The first row remains unchanged and the other rows are zero, and the first component in the frequency domain is obtained. At the same time, the frequency domain sub-image Perform a second transformation so that the frequency domain sub-image The first row is zero, and the other rows remain unchanged, and the second component in the frequency domain is obtained.

[0063] It can be understood that the first frequency domain component The second component in the frequency domain includes information such as stripe noise in the vertical direction. It usually contains the detailed information of the image and does not contain vertical stripe noise.

[0064] S33: Perform inverse Fourier transform on the T frequency-domain first components respectively to obtain corresponding T time-domain first original components, and use a second Gaussian convolution kernel to filter the T time-domain first original components respectively to obtain corresponding T time-domain first components.

[0065] Specifically, take the i-th frequency domain sub-image For example, first, the first frequency domain component Perform inverse Fourier transform to obtain the corresponding first original component L′ in the time domain i1，0 Next, use the second Gaussian convolution kernel G b For the first original component L′ in the time domain i1，0 Filter and obtain the corresponding first time domain component L′ i1 For example, when i=1, the first frequency domain component of the first layer Perform inverse Fourier transform to obtain the corresponding first original component L′ in the time domain 01，0 , and use the second Gaussian convolution kernel G b For the first original component L′ in the time domain 01，0 Filter and obtain the corresponding first time domain component L′ 01 The first component in the frequency domain The stripe noise information is included in the image, which is filtered and removed.

[0066] In an embodiment of the present invention, the second Gaussian convolution kernel G b The variance σ is set to 3. The relationship between the variance and size of the Gaussian convolution kernel is:

[0067] σ=0.3*((k-1)*0.5-1)+0.8(5)

[0068] Accordingly, according to formula (4) and the variance value, the corresponding second Gaussian convolution kernel G can be calculated b size.

[0069] S34: Perform inverse Fourier transform on the T second frequency-domain components respectively to obtain T second time-domain components corresponding to the original image.

[0070] Specifically, take the i-th frequency domain sub-image For example, for the second frequency domain component Perform inverse Fourier transform to obtain the second time domain component L′ i2 The second frequency domain component It does not contain stripe noise information and no filtering is performed.

[0071] S35: respectively fuse the T time-domain first components and the corresponding time-domain second components to obtain corresponding T clean Gaussian sub-images.

[0072] Specifically, take the i-th frequency domain sub-image For example, the first time domain component L′ i1 and the corresponding time domain second component L′ i2 Add together to get the corresponding clean sub-image L′ i , that is:

[0073] L′ i =L′ i1 +L′ i2 (6)

[0074] It should be understood that the first time domain component L′ i1 is the sub-image L i In the corresponding scale, the part of the image with stripe noise removed is converted into the first component L′ in the time domain. i1 and the second time domain component L′ i2 Add together to get the clean sub-image L′ i .

[0075] According to the above steps, for T sub-images L at different scales 0 , L 1 , L 2 , ..., L T-1 After filtering is completed, the corresponding T clean sub-images L′ at different scales are obtained. 0 , L′ 1 , L′ 2 …L′ T-1 .

[0076] S4: Fusing the T clean sub-images to obtain a corresponding strip-free image.

[0077] Specifically, the clean sub-images L′ of T different scales are 0 , L′ 1 , L′ 2 ...L′ T-1 Add them together to get the corresponding de-striped image I′, then:

[0078]

[0079] Where i∈[0,T-1].

[0080] The present invention removes stripe noise at different scales by decomposing the original image into multi-scale sub-images by Laplacian pyramid and filtering the sub-images, so that the stripe noise removal is more thorough and clean. The sub-image is Fourier transformed into the frequency domain, and the sub-image is decomposed into a part containing vertical stripe noise and a part not containing vertical stripe noise in the frequency domain, and the part containing vertical stripe noise is filtered in the time domain, and the part not containing vertical stripe noise is not filtered, so that the stripe noise is accurately removed and the image detail information can be effectively retained.

[0081] Figure 3 is a block diagram of a device for removing multi-scale stripe noise from remote sensing images provided by the present invention. Figure 3 As shown, the device 300 for removing multi-scale stripe noise from remote sensing images can be built into an electronic device, and includes an image acquisition module 310 , a multi-scale decomposition module 320 , a multi-scale filtering module 330 and a fusion module 340 .

[0082] The image acquisition module 310 is used to acquire an original image, wherein the original image contains stripe noise;

[0083] The multi-scale decomposition module 320 is used to perform T-layer Laplacian pyramid decomposition on the original image to obtain T sub-images of different scales;

[0084] The multi-scale filtering module 330 is used to perform filtering processing on the T sub-images respectively to obtain corresponding T clean sub-images;

[0085] The fusion module 340 is used to fuse the T clean sub-images to obtain a strip-free image corresponding to the original image.

[0086] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the aforementioned Figure 1 The multi-scale removal method of remote sensing image stripe noise is described in the illustrated embodiment.

[0087] Furthermore, the electronic device also includes: at least one input device; at least one output device. The above-mentioned memory, processor, input device and output device are connected via a bus. Among them, the input device can be a camera, a touch panel, a physical button or a mouse, etc. The output device can be a display screen. The memory can be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as a disk memory. The memory is used to store a set of executable program codes, and the processor is coupled to the memory.

[0088] The embodiment of the present invention further provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments. The computer-readable storage medium may be the electronic device in the above embodiments. The computer-readable storage medium stores a computer program, which is executed by a processor to implement the above Figure 1 The multi-scale removal method of remote sensing image stripe noise described in the illustrated embodiment. Further, the computer storable medium may also be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0089] It should be noted that the functional modules in the various embodiments of the present invention can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.

[0090] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] The above is a description of a method, device, electronic device and readable storage medium for removing multi-scale stripe noise from remote sensing images provided by the present invention. For technicians in this field, according to the ideas of the embodiments of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A multi-scale method for removing stripe noise from remote sensing images, characterized in that: The method includes: S1: Acquire an original image, wherein the original image contains stripe noise; S2: Decomposing the original image into T-layer Laplacian pyramids to obtain T sub-images of different scales; S3: performing filtering processing on the T sub-images respectively to obtain corresponding T clean sub-images; S4: Fusing the T clean sub-images to obtain a strip-free image corresponding to the original image.

2. The method according to claim 1, characterized in that The step S3 comprises: S31: Perform Fourier transform on the T sub-images respectively to obtain corresponding T frequency domain sub-images; S32: performing a first transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain first components, and performing a second transformation on each of the T frequency domain sub-images to obtain corresponding T frequency domain second components; S33: performing inverse Fourier transform on the T frequency-domain first components respectively to obtain corresponding T time-domain first original components, and using a second Gaussian convolution kernel to perform filtering processing on the T time-domain first original components respectively to obtain corresponding T time-domain first components; S34: performing inverse Fourier transform on the T frequency-domain second components respectively to obtain corresponding T time-domain second components; S35: respectively fuse the T time-domain first components and the corresponding time-domain second components to obtain corresponding T clean sub-images.

3. The method according to claim 2, characterized in that The first transformation is to make the first row of the frequency domain sub-image remain unchanged and the other rows are zero; the second transformation is to make the first row of the frequency domain sub-image zero and the other rows remain unchanged.

4. The method according to claim 2, characterized in that: The variance of the second Gaussian convolution kernel is 3.

5. The method according to claim 1, characterized in that The size of the sub-image of the topmost pyramid in the T-layer Laplacian pyramid is one pixel.

6. The method according to claim 1, characterized in that A first Gaussian convolution kernel is used for filtering in the Laplacian pyramid decomposition, and the size of the first Gaussian convolution kernel is 5×5.

7. A device for removing multi-scale stripe noise from remote sensing images, characterized in that: The device includes: An image acquisition module, used for acquiring an original image, wherein the original image contains strips; A multi-scale decomposition module, used for performing T-layer Laplace pyramid decomposition on the original image to obtain T sub-images of different scales; A multi-scale filtering module, used for filtering the T sub-images respectively to obtain corresponding T clean sub-images; The fusion module is used to fuse the T clean sub-images to obtain a corresponding de-striped image.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for removing multi-scale stripe noise from remote sensing images as described in any one of claims 1 to 6 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for removing multi-scale stripe noise from remote sensing images as described in any one of claims 1 to 6 is implemented.

Images