Blind image deblurring method robust to impulse noise

Abstract

The invention discloses a blind image deblurring method robust to pulse noise. Belongs to the technical field of computer vision and image processing. The method solves the technical problem that when a fuzzy kernel is unknown and an image is polluted by pulse noise, a traditional method is difficult to balance noise suppression and detail reservation, so that a restored image is distorted. The method comprises the following steps of: establishing an image pyramid # imgabs 1 # of a blurred image # imgabs 0 # from coarse to fine by using a fuzzy kernel estimation mode from coarse to fine; the method comprises the following steps: estimating a blurring kernel # imgabs3 # of a degraded image of a # imgabs2 # layer, performing up-sampling on # imgabs4 # to obtain an initial value of a blurring kernel of a # imgabs5 # layer, updating a blurring kernel # imgabs7 # of # imgabs6 #, repeatedly performing alternate iterative optimization, performing sparse logarithm prior constraint in a blind image deblurring model, stabilizing a noise detection process, avoiding the influence of an extreme weight value, and combining FFT acceleration calculation to obtain a blurring kernel of the blind image deblurring model. The time complexity is reduced, and the fuzzy kernel estimation precision in a complex scene is improved by the overall method.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and particularly relates to a blind image deblurring method that is robust to impulse noise. Background Art

[0002] Traditional blind image deblurring methods rely on edge detection or Gaussian noise assumptions, are sensitive to impulse noise, and easily lead to bias in blur kernel estimation. Moreover, existing methods suppress noise through a weight matrix, but there are instabilities in the design of the regularization term (such as the constraint fails when the weight approaches 0 or 1), and some algorithms require multiple iterations for optimization, which is time-consuming and difficult to meet the real-time requirement.

[0003] When the blur kernel is unknown and the image is contaminated by impulse noise, traditional methods are difficult to balance noise suppression and detail preservation, resulting in distorted restored images. Summary of the Invention

[0004] To solve the technical problem that traditional methods are difficult to balance noise suppression and detail preservation, resulting in distorted restored images when the blur kernel is unknown and the image is contaminated by impulse noise, the present invention provides a blind image deblurring method that is robust to impulse noise. The method includes the following steps: The method includes the following steps: S1. Establish a blurred image containing impulse noise Image pyramid from coarse to fine ; S2. Initialize the blur kernel , and for use the alternating optimization algorithm to solve the corresponding blur kernel ; S3. Upsample , and use the upsampling result of as the initialized blur kernel of , and for use the alternating optimization algorithm to solve the corresponding blur kernel ; S4. In the same way as in step S3, gradually obtain the corresponding blur kernel ; S5. Solve the potential clear image through the blur kernel .

[0005] Furthermore, the alternating optimization algorithm is specifically: S21. Set a blind image deblurring model, which includes a data fitting term, an image regularization term, a blur kernel regularization term, a weight matrix sparsity constraint term, and a weight matrix distribution constraint term; S22. Initialize the weight matrix of the blind image deblurring model ; S23. Fix the blur kernel , and update the latent sharp image through the fast Fourier transform ; S24. Fix the latent sharp image , update the blur kernel , and obtain the updated blur kernel ; S25. Update the weight matrix through the latent sharp image and the updated blur kernel . .

[0006] Further, the blind image deblurring model is specifically:

[0007] ; Wherein, represents norm, represents norm, represents the Hadamard product, represents the gradient operator, , , and represent weight terms, represents the image pyramid from coarse to fine in any one of them.

[0008] Further, update the latent sharp image through the fast Fourier transform , through: for; Wherein, , and are the FFT operator, the inverse FFT operator and the complex conjugate operator respectively, , wherein, represents the iteratively reweighted least squares method, represents the latent sharp image the negative 1.2 power of the gradient magnitude, represents the number of iterations of the iteratively reweighted least squares method, is a positive number, is used to avoid the situation where the image gradient value is 0.

[0009] Further, update the blur kernel , an updated blurred kernel is obtained , by: It is carried out.

[0010] Furthermore, the weight matrix is updated by the potential clear image and the updated blurred kernel , by: It is carried out. It is carried out.

[0011] Furthermore, the potential clear image is solved by the blurred kernel through: It is carried out; wherein, represents the weight matrix corresponding to , represents the potential clear image corresponding to .

[0012] The beneficial effects of this method are as follows: The method described in the present invention uses a coarse-to-fine blurred kernel estimation method to establish a coarse-to-fine image pyramid of the blurred image ; estimating the blurred kernel of the degraded image of the th layer, upsampling to obtain the initial value of the blurred kernel of the th layer, updating its blurred kernel , repeating the alternating iterative optimization until the blurred kernel with the highest fineness and its corresponding clear image are obtained, which can balance noise suppression and detail retention; and by imposing a sparse logarithmic prior constraint in the blind image deblurring model, stabilizing the noise detection process, avoiding the influence of extreme weight values, and combining FFT to accelerate the calculation, reducing the time complexity, the overall method improves the accuracy of blurred kernel estimation in complex scenes. Description of the Drawings

[0013] Figure 1 This is a comparison diagram of the clear image (a) corresponding to a picture in the Levin dataset in the embodiment of the present invention, its blurred image (b) containing 10% noise, and the results of various deblurring methods applied to the blurred image (b). Detailed Embodiment

[0014] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0015] Embodiment 1 This embodiment provides a blind image deblurring method that is robust to impulse noise. The method includes the following steps: S1. Establish a blurred image containing impulse noise Image pyramid from coarse to fine ; S2. Initialize the blur kernel , and use the alternating optimization algorithm to solve for the corresponding blur kernel ; ; S3. Upsample , and use the upsampling result of as the initial blur kernel of . Then use the alternating optimization algorithm to solve for the corresponding blur kernel ; ; S4. In the same way as in step S3, gradually obtain the corresponding blur kernel ; S5. Solve for the potential clear image through the blur kernel .

[0016] Embodiment 2 This embodiment further limits Embodiment 1. The alternating optimization algorithm is specifically as follows: S21. Set a blind image deblurring model, which sequentially includes a data fitting term, an image regularization term, a blur kernel regularization term, a weight matrix sparsity constraint term, and a weight matrix distribution constraint term; S22. Initialize the weight matrix of the blind image deblurring model ; S23. Fix the blur kernel (the blur kernel here can be understood as the initial blur kernel in Embodiment 1 . During the process of continuously solving for the potential clear image of , both the blur kernel and the weight matrix are iterated), and update the potential clear image through the fast Fourier transform; S24. Fix the potential clear image , update the blur kernel , obtain the updated blur kernel ; S25. Update the weight matrix using the potential clear image and the updated blur kernel .

[0017] The blind image deblurring model is specifically as follows:

[0018] ; wherein, denotes norm, denotes norm, denotes the Hadamard product, denotes the gradient operator, , , and denote weight terms, denotes an image pyramid from coarse to fine among them.

[0019] Update the potential clear image by fast Fourier transform, through: for implementation; wherein, , and are respectively the FFT operator, the inverse FFT operator and the complex conjugate operator, , wherein, denotes the iteratively reweighted least squares method, denotes the potential clear image the negative 1.2 power of the gradient magnitude, denotes the number of iterations of the iteratively reweighted least squares method, is a positive number, is used to avoid the situation where the image gradient value is 0.

[0020] Update the blur kernel , obtain the updated blur kernel , through: for implementation.

[0021] Update the weight matrix using the potential clear image and the updated blur kernel , by: carry out.

[0022] Through the blur kernel Solve for the potential clear image by: carry out; wherein, represents the weight matrix corresponding to ; represents the potential clear image corresponding to .

[0023] Example 3 This example further illustrates the beneficial effects of the present invention. On datasets such as Levin and natural images, the PSNR and SSIM of the method of the present invention are respectively increased by 2.5% - 5.2% and 0.7% - 2.7%. At the same time, the present invention has strong robustness and can still restore clear details under 10% impulse noise. The method of the present invention can converge quickly, and both the blind deblurring and non-blind deblurring processes can converge within a limited number of iterations.

[0024] In this example, the parameter settings are: .

[0025] Achieved effect: On the Levin dataset, the PSNR reaches 27.69 d (a 2.7% improvement over the sub-optimal method).

[0026] Such as Figure 1Shown are the clear image (a) corresponding to an image on the Levin dataset and its blurred image with 10% noise (b) (PSNR = 15.4855, SSIM = 0.1333), as well as a comparison chart of the results of applying various deblurring methods to the blurred image (b). Among them, the deblurring method used for image (c) (PSNR = 9.1338, SSIM = 0.0277) is the method described in the paper "Blind image deblurring with Gaussian curvature of the image surface" (Source: Signal Processing: Image Communication); the deblurring method used for image (d) (PSNR = 8.7570, SSIM = 0.0136) is the method described in the paper "Graph-based blind image deblurring from a single photograph" (Source: IEEE transactions on image processing); the deblurring method used for image (e) (PSNR = 23.4329, SSIM = 0.7845) is the method described in the paper "Blind image deblurring withoutlier handling" (Source: Proceedings of the IEEE International Conference on Computer Vision); the deblurring method used for image (f) (PSNR = 26.3822, SSIM = 0.8577) is the method described in the paper "Oid: Outlier identifying and discarding in blind image deblurring" (Source: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16. Springer International Publishing); the deblurring method used for image (g) (PSNR = 27.6918, SSIM = 0.8729) is the method described in the present invention.

[0027] As can be seen from the above comparison, the method described in the present invention performs excellently in retaining image details, with a better visual effect than other methods, and the restored images are clearer and more natural. The method described in the present invention obtains the highest average PSNR and SSIM values, which are 26.5575 dB and 0.8289 respectively. Compared with the sub-optimal method with 25.8576 dB and 0.8232, there are improvements of 2.7% and 0.7% in PSNR and SSIM respectively.

Claims

1. A blind image deblurring method robust to impulse noise, characterized in that, The method includes the following steps: S1. Establish a blurred image containing impulse noise Image pyramid from coarse to fine ; S2. Initialize the blur kernel , for , use the alternating optimization algorithm to solve for the corresponding blur kernel ; S3. Upsample and use the upsampling result as the initial blur kernel of Then, use the alternating optimization algorithm to solve for the corresponding blur kernel ; ​ S4. In the same way as in step S3, gradually obtain the corresponding blurred kernel ; S5. Through the blur kernel Solve for the potential clear image.

2. The blind image deblurring method robust to impulse noise according to claim 1, wherein The specific alternating optimization algorithm is as follows: S21. Set a blind image deblurring model, which includes a data fitting term, an image regularization term, a blur kernel regularization term, a weight matrix sparsity constraint term, and a weight matrix distribution constraint term; S22. Initialize the weight matrix of the blind image deblurring model ; S23. Fix the blur kernel , and update the potential clear image through fast Fourier transform ; S24. Fix the potential clear image , update the blur kernel , and obtain the updated blur kernel ; S25, update the weight matrix with the potential clear image and the updated blurred kernel 。 。 3. The blind image deblurring method robust to impulse noise according to claim 2, wherein The specific blind image deblurring model is as follows: ； Among them, denotes the norm, denotes the norm, denotes the Hadamard product, denotes the gradient operator, 、 、 and denote the weight terms, denotes any one of the coarse-to-fine image pyramids.

4. The blind image deblurring method robust to impulse noise according to claim 3, wherein Updating a potential clear image by fast Fourier transform , by: Conduct; Among them, , and are the FFT operator, the inverse FFT operator, and the complex conjugate operator respectively. , where represents the iterative reweighted least squares method, represents the potential clear image the negative 1.2 power of the gradient magnitude. represents the number of iterations of the iteratively reweighted least squares method, is a positive number, serves to avoid the situation where the image gradient value is 0.

5. The blind image deblurring method robust to impulse noise according to claim 4, wherein, Update the blur kernel to obtain the updated blur kernel by: Proceed.

6. The blind image deblurring method robust to impulse noise according to claim 5, wherein Via a potential clear image and an updated blur kernel update the weight matrix , by: Proceed.

7. The blind image deblurring method robust to impulse noise according to claim 6, wherein Through the blur kernel Solving for the potential clear image by: Conduct; Among them, represents the weight matrix corresponding to , and represents the potential clear image corresponding to .

8. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, it implements the steps of the method according to any one of claims 1-7.

Images