A blind image deblurring algorithm based on local binary patterns

Abstract

An image blind deblurring algorithm based on local binary patterns relates to the technical field of image blind deblurring, and solves the problem that the existing image blind deblurring algorithms have poor blurring effects on remote sensing images. Based on the characteristics of the local binary patterns of clear images and blurred images, the present invention uses local binary patterns to extract important pixels in the deblurring process, and accumulates them using the L1 norm; at the same time, the gradient information corresponding to these important pixels is processed using the L0 norm. The gradients of the remaining pixels are processed using the L2 norm, so that the present invention can blur tiny details while sharpening the main edges of the image, improve image quality, reduce image artifacts, and estimate a more accurate blur kernel. The present invention not only performs well in the task of removing blur from non-remote sensing images, but also has better processing results for blurring remote sensing images than existing algorithms, and has certain engineering significance for improving the image quality of daily life and space remote sensing imaging.

Description

Technical Field

[0001] The present invention relates to the technical field of image blind deblurring, and in particular to an image deblurring algorithm based on local binary pattern (LBP). Background Art

[0002] Whether in daily life or in remote sensing imaging, image blur can occur due to factors such as camera shake, imaging equipment accuracy, and complex imaging environments, leading to image quality degradation and loss of important details. Due to the uncertainty of the causes of blur, it is necessary to restore blurred images without prior knowledge. Blind image deblurring involves estimating the blur kernel and the clear image separately, given only the blurred image. However, due to its infinite number of solutions, this problem is a classic ill-posed problem, requiring the use of prior knowledge and the addition of certain constraints to obtain the optimal solution. Current methods based on extreme channels suffer from unsatisfactory processing results due to the presence of interfering pixels in areas affected by extreme channels in complex textured images, such as remote sensing images. Summary of the Invention

[0003] In order to solve the problems existing in the prior art, the present invention provides an image deblurring algorithm based on local binary pattern (LBP), which solves the problem that the prior art method using a priori model cannot achieve ideal imaging effects in actual imaging.

[0004] The technical solutions adopted by the present invention to solve the technical problems are as follows:

[0005] An image deblurring algorithm based on local binary patterns, the algorithm comprises the following steps:

[0006] Step 1: Get the blurred image;

[0007] Step 2: Constructing an objective function for the blurred image and initializing the parameters of the objective function;

[0008] The objective function described in step 2 is:

[0009]

[0010] Among them, α, β and γ are the weights of the regularization term; g, u and h are the blurred image, potential clear image and blur kernel respectively; L(u) is a nonlinear operator, is a function of the gradient term;

[0011] Step 3: solving the potential clear image function in the objective function;

[0012] Solving the potential clear image function in the objective function in step 3 includes the following steps:

[0013] Step A: Use the alternating projection minimization method for multiple iterations, use the semi-quadratic splitting method, and introduce auxiliary variables w and z to solve the equation. The equation can be expressed as:

[0014]

[0015]

[0016] Among them, λ1 and λ2 are penalty factors;

[0017] Step B: Solve for w, select key pixels through LBP threshold, and construct linear operator C. C is the mapping matrix from key pixels to the original image, which is defined as:

[0018]

[0019] Where LBPu is the LBP value of the image, (i, j) is the pixel coordinate;

[0020] The FISTA algorithm is used to solve w, and the contraction operator is as follows:

[0021] D e (w)=sgn(w)max(|we|,0);

[0022] Step C: Solve for z. When φ(z)=||z||0,

[0023]

[0024] when hour,

[0025]

[0026] Step D: Introduce the auxiliary variable v to obtain the split equations C and D:

[0027]

[0028] Where λ3 is the penalty factor;

[0029] Step E: Derivative v in function C and set it equal to 0 to solve for v:

[0030] v=(λ1C T C+λ3) -1 (λ1C T w+λ3u).

[0031] Take the derivative of u in function D and set it equal to 0, and use the fast Fourier transform to solve for u:

[0032]

[0033] Among them, F, F -1 and They represent FFT, inverse FFT and conjugate FFT respectively;

[0034] Step 4: solving the fuzzy kernel function in the objective function;

[0035] Step 5: Repeat steps 3 and 4 according to the set number of times;

[0036] Step 6: Obtain a clear image through non-blind restoration algorithm.

[0037] Preferably, the LBP values ​​of all pixels of the blurred image in step 1 are recorded as LBPu; the LBPu acquisition process is:

[0038] P-1

[0039] LBPu(x c ,y c )=∑2 p s(i p -i c )

[0040] p=0,

[0041] Among them, (x c ,y c ) is the center pixel, i c is the gray value of the center pixel, i p is the grayscale value of the 8-neighborhood pixel, P is 8, and s(x) is the sign function;

[0042]

[0043] After obtaining the LBP values ​​of all pixels in the blurred image, the key pixels for improving the image resolution are obtained by setting the LBP threshold, and the convex L1 norm is used for accumulation.

[0044] LBP(u)=||L(u)||1.

[0045] Preferably, in step 2, after obtaining the pixels that pass the LBP threshold screening, the gradients corresponding to these key pixels are sparsely processed using the L0 norm, and the gradients of the remaining pixels are processed using the L2 norm to obtain

[0046]

[0047] Among them, TL and TM are the upper and lower limits of the threshold respectively, and LBPu is the LBP value of the image.

[0048] Preferably, solving the fuzzy kernel function in the objective function in step 4 includes the following steps:

[0049]

[0050] Apply FFT to the function E to obtain the blur kernel:

[0051]

[0052] It is required that all elements in h are greater than 0 and the sum of all elements is 1.

[0053] The beneficial effects of the present invention are as follows: the present invention innovatively proposes a new image blind deblurring method for processing images during the imaging process, especially blurs that exist in remote sensing images with very rich textures. The theoretical model and algorithm effectively remove blurs and restore images while ensuring that image detail information is not lost and preventing over-sharpening. It has strong robustness for different types of images, is more suitable for processing remote sensing images than existing algorithms, and can effectively improve the image quality of acquired remote sensing images. The present invention can not only be applied to ordinary image processing required in daily life, but can also be competent for the restoration of space remote sensing images, solve the problem of image quality degradation caused by multiple factors such as imaging systems and imaging environments, and achieve the acquisition of high-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 The present invention provides a flow chart of an image deblurring algorithm based on local binary patterns. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0056] Since the grayscale invariant and rotational invariant operator LBP can effectively describe image texture and extract texture features, and the histogram distribution of LBP of clear image and blurred image is similar, blurring will not change the distribution of LBP value on a large scale, LBP is used to distinguish pixels in different situations in the image restoration process and perform corresponding processing respectively.

[0057] In a 3*3 window, the center pixel is compared with its eight neighboring pixels. If the neighboring pixels are smaller than the center pixel, they are set to 0; otherwise, they are set to 1. Binary encoding is then performed to obtain the LBP value of the center pixel in the window.

[0058] like Figure 1 As shown, the present invention proposes an effective optimization algorithm based on LBP, that is, an image deblurring algorithm based on local binary patterns, which includes the following steps:

[0059] Step 1: Get the blurred image;

[0060] Step 2: Constructing an objective function for the blurred image and initializing the parameters of the objective function;

[0061] The objective function is:

[0062]

[0063] Among them, α, β and γ are the weights of the regularization term; g, u and h are the blurred image, potential clear image and blur kernel respectively; L(u) is a nonlinear operator, is a function of the gradient term.

[0064] The LBP values ​​of all pixels of the blurred image in step 1 are recorded as LBPu. The process of obtaining the LBPu is as follows:

[0065]

[0066] Among them, (x c ,y c ) is the center pixel, i c is the gray value of the center pixel, i p is the grayscale value of the 8-neighborhood pixel, P is 8, and s(x) is the sign function;

[0067]

[0068] After obtaining the LBP values ​​of all pixels in the blurred image, the key pixels for improving the image resolution are obtained by setting the LBP threshold, and the convex L1 norm is used for accumulation.

[0069] LBP(u)=||L(u)||1.

[0070] Step 3: solving the potential clear image function in the objective function;

[0071] In step 2, after obtaining the pixels that pass the LBP threshold screening, the gradients corresponding to these key pixels are sparsely processed using the L0 norm, and the gradients of the remaining pixels are processed using the L2 norm to obtain

[0072]

[0073] Among them, TL and TM are the upper and lower limits of the threshold respectively, and LBPu is the LBP value of the image.

[0074] Solving the potential clear image function in the objective function in step 3 includes the following steps:

[0075] Step A: After multiple iterations, the alternating projection minimization method is used, and the semi-quadratic splitting method is used to solve the equation by introducing auxiliary variables w and z. The equation can be expressed as:

[0076]

[0077]

[0078] Among them, λ1 and λ2 are penalty factors;

[0079] Step B: Solve for w. Since L(u) is a nonlinear operator, a linear solution cannot be performed. Select key pixels through the LBP threshold and construct a linear operator C. C is the mapping matrix from key pixels to the original image, which is defined as:

[0080]

[0081] Where LBPu is the LBP value of the image, (i, j) is the pixel coordinate;

[0082] Use the FISTA algorithm to solve w, and set the parameters and solve the process as follows:

[0083] , D e (w) = sgn(w)max(|we|,0);

[0084] Step C: Solve for z. When φ(z)=||z||0,

[0085]

[0086] when hour,

[0087]

[0088] Step D: Introduce the auxiliary variable v to obtain the split equations C and D:

[0089]

[0090]

[0091] Where λ3 is the penalty factor.

[0092] Step E: Derivative v in function C and set it equal to 0 to solve for v:

[0093] v=(λ1C T C+λ3) -1 (λ1C T w+λ3u).

[0094] Take the derivative of u in function D and set it equal to 0, and use the fast Fourier transform to solve for u:

[0095]

[0096] Among them, F, F -1 and They represent FFT, inverse FFT and conjugate FFT respectively;

[0097] Step 4: solving the fuzzy kernel function in the objective function;

[0098] Solving the fuzzy kernel function in the objective function in step 4 includes the following steps:

[0099]

[0100] Apply FFT to the function E to obtain the blur kernel:

[0101]

[0102] It is required that all elements in h are greater than 0 and the sum of all elements is 1.

[0103] Step 5: Loop through steps 3 and 4 until the loop termination condition is met. In this embodiment, steps 3 and 4 are looped through five times, i.e., the alternating projection minimization method in step A is iterated five times.

[0104] Step 6: Obtain a clear image through non-blind restoration algorithm.

Claims

1. An image deblurring algorithm based on local binary patterns, characterized in that: The algorithm consists of the following steps: Step 1: Get the blurred image; Step 2: Construct an objective function of the blurred image and initialize the parameters of the objective function; the objective function in step 2 is: Among them, α, β and γ are the weights of the regularization term; g, u and h are the blurred image, potential clear image and blur kernel respectively; L(u) is a nonlinear operator, is a function of the gradient term; Step 3: solving the potential clear image function in the objective function; Solving the potential clear image function in the objective function in step 3 includes the following steps: Step A: Use the alternating projection minimization method for multiple iterations, use the semi-quadratic splitting method, and introduce auxiliary variables w and z to solve the equation. The equation can be expressed as: Among them, λ1 and λ2 are penalty factors; Step B: Solve w, select key pixels through LBP threshold, and construct linear operator C. C is the mapping matrix from key pixels to the original image, which is defined as: Where LBPu is the LBP value of the image, (i, j) is the pixel coordinate; The FISTA algorithm is used to solve w and the contraction operator is as follows: D e (w)=sgn(w)max(|w-e|,0); Step C: Solve for z. When φ(z)=||z||0, when hour, Step D: Introduce the auxiliary variable v to obtain the split equations C and D: Where λ3 is the penalty factor; Step E: Take the derivative of v in function C and set it equal to 0 to solve for v: v=(λ1C T C+λ3) -1 (λ1C T w+λ3u) Take the derivative of u in function D and set it equal to 0, and use the fast Fourier transform to solve for u: Among them, F, F -1 and They represent FFT, inverse FFT and conjugate FFT respectively; Step 4: Solve the fuzzy kernel function in the objective function; Step 5: Execute step 3 and step 4 repeatedly according to the set number of times; Step 6: Obtain a clear image through non-blind restoration algorithm.

2. The image deblurring algorithm based on local binary patterns according to claim 1, characterized in that: The LBP values ​​of all pixels of the blurred image in step 1 are recorded as LBPu. The acquisition process of LBPu is as follows: Among them, (x c ,y c ) is the center pixel, i c is the gray value of the center pixel, i p is the grayscale value of the 8-neighborhood pixel, P is 8, and s(x) is the sign function; After obtaining the LBP values ​​of all pixels in the blurred image, the key pixels for improving the image resolution are obtained by setting the LBP threshold, and the convex L1 norm is used for accumulation. LBP(u)=||L(u)||1.

3. The image deblurring algorithm based on local binary patterns according to claim 1 or 2, characterized in that: In step 2, after obtaining the pixels that pass the LBP threshold screening, the gradients corresponding to these key pixels are sparsely processed using the L0 norm, and the gradients of the remaining pixels are processed using the L2 norm to obtain Among them, TL and TM are the upper and lower limits of the threshold respectively, and LBPu is the LBP value of the image.

4. The image deblurring algorithm based on local binary patterns according to claim 1, characterized in that: Solving the fuzzy kernel function in the objective function in step 4 includes the following steps: Use FFT to obtain the blur kernel of function E: It is required that all elements in h are greater than 0 and the sum of all elements is 1.

Images