A total difference non-blind image deblurring method based on local extremum constraint

Abstract

The application discloses a total difference non-blind image deblurring method based on local extremum constraint, and comprises the following steps: obtaining a dark channel of a degraded image according to a dark channel definition of an image; assigning a minimum value greater than zero to a position of an element being zero in the dark channel to obtain a local extremum of the degraded image; constructing a local extremum constraint according to a relationship between the local extremum of the degraded image and a local structure of the image; strengthening the spatial local structure of the image according to the local extremum constraint to construct a total difference model of two local extremum constraints; proposing a total difference image deblurring method based on the local extremum constraint; and restoring the inherent spatial local structure of the image by using the method to obtain a restored image. The method uses the local extremum of the image to strengthen the structural information of the image, and restores the degraded image by combining the total difference model, so that the inherent spatial local structure of the image is effectively restored.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a total difference non-blind image deblurring method based on local extremum constraint. BACKGROUND

[0002] Image blur is one of the main reasons for poor image quality, which can delete important details in the image, such as image edges. Non-blind image deblurring is to recover the latent sharp image from the blurred image generated by a known blur kernel, which is a common but challenging inverse problem in imaging. Non-blind image deblurring is an ill-posed problem, and the existence of noise and band-limited blur kernel makes the solution of the problem not unique. This makes it difficult to estimate the latent sharp image from a single blurred observation, even if the blur kernel is known.

[0003] Image deblurring is a key step to improve image quality. Since image deblurring is essentially ill-posed, regularization techniques are adopted to make it well-posed. In the regularization framework, a good image prior model that can effectively represent the inherent properties of the original image can be used to obtain a high-quality restored image. In the past few decades, various regularization-based deblurring methods have emerged. Among them, the total difference model is widely used due to its excellent ability to preserve significant edges, but it also smooths the details of the image. SUMMARY

[0004] In view of the problems in the prior art, the present application discloses a total difference non-blind image deblurring method based on local extremum constraint, which specifically comprises the following steps:

[0005] According to the image dark channel definition, the dark channel of the degraded image is obtained, and a minimum value greater than zero is assigned to the position of the zero element in the dark channel to obtain the local extremum of the degraded image.

[0006] According to the relationship between the local extremum of the degraded image and the local structure of the image, a local extremum constraint is constructed, the spatial local structure of the image is strengthened according to the local extremum constraint, and a total difference model of two local extremum constraints is constructed.

[0007] A total difference image deblurring method based on local extremum constraint is proposed, and the inherent spatial local structure of the image is recovered by using the method to obtain a restored image.

[0008] Further, the local extremum constraint is obtained in the following way:

[0009] Assign a minimum value greater than zero to the position of the zero element in the dark channel

[0010]

[0011] Wherein B represents the number of wave bands, i and q represent the position of pixels, represents a pixel block centered at i, X j is the jth wave band τ is a constant greater than 0, τ = 10 -6 If X is a gray image, then min j∈{1,...,B} X j (q) = X j (q).

[0012] Further, the total difference model of the two local extremum constraints is established in the following manner:

[0013] For a gray image, the expression of the total difference model of the local extremum constraint is:

[0014]

[0015] For a hyperspectral image, the expression of the total difference model of the local extremum constraint is:

[0016]

[0017] Wherein, j = 1, 2, …, B represents the jth wave band, C0 is the image local extremum, which is defined as formula (1).

[0018] Due to the adoption of the above technical scheme, the total difference non-blind image deblurring method based on local extremum constraint provided by the application explores the relationship between the dark channel of the blurred image and the local structure of the image, that is, the local minimum pixel in the blurred degraded image can reflect the local spatial structure of the image to a certain extent, firstly, the image dark channel is extended to a multi-waveband hyperspectral image, which is called a generalized dark channel, and then a kind of image local extremum constraint is defined, and then the constraint is introduced into the total difference model to define the local extremum constraint total difference model, thereby solving the problem that the total difference model tends to smooth the image detail information, on this basis, the total difference non-blind image deblurring method based on local extremum constraint is proposed, which can effectively preserve the significant edge structure of the image and restore the detail information of the image by strengthening the structure information of the degraded image local minimum pixel, and the recovery effect is improved. The method can be used as an effective means for blurred degraded image (including natural image and hyperspectral image) recovery, and has important application value in blurred degraded image recovery and the like. BRIEF DESCRIPTION OF DRAWINGS

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a total difference non-blind image deblurring method based on local extremum constraints provided by the present invention.

[0021] Figure 2 For local extrema of the degraded image;

[0022] Figure 3 This is a schematic diagram illustrating how the correlation values ​​of local extrema reflect the image structure in this invention;

[0023] Figure 4 This is a schematic diagram of the structural information of the local extremum constraint enhancement image in this invention;

[0024] Figure 5 This is a schematic diagram of the restoration result of the local extremum constraint total difference non-blind image deblurring method provided by the present invention on a natural image in an embodiment;

[0025] Figure 6 This is a schematic diagram of the recovery result of the total difference non-blind image deblurring method with local extremum constraints provided by the present invention on a hyperspectral image. Detailed Implementation

[0026] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:

[0027] like Figure 1 The method shown is a total difference non-blind image deblurring method based on local extremum constraints, which specifically includes the following steps:

[0028] Step 101: Calculate the local extrema of the blurred degraded image based on the obtained blurred degraded image;

[0029] Specifically, we further generalize the dark channel, which describes the smallest pixel in a local neighborhood of an image, into a more general form applicable to multi-band images and hyperspectral images, as defined below.

[0030]

[0031] Where B represents the band number, and i and q represent the pixel positions. X represents the pixel block centered at point i. jis a constant greater than 0, in this paper τ = 10 -6 If X is a gray image, then min j∈{1,...,B} X j (q) = X j (q). According to the obtained blurred degradation image Y, the local extreme value of the degradation image is calculated by using the above formula, as shown in Fig. 2 (b), it can be seen that the dark channel of the blurred degradation image has fewer zero elements than the dark channel of the clear image. Figure 2

[0032] Step 102: Calculate the local extreme value related constraint according to the local extreme value of the blurred degradation image;

[0033] Specifically, for the extreme value of the degradation image obtained in step 101, we assign a minimum value greater than zero to the position of its zero element to obtain the local extreme value of the image. By taking the reciprocal of each position element, we get the related value of the local extreme value, which is called the related value of the local extreme value. Through the related value of the local extreme value, we explore the relationship between the local extreme value of the image and the structure of the image, as shown in Fig. 3 (b). It can be seen that the related value of the local extreme value can reflect the structural information of the image to some extent, so we consider using the related value of the local extreme value of the image to constrain the gradient of the degradation image. Figure 3

[0034] Step 103: According to the local extreme value related constraint obtained in step 103, constrain the image gradient generated in each iteration to strengthen the structural information of the image; as shown in Fig. 4 (b), it reflects the image gradient after strengthening by using the local extreme value constraint. Figure 4

[0035] Step 104: Introduce the local extreme value related constraint into the total difference model to define the local extreme value constraint total difference, and define the adaptive updating regularization term parameter.

[0036] Specifically, the constraint total difference model based on gray image and the constraint total difference model based on multispectral and hyperspectral image are defined respectively as follows:

[0037]

[0038]

[0039] Where j = 1, 2, …, B represents the jth waveband, C 0i is the local minimum value of the image, which is defined as follows:

[0040]

[0041] Where, B represents the number of wavebands, i and q represent the position of the pixel, X​​​j is the jth band τ is a constant greater than 0, in this paper τ = 10 -6 If X is a grayscale image, then min j∈{1,...,B} X j (q) = X j (q).

[0042] For a grayscale image, the parameter λ is defined as follows

[0043]

[0044] where ρ = 2 (α + θMN), MN is the total number of pixels in each band of the image, σ is the standard deviation of zero-mean Gaussian noise, the parameters (α, β) → 0 and θ = 0.4.

[0045] For multispectral and hyperspectral images, the parameter λ is defined as

[0046]

[0047] It can be seen that the value of λ in the present application is adaptive, it does not need to be manually specified, and for hyperspectral images, different bands have different λ values.

[0048] Step 105: Determine whether the iteration stopping condition is met, if yes, the iteration is ended, and an estimated latent sharp image is obtained, if not, continue iteration until the iteration stopping condition is met.

[0049] Embodiment:

[0050] The specific steps of the total difference non-blind image deblurring method based on local extremum constraint implemented by the present application are as follows:

[0051] According to the above method steps, for natural images and hyperspectral images, respectively, in several groups of different blur kernel degradation conditions, degraded images are generated to test the total difference non-blind image deblurring method based on local extremum constraint provided by the present application, and the application effect is analyzed and evaluated.

[0052] 1. Data set and degradation type

[0053] The test data consists of three natural image datasets and two hyperspectral images. Among them, the natural image test dataset 1 consists of 20 gray-scale images, dataset 2 consists of 9 color images and 6 gray-scale images, a total of 15 images, and dataset 3 is the Berkeley Segmentation Dataset, which consists of 100 images. The first set of hyperspectral data uses Pavia University, which is the Pavia University hyperspectral dataset acquired by the ROSIS (Reflective Optics System Imaging Spectrometer) sensor in the Pavia region of Italy in 2001. The image size is 610*340, a total of 207400 pixels, the spatial resolution is 1.3m, the band range is 0.43-0.86μm, and there are 103 bands. The second set of hyperspectral data Urban162 is obtained by the Hydice sensor, and the image size is 307*307. The original data has 210 bands, after removing noise and water absorption bands, generally leaving 162 bands for subsequent processing and analysis.

[0054] Given these datasets, the degraded images are generated by the following blur kernels and standard deviations of Gaussian noise:

[0055] 1) Gaussian blur kernel with standard deviation 1.6, noise level σ = 0.00006, denoted as GuassianA;

[0056] 2) Gaussian blur kernel with standard deviation 3, noise level σ = 0.0001, denoted as GuassianB;

[0057] 3) Uniform blur kernel with size 9, noise level σ = 0.00006, denoted as SquareA;

[0058] 4) Uniform blur kernel with size 13, noise level σ = 0.00006, denoted as SquareB;

[0059] 5) Motion blur kernel with displacement 20 pixels and motion angle 15 degrees, noise level σ = 0.0002, denoted as MotionA;

[0060] 6) Motion blur kernel with displacement 35 pixels and motion angle 45 degrees, noise level σ = 0.0002, denoted as MotionB;

[0061] 2. Experimental evaluation index

[0062] In order to give a comprehensive evaluation, the experimental results on natural images and hyperspectral images are evaluated quantitatively and visually. On natural images, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are used for quantitative measurement, and on hyperspectral data, the PSNR, SSIM, spectral angle mapper (SAM) and the synthetic global multidimensional relative error (ERGAS) are used for quantitative evaluation, wherein the PSNR and SSIM are used to evaluate the spatial quality of the image, the larger the value is, the better the quality of the restored image is, and the SAM and ERGAS evaluate the spectral quality of the image, the smaller the value is, the better the quality of the restored image is.

[0063] 3. Analysis and evaluation of experimental results

[0064] The total difference non-blind image deblurring method based on local extremum constraint provided in the application has the experimental results under multiple data sets and blur kernel degradation types as shown in Tables 1-3, and the best values of each evaluation index are marked in bold, and the corresponding restored result images are shown in Figs. 1-3. Figure 5 - Figs. Figure 6 .

[0065] In order to analyze and evaluate the restoration effect of the proposed total difference non-blind image deblurring method based on local extremum constraint (LMCTV CTV and LMCTV CHTV ), the traditional total difference (TV), the fast total difference image restoration based on derivative alternating direction optimization (ADMM-C and ADMM-H), the fast positive constraint deconvolution (FPD) of hyperspectral images and the hyperspectral image deconvolution based on spectral space total difference regularization (SSTV) are introduced. The experimental evaluation results on three natural image data sets and two hyperspectral images are obtained, as shown in Tables 1-3 below:

[0066] Table 1: Experimental evaluation results on data set 1

[0067]

[0068] Table 2: Experimental evaluation results on data set 2

[0069]

[0070] Table 3: Experimental evaluation results on data set 3

[0071]

[0072]

[0073] As can be seen from Tables 1-3, the total difference non-blind image deblurring method based on local extremum constraint (LMCTV CTVThe non-blind image deblurring method is superior to traditional total variation (TV) and derivative-based alternating direction optimization fast total variation image restoration (ADMM-C and ADMM-H) in recovery effect under multiple natural image data sets and blurring degradation types.

[0074] Experimental evaluation results on hyperspectral data are shown in Tables 4-5 as follows:

[0075] Table 4 Experimental evaluation results on Urban data

[0076]

[0077] Table 5 Experimental evaluation results on Pavia University data

[0078]

[0079] As can be seen from Tables 4 and 5, the total variation based on local extremum constraint (LMCTV CHTV The non-blind image deblurring method is superior to fast positive constraint deconvolution (FPD) of hyperspectral images and hyperspectral image deconvolution based on spectral space total variation regularization (SSTV) in recovery effect in experiments on two hyperspectral data of different degradation types.

[0080] The total variation based on local extremum constraint non-blind image deblurring method disclosed in the application uses the dark channel of a blurred image to reflect the spatial structure information of the image to some extent and the fact that blurring operation changes the local minimum of the image, uses image local extremum to strengthen the structure information of the image, and restores the degraded image by combining a total variation model, so that the inherent spatial local structure of the image, including significant edges and detail information, is effectively restored, and the method has good applicability to different blurring degradation types.

[0081] The above description is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent replacement or change according to the technical solution and inventive concept of the application within the technical range disclosed by the application, which should be covered in the protection scope of the application.

Claims

1. A total-variation based non-blind image deblurring method based on local extremum constraint, characterized in that The application relates to a total difference image deblurring method based on local extremum constraint. The application comprises the following steps: According to the relationship between the local extremum of the degraded image and the local structure of the image, a local extremum constraint is constructed, the spatial local structure of the image is strengthened according to the local extremum constraint, and a total difference model of two local extremum constraints is constructed; The application proposes a total difference image deblurring method based on local extremum constraint, and the inherent spatial local structure of the image is recovered to obtain a recovered image by using the method; The local extremum constraint is obtained in the following way: The zero element position in the dark channel is given a minimum value greater than zero wherein B denotes the number of wavebands, i and q denote the position of the pixel, denotes a block of pixels centered at i, X j is the jth waveband τ is a constant greater than 0, τ = 10 -6 If X is a grayscale image, then min j∈{1,...,B} X j (q) = X j (q); The total difference model of the two local extremum constraints is established in the following way: For a gray image, the expression of the total difference model of the local extremum constraint is as follows: For a hyperspectral image, the expression of the total difference model of the local extremum constraint is as follows: Wherein, j=1, 2,..., B represents the jth waveband, C0 is the local extremum of the image, and the definition is formula (1).

Images