Image restoration method

Abstract

A image restoration method comprises: inputting a noisy image into a computing device; configuring multiple iteration parameters for a main iterative module of the computing device; performing iterative image restoration on the noisy image by the main iterative control module of the computing device based on an image restoration objective function; and determining whether the iteration has converged based on the denoised image by the main iterative control module. If the iteration has converged, outputting the denoised image; otherwise, the main iterative control module performs further iterations using the image restoration objective function.

Description

Technical Field

[0001] The present invention relates to an image restoration method, in particular to an image restoration method combining first-order total variation and second-order total variation. Prior Art

[0002] Total variation regularization plays an important role in signal processing and image restoration. Total variation can be summarized into two categories: channel-by-channel calculation methods and vector bundle calculation methods. The channel-by-channel calculation method divides each color channel in the color image into independent channels, and then applies total variation to each independent channel to remove noise while retaining image details. However, the channel-by-channel calculation method will cause the correlation between different channels to be ignored, affecting the image restoration results. In addition, when processing images with complex structures and textures, it will not be able to fully preserve their details and textures.

[0003] Vector bundle computing methods treat an image as a multidimensional set of data points defined in a vector bundle. They use metrics to describe the relationships and structure between pixels, better preserving the image's geometric properties and details. The image is viewed as a cross-section of a vector bundle defined on an underlying manifold, and covariant derivatives are used to compute the image's gradient, thereby capturing the image's subtle structure and texture. However, because vector bundle computing methods need to handle complex geometric structures and high-dimensional data, they require a large number of matrix operations and mathematical operations in a very high-dimensional space, increasing the time and computing resources required to process the image. Summary of the Invention

[0004] The main purpose of the present invention is to combine first-order and second-order chroma-luminance isotropic total variation regularization terms to perform image restoration on noisy images, which can effectively remove noise in noisy images and maintain the detailed features in the noisy images.

[0005] An image restoration method of the present invention includes: inputting a noisy image into a computing device; a main iteration module of the computing device setting a plurality of iteration parameters; and the main iteration control module of the computing device performing image restoration iterations on the noisy image using an image restoration objective function, wherein the image restoration objective function is: ,in, is an image restoration model parameter, is a denoised image matrix, is a Frobenius norm, is the matrix of the noisy image, is a Gaussian blur convolution kernel, is a convolution operator, is the first-order isotropic total variation of chroma-brightness, is a SV-VBTV regularization parameter, is the total isotropic variation of chroma-brightness second order, is a SV-SOVBTV regularization term parameter, the main iterative control module of the computing device updates the matrix of the denoised image, multiple parameters in the chroma-luminance first-order isotropic total variation, and multiple parameters in the chroma-luminance second-order isotropic total variation according to the image restoration objective function; and the main iterative control module of the computing device determines whether the iteration has converged based on the denoised image, and if so, outputs the denoised image; if not, the main iterative control module iterates again using the image restoration objective function.

[0006] The main iterative control module of the computing device of the present invention combines the first-order and second-order isotropic total variation of chroma-luminance to perform image restoration on noisy images, which can effectively remove noise in the image and maintain the global consistency and detail characteristics of multiple channels. Simple diagram description

[0007] FIG1 is a flowchart of an image restoration method according to one embodiment of the present invention. Figures 2a to 2c: the original pattern; the noisy image with added noise; and the denoised image obtained by performing image restoration on the noisy image using the image restoration method of this embodiment. Figures 3a to 3c: the original pattern; the noisy image with added noise; and the denoised image obtained by performing image restoration on the noisy image using the image restoration method of this embodiment. Implementation Method

[0008] Referring to FIG. 1 , which is a flowchart of an image restoration method 10 according to an embodiment of the present invention, the method includes: inputting a noisy image to a computing device 11, setting iteration parameters 12, performing image restoration on the noisy image and updating a denoised image and parameters 13, determining whether convergence has occurred 14, and outputting a denoised image 15.

[0009] Please refer to FIG. 1. In step 11, a noisy image is input into a computing device. In this embodiment, the noisy image is a color image. The matrix of the noisy image is represented as , is a real number, M and N are the number of columns and rows of the noisy image, respectively, and 3 is the three channels of the color space. The computing device can be a desktop computer or other device capable of performing logical operations.

[0010] Please refer to Figure 1. In step 12, a main iteration module of the computing device sets a plurality of iteration parameters. These iteration parameters serve as initialization parameters for the main iteration module of the computing device during subsequent iterations. In this embodiment, these iteration parameters include a denoised image matrix, an auxiliary variable of the first-order isotropic total variation of chroma-luminance, a Lagrange multiplier of the first-order isotropic total variation of chroma-luminance, an auxiliary variable of the second-order isotropic total variation of chroma-luminance, a Lagrange multiplier of the second-order isotropic total variation of chroma-luminance, image restoration model term parameters, SV-VBTV regularization term parameters, SV-SOVBTV regularization term parameters, brightness parameters, tangent bundle metrics, and penalty parameters of Lagrange multipliers.

[0011] Referring to FIG. 1 , in step 13 , the main iterative control module of the computing device performs iterative image restoration on the noisy image using an image restoration objective function. Before the iteration, if the original color space of the noisy image is not the HSV (Hue, Saturation, Value) color space, the computing device first converts the color space of the noisy image to the SV (Saturation, Value) color space before performing the iterative image restoration.

[0012] If the original color space of the noisy image is the RGB color space, the formula for converting the color space of the noisy image by the computing device can be expressed as: in, is the matrix after the color space of the noisy image is converted, , is the matrix of the original color space of the noisy image, , and are the number of columns and rows of the matrix, is the brightness parameter. 、 and are the R, G, and B color components of the noisy image in the original color space, and This is the part used to calculate chroma. It is the part used to calculate brightness.

[0013] The image restoration objective function of the main iterative control module of the computing device is: in, is an image restoration model parameter, , is a denoised image matrix, , is the Frobenius norm, is the matrix of the noisy image, is a Gaussian blur convolution kernel, , is the convolution kernel size, is a convolution operator, is the first-order isotropic total variation of chroma-brightness, is a SV-VBTV regularization parameter, is the total isotropic variation of chroma-brightness second order, is a SV-SOVBTV regularization term parameter. The main iterative control module of the computing device updates the matrix of the denoised image, the multiple parameters in the chroma-luminance first-order isotropic total variation, and the multiple parameters in the chroma-luminance second-order isotropic total variation according to the image restoration objective function. The initial value in step 12 is set to the matrix of the noisy image.

[0014] The image restoration objective function The error between the denoised image and the noisy image is calculated using the Frobenius norm. It is the weight used to control the calculated error. Minimizing min can make the denoised image close to the noisy image. The first-order vector bundle total variation is used to calculate the gradient of the image, suppress the influence of small gradients in the image and retain large gradients to protect the edge details of the image. However, the first-order total variation term can easily lead to excessive smoothing of the restored image. Therefore, this embodiment adds a The second-order regularization term is used to calculate the second-order gradient of the image to process high-frequency components and complex structures through the second-order total variation. It can effectively remove noise and smooth the image while preserving the edges and details of the image. and It is the weight that adjusts the influence of the first-order regularization term and the second-order regularization term.

[0015] Preferably, in this embodiment, the main iterative control module of the computing device uses a linearized alternating direction method to solve the image restoration objective function. After the main iterative control module of the computing device uses the linearized alternating direction method to expand the image restoration objective function, it can be organized as follows: in, is the augmented Lagrange function, is a vector bundle on the Riemannian manifold, 、 is the tangent bundle metric, , is the positive definite metric of the vector bundle, is the covariant derivative, m and n are the index values ​​of the number of columns and rows of the input image, is a coordinate system on the Riemannian manifold, is the first auxiliary variable of the matrix of the denoised image, is a second auxiliary variable of the matrix of the denoised image, 、 are auxiliary variables for the regularization of the first-order isotropic total variation of chroma-brightness in the horizontal and vertical directions, , , , 、 are auxiliary variables for the regularization of the second-order isotropic total variation of chroma-brightness in the horizontal and vertical directions, , , , 、 are the convolution calculations of the first-order discrete gradient matrix in the horizontal and vertical directions, [、] are the convolution calculations of the second-order discrete gradient matrix in the horizontal and vertical directions, 、 、 、 is the Lagrange multiplier, , 、 is the penalty parameter of the Lagrange multiplier, .

[0016] Auxiliary variable for the regularization term of the first-order isotropic total variation of chroma-brightness in the horizontal and vertical directions 、 , horizontal and vertical chroma-brightness second-order isotropic total variation regularization term auxiliary variable 、 and Lagrange multipliers 、 、 、 At the beginning of step 12, all values ​​are set to zero matrix.

[0017] Among them, the augmented Lagrangian function It is a first-order regularization term that performs spatial smoothing on the image based on the first-order total variation of the SV color space to suppress noise and retain the first-order structural characteristics of the image. It is a second-order regularization term, which improves the protection of high-frequency components of the image based on the second-order total variation of the SV color space, and is used to repair complex textures and structural features to solve the problem of insufficient detail recovery of the first-order regularization term. It is an auxiliary variable penalty term, which is used to ensure that the auxiliary variable can represent the first-order and second-order gradients of the image to improve the stability of the optimization process. It is the Lagrange multiplier penalty correction term, which is used to correct the Lagrange multiplier and reduce the numerical error of the auxiliary variables to improve the convergence of the model.

[0018] The main iterative control module updates the parameters of the chroma-luminance first-order isotropic total variation and the parameters of the chroma-luminance second-order isotropic total variation according to the image restoration objective function. The optimization method of the auxiliary variables and Lagrange multipliers of the chroma-luminance first-order isotropic total variation regularization term is as follows.

[0019] When optimizing the matrix of the denoised image [Q], the other parameters are fixed, and the image restoration objective function can be written as: Then, through two-dimensional discrete Fourier transform and inverse Fourier transform, the matrix of the denoised image can be obtained. The optimization formula of [Q]: in, is the two-dimensional discrete Fourier transform, is the Hadamard product operator, and To control the penalty parameter, The convolution calculation form rotates 180 degrees to the horizontal discrete gradient matrix, The convolution calculation form is a 180-degree rotation of the vertical discrete gradient matrix.

[0020] Auxiliary variable for the regularization term of the first-order isotropic total variation of chroma-brightness in the horizontal and vertical directions 、 The analytical solution can be generated through the soft thresholding algorithm. According to the soft thresholding algorithm, the optimization formula for the chroma part can be obtained as follows: in, As the maximum function, set It means taking non-negative value, the number of channels For 1 and 2, the optimization formula for the brightness part is; .

[0021] Auxiliary variable for the regularization term of the second-order isotropic total variation of chroma-brightness in the horizontal and vertical directions 、 The analytical solution can also be generated through the soft threshold algorithm. According to the soft threshold algorithm, the optimization formula for the chroma part can be obtained as follows: Number of channels For 1 and 2, the optimization formula for the brightness is: .

[0022] Lagrange multipliers 、 、 、 The optimization formula is: In step 13, the matrix of the denoised image is completed in sequence through the above optimization formula [Q] [、] Auxiliary variable of the first-order isotropic total variation regularization term of chroma-brightness in the horizontal and vertical directions 、 , horizontal and vertical chroma-brightness second-order isotropic total variation regularization term auxiliary variable 、 and Lagrange multipliers 、 、 、 This embodiment uses a soft threshold algorithm to regularize the auxiliary variable of the first-order isotropic total variation of chroma-brightness in the horizontal and vertical directions. 、 and auxiliary variables of the second-order isotropic total variation regularization term of chroma-brightness in the horizontal and vertical directions 、 Optimizing the chroma channel in the denoising process can make it smoother and reduce the influence of noise. The auxiliary variable of the regularization term of the first-order isotropic total variation of chroma-luminance in the horizontal and vertical directions is used through the soft threshold algorithm. 、 and auxiliary variables of the second-order isotropic total variation regularization term of chroma-brightness in the horizontal and vertical directions 、 The brightness channel in the optimization is optimized and the brightness parameter Fine-tuning the brightness channel can smooth out brightness changes while preserving image details. 、 、 、 The optimization can be performed to adjust the constraints of the main variables to the auxiliary variables in the framework of the linear alternating direction method.

[0023] Please refer to FIG. 1. In step 14, the main iterative control module of the computing device determines whether the iteration has converged based on the denoised image. In this embodiment, the main iterative control module of the computing device determines whether the iteration has converged based on the denoised image. Whether the calculation result is less than a threshold value determines whether the iteration has converged. is the matrix of the denoised image updated at the tth iteration, is the matrix of the denoised image updated at the t-1th iteration. Preferably, to avoid non-convergence leading to incomplete iterations, the main iteration control module of the computing device determines whether the iteration has converged based on the denoised image. The main iteration control module also determines whether the number of iterations has reached an upper limit. If so, the denoised image is output. If the iteration has not converged and the upper limit has not been reached, step 13 is performed to iterate image restoration on the noisy image.

[0024] Referring to FIG. 1 , in step 15 , the denoised image is output. If the original color space of the noisy image is an RGB color space, the main iterative control module of the computing device further converts the color space of the denoised image to an RGB color space after completing the image restoration iteration. The formula for converting the color space of the noisy image by the computing device can be expressed as: in, The matrix that converts the denoised image to the original RGB color space.

[0025] Please refer to Figures 2a to 2c, which respectively show the original pattern, the noisy image with added noise, and the denoised image obtained by image restoration using the image restoration method 10 of this embodiment. The added noise is a Gaussian blur with a standard deviation of 1.00 and a Gaussian noise variance of 0.502. Figure 2b shows that the noisy image with added noise has almost lost its image details and object edges. However, Figure 2c shows that the image restoration of this embodiment has eliminated the noise and preserved some of the image details and object edges. Please refer to Figures 3a to 3c, which respectively show another original pattern, the noisy image with added noise, and the denoised image obtained by image restoration using the image restoration method 10 of this embodiment. Similarly, the added noise is a Gaussian blur with a standard deviation of 1.00 and a Gaussian noise variance of 0.502. Figure 3c shows that the image restoration of this embodiment has indeed eliminated the noise in the other image and preserved the image details and object edges.

[0026] The main iterative control module of the computing device of the present invention combines the first-order and second-order isotropic total variation of chroma-luminance to perform image restoration on noisy images, which can effectively remove noise in the image and maintain the global consistency and detail characteristics of multiple channels.

[0027] The scope of protection of the present invention shall be determined by the scope of the attached patent application. Any changes and modifications made by anyone familiar with this technology without departing from the spirit and scope of the present invention shall fall within the scope of protection of the present invention.

[0028] 10: Image restoration method 11: Input the noisy image to the computing device 12: Set iteration parameters 13: Perform image restoration on the noisy image and update the denoised image and parameters 14: Determine whether it converges 15: Output denoised image

Claims

1. An image restoration method, comprising: inputting a noisy image into a computing device; a main iteration module of the computing device setting a plurality of iteration parameters; and the main iteration control module of the computing device iteratively restoring the noisy image using an image restoration objective function, wherein the image restoration objective function is: is an image restoration model term parameter, is a matrix of the denoised image, is a Frobenius norm, is the matrix of the noisy image, is a Gaussian blur convolution kernel, is a convolution operator, is a chroma-luminance first-order isotropic total variation, is an SV-VBTV regularization term parameter, is a chroma-luminance second-order isotropic total variation, is a SV-SOVBTV regularization term parameter, the main iterative control module of the computing device updates the matrix of the denoised image, multiple parameters in the chroma-luminance first-order isotropic total variation, and multiple parameters in the chroma-luminance second-order isotropic total variation according to the image restoration objective function; and the main iterative control module of the computing device determines whether the iteration has converged based on the denoised image, and if so, outputs the denoised image; if not, the main iterative control module iterates again using the image restoration objective function.

2. The image restoration method of claim 1, wherein if the original color space of the noisy image is not the HSV (Hue, Saturation, Value) color space, the main iterative control module further includes the computing device converting the color space of the noisy image before iterating the image restoration to convert the noisy image into the SV (Saturation, Value) color space.

3. The image restoration method of claim 2, wherein the original color space of the noisy image is an RGB color space, and the formula for converting the color space of the noisy image by the computing device can be expressed as: wherein, is the matrix of the original color space of the noisy image, and are the number of columns and rows of the matrix respectively, and is a brightness parameter.

4. The image restoration method of claim 3, wherein if the original color space of the noisy image is an RGB color space, the main iterative control module of the computing device further includes converting the color space of the denoised image by the computing device after completing the iteration of image restoration, so as to convert the denoised image into an RGB color space.

5. The image restoration method of claim 4, wherein the formula for converting the color space of the noisy image by the computing device can be expressed as: wherein, The matrix that converts the denoised image to the original RGB color space.

6. The image restoration method of claim 1, wherein the main iterative control module of the computing device optimizes the image restoration objective function using a linearized alternating direction method.

7. The image restoration method of claim 6, wherein the main iterative control module of the computing device uses a linearized alternating direction method to solve the image restoration objective function, which can be expressed as: wherein, is the augmented Lagrange function, is a vector bundle on the Riemannian manifold, , are the tangent bundle metrics, is the positive definite metric of the vector bundle, is the covariant derivative, m, n are the index values ​​of the number of columns and rows of the input image, is the coordinate system on the Riemannian manifold, is a first auxiliary variable of the matrix of the denoised image, is a second auxiliary variable of the matrix of the denoised image, and are auxiliary variables of the first-order isotropic total variation regularization term of chroma-luminance in the horizontal and vertical directions, respectively, , , , are auxiliary variables of the second-order isotropic total variation regularization term of chroma-luminance in the horizontal and vertical directions, respectively, , , , are the convolution calculations of the first-order discrete gradient matrices in the horizontal and vertical directions, respectively, , , , are the convolution calculations of the second-order discrete gradient matrices in the horizontal and vertical directions, respectively, , , , , are Lagrange multipliers, and , are penalty parameters of the Lagrange multipliers.

8. The image restoration method of claim 7, wherein the main iterative control module updates the parameters of the chroma-luminance first-order isotropic total variation and the parameters of the chroma-luminance second-order isotropic total variation according to the image restoration objective function as auxiliary variables of the chroma-luminance first-order isotropic total variation regularization term and Lagrange multipliers.

9. The image restoration method of claim 1, wherein the main iterative control module of the computing device determines whether the iteration has converged based on whether the calculation result is less than a threshold value, wherein is the matrix of the denoised image updated by the t-th iteration, and is the matrix of the denoised image updated by the t-1-th iteration.

10. The image restoration method of claim 1, wherein the main iteration control module of the computing device determines whether the iteration has converged based on the denoised image, and further determines whether the number of iterations has reached an upper limit of the number of iterations, and if so, outputs the denoised image.

Images