L0 regularization image smoothing method with adaptive weighting matrix and application thereof

Abstract

The application belongs to the field of image processing, and discloses an L0 regularization image smoothing method of an adaptive weighted matrix and application, specifically designs an adaptive weighted matrix, and applies the adaptive weighted matrix to an L0 gradient minimization method, gives different weights in different gradient directions to achieve the purpose of depicting local features and preventing over-smoothing, secondly, uses an ADMM algorithm to efficiently solve the proposed model, finally, inputs a to-be-processed image into the proposed minimization model, outputs a final smoothed image, and extends the result to other applications. The application solves the problem that common methods cannot process low-resolution and high-amplitude structures, and can also obtain a better smoothed image, and compared with other advanced methods, the model of the application has significant improvement in visual effect and numerical value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to a new L0 regularization image smoothing method based on an adaptive weighting matrix and application. BACKGROUND

[0002] Many computer vision and image analysis tasks involve image smoothing methods, which mainly aim to smooth image texture and remove noise while ensuring the edge intensity of the image. On this basis, image smoothing extends to many applications as a pre-processing for other applications, such as edge detection, HDR tone mapping, JPEG image compression artifact removal, image detail enhancement, etc. At the same time, image smoothing can also be used to improve the performance of higher-level visual application tasks, such as image segmentation, stereo sound matching, etc.

[0003] Common image smoothing methods are mainly divided into three categories: filter-based methods, model-based methods, and learning-based methods. Compared with filter-based methods, model-based methods can consider all pixels in the image, and their edge protection capabilities are more prominent. Among the model-based algorithms, L0 gradient minimization is one of the most classic image smoothing algorithms. This method approximates prominent structures in a sparse control manner, globally locates the edges of the image, sharpens the main amplitude edges, and thus achieves good smoothing results. However, for low-resolution, high-amplitude structures, over-sharpening results may occur, and some redundant structures are retained, lacking robustness to noise. Although many methods have been developed in recent years to improve the L0 gradient minimization method, they cannot effectively solve this problem. SUMMARY

[0004] In order to solve the problems in the prior art, the present application provides a new L0 regularization image smoothing method based on an adaptive weighting matrix and application, which uses a weighting matrix to characterize the gradient in different directions, better solving the problems of over-sharpening and texture residue existing in the prior art models, and extending to its applications.

[0005] In order to achieve the above purpose, the present application is implemented by the following technical solutions:

[0006] The present application is a new L0 regularization image smoothing method based on an adaptive weighting matrix and application, and the specific L0 regularization image smoothing method comprises the following steps:

[0007] Step 1: establish a new L0 regularization image smoothing model based on an adaptive weighting matrix as follows:

[0008]

[0009] wherein u and f represent the smoothed image and the input image respectively, T represents the adaptive weighting matrix, and ▽ = (▽ x,▽ y ) denotes the gradient operator, λ denotes a non-negative penalty parameter, the first term is a fitting term for image smoothing, the second term is a regularization term, and ||·||0 denotes the 0-norm.

[0010] Step 2: A reasonable weighted matrix is designed to give different weights to different gradient directions, and to depict the local structure of the smooth image. Specifically, by giving different weights to the gradients in the x-axis and y-axis directions of the image, the local details of the smooth image are depicted, and the adaptive weighted matrix is defined as:

[0011]

[0012] where κ and σ are non-negative parameters, denotes a Gaussian kernel function, * is a convolution operator, and the value of the weight t1(x,y) and t2(x,y) is obtained by convolving the Gaussian filter G σ (x,y) and the gradient information ▽ x u(x,y) of the image. y u(x,y) are convolved, when the two weights are the same constant, the gradient information is isotropic, and by selecting appropriate weights, the weights are coupled with the gradient information to perform anisotropic L0 regularization, effectively depicting the structure information of the image.

[0013] Step 3: The L0 regularization image smoothing model designed in step 1 is non-smooth, and the ADMM algorithm is used to solve the L0 regularization image smoothing model, which specifically includes the following steps:

[0014] Step 3-1: An auxiliary variable w=(w1,w2) T is introduced, v=(v1,v2) T is listed, and each sub-problem is solved;

[0015] Step 3-2: Initialize w 0 , v 0 , u 0 , α 0 , β 0 , and set the maximum number of iterations i max ;

[0016] Step 3-3: Use the hard threshold formula to solve w k+1 ;

[0017] Step 3-4: Directly solve v1 and v2;

[0018] Step 3-5: Use the fast Fourier transform to solve u k+1 ;

[0019] Step 3-6: Update the multipliers α k+1 , βk+1 ;

[0020] Steps 3-7: Stop iterating after the iteration termination condition is met.

[0021] A further improvement of the present invention is that, in step 3-2, the initialization method specifically includes the following steps:

[0022] Step 3-2-1: For the Lagrange multiplier α 0 β 0 Here, the initial value is set to a zero matrix of the same size as the input image;

[0023] Step 3-2-2: For variable w 0 v 0 u 0 , there is w 0 =▽u 0 v 0 =▽u 0 u 0 =f.

[0024] The specific steps for solving the model in steps 3-4 are as follows:

[0025] Step 4-1: Optimize w1 and w2:

[0026]

[0027] Step 4-2: Optimize v1 and v2:

[0028]

[0029] Step 4-3: Optimize u:

[0030]

[0031] Step 4-4: Update multipliers α and β:

[0032]

[0033] The beneficial effects of this invention are: This invention relates to an adaptive weighting matrix and applies it to the L0 gradient minimization method. By assigning different weights to different gradient directions, it achieves the purpose of characterizing local features and preventing over-smoothing.

[0034] The novel model proposed in this invention solves the problem that common methods cannot handle low-resolution, high-amplitude structures, and can also obtain better smoothed images. Compared with other advanced methods, the model of this invention has significant improvements in both visual effects and numerical values. Attached Figure Description

[0035] Figure 1For texture images that require smoothing.

[0036] Figure 2 The diagram shows a comparison of four models and their magnified local details, where (a) is the L0 gradient minimization method, (b) is the result of the SSTF model, (c) is the result of the GFES model, and (d) is the result of the model proposed in this invention. The boxed areas are magnified local details.

[0037] Figure 3 The texture images used in the quantitative evaluation are: (a) the real image "Ball", (b) the added textures, which are four different textures from left to right: "35a8c", "4bada", "brickrg" and "weave", and (c) the obtained texture image.

[0038] Figure 4 The artifact image for JPEG image compression.

[0039] Figure 5 Comparison of artifact removal results for JPEG image compression and magnified local images: (a) L0 gradient minimization method, (b) SSTF model result, (c) GFES model result, and (d) model result proposed in this invention. The boxed areas represent magnified local details. Detailed Implementation

[0040] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0041] This invention relates to an adaptive weighted matrix L0 regularized image smoothing method and its application. The method specifically includes...

[0042] 1. L0 Regularization Model of Adaptive Weighted Matrix

[0043] Currently, L0 gradient minimization methods are widely used for image smoothing due to their sparsity, effectively removing image textures. However, these methods often struggle with low-resolution, high-amplitude structures. Anisotropic gradient minimization models, by assigning different weights to the gradients along the x and y axes, can effectively characterize local image details, making them more suitable for processing such images. The following presents an L0 regularization model based on an adaptive weighting matrix:

[0044]

[0045] The adaptive weighting matrix T is defined as follows:

[0046]

[0047] Where κ and σ are non-negative parameters, represents the Gaussian kernel function, and * represents the convolution operator. The Gaussian filter G is then applied. σ (x,y) and gradient information of the image ▽ x u(x,y) and ▽ y Convolution of u(x,y) yields the values ​​of weights t1(x,y) and t2(x,y). When both weights are constants, the gradient information is isotropic. By selecting appropriate weights and coupling them with the gradient information, anisotropic L0 regularization can be applied, effectively characterizing the structural information of the image.

[0048] The L0 norm can be understood as the number of non-zero elements in a matrix, and its specific definition is:

[0049]

[0050] The L0 minimization problem is an NP-hard optimization problem, which increases the complexity of solving the model to some extent.

[0051] For an image u∈X, the discrete gradient operator ▽ is defined as (▽u) j,k =((▽) x u) j,k ,(▽ y u) j,k ), j,k=1,...,N, where:

[0052]

[0053] 2. ADMM Algorithm

[0054] Since model (1) is a non-smooth model, the problem becomes more complex. This invention uses the ADMM (alternating direction method of multipliers) algorithm to solve it. This algorithm introduces auxiliary variables to transform the objective function into an equivalent form and decompose it into several sub-problems. At the same time, it solves each sub-problem. It is an efficient convex optimization algorithm. Its main steps are as follows:

[0055] (1) Introduce auxiliary variable w = (w1, w2) T v = (v1, v2) T :

[0056]

[0057] (2) List the Lagrangian functions of model (1):

[0058]

[0059] (3) For the w subproblem, it can be transformed into the following form:

[0060] The above problem can be solved directly using hard thresholding:

[0061]

[0062] in,

[0063] (4) For the subproblem v, which is smooth, it can be solved directly. After simple calculation, the solution to the subproblem v can be expressed as:

[0064]

[0065] (5) The u subproblem is a quadratic function, since ▽ T ▽ indicates a block loop, which can be solved quickly using the Fast Fourier Transform after diagonalizing the derivative operator:

[0066]

[0067] in, This represents the Fast Fourier Operator.

[0068] (6) Update the Lagrange multipliers in the algorithm:

[0069]

[0070] (7) After reaching the maximum number of iterations, the iteration terminates and the final smooth image is output.

[0071] Through the above process, the optimal solution of model (1) can be obtained. The specific algorithm details are shown in Algorithm 1.

[0072]

[0073]

[0074] 3. Extended Applications of Image Smoothing Algorithms

[0075] Image smoothing, as one of the fundamental tasks in computer vision, has a wide range of applications. JPEG image compression artifact removal is one such application. During image compression, traditional JPEG compression methods are easily limited by storage space and produce unsatisfactory artifacts. By inputting the compressed JPEG artifact image into the proposed model, we can obtain a compressed image with artifacts removed, achieving the goal of reducing storage space without compromising image quality.

[0076] 4. Numerical Experiments

[0077] To verify the effectiveness of the novel adaptive weighted matrix L0 regularization method of this invention and to make a more objective comparison, this invention conducts both qualitative and quantitative comparisons. Figure 1 A texture image with a resolution of 322×400×3 is displayed.

[0078] The parameters of the smoothing algorithm of this invention are selected as λ = 0.03, κ = 2, σ = 1, k = 40, γ1 = 0.06, and γ2 = 0.06. Figure 2 In the middle (a), the result is obtained by the traditional L0 gradient minimization method. Figure 2 In the middle (d), the result is obtained from the improved adaptive weighting matrix L0 regularization. Figure 2 (b) and (c) are the results obtained from the SSTF and GFES models, respectively.

[0079] according to Figure 2 As can be seen from the magnified area, compared with the L0 gradient minimization method, the model of this invention effectively removes texture and obtains block-by-block smooth regions, overcoming the problem that such models cannot handle low-resolution, high-amplitude structures, while avoiding over-sharpening results. In contrast, advanced models such as SSTF and GFES have the problem of over-smoothing, which cannot protect image edges and details.

[0080] For quantitative evaluation indicators, this invention selected the comparison method provided by the G-norm model and compared it with it, such as... Figure 3 As shown, this invention utilizes a 256×256×3 resolution "Ball" image, applies different textures to it, and evaluates it using Peak Signal-to-Noise Ratio (PSNR), which is specifically defined as follows:

[0081]

[0082] Where u represents the smoothed image and u0 represents the original image.

[0083] As can be seen from Table 1, regardless of the texture type, the PSNR value using the adaptive weighted L0 regularization proposed in this invention is the highest, indicating that the model can obtain better smoothing results.

[0084] Table 1 PSNR values ​​for the three models

[0085]

[0086] Next, the proposed model will be applied to the removal of artifacts from JPEG compression: the JPEG compression artifact image will be input into the model of the present invention, and in particular, the parameter λ = 0.01 will be set here.

[0087] pass Figure 5 As can be seen from the magnified images, the traditional L0 model and GFES cannot completely remove artifacts from the image, and the image quality still needs improvement. While the SSTF model does not produce artifacts, its results are overly smoothed and do not preserve the image edges and details. In contrast, the model proposed in this invention can not only remove artifacts but also clearly delineate the image edges.

[0088] In summary, this invention addresses the problem that traditional L0 models cannot handle low-resolution, high-amplitude structures. It proposes a novel L0 regularization method based on an adaptive weighting matrix, which preserves image edges to the greatest extent possible and obtains block-by-block smooth regions. Furthermore, this invention utilizes the ADMM algorithm to efficiently solve the proposed model and applies it to JPEG compression artifact removal. Numerical experiments demonstrate the effectiveness of this invention from both subjective and objective perspectives.

[0089] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. An adaptive weighted matrix L0 regularization image smoothing method, used for artifact removal in JPEG image compression, characterized in that: The L0 regularized image smoothing method includes the following steps: Step 1: Establish an L0 regularized image smoothing model based on an adaptive weighting matrix; Step 2: Design a weighting matrix to assign different weights to different gradient directions to characterize the local structure of a smooth image; Step 3: The L0 regularized image smoothing model designed in Step 1 is non-smooth. The ADMM algorithm is used to solve the L0 regularized image smoothing model, where... In step 1, the L0 regularized image smoothing model based on the adaptive weighting matrix is ​​as follows: ， in, and These represent the smoothed image and the input image, respectively. Represents an adaptive weighted matrix. Represents the gradient operator, The first term represents the non-negative penalty parameter, and the second term is the image smoothing fitting term, while the third term is the regularization term. Represents the 0-norm; In step 2, by giving the image shaft and The gradient along the axis is assigned different weights to characterize the local details of the smooth image. The adaptive weighting matrix is ​​defined as follows: ， in, and For non-negative parameters, Represents the Gaussian kernel function. For the convolution operator, pass through a Gaussian filter Gradient information of the image and Perform convolution to obtain weights and When the two weights are the same constant, the gradient information is isotropic. By selecting appropriate weights, the weights are coupled with the gradient information to perform anisotropic L0 regularization, effectively characterizing the structural information of the image.

2. The L0 regularization image smoothing method with adaptive weighting matrix according to claim 1, characterized in that: Step 3 uses the ADMM algorithm to solve the L0 regularized image smoothing model, specifically including the following steps: Step 3-1: Introduce auxiliary variables , List its Lagrangian function and solve each subproblem; Step 3-2: Initialization , , , , Set the maximum number of iterations. ; Step 3-3: Use the hard threshold formula to calculate ; Steps 3-4: Solve directly , ; Steps 3-5: Solve using Fast Fourier Transform ; Steps 3-6: Update the multipliers , ; Steps 3-7: Stop iterating after the iteration termination condition is met.

3. The L0 regularization image smoothing method with adaptive weighting matrix according to claim 2, characterized in that: In step 3-2, the initialization method specifically includes the following steps: Step 3-2-1: For Lagrange multipliers , Here, the initial value is set to a zero matrix of the same size as the input image; Step 3-2-2: For variables , , ,have , , .

4. The L0 regularization image smoothing method with adaptive weighting matrix according to claim 2, characterized in that: The specific steps for solving the model in steps 3-4 are as follows: Step 4-1: Optimization , : ， Step 4-2: Optimization , : ， Step 4-3: Optimization : ， Step 4-4: Update the multipliers , : 。

Images