A global sparse texture filtering method based on edge structure preservation

Abstract

The application provides a global sparse texture filtering method based on edge structure preservation, including introducing a texture inhibition function in a penalty term, and constraining the gradient of an output image, the texture inhibition function inhibits texture, noise and unnecessary detail information in the image by setting two threshold values, then using the inhibited gradient as the input of the denominator of the penalty term, so that the penalty term can sufficiently distinguish texture and structure; sparse regular L1 norm is used to constrain the penalty term, non-convex optimization is converted into a convex optimization problem by introducing a sub-gradient, and an alternating direction multiplier method is used for iterative solution, so that better edge preservation is achieved; sparse L p Norm is used to constrain the penalty term and a preconditioned conjugate gradient method is used to accelerate and improve the calculation efficiency, so that more robust and sparse image smoothing effect is achieved. The application can improve the robustness of the algorithm in distinguishing texture and structure, retain better semantic information, and achieve better edge structure preservation and smoothing performance.

Description

Technical Field

[0001] This invention relates to the field of edge-preserving image smoothing technology, and more specifically to a global sparse texture filtering method based on edge structure preservation. Background Technology

[0002] Edge-preserving image smoothing has been a research hotspot in image processing tasks, primarily used to filter out textures and noise while preserving important structures and edges. Various algorithms have been proposed over the past few decades, and edge-preserving image smoothing has wide applications in many computer vision tasks, including detail enhancement, edge extraction, HDR mapping, image denoising, cropping artifact removal, and image / anime stylization.

[0003] Existing edge-preserving image smoothing methods are mainly divided into three categories: smoothing algorithms based on local filters, smoothing algorithms based on global optimization, and smoothing algorithms based on deep learning. Among them, smoothing algorithms based on global optimization typically establish a global optimization framework, which consists of a data term and a penalty term. The data term ensures that the smoothed output image S is as close as possible to the input image I, using the L2 norm to constrain the deviation between the output and input. The penalty term uses various norms to constrain the gradient of the output image, thereby achieving the edge-preserving smoothing effect. The general framework of the global optimization-based algorithm is as follows:

[0004]

[0005] Where λ is a smoothing parameter used to control the smoothing intensity of the image; p is a two-dimensional pixel of the indexed image; δ(S p This is a gradient penalty applied to the output image. In other words, it produces a large response for areas with significant texture and noise, while imposing the least possible constraint on structural edge regions.

[0006] Among existing methods, the Total Variation (TV) model is the earliest texture structure decomposition algorithm. Subsequent proposed methods such as Weighted Least Squares (WLS), L0 gradient minimization, and Relative Total Variation (RTV) models, while effective in removing large-scale noise or texture, smooth out small structural edges and weak structures, making it difficult to achieve efficient smoothing while preserving edges. The Relative Total Variation (RTV) model uses the L2 norm to constrain the structural similarity between the input and output images, applying the L2 norm to the gradient of the output image to achieve image smoothing. Although the Relative Total Variation (RTV) model boasts high-quality smoothing performance, its ability to preserve weak structural information is not favorable, and it may even cause the loss of small structures. On the one hand, the penalty term using the L2 norm constraint finds an approximation of the true value by minimizing it; its sparsity is weak, and it may not be optimal at the critical point distinguishing texture and structure, potentially resulting in the coexistence of texture and structure. Achieving high-quality smoothing comes at the cost of sacrificing small structures and weak edges. On the other hand, the penalty term of the Relative Total Variation (RTV) model consists of two terms. Since texture and structure have different characteristics within the window, texture and structure are distinguished by the ratio of the sum of the absolute values ​​of gradients within the window to the absolute value of the sum of gradients within the window. This works well for smoothing ordinary texture images, but it can cause undersmoothing for textures of different scales. This is because there are some large-scale textures that are difficult to filter out and are comparable to significant structures. Smoothing large-scale textures would lead to oversmoothing, i.e., loss of weak structures and structural edges. In summary, the L2 norm-based Relative Total Variation (RTV) model has weak sparsity and its edge-preserving ability needs improvement, lacking robustness. Summary of the Invention

[0007] To address the technical issues of existing L2-norm-based relative total variational models, which suffer from oversmoothing when large-scale textures with significant structures are present, resulting in the loss of weak structures and structural edges, weak sparsity, and insufficient edge preservation capabilities, this invention provides a global sparse texture filtering method based on edge structure preservation. This method fully considers the case of mixed structures and large-scale textures to improve the algorithm's robustness in distinguishing between textures and structures, while preserving better semantic information and achieving better smoothing performance in terms of edge structure preservation.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A global sparse texture filtering method based on edge structure preservation, the method comprising the following steps:

[0010] S1. A texture suppression function is introduced into the penalty term of the global optimization framework to constrain the gradient of the output image. The texture suppression function suppresses texture, noise, and unnecessary details in the image by setting two thresholds, σ1 and σ2. σ1 is used to suppress smaller-scale textures, noise, and unnecessary details, while σ2 is used to suppress larger-scale textures comparable to significant structures, thereby achieving a better texture image smoothing effect. The formula for the texture suppression function is shown below:

[0011]

[0012] Where Δ is the gradient of the output image, σ1 and σ2 are the thresholds, and exp(·) is the exponential function;

[0013] Then, the gradient after suppression by the texture suppression function is used as the input of the denominator of the penalty term, so that the penalty term can fully distinguish between texture and structure, and apply different degrees of penalty to the two, thereby better preserving the semantic information of the image.

[0014] S2. The L1 norm, which has strong sparsity, is used to constrain the penalty term. Using the L1 norm can achieve a more sparse effect. Although the L1 norm is not differentiable for sparse L1, the non-convex optimization problem is transformed into a convex optimization problem by introducing the subgradient and iteratively solving it using the alternating direction multiplier method. This can achieve a better edge-preserving image smoothing algorithm and avoid the fact that the L2 norm has weak sparsity and cannot better preserve the semantic information of the image.

[0015] S3, using L with strong sparsity p The norm is used to constrain the penalty term, where 0 < p < 1, for L p To effectively solve the norm problem, an iterative reweighted L1 norm approach is adopted, and a preprocessing conjugate gradient method is used to accelerate and improve computational efficiency, thereby achieving a more robust and sparse image smoothing effect.

[0016] Furthermore, step S2 includes the following steps:

[0017] S21. The objective function of the edge-preserving image smoothing algorithm is defined as follows:

[0018]

[0019] Where S is the output image; I is the input image; λ is the balancing parameter, used to adjust the relationship between the data terms and the penalty terms, which can directly affect the smoothing performance; ξ is a very small positive number to prevent it from being divisible by zero. Represents the L2 norm; p indicates the index position; x and y represent the x and y directions of the image; D * It is the sum of the absolute values ​​of the weighted gradients within the window; M *It is the absolute value of the weighted gradient sum within the window; D * and M * The derivation formula is as follows:

[0020]

[0021] Where *∈{x,y} represents the x-direction and y-direction; Ω(p) is the neighborhood space centered at p with a window size of r×r, where r is represented as 2×k+1; g k (||pq||) is the Gaussian kernel function; k is the Gaussian kernel size; h is the texture suppression function at different scales; This indicates the process of finding the differential;

[0022] S22. Transform the objective function formula (2) into an L1 norm constraint penalty term:

[0023] First, solve for the x-direction; the y-direction is handled in the same way as the x-direction. Expand the penalty term in formula (2) as follows:

[0024]

[0025] Here, the penalty term in the x-direction is decomposed into a linear term. and a non-linear weighting component The processing in the y-direction is the same. It is expressed as follows:

[0026]

[0027] Finally, the objective function is rewritten and matrix-reduced as follows:

[0028]

[0029] Among them, C x C y It is a backward difference matrix, and Both are diagonal matrices of size N×N, where N represents the total number of pixels in the image;

[0030] S23. Solve the objective function using the alternating direction multiplier method:

[0031] To effectively solve equation (7) above, let ψ x =U x C x S, ψ y =U y C y S, by constructing the augmented Lagrangian function, yields the following objective function:

[0032]

[0033] To solve the final objective function (8), the objective function only contains the L1 norm and L2 norm and has no constraints, so we can directly perform differentiation to minimize it.

[0034] Furthermore, step S3 includes the following steps:

[0035] S31. The penalty term of the objective function of the edge-preserving image smoothing algorithm is adopted using L. p Norm constraints are defined as follows:

[0036]

[0037] S32, Using an iterative reweighting method to adjust L p The norm problem is transformed into solving the L1 norm problem. The weights in each round are dynamically weighted using the values ​​from the previous iteration, and the L1 norm is derived based on the L1 norm. p The method for finding the norm involves iteratively minimizing a non-convex function to transform it into a convex function, with weights W. x pass To adjust, here is the intermediate value in the x-direction for the k-th iteration:

[0038]

[0039] For weight terms After each iteration, the weights are updated in the next iteration. Here, ξ is a very small positive number to prevent divisibility by zero, and the same applies to the y-direction; rewriting equation (10) and matrixing it yields:

[0040]

[0041] in, and All are the second term weights. The solution of equation (11) is similar to the solution of equation (7) above, and will not be repeated here.

[0042] Compared with existing technologies, the global sparse texture filtering method based on edge structure preservation provided by this invention constrains the gradient of the output image by introducing a texture suppression function. First, two appropriate gradient thresholds are set to suppress large-scale textures comparable to salient structures in the image, as well as unnecessary details, noise, and small-scale textures. Then, the weights in the penalty term are processed using the gradient suppressed by the texture suppression function to ensure efficient smoothing without overfitting or underfitting. Finally, the L1 norm, which has strong sparsity, is used to constrain the penalty term, so that the proposed method model is as close as possible to the true value when minimized, improving the robustness of the algorithm. Therefore, the sparse edge-preserving image smoothing algorithm with texture suppression function proposed in this invention achieves good smoothing performance while preserving structural edges, and also has the following advantages:

[0043] 1. This invention proposes a novel global optimization model framework that can achieve high-quality smooth results while effectively preserving edges, and adopts an L1 norm constraint penalty term, giving the model a certain degree of sparsity.

[0044] 2. This invention fully considers the problem of texture image smoothing, introduces a texture suppression function, suppresses multi-scale textures that are difficult to remove, and achieves a better smoothing effect.

[0045] 3. This invention uses L p The norm constraint penalty term achieves a more robust and sparse image smoothing effect, and it employs the idea of ​​iterative reweighting to adjust L... p The norm problem is transformed into solving the L1 norm problem. Compared to the penalty term for L1 norm constraints, L... p The model with norm-constrained penalty terms exhibits better sparsity. Attached Figure Description

[0046] Figure 1 This is a demonstration of the results of extracting structures from different texture images provided by the present invention.

[0047] Figure 2 This is a texture suppression function provided by the present invention.

[0048] Figure 3 This is a comparison chart of the smoothing effect of the present invention with other penalty items.

[0049] Figure 4 The Global Optimization Tactics (GSTF) algorithm provided by this invention is based on L1 and L2. p (p=0.4) and L p (p=0.8) Comparison of smoothing results after regularization.

[0050] Figure 5 This is a diagram showing the effect of adjusting different parameters k on the smoothing results of texture images according to the present invention.

[0051] Figure 6 This is a diagram showing the effect of different parameters of the present invention on the smoothing results of natural images.

[0052] Figure 7 This is a diagram comparing the present invention with RTV by setting different λ and ρ (adjusting the smoothing intensity).

[0053] Figure 8 This is a comparison chart of the smoothing results of the present invention and texture filters on large-scale textured images.

[0054] Figure 9 This is a comparison chart of the smoothing results of the present invention and texture filters on complex texture images.

[0055] Figure 10 This is a comparison chart of the smoothing results of texture images at different scales using the present invention and texture filters.

[0056] Figure 11 This is a comparison chart of the visual effects of the present invention and various filters in edge extraction.

[0057] Figure 12 This is a comparison chart of the visual effects of the present invention and various filters in HDR tone mapping.

[0058] Figure 13 This is a comparison chart of the visual effects of the present invention and various filters in removing clipart compression artifacts.

[0059] Figure 14 This is a comparison chart of the image abstraction visual effects of the present invention and various filters. Detailed Implementation

[0060] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0061] Please refer to Figure 1 The image shown illustrates the edge-preserving smoothing effect of a texture image using a global sparse texture filtering method based on edge structure preservation provided by this invention. The method of this invention includes the following steps:

[0062] S1. A texture suppression function is introduced into the penalty term of the global optimization framework to constrain the gradient of the output image. The texture suppression function suppresses texture, noise, and unnecessary details in the image by setting two thresholds, σ1 and σ2. σ1 is used to suppress smaller-scale textures, noise, and unnecessary details, while σ2 is used to suppress larger-scale textures comparable to significant structures, thereby achieving a better texture image smoothing effect. The formula for the texture suppression function is shown below:

[0063]

[0064] Where Δ is the gradient of the output image, σ1 and σ2 are the thresholds, and exp(·) is the exponential function;

[0065] Here, Δ represents the gradient of the output image, and its effect on texture gradient suppression is as follows: Figure 2As shown, when processing textures at different scales, the edges are preserved as much as possible while compressing textures between small and large structures. Unnecessary details, noise, and small-scale textures are filtered out, thus achieving better preservation of weak and small-scale structure edges. Here, the texture suppression function is used to suppress large-scale textures that the RTV model cannot filter out. This is based on the characteristics of the RTV model: the RTV model has a large response in textured regions and a small response in structural regions (i.e., different weights are given to texture and structure). By processing the image gradient in the denominator, large-scale texture gradients are suppressed, or small-scale textures, noise, and unnecessary detail gradients are set to 0, resulting in a larger response to these regions and thus better smoothing performance for textures at different scales.

[0066] Then, the gradient after suppression by the texture suppression function is used as the input of the denominator of the penalty term, so that the penalty term can fully distinguish between texture and structure, and apply different degrees of penalty to the two, thereby better preserving the semantic information of the image.

[0067] S2. Subsequently, since the L2 norm has weak sparsity and cannot preserve the semantic information of the image well, this invention uses the L1 norm, which has stronger sparsity, to constrain the penalty term. Using the L1 norm can achieve a more sparse effect. Although the sparse L1 norm is not differentiable, by introducing the subgradient to transform the non-convex optimization problem into a convex optimization problem, and using the Alternating Direction Multiplier Method (ADMM) for iterative solution, a better edge-preserving image smoothing algorithm can be achieved, avoiding the problem that the L2 norm has weak sparsity and cannot preserve the semantic information of the image well.

[0068] As a specific embodiment, step S2 includes the following steps:

[0069] S21. The objective function of the edge-preserving image smoothing algorithm is defined as follows:

[0070]

[0071] Where S is the output image; I is the input image; λ is the balancing parameter, used to adjust the relationship between the data terms and the penalty terms, which can directly affect the smoothing performance; ξ is a very small positive number to prevent it from being divisible by zero. Represents the L2 norm; p indicates the index position; x and y represent the x and y directions of the image; D * It is the sum of the absolute values ​​of the weighted gradients within the window; M * It is the absolute value of the weighted gradient sum within the window; D * and M * The derivation formula is as follows:

[0072]

[0073] Where *∈{x,y} represents the x-direction and y-direction; Ω(p) is the neighborhood space centered at p with a window size of r×r, where r is represented as 2×k+1; g k (||pq||) is the Gaussian kernel function; k is the Gaussian kernel size; h is the texture suppression function at different scales; This indicates the calculation of the differential.

[0074] Regarding the characteristics of texture and structure, the D of both... * The sizes are quite similar. Since the gradient of the structure within the window is almost unidirectional and the gradient of the texture within the window diverges outwards, the M of the structure region... * M is much larger than the texture area * Based on this characteristic, the present invention combines the two to maximize the application of different levels of punishment to texture and structure, namely D. * As a molecule, M * As the denominator, we have formula (2).

[0075] like Figure 3 As shown, the proposed GSTF1 is compared with L1-RTV and RTV methods in two types of graphs. Figure 3 The first line clearly shows that L1-RTV retains details well, but it also reveals significant shortcomings in handling small-scale textures, such as... Figure 3 As shown in (c); GSTF1 achieves a good balance in both preserving weak and small structure edges and removing small textures. It not only filters out textures and noise but also identifies weak and small structure edges with minimal disruption. Figure 3 As shown in (d); RTV, while effectively smoothing, disrupts weak structures, such as... Figure 3 As shown in (b). In Figure 3 The second line, as Figure 3 As shown in (e), there is a significant amount of noise mixed in with the structure. Even when the RTV method effectively smooths the edges, the structure is not very clear, and artifacts may even exist, such as... Figure 3 As shown in (f); although the RTV model using L1 norm constraints solves the problem of edge presence, artifacts are not eliminated, such as Figure 3 As shown in (g); in contrast, the proposed GSTF1 solves the existing edge and artifact problems, as shown in (g). Figure 3 As shown in (h). In general, the GSTF1 method is useful in preserving small structures and has a good effect on suppressing unwanted details, textures, noise and artifacts.

[0076] To ensure the sparsity of the obtained solution, L1 norm constraints are used for the regularization term, and the Alternating Direction Multiplier Method (ADMM) is employed for iterative solution of the proposed algorithm. The global optimization model based on GSTF1 proposed in this invention consists of a quadratic term and a linear term, and by minimizing the objective function, it is ultimately transformed into solving a series of linear equations.

[0077] S22. Transform the objective function formula (2) into an L1 norm constraint penalty term:

[0078] This invention first solves for the x-direction, and the y-direction is processed in the same way as the x-direction; here the penalty term of formula (2) is expanded as follows:

[0079]

[0080] Here, the penalty term in the x-direction can be decomposed into a linear term. and a non-linear weighting component The processing in the y-direction is the same. It is expressed as follows:

[0081]

[0082] Finally, the objective function is rewritten as follows:

[0083]

[0084] Here, S and I are both N×1 vectors, where N represents the total number of pixels in the image. The above formula can then be rewritten in matrix form as follows:

[0085]

[0086] Among them, C x C y It is a backward difference matrix, and They are all diagonal matrices of size N×N.

[0087] S23. Solve the objective function using the alternating direction multiplier method:

[0088] To effectively solve equation (7) above, let ψ x =U x C x S, ψ y =U y C y S, by constructing the augmented Lagrangian function, yields the following objective function:

[0089]

[0090] To solve the final objective function (8), the objective function only contains the L1 norm and L2 norm and has no constraints, so it can be directly minimized by differentiation. Specifically, here we respectively define S and ψ x ψ y η x η y The solution process is as follows:

[0091] 1) First, fix ψ x ψ y η x η y S can be solved:

[0092]

[0093] The above formula is solved by iterative minimization, and the iterative formula is as follows:

[0094]

[0095] Where E is the identity matrix, here Correspondingly L here t It is a five-point non-homogeneous sparse Laplacian matrix. The linear equations can be solved using the fast preprocessing conjugate gradient method (PCG), and the output image S is finally obtained. t+1 .

[0096] 2) Secondly, fix S and η. x η y Solve for ψ respectively x ψ y :

[0097]

[0098] The shrink operation here can be written as shrink(θ,τ)=sign(θ)max(|θ|-τ,0), where θ represents... τ is represented as sign(·) and max(·) are the sign function and the maximum value function, respectively.

[0099] 3) Finally, fix S and ψ. x ψ y Solve for η respectively x η y :

[0100]

[0101] Finally, based on the above formula derivation, the image smoothing algorithm proposed in this invention can be described as follows:

[0102] Input: Input image I, balancing parameters λ and ρ, convergence error ε

[0103] Output: Filtered image S

[0104] 1: Initialization: S {0} =I;

[0105] 2: while do

[0106] 3: Compress the gradient of the input image I;

[0107] 4: Calculate using formula (6)

[0108] 5: Calculate S using formula (12) {t+1} ;

[0109] 6: Calculate using formula (13)

[0110] 7: Calculate using formula (14)

[0111] 8: The calculation formula (15) yields the result.

[0112] 9: The result is obtained from formula (16).

[0113] 10: end while

[0114] 11: return S {t}

[0115] S3, using L with strong sparsity p The norm is used to constrain the penalty term, where 0 < p < 1, for L p To effectively solve the norm problem, an iterative reweighted L1 norm approach is adopted, and a preprocessing conjugate gradient method is used to accelerate and improve computational efficiency, thereby achieving a more robust and sparse image smoothing effect.

[0116] In a specific embodiment, step S3 includes the following steps:

[0117] S31. The penalty term of the objective function of the edge-preserving image smoothing algorithm is adopted using L. p Norm constraints are defined as follows:

[0118]

[0119] S32, Using an iterative reweighting method to adjust L p The problem of finding the norm is transformed into solving the L1 norm problem.p The solution of the norm is a non-convex optimization problem. Therefore, solving this objective function should be transformed into a simpler problem that can be solved currently. The present invention will adopt an iterative reweighting method. The weight of each round is dynamically weighted by the values in the previous round of iteration, and based on the L1 norm, the L p norm solution method is derived. The non-convex function is transformed into a convex function by iterative minimization, and the weight W x is adjusted by . The following is the intermediate value in the x direction at the k-th iteration:

[0120]

[0121] For the weight term After each iteration ends, the weight is updated in the next round of iteration Here, ξ is a very small positive number to prevent division by zero. The same treatment is applied to the y direction; rewrite Equation (10) and matrixize it as:

[0122]

[0123] where and are both the weights of the second term, and they are also diagonal matrices of size N×N. The solution of Equation (11) is similar to the solution process of Equation (7) above, which will not be elaborated here. Minimizing the objective function is equivalent to iteratively solving a series of linear equations. When iterating, pay attention to calculating the weight W. To accelerate the solution of the linear equations, the preconditioned conjugate gradient method (PCG) is used to optimize the objective function here. The algorithm of the present invention is easy to solve, and finally a good operation efficiency is obtained.

[0124] Figure 4 Shows the smoothing results of the algorithm of the present invention based on L1 and L p (p = 0.4) and L p (p = 0.8) regularization. It can be clearly seen from the figure that the image smoothed by the GSTF model with L1 regularization is more flattened, while the GSTF model with L p (p = 0.8) constraint does not cause the destruction of small structure edges and weak structures while effectively smoothing, which is better than the GSTF models with L1 and L p (p = 0.4) norm constraints. Therefore, in order to achieve a more accurate edge preservation effect, for the GSTF model with L p (0 < p < 1) constraint, the present invention will adopt p = 0.8 for subsequent experimental comparison.

[0125] The method proposed in this invention mainly adjusts two parameters: the Gaussian kernel size k and the Lagrange penalty parameter ρ. A larger Gaussian kernel size k results in a smoother image, but the smoothed image retains almost only the main structure, causing smaller, more important structures to be smoothed out. In practice, the Gaussian kernel size k is generally controlled between [1, 3]. A larger Gaussian kernel size is used for textured images, while the opposite is true for non-textured images. The Lagrange penalty parameter ρ is a crucial parameter for controlling the smoothing intensity in the proposed method. A larger ρ results in a stronger smoothing effect, while a smaller ρ results in more noticeable detail retention. Generally, ρ is set to [3, 4]. 2 ,15e 2 ]among.

[0126] In addition, the method of the present invention also requires adjusting the compression thresholds σ1 and σ2 of the output image gradient S. Here, σ1 and σ2 play a key role in suppressing large-scale textures and effectively suppressing some noise, textures, unnecessary details, etc., which can directly affect the final smoothing effect. Usually, σ1 and σ2 are set to multiples of k, where σ1 is set to 0.5 times k and σ2 is set to 2-5 times k. Figure 5 By adjusting the Gaussian kernel size k, the smoothing effect of different Gaussian kernel sizes k on texture images was demonstrated. Here, the convolution kernel sizes were set to k=1.5, k=2.5, k=3, and k=3.5. It can be seen that when other parameters are fixed, a larger k results in a better smoothing effect, while a smaller k preserves more details and texture.

[0127] Figure 6 This demonstrates the effect of adjusting the Gaussian kernel size *k* and the Lagrange penalty parameter *ρ* on the smoothing results of natural images. It is evident that with *k* = 1, a larger *ρ* results in greater smoothing strength. However, with *ρ* = 300, adjusting *k* does not have a particularly significant effect, indicating that controlling *ρ* is more effective. Of course, a balanced combination of both parameters yields the optimal smoothing result.

[0128] Since this invention employs the ADMM solution method, the proposed method mainly consists of three stages in each iteration. The first stage is solving the linear system S, which includes gradient compression, weight calculation, and solving the linear system. Gradient compression involves two one-dimensional linear operations with a complexity of O(N), where N is the total number of pixels in the image. Weight calculation involves two convolution operations with a complexity of O(k). 2The first step involves solving for the linear system of the five-point non-homogeneous Laplacian matrix. Direct inversion is time-consuming, so several acceleration methods are considered, such as preprocessing conjugate gradient (PCG) and LU decomposition, which can achieve linear speed. This invention uses PCG acceleration, achieving good smoothing results in a short time. The second step is to solve for the constrained L1 norm. For non-convex L1 norms, a subgradient is introduced to obtain its closed-form solution. Since it is solved element-wise, the complexity is also linear at O(N). The third step is to solve for the Lagrange dual multipliers, using gradient ascent to approximate the solution, with a complexity of O(N).

[0129] All algorithms were tested on an i5-9300H CPU, 16GB of RAM, and MATLAB 2018b platform. This invention was evaluated using three color images at different resolutions and compared with seven other algorithms, as shown in Table 1 below. The proposed method is slightly slower than L0, RTV, and ROG, but performs better in processing the details of the smoothed results. The algorithm is faster than G-smooth, SDF, and SSF.

[0130] Table 1 compares the running times of different algorithms on three-channel color images (measured in seconds).

[0131]

[0132] During algorithm execution, both RTV and GSTF1 methods solve linear systems to obtain the final smoothing result; both are globally optimized filters. Figure 7 As shown, a comparison of the results of RTV and GSTF1 under different smoothing intensities is presented. It is evident that as the smoothing intensity increases, the GSTF1 method causes less damage to small edges and weak structures, while RTV blurs the edges of small and weak structures. This indicates that the algorithm proposed in this invention performs better in protecting edges and smoothing details; therefore, the proposed GSTF1 has a better edge structure preservation effect.

[0133] This invention evaluates the proposed edge-preserving smoothing algorithm on various image processing tasks (image smoothing, HDR mapping, edge extraction, image abstraction, artifact removal). Extensive experiments were conducted on the NKS dataset, SPS dataset, and datasets provided by Xu et al., including GSTF1 and GSTF2. 0.8The proposed algorithm was compared with current state-of-the-art algorithms, including local filters (GF, BTF, RGF, TF), global optimization algorithms (WLS, L0, RTV, FGS, SDF, SGWLS, BLF-LS, G-smooth, L1-R, SSF, BTV), and deep learning methods (DEAF, ResNet, DeepFSps). The comparison primarily focused on image smoothing quality. Furthermore, the invention employed quantization metrics PSNR and SSIM for comparison, and a series of experiments validated the effectiveness of the proposed algorithm.

[0134] like Figure 8 As shown in (a), texture is ubiquitous in the image. The BTF method, while smoothing the texture, blurs the structural edges, resulting in artifacts in the smoothed image, such as... Figure 8 As shown in (b). However, the RGF method has difficulty filtering out large-scale textures, such as... Figure 8 As shown in (c). TF and G-smooth filters out more obvious textures, but some obvious texture lines remain, such as... Figure 8 As shown in (d) and (e), the RTV, BTV, and DeepFsps methods can smooth textures relatively well, but small structures and their edges are damaged and become very blurry, such as... Figure 8 As shown in (f), (g), and (h). In contrast, the method GSTF1 of the present invention and the proposed GSTF... 0.8 The method not only effectively filters out large-scale textures and lines, but also preserves these structures and their edges, achieving effective smoothing. Figure 8 As shown in (i) and (j).

[0135] like Figure 9 As shown in the fabric texture in (a), the small pattern structures and block textures in the fabric are roughly the same size. How to remove the texture while preserving the structural edges and small structural elements is a challenging problem. BTF and G-smooth methods are effective at smoothing the fabric background texture, but they destroy the small structural elements and produce artifacts, such as... Figure 9 As shown in (b) and (f). L0, SGWLS, FGS, and DEAF do not adequately consider the relationship between texture and structure; although they preserve small structures very well, their smoothing effect is poor, such as... Figure 9 As shown in (c), (d), (e), and (i), RTV and BTV achieve better texture removal, but severely damage the small-structure wick and its edges, such as... Figure 9 As shown in (g) and (h), the deep learning method DeeFsps effectively smooths out textures, but it also penalizes gradients of small structures, resulting in oversmoothing, as shown in... Figure 9As shown in (j). It is evident that the proposed GSTF1 method fully considers the trade-off between texture and structure, effectively filtering out texture while preserving these small structures, such as... Figure 9 As shown in (k), but it also weakens the effect. Figure 9 The wick in the lower right corner of (k). In contrast, the proposed GSTF1 fully considers this issue. Due to its better sparsity than methods using the L1 norm, it achieves better smoothing of textured images without destroying small structural edges, such as... Figure 9 As shown in (l).

[0136] like Figure 10 The image shown in (a) exhibits a variety of complex textures at different scales, with the marked boxes containing small-scale textures and structures, posing a significant challenge to the algorithm. The RGF method produces artifacts after filtering, such as... Figure 10 As shown in (b). While the SDF, RTV, and BTV methods remove large-scale textures, they all more or less destroy the edges of small structures, such as... Figure 10 As shown in (c), (d), and (e), the deep learning method DeepFsps lacks scale awareness. Although it achieves good smoothing, small structures are severely damaged, resulting in oversmoothing. In contrast, the proposed GSTF1 method of this invention filters out texture while preserving... Figure 10 (g) The small structure in the method box, but the edges of the small structure are slightly damaged. Based on the GSTF proposed in this invention. 0.8 A more sparse method can solve this problem, such as Figure 10 As shown in (h), the superiority of its edge-preserving smoothness algorithm is demonstrated.

[0137] The method of this invention exhibits good processing performance for multi-scale texture images, avoiding undersmoothing phenomena. Therefore, this method is highly effective in handling complex tasks. Table 2 below shows a comparison of quantitative results for various methods on the NKS and SPS datasets. It is evident that the GSTF1 algorithm proposed in this invention and GSTF... 0.8 The algorithm has certain advantages in both PSNR and SSIM metrics.

[0138] Table 2 shows the quantitative analysis results on the NKS and SPS datasets.

[0139]

[0140] Edge extraction is an application of image smoothing. For natural images with a lot of noise or many details, direct edge extraction often yields unsatisfactory results. However, image smoothing filters out unnecessary textures, details, or noise, preserving the main structural edges before edge extraction. This invention uses the Canny operator for edge detection, achieving more ideal results.

[0141] Figure 11 The visual effect of an algorithm for edge extraction is demonstrated. Figure 11 (a) contains both texture and structural information. Visually, RGF, L0, and SSF perform poorly in edge extraction, as these methods preserve a large amount of texture or detail. BTF, G-smooth, RTV, and BTV, on the other hand, show better overall edge extraction, but... Figure 11 In (b), (e), (f), and (g), the edges of the small structures within the lower half of the green boxes are all damaged to some extent. In contrast, the method proposed in this invention exhibits some artifacts at the edges of GSTF1, leading to inaccurate edges, such as... Figure 11 As shown in (i). GSTF 0.8 This method effectively solves the problem without causing any damage during edge extraction, such as... Figure 11 As shown in (j). Therefore, the method proposed in this invention is effective.

[0142] HDR tone mapping is another application that preserves edges while increasing brightness. HDR aims to compress the intensity of a high dynamic range image while maintaining the original details and colors to create a low dynamic range image. It consists of a base layer and a detail layer. The base layer is a smoothed output of the input HDR image. The base layer is then non-linearly mapped to the low dynamic range and recombined with the detail layer to obtain the final tone-mapped image.

[0143] Figure 12 The comparison results of various HDR tone mapping algorithms are presented. It is evident that the LDR image generated by WLS is not the expected result, being generally dark. L0 exhibits artifacts, resulting in a very blurry LDR image. The LDR image generated by G-smooth shows halos, and details are not clearly visible. The proposed algorithm GSTF1 overcomes these shortcomings, but the processing of detail edges in GSTF1... 0.8 The algorithm is superior. This invention uses TMQI to quantitatively evaluate the image quality of HDR tone mapping, and uses 15 HDR images provided by Yeganeh et al. The specific quantification results are shown in Table 3 below. The image smoothing algorithm of this invention has certain advantages over various methods.

[0144] Table 3 shows the quantitative analysis results of each method on the dataset provided by Yeganeh.

[0145]

[0146] Image compression artifact removal aims to eliminate artifacts that appear during the compression process. Such images are segmented and constant with sharp edges; when compressed in JPEG format, artifacts can appear near these edges. Figure 13 As shown in (a), texture filtering algorithms are well-suited for this application, smoothing out unnecessary details while preserving edge structures. Therefore, the proposed algorithm can be used to reduce artifacts caused by image compression. The L0 algorithm can eliminate most compression artifacts, but it directly smooths out weak structures and their edges, such as… Figure 13 As shown in (b), the SDF and BTV algorithms blur rather than sharpen edges when dealing with clipart compression artifacts, as... Figure 13 As shown in (c) and (e), the RTV algorithm was used to remove compression artifacts, achieving some success, but it caused some damage to the structural edges, such as... Figure 13 As shown in (d), the G-smooth method proposed by Liu et al. can sharpen edges, but it introduces artifacts in small structural regions, such as... Figure 13 (f) As shown in the blue box. The methods GSTF1 and GSTF proposed in this invention. 0.8 Optimal performance was achieved in both edge sharpening and preservation of small structures, such as Figure 13 As shown in (g) and (h).

[0147] Image abstraction is one application of image smoothing. Image abstraction aims to remove unnecessary details from an image through filtering, while abstracting edges and structures to create a cartoon-like image. To better represent the information the image intends to convey, the result of image abstraction is as follows: Figure 14 As shown.

[0148] The proposed method has been extensively evaluated and shows good performance on both textured and non-textured images, with edge preservation capabilities superior to existing filtering algorithms.

[0149] Compared with existing technologies, the global sparse texture filtering method based on edge structure preservation provided by this invention constrains the gradient of the output image by introducing a texture suppression function. First, two appropriate gradient thresholds are set to suppress large-scale textures comparable to salient structures in the image, as well as unnecessary details, noise, and small-scale textures. Then, the weights in the penalty term are processed using the gradient suppressed by the texture suppression function to ensure efficient smoothing without overfitting or underfitting. Finally, the L1 norm, which has strong sparsity, is used to constrain the penalty term, so that the proposed method model is as close as possible to the true value when minimized, improving the robustness of the algorithm. Therefore, the sparse edge-preserving image smoothing algorithm with texture suppression function proposed in this invention achieves good smoothing performance while preserving structural edges, and also has the following advantages:

[0150] 1. This invention proposes a novel global optimization model framework that can achieve high-quality smooth results while effectively preserving edges, and adopts an L1 norm constraint penalty term, giving the model a certain degree of sparsity.

[0151] 2. This invention fully considers the problem of texture image smoothing, introduces a texture suppression function, suppresses multi-scale textures that are difficult to remove, and achieves a better smoothing effect.

[0152] 3. This invention uses L p The norm constraint penalty term achieves a more robust and sparse image smoothing effect, and it employs the idea of ​​iterative reweighting to adjust L... p The norm problem is transformed into solving the L1 norm problem. Compared to the penalty term for L1 norm constraints, L... p The model with norm-constrained penalty terms exhibits better sparsity.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A global sparse texture filtering method based on edge structure preservation, characterized in that, The method includes the following steps: S1. Introduce a texture suppression function into the penalty term of the global optimization framework to constrain the gradient of the output image. The texture suppression function is defined by... and Two thresholds are used to suppress texture, noise, and unwanted detail in the image, where Used to suppress smaller-scale textures, noise, and unwanted details, while This is used to suppress large-scale textures comparable to salient structures, thereby achieving better texture image smoothing. The formula for the texture suppression function is shown below: Equation (1) in, To output the image gradient, and For the threshold, It is an exponential function; Then, the gradient after suppression by the texture suppression function is used as the input of the denominator of the penalty term, so that the penalty term can fully distinguish between texture and structure, and apply different degrees of penalty to the two, thereby better preserving the semantic information of the image. S2, using a high degree of sparsity Norms are used to constrain penalty terms. Norms can achieve more sparsity results, although for sparsity... Although the norm is not differentiable, by introducing subgradients to transform the non-convex optimization problem into a convex one, and using the alternating direction multiplier method for iterative solution, a better edge-preserving image smoothing algorithm can be achieved, avoiding... Norm sparsity is weak, and it cannot preserve the semantic information of the image well; S3, employing a high degree of sparsity. Norms are used to constrain penalty terms, where ,for Norm problems are solved efficiently using iterative reweighting. The solution is obtained by using the norm method, and the preprocessing conjugate gradient method is used to accelerate and improve the computational efficiency, thereby achieving a more robust and sparse image smoothing effect. Step S3 includes the following steps: S31, The penalty term of the objective function of the edge-preserving image smoothing algorithm is adopted. Norm constraints are defined as follows: Equation (9) S32. Using an iterative reweighting method to... Norm solving is transformed into solving The norm problem involves dynamically weighting the values ​​from the previous iteration to obtain the weights for each round, and... Based on norm, it was derived The method for solving norms involves iteratively minimizing non-convex functions to transform them into convex functions, with weights... pass To adjust, the following is the first... The next iteration The median value of the direction: Equation (10) For weight terms After each iteration, the weights are updated in the next iteration. , here It is a very small positive number to prevent it from being divisible by zero. The same applies to the handling of direction; rewrite equation (10) and matrix it as follows: Equation (11) in, and All are the second weight.

2. The global sparse texture filtering method based on edge structure preservation according to claim 1, characterized in that, Step S2 includes the following steps: S21. The objective function of the edge-preserving image smoothing algorithm is defined as follows: Equation (2) in, It outputs an image; It is the input image; It is a balancing parameter used to adjust the relationship between data items and penalty items, and can directly affect smoothing performance; It is a very small positive number, to prevent it from being divisible by zero; represent Norm; Indicates the index position; and Representing an image direction and direction; It is the sum of the absolute values ​​of the weighted gradients within the window; It is the absolute value of the weighted gradient sum within the window; and The derivation formula is as follows: Equation (3) Equation (4) in, express direction and direction; Therefore Centered on, the window size is The domain space, in which Represented as ; It is a Gaussian kernel function; The size of the Gaussian kernel; These are texture suppression functions at different scales; This indicates the process of finding the differential; S22. Transform the objective function formula (2) into Norm constraint penalty term: First to The direction is used to solve the problem. direction and The direction is handled the same way; the penalty term in formula (2) is expanded as follows: Equation (5) here The penalty term for direction is decomposed into a linear term. and a non-linear weighting component , The direction is handled the same way. It is expressed as follows: Equation (6) Finally, the objective function is rewritten and matrix-reduced as follows: Equation (7) in, , It is a backward difference matrix, and , All A diagonal matrix of size, where Represents the total number of pixels in the image; S23. Solve the objective function using the alternating direction multiplier method: To effectively solve equation (7) above, let , By constructing the augmented Lagrange function, the objective function is obtained as follows: Equation (8) For solving the final objective function (8), the objective function only contains norm, The norm is unconstrained, so we can directly minimize it by taking the derivative.

Images