A color image salt-and-pepper noise removing method based on tensor low-rank prior and implicit regularization technique

Abstract

The application provides a color image salt and pepper noise removing method based on tensor low-rank prior and implicit regularization technology, including: detecting salt and pepper noise position information by detecting pixel amplitude of an image; regarding a color image contaminated by salt and pepper noise as a sum of a tensor with missing data and a salt and pepper noise tensor; and reconstructing the tensor with missing data by using a model-driven parallel matrix decomposition low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network; the method provided by the application detects noise position information by detecting and processing pixel amplitude of an image, and introduces a model-driven parallel matrix decomposition low-rank tensor reconstruction algorithm and a data-driven FFDNet denoising network, so that the denoising effect is more superior.

Description

Technical Field

[0001] This invention relates to the field of image denoising, and in particular to a salt and pepper denoising method for color images based on tensor low-rank prior and implicit regularization techniques. Background Technology

[0002] Salt-and-pepper noise is a noise source with significant sparse statistical characteristics. The "pepper" noise refers to noise points with zero amplitude, typically caused by data loss. The "salt" noise refers to noise points with impulse values, usually caused by strong impulse interference during image transmission. The goal of salt-and-pepper noise denoising is to estimate a clean image from degraded observations. Existing salt-and-pepper denoising methods are mainly divided into two categories: model-driven methods and data-driven methods. Although existing methods have achieved good denoising performance, they suffer from two main limitations.

[0003] 1) In existing methods, the amplitude (zero or impulse value) of salt-and-pepper noise is usually ignored. However, this feature is important for noise localization and is also easy to identify.

[0004] 2) Existing methods often treat color images as three independent matrices extracted from RGB channels, thus ignoring the similarity of signals between channels.

[0005] The existing solutions have not been able to adequately address the two limitations mentioned above. Summary of the Invention

[0006] The main objective of this invention is to overcome the aforementioned deficiencies in the prior art and propose a salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques. First, the noise location information is detected by detecting the pixel amplitude of the processed image. Then, a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm and a data-driven FFDNet denoising network are introduced, resulting in a superior denoising effect.

[0007] The present invention adopts the following technical solution:

[0008] A salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques includes:

[0009] The location information of salt and pepper noise is detected by detecting the pixel amplitude of the image;

[0010] The color image contaminated with salt and pepper noise is used as the sum of the tensor of the missing data and the salt and pepper noise tensor;

[0011] A model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors of missing data.

[0012] Specifically, a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors of missing data. The specific model includes:

[0013]

[0014] Where μ is the regularization parameter, used to balance the low-rank model driving force. The regularization term implemented with the FFDnet denoising network; For data-driven items, It is the reconstructed low-rank tensor. It is the observation tensor, and Ω is the index set corresponding to the observed entries. It is a projection function that will The data in X remains on Ω. n and Y n These represent the TMac factor matrix, R (n) This is the unfolded mode.

[0015] Specifically, the method utilizes a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network to reconstruct tensors from missing data. It also includes low-rank tensor reconstruction based on model-driven parallel matrix factorization low-rank tensors, specifically comprising:

[0016]

[0017] in It is the reconstructed low-rank tensor. It is the observation tensor, and Ω is the index set corresponding to the observed entries. It is a projection function that will The data in the Ω remains;

[0018]

[0019] in, X n and Y n Let TMac be the factor matrix, and R be the factor matrix. (n) For the expansion mode, the expanded tensor is

[0020] Specifically, the model is solved as follows:

[0021]

[0022] Where is the learning rate, i represents the number of iterations, and n represents the pattern number to be expanded;

[0023] This X nSubproblems and Y n The sub-problems are as follows:

[0024]

[0025] J represents the objective function;

[0026] make and The solution to (5) is:

[0027]

[0028] Among them, symbols It is the Moore-Penrose pseudo-inverse operator;

[0029] The sub-problems are as follows:

[0030]

[0031] Frobenius norm It calculates the square root of the sum of the squares of all elements; Defined as a tensor The expansion of the matrix, the Frobenius norm. With tensor Frobenius norm They are equal; therefore, we obtain Subproblems are as follows;

[0032]

[0033] We expand equation (8) and then combine like terms to get:

[0034]

[0035] Equation (9) can be optimized as follows:

[0036]

[0037] Equation (10) is rounded and then simplified as follows:

[0038]

[0039] Will and The subproblem is rewritten as:

[0040]

[0041] in, The regularization term is implemented using FFDnet; then, equation (12) is standardized; finally, the recovered color image model is as follows:

[0042]

[0043] Where Ω c As the complement of Ω, FFDNet is a data-driven denoising neural network; σ is a denoising parameter related to the salt-and-pepper noise level, which is related to... It is related to systematic errors.

[0044] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0045] This invention provides a salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques. The method includes: detecting the location information of salt-and-pepper noise by detecting the pixel amplitude of the image; using the salt-and-pepper noise-contaminated color image as the sum of the tensor of the missing data and the salt-and-pepper noise tensor; and reconstructing the tensor of the missing data using a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network. The method provided by this invention detects the noise location information by detecting the pixel amplitude of the processed image, and introduces a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm and a data-driven FFDNet denoising network, resulting in superior denoising performance. Attached Figure Description

[0046] Figure 1 A schematic diagram of a model provided for an embodiment of the present invention;

[0047] Figure 2 The images show a comparison of the denoising effects of different algorithms provided in this embodiment of the invention, where: (a) original image; (b) salt and pepper noise image with a noise level of 0.1; (c) denoising using the TNN method; (d) denoising using the TSF method; (e) denoising using the IMF method; (f) denoising using the SFT_Lp method; (g) denoising using the TMac method; (h) denoising using the TNN_DP3 method; and (i) denoising using the TMac_DP3 method of this invention.

[0048] Figure 3 Comparison of denoising effects of different algorithms provided in embodiments of the present invention; wherein: (a) original image; (b) salt and pepper noise image with a noise level of 0.5; (c) denoising using TNN method; (d) denoising using TSF method; (e) denoising using IMF method; (f) denoising using SFT_Lp method; (g) denoising using TMac method; (h) denoising using TNN_DP3 method; (i) denoising using the TMac_DP3 method of the present invention;

[0049] Figure 4Comparison of denoising effects of different algorithms provided in embodiments of the present invention; wherein: (a) original image; (b) salt and pepper noise image with a noise level of 0.9; (c) denoising using TNN method; (d) denoising using TSF method; (e) denoising using IMF method; (f) denoising using SFT_Lp method; (g) denoising using TMac method; (h) denoising using TNN_DP3 method; (i) denoising using the TMac_DP3 method of the present invention;

[0050] Figure 5 The dynamic iterative curves of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) values ​​for different algorithms provided in embodiments of the present invention are shown below; where (a) PSNR value with a noise level of 0.1; (b) PSNR value with a noise level of 0.5; (c) PSNR value with a noise level of 0.9; (d) SSIM value with a noise level of 0.1; (e) SSIM value with a noise level of 0.5; and (f) SSIM value with a noise level of 0.9.

[0051] Figure 6 This is a comparison chart of the denoising and stabilization effects of different algorithms provided in the embodiments of the present invention;

[0052] Figure 7 The figure shows the parameter sensitivity curves of different algorithms provided in the embodiments of the present invention; wherein Figure (a) is the PSNR value and Figure (b) is the SSIM value;

[0053] Figure 8 This is a comparison chart showing the noise reduction sensitivity of different algorithms provided in the embodiments of the present invention.

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0055] This invention provides a salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques. It detects noise location information by detecting the pixel amplitude of the processed image, and introduces a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm and a data-driven FFDNet denoising network, resulting in superior denoising performance.

[0056] The technical solution adopted in this invention is as follows:

[0057] A salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques includes:

[0058] The location information of salt and pepper noise is detected by detecting the pixel amplitude of the image;

[0059] The color image contaminated with salt and pepper noise is used as the sum of the tensor of the missing data and the salt and pepper noise tensor;

[0060] A model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors of missing data.

[0061] Specifically, a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors of missing data. The specific model includes:

[0062]

[0063] Where μ is the regularization parameter, used to balance the low-rank model driving force. The regularization term implemented with the FFDnet denoising network; For data-driven items, It is the reconstructed low-rank tensor. It is the observation tensor, and Ω is the index set corresponding to the observed entries. It is a projection function that will The data in X remains on Ω. n and Y n These represent the TMac factor matrix, R (n) For the unfolded mode, such as Figure 1 This is a schematic diagram of a model provided for an embodiment of the present invention.

[0064] Specifically, the method utilizes a model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network to reconstruct tensors from missing data. It also includes low-rank tensor reconstruction based on model-driven parallel matrix factorization low-rank tensors, specifically comprising:

[0065]

[0066] in It is the reconstructed low-rank tensor. It is the observation tensor, and Ω is the index set corresponding to the observed entries. It is a projection function that will The data in the Ω remains;

[0067]

[0068] in, X n and Y n Let TMac be the factor matrix, and R be the factor matrix. (n) For the expansion mode, the expanded tensor is

[0069] Specifically, the model is solved as follows:

[0070]

[0071] Where is the learning rate, i represents the number of iterations, and n represents the pattern number to be expanded;

[0072] This X n Subproblems and Y n The sub-problems are as follows:

[0073]

[0074] J represents the objective function;

[0075] make and The solution to (5) is:

[0076]

[0077] Among them, symbols It is the Moore-Penrose pseudo-inverse operator;

[0078] The sub-problems are as follows:

[0079]

[0080] Frobenius norm It calculates the square root of the sum of the squares of all elements; Defined as a tensor The expansion of the matrix, the Frobenius norm. With tensor Frobenius norm They are equal; therefore, we obtain Subproblems are as follows;

[0081]

[0082] We expand equation (8) and then combine like terms to get:

[0083]

[0084] Equation (9) can be optimized as follows:

[0085]

[0086] Equation (10) is rounded and then simplified as follows:

[0087]

[0088] Will and The subproblem is rewritten as:

[0089]

[0090] in, The regularization term is implemented using FFDnet; then, equation (12) is standardized; finally, the recovered color image model is as follows:

[0091]

[0092] Where Ω c As the complement of Ω, FFDNet is a data-driven denoising neural network; σ is a denoising parameter related to the salt-and-pepper noise level, which is related to... It is related to systematic errors.

[0093] Finally, the proposed method is summarized in Algorithm 1.

[0094]

[0095]

[0096] Where, tol = 10 -5 . It is a matrix containing random value terms. Similarly, It is also a matrix that contains random value terms.

[0097] The advantages of the method of the present invention will be illustrated below through specific experiments;

[0098] This section evaluates the performance of the proposed salt-and-pepper noise denoising model. To validate the proposed model, this invention primarily compares four denoising algorithms: TNN, TMac, TMac_DP3, and TNN_DP3. Then, ablation experiments are conducted to verify the stability of TMac and TNN. The parameter sensitivity of the model is then verified by changing the values ​​of key parameters. To better observe the performance of the proposed model, this invention also compares TSF, SFT_Lp, and IMF, respectively, using peak signal-to-noise ratio (PSNR).

[48] Structural Similarity Index (SSIM)

[49] Measurement.

[0099] Peak signal-to-noise ratio (PSNR) is usually expressed in decibels (dB), with values ​​typically between 20 and 50. A higher value indicates a better recovered image. The formula for PSNR is as follows:

[0100]

[0101] Where X is the original image, Y is the restored image, and max(X) is the maximum possible pixel value of the image.

[0102] Structural similarity (SSIM) is an objective data metric for evaluating the similarity between two images, with a value ranging from 0 to 1. When the SSIM value is 1, the two images are identical. The formula for SSIM is:

[0103]

[0104] Where u X Let X be the mean, u Y Let σ be the mean of Y. X 2 Let X be the variance, and σ be the variance. Y 2 Let (Lk1) be the variance of Y. 2 and (Lk2) 2 These are two constants, to avoid division by zero.

[0105] 1) Experimental Results - Comparison of Salt-and-Pepper Denoising Results for Color Images:

[0106] To better observe the denoising capabilities of salt-and-pepper noise reduction in the images, three different levels of salt-and-pepper noise were added to the images mentioned above. Then, this embodiment of the invention compared images restored using seven algorithms; such as... Figure 2 The images show a comparison of the denoising effects of different algorithms provided in this embodiment of the invention, where: (a) original image; (b) salt and pepper noise image with a noise level of 0.1; (c) denoising using the TNN method; (d) denoising using the TSF method; (e) denoising using the IMF method; (f) denoising using the SFT_Lp method; (g) denoising using the TMac method; (h) denoising using the TNN_DP3 method; and (i) denoising using the TMac_DP3 method of this invention. Figure 3 Comparison of denoising effects of different algorithms provided in embodiments of the present invention; wherein: (a) original image; (b) salt and pepper noise image with a noise level of 0.5; (c) denoising using TNN method; (d) denoising using TSF method; (e) denoising using IMF method; (f) denoising using SFT_Lp method; (g) denoising using TMac method; (h) denoising using TNN_DP3 method; (i) denoising using the TMac_DP3 method of the present invention; Figure 4 Comparison of denoising effects of different algorithms provided in embodiments of the present invention; wherein: (a) original image; (b) salt and pepper noise image with a noise level of 0.9; (c) denoising using TNN method; (d) denoising using TSF method; (e) denoising using IMF method; (f) denoising using SFT_Lp method; (g) denoising using TMac method; (h) denoising using TNN_DP3 method; (i) denoising using the TMac_DP3 method of the present invention;

[0107] from Figure 2It can be seen that when the noise level of "Lena" is 0.1, the tensor-based low-rank method outperforms the local self-similarity denoising algorithm. From the magnified local images, mosaic artifacts exist in both the median filtering method and the TV regularization recovery method. The tensor-based low-rank method exhibits high smoothness, indicating that the low-rank tensor reconstruction method can achieve good denoising results. Figure 4 It can be seen that TMac_DP3 achieves the best restoration effect, with the clearest eye area and smooth and natural processing of overall color and details. TNN_DP3 performs slightly worse, but both algorithms are significantly better than other methods. The results show that the network combining TMac and TNN with FFDNet for denoising outperforms other algorithms in both high and low noise levels. Therefore, single low-rank tensor reconstruction based on TNN or TMac cannot guarantee regularization at low rank. However, adding an FFDNet denoising network based on depth priors can greatly improve the denoising effect. Figure 3 It can be observed that the aircraft outline is distorted after restoration using median filtering and TV regularization. TNN, TMac, and TNN_DP3 show noticeable texture blurring in the partially magnified wing sections. However, TMac_DP3 exhibits the best restoration results in terms of color and sharpness, further demonstrating its superior performance.

[0108] (2) Experimental Results - Quantitative Analysis:

[0109] In this section, embodiments of the invention perform PSNR and SSIM calculations and statistics on eight pairs of graphs under nine noise levels. The bolded values ​​for PSNR and SSIM represent the optimal values. Experimental comparison methods include TSF, IMF, TNN, TMac, SFT_FP, and TNN_DP3.

[0110] Table 1 PSNR values ​​from algorithm experiments

[0111]

[0112]

[0113]

[0114] Table 2 SSIM values ​​from algorithm experiments

[0115]

[0116]

[0117]

[0118] From Table 1, we can see that under low noise conditions for the "Lena" image, the PSNR of the new algorithm TMac_DP3 is 2dB higher than the other five algorithms, and TMAC_DP3 outperforms TNN_DP3 at most noise levels. The PSNR value for "Opera" completely surpasses the other algorithms. When testing "Baboon," the PSNR value of TMac_DP3 is 2dB to 3dB higher than the other algorithms at low noise levels. This indicates that the algorithm has a strong ability to process images with high color complexity. From Table 2, we can see the corresponding SSIM values. TMac_DP3 performs well at most levels. In the overall denoising results, TMac_DP3 performs best, followed by TNN_DP3. The experimental results further demonstrate the superior performance of this model.

[0119] (3) Experimental Results - Ablation Experiment:

[0120] In this section, embodiments of the invention explore the stability of TNN, TMac, TNN_DP3, and TMac_DP3 during the iteration process. Dynamic iteration curves of PSNR and SSIM values ​​are plotted for four noise levels.

[0121] Figure 5 The dynamic iterative curves of PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) values ​​for different algorithms provided in embodiments of the present invention are shown below; where (a) PSNR value with a noise level of 0.1; (b) PSNR value with a noise level of 0.5; (c) PSNR value with a noise level of 0.9; (d) SSIM value with a noise level of 0.1; (e) SSIM value with a noise level of 0.5; and (f) SSIM value with a noise level of 0.9.

[0122] from Figure 5 It can be seen that the TNN algorithm exhibits degradation when the number of iterations reaches around 60. However, the TMac algorithm shows smooth and stable performance. Here, we can conclude that TMac is more stable than TNN. Compared to the TNN curve, the TNN_DP3 and TMac_DP3 curves are smoother and more stable, and their performance is superior to TMac. This indicates that the introduction of FFDnet can improve not only model performance but also model stability. It is worth noting that TNN_DP3 exhibits severe degradation after 20 iterations. As the number of iterations increases, the performance of TNN_DP3 increases slowly, eventually leading to performance inferior to TMac_DP3. For the TMac_DP3 model, the curve is smooth and stable, and its performance becomes more prominent with increasing iterations.

[0123] Figure 6 This is a comparison chart of the denoising and stabilization effects of different algorithms provided in the embodiments of the present invention;

[0124] By comparing these four algorithms, we can... Figure 7 The fully enlarged image shows the complete outline of the theater. The images restored by the TNN and TMac models appear blurry. The images restored by TNN_DP3 and TMac_Dp3 are quite similar; they restore details, and the overall image is sharper and smoother. However, TMac_DP3 is more stable than TNN_DP3.

[0125] (3) Experimental Results - Parameter Sensitivity Analysis:

[0126] The sensitivity of the TMAC_DP3 model to the parameter σ was discussed in the embodiments of this invention; from the above... It can be seen that σ is adjusted by μ; as Figure 7 The figure shows the parameter sensitivity curves of different algorithms provided in the embodiments of the present invention; wherein Figure (a) is the PSNR value and Figure (b) is the SSIM value; Figure 8 This is a comparison chart showing the noise reduction sensitivity of different algorithms provided in the embodiments of the present invention.

[0127] from Figure 7 It can be seen that when other parameters remain constant, the value of μ is adjusted by approximately a factor of 10. The iterative curves of the experimental results show relatively small changes, therefore the TMac_DP3 model is quite stable and less affected by the parameter μ. Figure 8 As can be seen, the recovery results are consistent. This further proves that the TMac_DP3 model is less affected by the parameter μ.

[0128] In summary, this invention combines model-driven and data-driven regularization to propose a novel salt-and-pepper noise denoising model for color images. This model treats the contaminated color image as the sum of a tensor containing missing data and a salt-and-pepper noise tensor determined by amplitude detection. Thus, color image denoising is cleverly transformed into a low-rank tensor recovery problem. Then, a model-driven TMac regularization term is introduced to characterize the global low-rank features of the multi-dimensional image, enhancing global correlation within the image. Simultaneously, a data-driven FFDnet is used to explore deep image priors for each channel of the color image. FFDnet effectively represents the details that are difficult to capture in TMac regularization. The new model is then solved using the BSUM algorithm. Finally, extensive experiments were conducted at different levels of salt-and-pepper noise. Experimental results show that the proposed model outperforms other models at most salt-and-pepper noise levels.

[0129] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A salt-and-pepper denoising method for color images based on tensor low-rank priors and implicit regularization techniques, characterized in that, include: The location information of salt and pepper noise is detected by detecting the pixel amplitude of the image; The color image contaminated with salt and pepper noise is used as the sum of the tensor of the missing data and the salt and pepper noise tensor; A model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors of missing data. A model-driven parallel matrix factorization low-rank tensor reconstruction algorithm combined with a data-driven FFDNet denoising network is used to reconstruct tensors from missing data. Specifically, the model includes: in, It is a regularization parameter used to balance the low-rank model driver. The regularization term implemented with the FFDnet denoising network; For data-driven items, It is the reconstructed low-rank tensor. It is the observation tensor. It is the set of indexes corresponding to the observed entries. It is a projection function that will The data in the middle remains superior, and These represent the TMac factor matrix, , For unfolding mode; This paper describes a method for reconstructing low-rank tensors from missing data using a model-driven parallel matrix factorization algorithm combined with a data-driven FFDNet denoising network. It also includes low-rank tensor reconstruction based on model-driven parallel matrix factorization, specifically comprising: The expanded tensor is , n represents the pattern number for expansion.

2. The salt-and-pepper denoising method for color images based on tensor low-rank prior and implicit regularization techniques according to claim 1, characterized in that, The model solution is as follows: in, This is the learning rate, and i represents the number of iterations. this Subproblems and The sub-problems are as follows: J represents the objective function; make and ,calculate The solution is: Among them, symbols It is the Moore-Penrose pseudo-inverse operator; The sub-problems are as follows: Frobenius norm It calculates the square root of the sum of the squares of all elements; Defined as a tensor The expansion of the matrix, the Frobenius norm. With tensor Frobenius norm They are equal; therefore, we obtain Subproblems are as follows; We unfold Formula, then combine like terms to get: Equation Optimized to: Equation The formula is prepared and then simplified as follows: Will and , The subproblem is rewritten as: in, It's a regularization term, implemented using FFDnet; then, The formula is standardized; finally, the recovered color image model is as follows: in for The supplement, It is a data-driven denoising neural network; It is a noise reduction parameter related to the salt-and-pepper noise level, and it is related to... It is related to systematic errors.

Images