Image restoration method, device and product based on double-regularized FCTN decomposition

Abstract

This invention provides an image restoration method, device, and product based on dual-regularized FCTN decomposition. The method includes: introducing a gradient factor regularization term and a structural sparsity regularization term based on FCTN decomposition to obtain an image restoration model; restoring the image to be restored using the image restoration model; the gradient factor regularization term is used to perform low-rank decomposition in the gradient domain of the FCTN factors to constrain local continuity; the structural sparsity regularization term is used to apply sparsity constraints to the shared dimensions among the FCTN factors. This invention can effectively improve the restoration accuracy of fine details and complex textures in images with high missing rates.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to an image restoration method, device, and product based on dual regularized FCTN decomposition. Background Technology

[0002] With the rapid development of science and technology, more and more high-dimensional and complex images, such as color images, multispectral images, and magnetic resonance imaging (MRI), are being acquired in the real world. During image acquisition and transmission, high-dimensional images inevitably suffer from loss or damage due to environmental limitations, unstable imaging equipment, or communication failures. High-dimensional image restoration (or tensor completion) aims to accurately reconstruct a clean and complete image from partially incomplete observation images. Its core idea lies in fully exploiting the potential prior knowledge inherent in the image itself (such as global low-rank property and local continuity), transforming an ill-posed problem into a well-posed problem that can be efficiently computed.

[0003] To effectively characterize the multidimensional structural properties of high-dimensional images, tensors, as a high-dimensional extension of vectors and matrices, have been introduced into this field. However, directly processing high-order tensors faces the challenge of the "curse of dimensionality," where storage and computational costs increase exponentially with tensor order. This has spurred the development of tensor decomposition techniques, which decompose high-order tensors into a series of low-order latent factors through multilinear operations, thereby effectively reducing computational overhead. Early classical methods such as CP decomposition and Tucker decomposition laid the foundation for tensor decomposition, but CP decomposition suffers from limitations such as difficulty in determining the rank and computational instability, while the core tensor parameters of Tucker decomposition increase exponentially with order, and the original data structure is lost when the tensor is expanded into a matrix, making it difficult to capture the global correlation between different patterns.

[0004] Subsequently, tensor network theory was introduced into the fields of computational science and signal processing. Tensor chain (TT) decomposition and tensor ring (TR) decomposition, by constructing "chains" or "rings" of topological structures, alleviated computational complexity to some extent. However, these two methods can only establish multilinear operations between adjacent factors, have limited ability to represent the correlations of the original tensors, and are highly sensitive to the order of modes. To overcome this limitation, Fully-Connected Tensor Network (FCTN) decomposition was proposed. FCTN decomposes high-order tensors into multiple factor tensors and establishes multilinear operations between any two core tensors, which can fully characterize the relationship between any two modes, better capture the global correlations of data, and possess transpose invariance.

[0005] Despite the significant advantages of the FCTN model in characterizing the global correlation of high-order tensors, it still faces the following serious challenges in practical applications: First, the complex network topology of FCTN results in a large number of rank parameters (e.g., the FCTN rank of an N-order tensor contains N(N-1) / 2 parameters), making the model unable to adaptively determine the rank and extremely sensitive to the choice of rank parameters. Second, FCTN mainly mines global low-rank properties in the factor domain, ignoring the inherent local smoothness prior and texture diversity of images. As a result, when the data missing rate is extremely high, relying solely on global low-rank properties cannot provide satisfactory recovery results, making it difficult to reconstruct fine local details and complex textures. Summary of the Invention

[0006] This invention provides an image restoration method, device, and product based on dual regularized FCTN decomposition.

[0007] An image restoration method based on double-regularized FCTN decomposition according to at least one embodiment of the present invention includes: obtaining an observation tensor of an image to be restored; introducing a gradient factor regularization term and a structural sparsity regularization term on the basis of FCTN decomposition to obtain an image restoration model; restoring the image to be restored based on the observation tensor using the image restoration model to obtain a target tensor; and obtaining an image restoration result of the image to be restored based on the target tensor; wherein the gradient factor regularization term is used to perform low-rank decomposition in the gradient domain of the FCTN factor to constrain local continuity, and the construction process of the gradient factor regularization term includes: for each FCTN factor, using difference moments... The gradient domain matrix is ​​obtained by applying the modulus expansion matrix of the FCTN factors; the gradient domain matrix is ​​decomposed into the product of two gradient factor matrices; Frobenius norm penalties are applied to the two gradient factor matrices respectively to construct the gradient factor regularization term; wherein, the structural sparse regularization term is used to impose sparse constraints on the shared dimension among the FCTN factors, and the construction process of the structural sparse regularization term includes: based on the variational approximation of the Schatten-1 / 2 norm, the FCTN factors are divided into multiple sub-tensor groups along the shared dimension; an iterative weighted constraint is applied to the sum of squared Frobenius norms of the sub-tensor groups to construct the structural sparse regularization term.

[0008] An image restoration method based on double-regularized FCTN decomposition according to at least one embodiment of the present invention applies an iterative weighting constraint to the sum of squares of the Frobenius norm of the sub-tensor group, comprising: in each iteration, calculating and updating the weight coefficients of the current iteration based on the Frobenius norm of the corresponding sub-tensor group in the previous iteration, wherein the weight coefficients are inversely proportional to the square root of the sum of squares of the Frobenius norm of the sub-tensor group.

[0009] According to at least one embodiment of the present invention, the image restoration method based on double regularized FCTN decomposition is provided. The image restoration model is solved by a proximal alternating minimization algorithm. The solution process includes: alternatingly updating the FCTN factor, the gradient factor matrix and the target tensor, and performing a near-zero element pruning operation when updating the FCTN factor until the convergence condition is met.

[0010] According to at least one embodiment of the present invention, the image restoration method based on double regularized FCTN decomposition has the convergence condition being: the relative error between the target tensors obtained in two adjacent iterations is less than an error threshold.

[0011] An image restoration method based on double-regularized FCTN decomposition according to at least one embodiment of the present invention alternately updates the FCTN factor, comprising: transforming the update subproblem of the FCTN factor in the image restoration model into matrix form; constructing the Sylvester matrix equation based on the matrix form; diagonalizing the coefficient matrix in the Sylvester matrix equation using matrix eigenvalue decomposition and one-dimensional fast Fourier transform; and solving the Sylvester matrix equation based on the diagonalized coefficient matrix to obtain the closed-form solution of the FCTN factor in the current iteration round.

[0012] An image restoration method based on double-regularized FCTN decomposition according to at least one embodiment of the present invention performs a near-zero element pruning operation when updating the FCTN factor, including: calculating the row sum and column sum of the modulus expansion matrix of the FCTN factor in the current iteration round; if the row sum or column sum is lower than a preset sparsity threshold, then clearing the corresponding entire row or column elements to zero.

[0013] An image restoration method based on double-regularized FCTN decomposition according to at least one embodiment of the present invention alternately updates the gradient factor matrix, including: constructing a least squares problem based on the FCTN factor, difference matrix and preset regularization parameters of the current iteration; solving the least squares problem to obtain closed-form solutions of the updated two gradient factor matrices.

[0014] According to another aspect of the present invention, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs an image restoration method based on dual regularized FCTN decomposition according to any embodiment of the present invention.

[0015] According to another aspect of the present invention, a readable storage medium is provided, wherein execution instructions are stored therein, which, when executed by a processor, are used to implement the image restoration method based on double regularized FCTN decomposition according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements an image restoration method based on double-regularized FCTN decomposition according to any embodiment of the present invention.

[0017] The present invention has the following beneficial effects: (1) By jointly introducing gradient factor regularization term and structural sparsity regularization term under the FCTN decomposition framework to construct the image restoration model, the model can be constrained from the dual levels of factor representation ability and network topology, thereby comprehensively improving the accuracy and stability of image restoration under high missing rate.

[0018] (2) By extracting the gradient domain matrix for each FCTN factor using the difference matrix and decomposing it into the product of two gradient factor matrices with low rank and applying Frobenius norm penalty respectively, low rank constraints can be applied to the factor gradient domain to accurately characterize the local continuity of the image, thereby effectively restoring the fine details and complex textures of the image under high missing rate.

[0019] (3) By dividing the FCTN factors into multiple sub-tensor groups along the shared dimension through the variational approximation based on the Schatten-1 / 2 norm, and applying iterative weighted constraints on the sum of squares of their Frobenius norms, the FCTN factors can be subjected to sparse constraints to shrink near-zero elements to zero, thereby effectively eliminating redundant network parameters and significantly enhancing the robustness of the model to the selection of FCTN rank parameters. Attached Figure Description

[0020] The accompanying drawings illustrate exemplary embodiments of the invention and, together with the description thereof, serve to explain the principles of the invention. These drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification.

[0021] Figure 1 This is a flowchart illustrating an image restoration method based on dual-regularized FCTN decomposition according to an embodiment of the present invention.

[0022] Figure 2 This is a flowchart illustrating an FCTN factor alternation update method according to an embodiment of the present invention.

[0023] Figure 3 This is a flowchart illustrating an alternating update method for gradient factor matrix according to an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of a test image according to an embodiment of the present invention.

[0025] Figure 5This is a schematic diagram illustrating the PSNR index in a color image completion task using different methods according to an embodiment of the present invention.

[0026] Figure 6 This is a schematic diagram of the SSIM index in a color image completion task using different methods according to an embodiment of the present invention.

[0027] Figure 7 This is a schematic diagram of the reconstructed visual effect of a "castle" image when MR=30% according to one embodiment of the present invention.

[0028] Figure 8 This is a schematic diagram of the reconstructed visual effect of a "mushroom" image when MR=30% according to one embodiment of the present invention.

[0029] Figure 9 This is a schematic diagram of the reconstructed visual effect of a "human" image when MR=30% according to one embodiment of the present invention.

[0030] Figure 10 This is a schematic diagram illustrating the reconstructed visual effect of the letter graffiti "spire" according to one embodiment of the present invention.

[0031] Figure 11 This is a schematic diagram illustrating the reconstructed visual effect of an irregular graffiti "pottery" according to one embodiment of the present invention.

[0032] Figure 12 This is a schematic diagram illustrating the reconstructed visual effect of a water-patterned graffiti "water" according to an embodiment of the present invention.

[0033] Figure 13 This is a schematic diagram illustrating the quantitative analysis results of various graffiti removal applications according to one embodiment of the present invention.

[0034] Figure 14 This is a schematic diagram of the reconstructed visual effect of a multispectral image with MR=80% according to an embodiment of the present invention and the corresponding residual image.

[0035] Figure 15 This is a schematic diagram of the reconstructed visual effect of an MRI image with MR=80% and the corresponding residual image according to an embodiment of the present invention.

[0036] Figure 16 This is a schematic diagram of the PSNR curves for different color images at MR=90% according to various methods of one embodiment of the present invention.

[0037] Figure 17 This is a schematic diagram illustrating the impact of gradient factor regularization and structural sparsity regularization on model performance under different MR conditions for a "human" image according to an embodiment of the present invention.

[0038] Figure 18 This is a schematic structural block diagram of an electronic device employing a processor-based hardware implementation according to an embodiment of the present invention. Detailed Implementation

[0039] The present invention will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the accompanying drawings.

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other. The technical solution of this invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] Figure 1 A schematic diagram illustrating the overall flow of an image restoration method based on dual-regularized FCTN decomposition according to one embodiment of the present invention is shown. Figure 1 The method shown includes steps S110 to S140. This method can be executed by electronic devices such as mobile phones and tablets.

[0042] In step S110, the observation tensor of the image to be restored is obtained.

[0043] The observation tensor can be a multidimensional data representation of the image to be recovered.

[0044] In step S120, based on the FCTN decomposition, a gradient factor regularization term and a structural sparsity regularization term are introduced to obtain the image restoration model.

[0045] The gradient factor regularization term is used to perform low-rank decomposition of the gradient domain of the FCTN factor to constrain local continuity. The construction process of the gradient factor regularization term includes: for each FCTN factor, the difference matrix is ​​applied to the modulus expansion matrix of the FCTN factor to obtain the gradient domain matrix; the gradient domain matrix is ​​decomposed into the product of two gradient factor matrices; and Frobenius norm penalties are applied to the two gradient factor matrices respectively to construct the gradient factor regularization term.

[0046] The structural sparse regularization term is used to impose sparse constraints on the shared dimension among FCTN factors. The construction process of the structural sparse regularization term includes: dividing the FCTN factors into multiple sub-tensor groups along the shared dimension based on the variational approximation of the Schatten-1 / 2 norm; applying iterative weighted constraints on the sum of squared Frobenius norms of the sub-tensor groups to construct the structural sparse regularization term.

[0047] As one possible implementation, an iterative weighting constraint is applied to the sum of squares of the Frobenius norm of the sub-tensor set, including: in each iteration, calculating and updating the weight coefficients of the current iteration based on the Frobenius norm of the corresponding sub-tensor set in the previous iteration, wherein the weight coefficients are inversely proportional to the square root of the sum of squares of the Frobenius norm of the sub-tensor set.

[0048] In step S130, based on the observation tensor, the image to be restored is restored using an image restoration model to obtain the target tensor.

[0049] In step S140, the image restoration result of the image to be restored is obtained based on the target tensor.

[0050] This invention introduces gradient factor regularization and structural sparsity regularization terms within the FCTN decomposition framework, promoting the sparsity of the factor tensor from both the FCTN factor itself and the FCTN network structure. Simultaneously, it improves the robustness of rank selection by adaptively constructing near-zero sets.

[0051] The following section details the construction process of the image restoration model (FCTN-FDS).

[0052] Real-world images typically exhibit repetitive textures and structures, thus they are generally considered low-rank. FCTN decomposition effectively captures the global correlations of high-dimensional images, but when the missing rate is high, considering only global correlations does not yield satisfactory recovery results. To further improve the representational power of FCTN decomposition, this embodiment introduces two regularization terms: a gradient factor regularization term and a structural sparsity regularization term.

[0053] (1) Gradient factor regularization term.

[0054] The purpose of gradient factor regularization is to introduce the prior information of local continuity into the FCTN model, so as to enhance the effect of the FCTN model in restoring image details and textures when the missing rate is high.

[0055] Property 1: Let... express Target Tensor If the FCTN factor is greater than or equal to the FCTN factor, then the following inequality holds: (1) in, Indicates the index of the FCTN factor; N represents the total number of FCTN factors; Describes the rank of a matrix; Represents the target tensor The expansion matrix of modulus k; Indicates FCTN factor The modulus k expansion matrix; i represents the index of the FCTN factor; Indicates multiplication; Indicates FCTN factor With FCTN factor The size of the shared dimension between them; Indicates FCTN factor With FCTN factor The size of the shared dimension between them; if Then there is .

[0056] According to property 1, this embodiment can be achieved by assuming the modulus of the factor tensor. Expanding matrix This low-rank assumption is achieved using a low-rank domain. Furthermore, since the sparsity of the gradient domain is generally considered equivalent to the local continuity of the original domain, this embodiment specifies a regularization term in the gradient domain to constrain local continuity. The gradient operator consists of the following difference matrix. express: (2) in, Represents the set of real numbers; Representing the difference matrix The number of rows or columns.

[0057] Due to the difference matrix The rank is Therefore, we can obtain: (3) Then we have: (4) Inspired by the above analysis, this embodiment, within the FCTN decomposition framework, [details about the implementation]. Perform low-rank decomposition. Specifically, let... ,in , Both and represent the gradient factor matrix obtained after low-rank decomposition. Furthermore, in all experiments... Set all to ,in, Indicates the rank of the gradient factor decomposition; 'min' indicates that the value is much smaller than 'min'; 'max' indicates that the value is the minimum; 'max' indicates that the value is the maximum. From property 1, we know that... yes The upper bound of the rank. (By) Therefore, this embodiment is for Performing low-rank decomposition can further enhance The low rank property of the gradient factor matrix improves the robustness of FCTN rank selection. Furthermore, this embodiment addresses the gradient factor matrix... and Applying Thikhonoff regularization to facilitate the target tensor Local continuity.

[0058] (2) Sparse regularization term.

[0059] Structural sparsity regularization indirectly promotes the low-rank nature of the target tensor by facilitating the low-rank nature of the shared dimensions among the interaction factor expansion matrices. Furthermore, zeroing out groups can prevent information loss. In this embodiment, the sum of the squares of the Frobenius norms of the corresponding sub-factor tensors is called a group. The detailed derivation process is as follows.

[0060] Theorem 2: Given a... Non-zero singular values matrix There are two factor matrices and ,in , making The Schatten-1 / 2 norm is then It can be defined as ,satisfy: (5) And when Perform singular value decomposition , and When this condition is met, the equation holds true. Furthermore, the above can be generalized to the tensor version definition as follows: (6) in, This represents the given original matrix, which can correspond to the modulus expansion matrix of the target tensor or FCTN factor; Let represent the Schatten-1 / 2 norm; A represents the left factor matrix obtained by factoring matrix X; B represents the right factor matrix obtained by factoring matrix X. This represents the k-th column vector of the left factor matrix A; This represents the k-th row vector of the right factor matrix B; denoted by L2 norm; d denotes the intermediate dimension (rank) of the factorization; the superscript T denotes the transpose of the matrix; Let X represent the i-th non-zero singular value of matrix X; U and V represent the left orthogonal matrix and the right orthogonal matrix obtained after singular value decomposition of matrix X, respectively; Σ represents the singular value diagonal matrix obtained after singular value decomposition of matrix X, whose diagonal elements are... m represents the number of rows in matrix X; n represents the number of columns in matrix X; r represents the rank of matrix X (the total number of non-zero singular values ​​in matrix X). This represents the k-th subtensor obtained after tensor decomposition of matrix A; This represents the k-th subtensor obtained after tensor decomposition of matrix B. This represents multilinear operations between tensors.

[0061] Inspired by Theorem 2, this embodiment proposes a structural sparsity regularization for tensors under the FCTN decomposition framework, in the following form: (7) in, Represents a sparse structure regularization operator; Represents the i-th FCTN factor Along the first Victor Individual tensor; Represents the j-th FCTN factor Along the i-th dimension, the r-th subtensor. The summation of the first two corresponds to... One FCTN rank value, the rest Reflects and The structural sparsity of the expansion matrices of two FCTN factors. Minimizing the structural sparsity regularization term enhances this. and The low-rank nature of the contracted FCTN factor tensor results promotes the overall low-rank nature of the target tensor. Specifically, structural sparsity regularization leads to several rows or columns in the expanded factor matrix being close to zero. Therefore, structural sparsity regularization can be used to make the factor tensor low-rank without incurring the computational burden of singular value decomposition. Furthermore, pruning the corresponding near-zero rows and columns (collectively referred to as near-zero groups) can adaptively enhance the robustness of FCTN rank selection.

[0062] (3) Tensor completion model.

[0063] Gradient factor regularization and structural sparsity regularization operate in the orthogonal direction of the factor tensor expansion matrix, thus preventing the situation where two regularization terms influence each other and cause the loss of important information.

[0064] Assumption It is the target tensor The partial observations (i.e., the observation tensors obtained from the image to be restored) can be represented by the image restoration model proposed in this embodiment as follows: (8) Wherein, min represents taking the minimum value; Represent the target tensor; Indicates FCTN decomposition; Denotes the Frobenius norm; This represents the FCTN factor sequence obtained after performing FCTN decomposition on the target tensor. This represents the k-th FCTN factor obtained after FCTN decomposition; k, i, and j all represent the index of the FCTN factor; N represents the total number of FCTN factors. This represents the structural sparsity regularization parameter corresponding to the kth FCTN factor; Represents a sparse structure regularization operator; Represents the i-th FCTN factor Along the r-th subtensor of the j-th dimension; Represents the j-th FCTN factor Along the r-th subtensor of the i-th dimension; Represents the i-th FCTN factor Along the first The total number of subtensors of dimension; r represents the subtensor index; This represents the gradient factor regularization parameter corresponding to the k-th FCTN factor; Indicates constraints; Indicates FCTN factor The expansion matrix of modulus k; Represents the difference matrix; and They represent respectively to The gradient factor matrix obtained after low-rank decomposition. Represents the index set of observed elements; This represents the projection operator used to preserve the index set. The elements in the index set The elements in the complement set are set to zero; Represents the observation tensor; This indicates a summation operation.

[0065] In equation (8), the regularization term By We decompose the target tensor into smaller factor tensors to exploit its global low-rank property. For the gradient factor regularization term... The detailed explanation of how to enhance local continuity is as follows: Based on the equation achievable Therefore minimize and It can get smaller This is achieved by left-multiplying the factor tensor expansion matrix by a difference matrix. It can enhance Local continuity of each column. Furthermore, due to... Each column is A linear combination of columns, then enhances The local continuity of the column can promote Local continuity of columns. Structural sparsity regularization term. The low-rank structure of the precise target tensor is further determined by the sparsity among the joint constraint factor tensor networks.

[0066] By introducing penalty items Indicator functions ( (8) can be rewritten as the unconstrained problem shown in Equation 9: (9) in, This represents the penalty parameter corresponding to the k-th FCTN factor; This indicates the function that indicates the penalty term.

[0067] Sparse regularization terms The calculation involves radicals, so direct minimization is difficult. However, we can obtain the following from the inequality between the arithmetic mean and the geometric mean: (10) For any auxiliary variable ,when The equality holds. Therefore, this embodiment adopts an iterative weighted least squares strategy to solve the following model: (11) in, This represents the weight coefficients in the iterative weighting process.

[0068] As one possible implementation, the image restoration model can be solved using a proximal alternating minimization algorithm. The solution process includes: alternately updating the FCTN factor, gradient factor matrix, and target tensor, and performing near-zero element pruning operations when updating the FCTN factor, until the convergence condition is met.

[0069] Figure 2 A flowchart illustrating an embodiment of the FCTN factor alternation update method of the present invention is shown, as follows: Figure 2 The method shown includes steps S210 to S240.

[0070] In step S210, the update subproblem of the FCTN factor in the image restoration model is transformed into matrix form.

[0071] In step S220, the Sylvester matrix equation is constructed based on matrix form.

[0072] In step S230, the coefficient matrix in the Sylvester matrix equation is diagonalized using matrix eigenvalue decomposition and one-dimensional fast Fourier transform.

[0073] In step S240, based on the diagonalized coefficient matrix, the Sylvester matrix equation is solved to obtain the closed-form solution of the FCTN factor for the current iteration round.

[0074] As one possible implementation, a near-zero element pruning operation is performed when updating the FCTN factor, including: calculating the row sum and column sum of the modulus expansion matrix of the FCTN factor in the current iteration round. If the row sum or column sum is lower than a preset sparsity threshold, the corresponding entire row or column element is cleared to zero.

[0075] Figure 3 A flowchart illustrating an alternating update method for the gradient factor matrix according to one embodiment of the present invention is shown, as follows: Figure 3 The method shown includes steps S310 to S320.

[0076] In step S310, a least squares problem is constructed based on the FCTN factor, difference matrix, and preset regularization parameters of the current iteration round.

[0077] In step S320, the least squares problem is solved to obtain the closed-form solution of the updated two gradient factor matrices.

[0078] As one possible implementation, the convergence condition of the proximal alternating minimization algorithm is: the relative error between the target tensors obtained in two adjacent iterations is less than an error threshold. For example, the convergence condition can be expressed as: , and Let S represent the target tensors obtained in the s-th and s+1-th iterations, respectively. S represents the error threshold.

[0079] Simulation experiment To comprehensively evaluate the performance of the proposed FCTN-FDS in representative high-dimensional image restoration tasks, the following simulation experiments were conducted. Experiments were performed on color images, multispectral imaging (MSI) images, and magnetic resonance imaging (MRI) images. Random sampling was used to obtain incomplete observations. Typically, the sampling ratio (SR) is defined as the ratio of the number of known elements sampled. The competing methods used in this embodiment include: Tucker decomposition-based methods; t-SVD-based methods; TR decomposition-based methods; and FCTN decomposition-based methods. It is important to note that for third-order data, FCTN decomposition and TR decomposition are essentially the same; they have the same topological structure and therefore their performance is identical.

[0080] To objectively evaluate the performance of different methods, this embodiment selects Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) as quality metrics. Generally, the higher the values ​​of PSNR and SSIM, the better the quality of the recovered image.

[0081] In the method proposed in this embodiment, the parameters are set as follows: ,Will of Set to the same value 0.5, of Set to the same value 0.01, of Set the threshold to the same value of 1. Set to 0.001, near-end parameter Set it to 0.1.

[0082] 1. Color image completion This embodiment demonstrates the experimental results of six color images from three aspects: quantitative evaluation, qualitative evaluation, and application to graffiti removal. The test images are as follows: Figure 4 As shown. In the color image completion experiment, each image was adjusted to a spatial resolution of 256×256 pixels with three color channels (RGB). Each algorithm iterated 200 times. This embodiment uses PSNR and SSIM metrics to evaluate the quality of the restoration results.

[0083] (1) Color image restoration In the experiment on random missing color images, this example demonstrates the experimental results for three color images. The missing rates were set to 20%, 30%, 40%, 50%, 60%, 70%, 80%, and 90%. Figure 5 and Figure 6 The quantitative restoration results (PSNR and SSIM) of the restored color image are shown. From Figure 5 and Figure 6 The results show that the proposed FCTN-FDS achieves good recovery results for all images and all missing rates, is applicable to different scenarios, and has strong generalization ability. This quantitatively demonstrates that the proposed FCTN-FDS has a powerful representational ability in recovering these color images.

[0084] To make a better visual comparison, Figure 7 , Figure 8 and Figure 9The images show the reconstructed visual effects of the "castle" at MR=30%, the "mushroom" at MR=50%, and the "person" at MR=80%. It can be observed that the proposed FCTN-FDS can effectively preserve local details, textures, and image colors while restoring the global structure at different missing rates. Therefore, the proposed method demonstrates excellent reconstruction performance in visual restoration.

[0085] (2) Graffiti Removal Application This example demonstrates the experimental results of graffiti removal on three color images. Figure 10 , Figure 11 and Figure 12 The images showcase the reconstructed visual effects of three graffiti images: a "spire" (letter graffiti), a "pottery" (irregular graffiti), and a "water" (mesh graffiti). Experiments demonstrate that FCTN-FDS exhibits a significant advantage in subjective quality. Particularly in the "water" image, this method effectively recovers complex natural textures, avoiding the blurring issues common in other methods. In the "spire" image, it demonstrates the strongest edge preservation capability, resulting in the clearest and sharpest reconstructed architectural structure. Overall, the FCTN-FDS method achieves optimal visual consistency and detail recovery when processing images with complex structures and rich textures.

[0086] Figure 13 The quantitative restoration results (PSNR and SSIM) of the graffiti removal application on the restored color image are shown. Combining quantitative and qualitative analysis, the FCTN-FDS method proposed in this embodiment exhibits the best overall performance. Specifically, FCTN-FDS achieved the highest PSNR and SSIM in both the "Spire" and "Water" tests, demonstrating its powerful ability to restore fine details and natural textures. While FCTN-FDS did not achieve the highest PSNR and SSIM values ​​in the "Pottery" test, it still maintained high reconstruction quality. Overall, the FCTN-FDS method demonstrates excellent robustness and restoration performance in most scenarios.

[0087] (3) Multispectral image completion This embodiment evaluates the robustness of various methods under extreme conditions of 80% missing rate in multispectral images. The number of iterations is 100. Visual reconstruction results are analyzed. Figure 14The 3DLogTNN, FCTN-FDS, and FCTNFR methods all demonstrated excellent reconstruction capabilities, significantly outperforming other methods in restoring the transparency of the beer glass, color reproduction of the color chart, and overall structural integrity. For FCTN-FDS and FCTNFR, although their visual effects are similar, careful comparison of the residual images at the bottom reveals that FCTN-FDS exhibits fewer artifacts at the edges of the glass and in the details of the color chart. This indicates that the structural sparsity constraint introduced by FCTN-FDS plays a positive role in further optimizing detail fidelity, enabling more accurate recovery of subtle textures and spectral consistency. In conclusion, the FCTN-FDS method demonstrates reconstruction performance comparable to or even slightly better than FCTNFR in processing multispectral images with high missing rates, validating the effectiveness of this model in complex spectral image processing.

[0088] (4) Magnetic resonance imaging image completion This embodiment addresses the reconstruction task of MRI brain images with a high missing rate of 80%, selecting band 90 for visualization. The number of iterations was 100. By comparing the visual effects and residual images of different methods, the performance of the FCTN-FDS method in medical image restoration was systematically evaluated. Figure 15 Observations show that the 3DLogTNN, FCTN-FDS, and FCTNFR methods all exhibit a significant advantage in visual reconstruction, successfully restoring the fine structure of brain tissue, including the boundaries between gray and white matter, the texture of gyri and sulci, and the contours of the skull. The overall image is clear and free of obvious artifacts. In contrast, the Tmac method shows severe blocky artifacts and structural distortion in the missing areas. While the TNN method retains the general outline, details are blurred and there is significant noise residue. The METNN method exhibits over-smoothing within the brain tissue, leading to the loss of texture details. Although the TRLRF and TRGFR methods restore the structure to some extent, the edge sharpness is insufficient, and there is a slight color shift. Although the overall performance of FCTN-FDS and FCTNFR is very similar, with residual images remaining at extremely low levels, detailed comparison reveals that FCTN-FDS has less noise response and cleaner edge details in the textured areas within the brain tissue. In summary, FCTN-FDS demonstrates top-tier reconstruction capabilities comparable to FCTNFR and slightly superior in some detail preservation when processing medical images with high missing rates, validating the effectiveness and robustness of this model in the restoration of complex medical images.

[0089] (5) Convergence analysis Figure 16The PSNR convergence curves of various methods for different color images are shown under a 90% missing rate. Overall, all methods exhibit a convergence characteristic where the PSNR value initially rises rapidly and then stabilizes with increasing iterations. Among them, the FCTN-FDS method proposed in this embodiment demonstrates the fastest convergence speed and the highest final PSNR value on most test images, indicating stronger robustness and reconstruction capability when handling images with high missing rates. In contrast, while FCTNFR and FCTNGS also achieve high PSNR on some images, their convergence speed and final accuracy are slightly inferior to FCTN-FDS. Furthermore, the basic FCTN method performs only moderately on all images, further validating the importance of introducing structural sparsity constraints and gradient factor regularization for improving model performance. In conclusion, FCTN-FDS not only outperforms other methods in visual effects but also demonstrates significant advantages in quantitative metrics and convergence efficiency, proving its effectiveness and superiority in high missing rate image inpainting tasks.

[0090] (6) Ablation test To evaluate the different contributions of gradient factor regularization and structural sparsity regularization, this embodiment tests the performance of FCTN (baseline), FCTNGS (with structural sparsity regularization), FCTNFR (with gradient factor regularization), and FCTN-FDS (with gradient factor regularization and structural sparsity regularization).

[0091] exist Figure 17 The image shows the PSNR and SSIM histograms of a color image of a person under MR conditions of 70%, 80%, and 90%. Experimental results show that, for each data missing rate in the test image, the PSNR and SSIM values ​​of the three reconstruction methods are improved to varying degrees compared to the baseline algorithm FCTN. Taking PSNR as an example, FCTNGS shows a minimum improvement of 0.5071 dB and a maximum improvement of 1.3593 dB compared to the baseline algorithm FCTN. FCTNFR shows a minimum improvement of 3.8502 dB and a maximum improvement of 4.3401 dB compared to the baseline algorithm FCTN. FCTN-FDS shows a minimum improvement of 4.2137 dB and a maximum improvement of 4.5288 dB compared to the baseline algorithm FCTN. This indicates that introducing gradient factor regularization and structural sparsity regularization in the FCTN framework can effectively induce the sparsity of the factor tensor, thereby improving the robustness of rank selection and enabling it to recover finer details and textures. Furthermore, the performance of FCTN-FDS is improved compared to both FCTNFR and FCTNGS. Therefore, the combined effect of the two regularization terms is beneficial, proving that the proposed FCTN-FDS is effective.

[0092] According to a further embodiment of the present invention, an electronic device is also provided. Figure 18 This diagram illustrates a schematic block diagram of an electronic device employing a processor-based hardware implementation according to an embodiment of the present invention. The hardware structure of the electronic device of the present invention can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application and overall design constraints of the hardware. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc. Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one connecting line is used in this diagram, but this does not indicate that there is only one bus or one type of bus. The memory 1300 stores a computer program. When the processor 1200 executes the computer program, the processor 1200 is able to execute the image restoration method based on dual regularized FCTN decomposition described in the above embodiments of the present invention.

[0093] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection (electronic device) having one or more wires, a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0094] This invention also provides a computer program product. The method of this invention can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the processes or functions of this invention are performed wholly or partially.

[0095] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0096] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0101] Those skilled in the art should understand that the above embodiments are merely for illustrating the present invention and are not intended to limit the scope of the invention. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present invention.

Claims

1. An image restoration method based on double-regularized FCTN decomposition, characterized in that, include: Obtain the observation tensor of the image to be restored; Based on FCTN decomposition, gradient factor regularization term and structural sparsity regularization term are introduced to obtain image restoration model; Based on the observed tensor, the image to be restored is restored using the image restoration model to obtain the target tensor; The image restoration result of the image to be restored is obtained based on the target tensor; The gradient factor regularization term is used to perform low-rank decomposition in the gradient domain of the FCTN factor to constrain local continuity. The construction process of the gradient factor regularization term includes: for each FCTN factor, applying the difference matrix to the modulus expansion matrix of the FCTN factor to obtain the gradient domain matrix; decomposing the gradient domain matrix into the product of two gradient factor matrices; applying Frobenius norm penalties to the two gradient factor matrices respectively to construct the gradient factor regularization term. The structural sparse regularization term is used to impose sparse constraints on the shared dimension among FCTN factors. The construction process of the structural sparse regularization term includes: dividing the FCTN factors into multiple sub-tensor groups along the shared dimension based on the variational approximation of the Schatten-1 / 2 norm; applying iterative weighted constraints on the sum of squared Frobenius norms of the sub-tensor groups to construct the structural sparse regularization term.

2. The image restoration method based on double-regularized FCTN decomposition as described in claim 1, characterized in that, Iterative weighted constraints are imposed on the sum of squares of the Frobenius norm of the subtensor set, including: In each iteration, the weight coefficients of the current iteration are calculated and updated based on the Frobenius norm of the corresponding sub-tensor group in the previous iteration. The weight coefficients are inversely proportional to the square root of the sum of squares of the Frobenius norms of the sub-tensor group.

3. The image restoration method based on double-regularized FCTN decomposition as described in claim 1, characterized in that, The image restoration model is solved using a proximal alternating minimization algorithm. The solution process includes: alternatingly updating the FCTN factor, gradient factor matrix, and target tensor, and performing near-zero element pruning operations when updating the FCTN factor, until the convergence condition is met.

4. The image restoration method based on double-regularized FCTN decomposition as described in claim 3, characterized in that, The convergence condition is that the relative error between the target tensors obtained in two adjacent iterations is less than the error threshold.

5. The image restoration method based on double-regularized FCTN decomposition as described in claim 3, characterized in that, Alternate updates to the FCTN factor include: The update subproblem regarding the FCTN factor in the image restoration model is transformed into matrix form; Construct the Sylvester matrix equation based on the aforementioned matrix form; The coefficient matrix in the Sylvester matrix equation is diagonalized using matrix eigenvalue decomposition and one-dimensional fast Fourier transform. Based on the diagonalized coefficient matrix, the Sylvester matrix equation is solved to obtain the closed-form solution of the FCTN factor in the current iteration round.

6. The image restoration method based on double-regularized FCTN decomposition as described in claim 3, characterized in that, Perform near-zero element pruning when updating the FCTN factor, including: Calculate the row sum and column sum of the modulus expansion matrix of the FCTN factor in the current iteration round; If the sum of the rows or columns is lower than a preset sparse threshold, then the corresponding entire row or column elements will be cleared to zero.

7. The image restoration method based on double-regularized FCTN decomposition as described in claim 3, characterized in that, Alternately update the gradient factor matrix, including: Based on the FCTN factor, difference matrix, and preset regularization parameters of the current iteration, construct the least squares problem; Solve the least squares problem to obtain a closed-form solution for the updated two gradient factor matrices.

8. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the image restoration method based on double-regularized FCTN decomposition as described in any one of claims 1 to 7.

9. A readable storage medium, characterized in that, The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the image restoration method based on double-regularized FCTN decomposition as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image restoration method based on double regularized FCTN decomposition as described in any one of claims 1 to 7.

Images