A Visual Data Completion Method Based on the Fusion of Low-Rank Total Variation Depth Prior

Abstract

The present invention belongs to the technical field of tensor completion, and particularly relates to a visual data completion method based on the fusion of low-rank total variation depth prior. The present invention includes constructing a low-rank total variation depth prior fusion algorithm, inputting an observed tensor into the low-rank total variation depth prior fusion algorithm, and using the ADMM algorithm for the variables of the low-rank total variation depth prior fusion algorithm to output a restored tensor. By means of the conversion tensor nuclear norm based on the log norm, the weighted promotion scheme of 3DTV, and the implicit prior provided by a convolutional neural network trained using grayscale images, to simultaneously capture low-rank, piecewise smooth, and data-driven priors; selecting the conversion tensor nuclear norm based on unitary matrix transformation to mine the essential low-rank attributes of the tensor; introducing a non-convex log function to replace the l<subgt;1< / subgt; norm in the TNN to effectively slow down the deviation of the rank function, so as to construct a non-convex conversion tensor nuclear norm regular term based on unitary matrix transformation to provide a better approximation of the essential rank function.

Description

Technical Field

[0001] The present invention belongs to the technical field of tensor completion, and particularly relates to a visual data completion method based on the fusion of low-rank total variation depth prior. Background Art

[0002] With the rapid development of scientific computing, data in the real world can present richer information and be stored in multi-dimensional arrays, i.e., tensor structures. However, due to environmental limitations, such as the transmission bandwidth of devices or the imaging conditions for visual data acquisition, some pixels of the data may be lost. Therefore, tensor completion is a typical and important problem in the field of visual processing. An ideal property of this method is to restore visual objects without losing their original features, such as edge sharpness, local smoothness, corner preservation, and high contrast. Its related research is promoted by many applications, including pattern recognition, video surveillance, and medical analysis, etc. The key problem of tensor completion is to explore the prior knowledge of data, which can be roughly divided into two branches, namely explicit prior and depth prior. The former refers to the mathematical expressions from theory or practice, while the latter is mostly data-driven. In recent years, low-rank regularization optimization models have been widely applied in image processing and computer vision computing systems. Due to the close correlation of scattered pixels and the high correlation of the spectral profiles of the tensor space dimensions, multi-dimensional data can be naturally perceived as having low-rank properties. The tubal-rank based on tensor singular value decomposition (t-SVD) has been rapidly developed because it can accurately characterize multi-dimensional characteristics. t-SVD can be equivalently expressed as the matrix nuclear norm depending on transform domain transformation techniques, such as Fourier, framelet, unitary matrix, and discrete cosine transform (DCT), etc.

[0003] The low-rank property mainly targets the global structure of visual data and cannot capture local structure information. For local smoothness, the total variation (TV) regularization term is a good supplementary constraint to characterize phenomena such as spatial smoothness and spectral band inheritance. Currently, traditional TV imposes similar sparsity on all bands of the spectrum, spatial width, and height. That is to say, the sparsity of all bands of these gradient maps is implicitly assumed to be the same and independent, but this method deviates from the situation where the sparsity of gradient maps in different bands of real data is different.

[0004] In the tensor completion problem in a simple environment, good results have been achieved using carefully designed explicit priors and optimization algorithms. However, when faced with real applications in a complex environment, manually crafted regularization terms cannot effectively capture the intrinsic characteristics. Recently, deep-learning (DL)-based models, especially multi-layer convolutional neural networks (CNNs), have achieved significant performance improvements by incorporating data-driven priors. Using the variable splitting technique of the Plug-and-Play (PnP) framework, well-trained DL algorithms can be flexibly inserted and iteratively handle sub-problems. So far, although PnP provides a flexible way to connect explicit priors and implicit priors, only one manually crafted prior is not sufficient to reveal the potential properties. Summary of the Invention

[0005] To solve the tensor completion problem, the present invention proposes a visual data completion method based on the fusion of low-rank total variation deep prior. The discrete cosine transform is used to transform the tensor nuclear norm and enhance the three-dimensional total variation as explicit priors, and a non-convex scheme is proposed to better characterize the global low-rank correlation and local smoothness. A weighted scheme is adopted to balance the penalties for different elements. In addition, a data-driven denoiser is incorporated into the Plug-and-Play framework as an implicit prior to provide implicit information that cannot be captured by explicit priors. Finally, the alternating direction method of multipliers (ADMM) algorithm is used to optimize each variable.

[0006] To achieve the above object, the technical solution provided by the present invention is: a visual data completion method based on the fusion of low-rank total variation deep prior, including:

[0007] The visual data completion method based on the fusion of low-rank total variation deep prior includes:

[0008] Construct a low-rank total variation deep prior fusion algorithm, input the observed tensor into the low-rank total variation deep prior fusion algorithm, and the low-rank total variation deep prior fusion algorithm introduces three auxiliary variables and constraint conditions and The objective function is expressed by the following formula:

[0009]

[0010] where L β () represents the objective function, represents the restored tensor, and Λ n are Lagrange multipliers, and β i, where \(i = 1,\ldots,4\) are Lagrange multiplier coefficients, and \(\|\cdot\|\) LT denotes the non-convex transformed tensor nuclear norm based on the log function, \(n_1\) represents the width, \(n_2\) represents the height, and \(n_3\) represents the number of channels, denotes the enforced restored tensor and the observed tensor in the projection operator that keeps them equal in the index set \(\Omega\), and \(\langle\cdot,\cdot\rangle\) is the inner product, is the \(F\)-norm, \(T\) represents the matrix transpose, denotes the prior of local continuity, \(\lambda\) and \(\tau\) are two trade-off parameters, \(n = 1,2,3\), and \(\|\cdot\|\) 21 is the \(l\) 21 norm, and \(U\) n , \(V\) n are matrix factors of denotes the gradient of \(P\), and \(P\) represents the two-dimensional form of n \(U\) n \(V\) T n denotes the matrix product of \(U\)

[0011] The auxiliary variables \(y\), auxiliary variable auxiliary variable restored tensor matrix factor \(U\) n and matrix factor \(V\) n are optimized using the ADMM algorithm to update the Lagrangian parameters;

[0012] The restored tensor is output.

[0013] Furthermore, the construction of the low-rank total variation depth prior fusion algorithm includes:

[0014] Converting the observed tensor into a likelihood term and a regularization term;

[0015] Approximating the transformed tensor nuclear norm through a non-convex log function;

[0016] Introducing a non-convex \(l\) 21 norm to construct a non-convex enhanced three-dimensional total variation regularization term;

[0017] Inserting a CNN model in the PnP framework to obtain the low-rank total variation depth prior fusion algorithm.

[0018] Furthermore, the approximation of the transformed tensor nuclear norm through a non-convex log function includes:

[0019] Based on the transformed tensor nuclear norm, the log function is introduced to measure the singular values and serve as the non-convex relaxation of the rank function, which is expressed by the formula:

[0020]

[0021] where, represents the intermediate variable, is the j-th singular value of the i-th forward section plane, is the discrete cosine transform form of , ε > 0 is a constant;

[0022] The minimization model of the non-convex relaxation can be expressed as:

[0023]

[0024] where, ||·|| LT is the non-convex transformed tensor nuclear norm based on the log function, represents the projection operator that enforces and to be equal on the index set Ω.

[0025] Furthermore, the non-convex l 21 norm is introduced to construct a non-convex enhanced three-dimensional total variation regularization term, which is expressed by the formula as follows:

[0026]

[0027]

[0028] where, represents the non-convex enhanced three-dimensional total variation regularization term, r is the matrix factorization dimension, and I represents the identity matrix.

[0029] Furthermore, the ADMM algorithm is used to optimize the auxiliary variables auxiliary variable auxiliary variable restored tensor matrix factor U n and matrix factor V n , including:

[0030] Solving the auxiliary variable

[0031] Removing all irrelevant factors, The related sub-problem can be expressed as:

[0032]

[0033] Among them, denotes the i-th forward slice of denotes the non-convex transformed tensor nuclear norm of the i-th forward slice of t represents the number of iterations, fft() represents the Fourier operation,

[0034] Solving the above equation requires calculating each forward slice separately, and each forward slice has the following closed-form solution:

[0035]

[0036] By performing singular value decomposition on each forward slice U2 and V2 can be obtained, is a diagonal matrix, and the elements on the diagonal are denoted as and solving the operator E α,ε (·) has the following specific form:

[0037]

[0038] where c1 = |x| - ε, ε represents a constant, and finally performing the inverse Fourier operation on i.e.,

[0039] Solving the auxiliary variable

[0040] The related sub-problem is:

[0041]

[0042] where denotes the value of the auxiliary variable in the (t + 1)-th iteration of denotes the value of the restored tensor in the t-th iteration of denotes the value of the Lagrange multiplier in the t-th iteration of

[0043] Taking as the data to be restored, the spatial slice can be provided to the denoiser FFDNet for optimized solution, and the specific form is:

[0044]

[0045] where denotes the i-th forward slice of denotes the i-th forward section;

[0046] Solve the auxiliary variable

[0047] By extracting all terms containing from the objective function of the low-rank total variation depth prior fusion algorithm, it is necessary to solve:

[0048]

[0049] where, represents the value of the auxiliary variable at the (t + 1)-th iteration ; represents U n t and matrix product, U n t represents the value of the matrix factor U at the t-th iteration n ; represents the value of the matrix factor V at the t-th iteration n ; represents the value of the Lagrange multiplier Λ at the t-th iteration n ;

[0050] By performing the fast Fourier transform and convolution theorem on the equation, the closed-form solution of is:

[0051]

[0052] where, is an intermediate variable, ⊙ represents element-wise product, is the Fourier transform, represents the transpose transform, |·| 2 is the square in the element direction, fold(·) represents the convolution operation, represents the difference matrix, represents the inverse operator, 1 represents the tensor with all elements being 1;

[0053] Solve the restored tensor

[0054] By collecting all terms containing from the objective function of the low-rank total variation depth prior fusion algorithm, we can obtain:

[0055]

[0056] where, represents the value of the restored tensor at the (t + 1)-th iteration ; Denote the value of the auxiliary variable in the \((t + 1)\)-th iteration , Denote the value of the Lagrange multiplier in the \(t\)-th iteration , Denote the value of the Lagrange multiplier in the \(t\)-th iteration ;

[0057] Assume That is The sub-problem Closed-form solution:

[0058]

[0059] Where Denote the complement of \(\Omega\);

[0060] Solve for the matrix factor \(U\) n , \(n = 1, 2, 3\):

[0061] By removing the terms irrelevant to \(U\) from the objective function, the sub-problem of \(U\) n Can be formulated as: n

[0062]

[0063] Where Denote the value of the matrix factor \(U\) in the \((t + 1)\)-th iteration n , Denote the value of the variable in the \((t + 1)\)-th iteration ;

[0064] Since \(V\) n Is an orthogonal matrix, the above equation can be equivalently reformulated as:

[0065]

[0066] Assume the variable Its closed-form solution is:

[0067]

[0068] Where \(\|\cdot\|_2\) denotes the \(l_2\) norm;

[0069] Solve for the matrix factor \(V\) n , \(n = 1, 2, 3\):

[0070] By extracting all the terms containing \(V\) from the objective function of the low-rank total variation depth prior fusion algorithm, we need to solve: n

[0071]

[0072] Where​​ Denote the matrix factor V in the (t + 1)-th iteration n value;

[0073] Its closed-form solution is:

[0074]

[0075] where svd( ) represents singular value decomposition, D represents a diagonal matrix, and B, C represent unitary matrices.

[0076] Compared with the prior art, the significant advantages of the present invention are: the present invention respectively uses non-convex functions log, l 21 function to enhance the transformation of the tensor nuclear norm and the ability of three-dimensional total variation learning for low rank and local smoothness, and explores implicit priors in the CNN model under the PnP framework to construct a low-rank total variation deep prior fusion algorithm to achieve a low-rank total variation deep prior fusion strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 is a flowchart of a visual data completion method based on low-rank total variation deep prior fusion according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0078] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0079] As Figure 1 shown, a visual data completion method based on low-rank total variation deep prior fusion includes the following steps:

[0080] Step 1, input a damaged multi-spectral image where n1, n2, n3 are respectively the spatial dimensions and the number of channels of the multi-spectral image, n1 represents the width, n2 represents the height, and n3 represents the number of channels. From the perspective of the maximum a posteriori (MAP) estimation of Bayesian rules, the tensor completion problem can be divided into a likelihood term and multiple regularization terms. The former is the data fidelity, represented as a constraint of a projection operator to keep the observation positions consistent, especially the pixel points consistent with the observation results. The latter involves deeply mining the prior knowledge of real-world data. The basic mapping formula of tensor completion can be written as:

[0081]

[0082] where represents the likelihood term, Let the observed tensor and the restored tensor be such that the observed tensor is obtained by converting a damaged multi-spectral image, and the restored tensor can be converted to obtain the restored image. P(·) is the probability distribution function.

[0083] Assume that each term of the observation is independent, and the likelihood term is expressed as:

[0084]

[0085] where δ(·) represents the Dirac delta function. When performing the maximization operation, the estimated value is strictly consistent with the observed value in the index set Ω, and its form of the Dirac delta function can be defined as:

[0086]

[0087] where, represents the Dirac delta function based on the index set Ω, represents the observed tensor and the restored tensor based on the index set Ω.

[0088] Due to the logarithmic invariance of δ(x), the likelihood term can be reformulated as The minimization problem of the proposed objective function, which is the opposite of the maximization problem of the posterior probability, can be expressed as:

[0089]

[0090] where, is the projection operator that enforces and to be equal in the index set Ω. To obtain a comprehensive constraint and characterize the intrinsic information of the underlying data, in this embodiment, by incorporating tensor low-rankness, local continuity, and data-driven priors into the PnP framework, it focuses on the fusion of explicit priors and implicit priors. Following the MAP framework, the constraint group can be expressed as:

[0091]

[0092] where, represents finding the probability distribution function of , respectively represent tensor low-rankness, local continuity, and data-driven priors, and λ and τ are two trade-off parameters.

[0093] Step 2, Convert the tensor nuclear norm as a convex relaxation of the tensor fiber rank. Use the l1 norm to measure the non-zero singular values. Due to the pursuit of convexity, an approach of equally treating each singular value is adopted. However, this strategy usually obtains the global optimal solution of the algorithm at the cost of losing feature information. To solve the above problems, a log function is introduced based on TTNN to measure the singular values and serve as a non-convex relaxation of the rank function, and its definition is as follows:

[0094]

[0095] where, represents the intermediate variable, is the j-th singular value of the i-th forward slice, is in the form of the discrete cosine transform, and ε > 0 is an extremely small constant. As a linear function, the value of the l1 norm will grow infinitely as the singular value increases, resulting in a gradual deviation from the essential rank function. The log function can slow down this effect, and its function value gradually becomes stable as the singular value increases. Therefore, it can be closer to the essential l0 rank constraint. Based on the above analysis, for a three-dimensional tensor the minimization model of the proposed non-convex relaxation can be expressed as:

[0096]

[0097] where, ||·|| LT is the non-convex transformed tensor nuclear norm based on the log function.

[0098] Since the first requirement for low-rank tensor recovery is low-rank guarantee, and appropriate unitary matrix transformation has deeper low-rank properties, this patent selects the transformed tensor nuclear norm based on unitary matrix transformation to explore the essential low-rank properties of the tensor. In addition, a non-convex log function is introduced to replace the l1 norm in TNN to effectively slow down the deviation of the rank function, so as to construct a non-convex transformed tensor nuclear norm regular term based on unitary matrix transformation and provide a better approximation of the essential rank function.

[0099] Step 3, The three-dimensional total variation (3DTV) reflects the profound configuration (i.e., sparsity feature) of the natural image gradient map. It naturally encodes the correlation between all these channels and has obtained excellent results in the field of image processing. However, due to the excessive penalty of the l1 norm on large coefficients and the different intensity changes in the smooth area under the three unfolding modes, it may not be able to recover the edges and boundaries of the real image. In addition, tensor data all have group sparsity and images have shared sparsity in two spatial dimensions. Therefore, l is introduced into the 3DTV model 21The norm is used to construct a non-convex enhanced 3DTV regularization term (Non-convertE3DTV, NonE3DTV) to preserve the edge structure under different smooth backgrounds and explore the group sparsity characteristics. Its specific form is as follows:

[0100]

[0101] Among them, represents the non-convex enhanced three-dimensional total variation regularization term, and U n , V n are the matrix factors of . represents the gradient of P, and P represents the two-dimensional form of . r is the matrix decomposition dimension, I represents the identity matrix, and n represents the dimension.

[0102] Step 4: As described in Steps 1-3, by deeply exploring the physical meaning between the singular values of each frontal slice and the three-dimensional subspace gradient domain, TTNN and NonE3DTV based on the log and l 21 functions are respectively proposed. Although these handcrafted regularizers have good interpretability and solid theoretical support, they cannot fully and satisfactorily represent complex image structures. As a supplement to the non-convex transformed tensor nuclear norm regularization term in parallel Step 2 and the non-convex enhanced 3DTV regularization term in Step 3, in this embodiment, a flexible and fast CNN model (FFDNet is used in this embodiment) is inserted in the PnP framework to explore implicit priors, and a low-rank total variation deep prior fusion algorithm is constructed to simultaneously encode the global and potentially local relevant general tensors. Therefore, the objective function of the low-rank total variation deep prior fusion algorithm (low-rank, total variation, and deep prior for low-rank tensor completion, LRTVDP) is:

[0103]

[0104] Among them, λ and τ are two trade-off parameters, represents the prior of local continuity, and ||·|| 21 is the l 21 norm.

[0105] Step 5: The classical ADMM optimization is used to solve each unknown variable in the algorithm of Step 4 respectively. First, by introducing three auxiliary variables and the constraint conditions and , Equation (9) is rewritten in the form of an augmented Lagrangian function:

[0106]

[0107] Among them, L β () represents the objective function, and Λ n are Lagrange multipliers, and β i , i = 1, …, 4 are Lagrange multiplier coefficients, <·,·> is the inner product, and ||·|| is the F-norm.

[0108] Step 5.1, solve the sub-problem

[0109] Remove all irrelevant factors, The relevant sub-problem can be expressed as:

[0110]

[0111] Among them, represents the i-th forward section of represents the non-convex transformed tensor norm of the i-th forward section of t represents the iteration number, fft() represents the Fourier operation, Therefore, to solve equation (11), it is necessary to calculate each of its forward sections separately, and each of its forward sections has the following closed-form solution:

[0112]

[0113] By performing singular value decomposition on each forward section U2 and V2 can be obtained, is a diagonal matrix, and the elements on the diagonal are represented as and solve the operator E α,ε (·) in the specific form of:

[0114] <A

[0115] Among them, E α,ε (·) is the general formula for solving the operator, x is ∑2(i,i), α is 1 / β1, c1 = |x| - ε, ε represents a constant, and finally perform the inverse Fourier operation on i.e.,

[0116] Step 5.2, solve the sub-problem

[0117] The relevant sub-problem is:

[0118]

[0119] Among them, represents the value of the iteration variable at the (t + 1)-th round , represents the value of the restored tensor at the t-th round of iteration , represents the value of the Lagrange multiplier at the t-th round of iteration .

[0120] Based on the plug-and-play strategy, this embodiment adopts a well-trained FFDNet, a flexible and fast CNN-based method, as the deep denoising engine to represent implicit prior features. Taking as the data to be restored, its spatial slices can be provided to the denoiser FFDNet for optimal solution, and the specific form is:

[0121]

[0122] Among them, represents the i-th forward section of represents the i-th forward section of

[0123] Step 5.3, solve the sub-problem

[0124] By extracting all the terms containing from formula (10), it is necessary to solve:

[0125]

[0126] Among them, represents the value of the auxiliary variable at the (t + 1)-th round of iteration , U n t (V n t ) T represents the matrix product of U n t and , represents the value of the Lagrange multiplier Λ n at the t-th round of iteration.

[0127] By performing the fast Fourier transform and convolution theorem on the equation, the closed-form solution of

[0128]

[0129] Among them, is an intermediate variable, and ⊙ represents element-wise multiplication, is the Fourier transform, represents the transpose transform, |·| 2 is the square in the element direction, fold(·) represents the convolution operation, represents the difference matrix, represents the inverse operator, 1 represents the tensor with all elements being 1.

[0130] Step 5.4, sub - problem

[0131] By collecting all terms containing from formula (10), we can obtain:

[0132]

[0133] where, represents the value of the restored tensor at the (t + 1)-th iteration , represents the value of the auxiliary variable at the (t + 1)-th iteration , represents the value of the Lagrange multiplier at the t-th iteration , represents the value of the Lagrange multiplier at the t-th iteration .

[0134] Assume that is sub - problem has a closed - form solution:

[0135]

[0136] where, represents the complement of Ω.

[0137] Step 5.5, U n , n = 1, 2, 3 sub - problem

[0138] By removing the terms unrelated to U n , the sub - problem of U n can be expressed as:

[0139]

[0140] where, represents the value of the matrix factor U at the (t + 1)-th iteration n , represents the value of the variable at the (t + 1)-th iteration .

[0141] Since V n is an orthogonal matrix, so there is Then formula (20) is equivalently reformulated as:

[0142]

[0143] Assume variable Its closed-form solution is:

[0144]

[0145] where, || ||2 represents the l2 norm.

[0146] Step 5.6, for V n , n = 1, 2, 3 sub-problem

[0147] By extracting all terms containing V from formula (10) n then it is necessary to solve:

[0148]

[0149] where, represents the value of the matrix factor V at the (t + 1)-th iteration n value.

[0150] Its closed solution is:

[0151]

[0152] where, svd( ) represents singular value decomposition, D represents a diagonal matrix, and B, C represent unitary matrices.

[0153] Step 5.7, update the Lagrange parameter. According to the principle of general ADMM, the update rule of the Lagrange multiplier is:

[0154]

[0155] where, represents the value of the Lagrange multiplier at the (t + 1)-th iteration value, represents the value of the Lagrange multiplier at the (t + 1)-th iteration value, represents the value of the Lagrange multiplier at the (t + 1)-th iteration value, and μ represents the step size.

[0156] Summarize the above algorithm as follows:

[0157] Input the observed tensor Parameter initial values β1, β2, β3, β4; τ, μ, σ and the maximum number of iterations t max ; the index set of the observed term Ω.

[0158] Initialize the hypothesis and Λ n is the zero tensor, and the current iteration number is t = 0; is an auxiliary variable.

[0159] 1. t ← t + 1;

[0160] 2. Update according to (12)

[0161] 3. Update according to (15)

[0162] 4. Update according to (17)

[0163] 5. Update U n , V n ;

[0164] 6. Update the Lagrange multipliers according to (25) and Λ n ;

[0165] until convergence;

[0166] Output the restored tensor

[0167] Step 6: Output the repaired image.

[0168] The present invention establishes a fusion model that includes explicit priors such as low-rank and local smoothness priors and an implicit prior provided by deep learning, named the low-rank total variation deep prior fusion algorithm (LRTVDP). Specifically, through the transformed tubal nuclear norm (TTNN) based on the log-norm, the weighted lifting scheme of 3DTV, and the implicit prior provided by a convolutional neural network trained using grayscale images, it simultaneously captures low-rank, piecewise smooth, and data-driven priors.

[0169] The above-described embodiments only represent one or several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. A visual data completion method based on the fusion of low-rank total variation depth prior, characterized in that, The visual data completion method based on low-rank total variation depth prior fusion includes: Construct a low-rank total variation depth prior fusion algorithm, input the observed tensor into the low-rank total variation depth prior fusion algorithm, and the low-rank total variation depth prior fusion algorithm introduces three auxiliary variables and constraint conditions and The objective function is expressed by the following formula: Among them, L β () represents the objective function, represents the restored tensor, and Λ n are Lagrange multipliers, and β i , i = 1, …, 4 are Lagrange multiplier coefficients, ||·|| LT represents the non-convex transformed tensor nuclear norm based on the log function, n1 represents the width, n2 represents the height, and n3 represents the number of channels, represents enforcing the restored tensor and the observed tensor to be equal in the index set Ω, <·, ·> is the inner product, is the F-norm, T represents the matrix transpose, represents the prior of local continuity, λ and τ are two trade-off parameters, n = 1, 2, 3, ||·|| 21 is the l 21 norm, and U n , V n are matrix factors of, represents the gradient of P, and P represents in two-dimensional form, represents the matrix product of U n and , where n represents the dimension; Use the ADMM algorithm to optimize the auxiliary variables of the low-rank total variation depth prior fusion algorithm Auxiliary variable Auxiliary variable Restored tensor Matrix factor U n And matrix factor V n Perform optimization and update the Lagrangian parameter; Output the restored tensor.

2. The visual data completion method based on low-rank total variation depth prior fusion according to claim 1, wherein The construction of the low-rank total variation depth prior fusion algorithm includes: Convert the observed tensor into a likelihood term and a regularization term; Approximate the nuclear norm of the transformed tensor through a non-convex log function; Introduce non-convex l 21 norm, and construct a non-convex enhanced three-dimensional total variation regularization term; Insert a CNN model in the PnP framework to obtain the low-rank total variation depth prior fusion algorithm.

3. The visual data completion method based on low-rank total variation depth prior fusion according to claim 2, wherein The approximation of the nuclear norm of the transformed tensor through a non-convex log function includes: Introduce a log function on the basis of the nuclear norm of the transformed tensor to measure the singular value and use it as a non-convex relaxation of the rank function, which is expressed by the formula: Among them, represents an intermediate variable, is the j-th singular value of the i-th forward section, is the discrete cosine transform form of, where ε > 0 is a constant; The minimization model of the non-convex relaxation can be expressed as: Among them, ||·|| LT is the non-convex transformed tensor nuclear norm based on the log function, represents the enforcement of and the projection operator that remains equal in the index set Ω.

4. The visual data completion method based on low-rank total variation depth prior fusion according to claim 2, wherein Introduce the non-convex l 21 norm to construct a non-convex enhanced three-dimensional total variation regularization term, which is expressed by the following formula: Among them, represents the non-convex enhanced three-dimensional total variation regularization term, r is the matrix decomposition dimension, and I represents the identity matrix.

5. The visual data completion method based on low-rank total variation depth prior fusion according to claim 1, characterized in that, The auxiliary variables of the low-rank total variation depth prior fusion algorithm using the ADMM algorithm Auxiliary variable Auxiliary variable Restored tensor Matrix factor U n and matrix factor V n are optimized, including: Solve for auxiliary variables Remove all irrelevant factors, The relevant sub-problems can be expressed as: Among them, represents the i-th forward section of represents the non-convex transformed tensor nuclear norm of the i-th forward section of t represents the number of iterations, fft() represents the Fourier operation, To solve the above formula, each forward slice needs to be calculated separately, and each forward slice has the following closed-form solution: By performing singular value decomposition on each forward section U2 and V2 can be obtained. is a diagonal matrix, and the elements on the diagonal are represented as And solve the operator E α,ε (·) has the specific form of: where c1 = |x| - ε, ε represents a constant, and finally perform the inverse Fourier transform, that is Solve for auxiliary variables The related sub-questions are as follows: Among them, represents the value of the auxiliary variable in the (t + 1)-th round of iteration , represents the value of the restored tensor in the t-th round of iteration , represents the value of the Lagrange multiplier in the t-th round of iteration ; Taking as the data to be restored, the spatial slices can be provided to the denoiser FFDNet for optimizing the solution, in the specific form of: Among them, represents the i-th forward section of represents the i-th forward section of Solve for auxiliary variables By extracting all the terms containing from the objective function of the low-rank total variation depth prior fusion algorithm, it is necessary to solve: Among them, represents the value of the auxiliary variable in the (t + 1)-th iteration ; represents the matrix product of U n t and , where U n t represents the value of the matrix factor U in the t-th iteration n ; represents the value of the matrix factor V in the t-th iteration n ; represents the value of the Lagrange multiplier Λ in the t-th iteration n ; By performing a fast Fourier transform and convolution theorem on the equation, The closed-form solution is: Among them, is an intermediate variable, and ⊙ represents element-wise multiplication, is the Fourier transform, represents the transpose transform, |·| 2 is the square in the element direction, and fold(·) represents the convolution operation, represents the difference matrix, represents the inverse operator, and 1 represents the tensor with all elements being 1; Solve for the restored tensor By collecting all the terms containing from the objective function of the low-rank total variation depth prior fusion algorithm, we can obtain: Among them, represents the value of the restored tensor in the (t + 1)-th iteration , represents the value of the auxiliary variable in the (t + 1)-th iteration , represents the value of the Lagrange multiplier in the t-th iteration , represents the value of the Lagrange multiplier in the t-th iteration ; Hypothesis That is Sub - problem Closed - form solution of: Among them, represents the complement set of Ω; Solve for matrix factor U n , n = 1, 2, 3: By removing the terms unrelated to U from the objective function, n the sub-problem of U n can be formulated as: Among them, represents the value of the matrix factor U in the (t + 1)-th round of iteration n ; represents the value of the iterative variable in the (t + 1)-th round of iteration; Since V n is an orthogonal matrix, the above equation can be equivalently reformulated as: Assume variables Its closed-form solution is as follows: Among them, || ||2 represents the l2 norm; Solve for matrix factor V n , n = 1, 2, 3: By extracting all terms containing V from the objective function of the low-rank total variation depth prior fusion algorithm, it is necessary to solve: n as follows: Among them, represents the value of the matrix factor V in the (t + 1)-th round of iteration n ; Its closed-form solution is: Among them, svd() represents singular value decomposition, D represents a diagonal matrix, and B and C represent unitary matrices.

Images