A Low-Rank Tensor Completion Method Based on Total Variation Regularization
By combining total variational regularization and the Schatten-p norm of the transform tensor into a low-rank tensor completion model, the problem of ignoring the local smoothness of tensors in existing technologies is solved, and better image restoration results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2023-03-02
- Publication Date
- 2026-07-31
AI Technical Summary
Existing low-rank tensor completion methods ignore the local smoothness of tensors along spatial and tubular dimensions when restoring images, resulting in poor image restoration performance.
By combining total variational regularization and the Schatten-p norm of the transformation tensor, a low-rank tensor completion model is constructed, and the alternating direction multiplier method is used for optimization to preserve the low-rank and local smoothness of the tensor.
It improves the image restoration effect, especially under low sampling rate conditions, restoring local details and overall structure of the image, and improving peak signal-to-noise ratio and structural similarity.
Smart Images
Figure CN116245761B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image processing, and in particular to a low-rank tensor completion method based on total variational regularization. Background Technology
[0002] The rapid development of computer technology has generated an increasing amount of multidimensional data, which, such as color images, is now often stored in the form of tensors. Tensors, with their multidimensional structure and inherent properties, are widely used in computer vision and image processing. However, in the real world, tensor data often contains missing values due to various reasons, so how to fill in these missing values for image restoration is a problem that needs to be solved.
[0003] Low-rank tensor completion is one of the most commonly used methods for completing missing values in images. This method utilizes the property that the underlying tensor is low-rank to construct a model that minimizes the tensor rank function. However, minimizing the tensor rank function is an NP-hard problem, so researchers use convex or non-convex substitution functions to replace the tensor rank function for solving the problem. The classic convex substitution tensor kernel norm model has a strong penalty for large singular values that represent the main information of the image, and the tensor kernel norm is not the tightest envelope for estimating the tensor rank function. The recovery effect of this classic low-rank tensor completion model is not optimal. Moreover, many current low-rank tensor completion studies are based on tensor kernel norms, such as transform tube kernel norms and tensor kernel norms based on nonlinear transforms.
[0004] However, the existing low-rank tensor completion methods mentioned above have shortcomings. They are based only on the global low-rank characteristics of tensors and ignore the local smoothness of tensors along spatial and tube dimensions, resulting in the loss of local smoothness information in the restored image. Ultimately, the image restoration effect needs to be further improved. Summary of the Invention
[0005] Purpose of the invention: The purpose of this invention is to provide a low-rank tensor completion method based on total variational regularization that can protect the local smoothness of tensors along spatial and tube dimensions while satisfying global low-rank properties, thereby improving the effect and performance of low-rank tensor completion.
[0006] Technical solution: The low-rank tensor completion method of the present invention includes the following steps:
[0007] S1. The low-rank property of tensors is characterized by the Schatten-p norm of the transformation tensor. The total variational regularization and the Schatten-p norm of the transformation tensor are combined to construct a low-rank tensor completion model.
[0008] S2 introduces auxiliary variables and uses the framework of the alternating direction multiplier method to construct the augmented Lagrangian function of the low-rank tensor completion model. It solves multiple subproblems separately and iterates until the convergence condition is met, finally obtaining the completed tensor.
[0009] Furthermore, in step S1, the total variational regularization and the Schatten-p norm of the transformation tensor are combined to construct the following expression for the low-rank tensor completion model:
[0010]
[0011]
[0012] in, This represents the tensor that needs to be completed. Tensor representing observation, tensor to be completed and the tensor of observation The size of each is n1×n2×n3; Ω represents the observation set. Tensor to be completed and the tensor of observation The element values are equal in the observation set; It transforms the Schatten-p norm of the tensor to characterize the tensor to be completed. The low-rank nature; It is the total variational regularization term; α is the regularization parameter;
[0013] Simultaneously, based on the definition of total variation, the tensor to be completed is calculated. Summing the differences between adjacent elements along each of the three dimensions:
[0014]
[0015]
[0016] Where F represents the total difference operator between adjacent elements, which is decomposed into difference operators F along three different dimensions. h F v and F t F = [F h ,F v ,F t ]); Tensor to be completed The solution is obtained by integral Φ, which is then divided into n subtensors u. i ;0 <p<1,p i >0 and
[0017] Furthermore, in step S2, the implementation process of constructing the augmented Lagrangian function of the low-rank tensor completion model is as follows:
[0018] Introducing three auxiliary variable tensors and Tensors to be completed The formula for summing the differences of adjacent elements along each of the three dimensions can be transformed into the following formula:
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
[0025] Where, F = [F h ,F v ,F t ];0 <p<1,p i >0 and i = 1, 2, ..., n;
[0026] The expression for the augmented Lagrange function is as follows:
[0027]
[0028] in, and is a Lagrange multiplier, i = 1, 2, ..., n; μ is a penalty parameter.
[0029] Furthermore, the first tensor is updated. The subproblem can be represented as:
[0030]
[0031] in and
[0032] Based on the definition of the Φ product and the unitary invariance of the Frobenius norm, the above equation is further transformed into:
[0033]
[0034] With tensor Each forward slice If we consider it as a variable, then we have:
[0035]
[0036] Next, solve each matrix using matrix differentiation. Then the matrix Combining tensors along the tube dimension Then on the tensor Expand the matrix U according to the third mode, then multiply the left side of matrix U by the transpose of the transformation matrix Φ, where the size of the transformation matrix Φ is n³ × n³; then perform the inverse operation of the third mode expansion of (ΦU) to obtain
[0037] Furthermore, the second updated tensor The subproblem is represented as:
[0038]
[0039] According to the definition of the Schatten-p norm of the transformation tensor, we have:
[0040]
[0041] Solve
[0042]
[0043] Among them, prox λf (·) is an approximation operator, defined as follows:
[0044] The matrix obtained by solving Combining tensors along the tube dimension Then on the tensor Expanded into a matrix according to the third mode Then in the matrix Multiply the left side by the transpose of the transformation matrix Φ, where the size of the transformation matrix Φ is n³ × n³; then... The inverse operation of the third mode expansion yields
[0045] Furthermore, the third update tensor The subproblem is represented as:
[0046]
[0047] Let tensor Solving tensors
[0048]
[0049] Where sign(·) is the sign function, |·| is the absolute value of the elements, and ° represents the product between elements.
[0050] Furthermore, the fourth update tensor The subproblem is represented as:
[0051]
[0052] Based on the properties of the Frobenius norm, the solution can be obtained by differentiation. The differentiated expression is as follows:
[0053]
[0054] Among them, F * Denotes the adjoint operator of F, Solving using FFT3 and iFFT3:
[0055]
[0056] in, This represents a tensor whose elements are all 1.
[0057] Furthermore, the tensor to be completed The subproblem is represented as:
[0058]
[0059] Based on the properties of the Frobenius norm, by differentiating and solving the subproblem, the final recovered tensor to be completed can be calculated.
[0060] Update the Lagrange multipliers according to the following formula. and y n+3 k+1 :
[0061]
[0062]
[0063]
[0064]
[0065] Each iteration updates every variable, and the iteration stops when the maximum number of iterations is reached. The output is the final completed tensor.
[0066] Alternatively, the iteration can stop and the final completed tensor can be output when the following condition is met.
[0067] max(Con1,Con2,Con3,Con4)<∈
[0068] in,
[0069]
[0070] ∈ is the set threshold.
[0071] Compared with the prior art, the significant advantages of this invention are as follows:
[0072] 1. For characterizing the low rank of tensors, the commonly used convexity is no longer used to replace the tensor kernel norm. Instead, the Schatten-p norm of the transformed tensor is used. This norm has a lower penalty for large singular values than the tensor kernel norm and can estimate the tensor rank more tightly than the tensor kernel norm.
[0073] 2. Existing low-rank tensor completion methods only consider the low-rank property of tensors and ignore the local smoothness of tensors along spatial and tube dimensions. Total variation is a suitable tool for studying the local smoothness of tensors. This invention combines total variation regularization and the Schatten-p norm of the transformed tensor to propose a low-rank tensor completion model, which enables the recovered tensor to further preserve the local smoothness of tensors along spatial and tube dimensions while protecting the low-rank property.
[0074] 3. The low-rank tensor completion model proposed in this invention is optimized and solved using the alternating direction multiplier method. Compared with existing classical methods, the tensor completion effect obtained by this invention is further improved. Attached Figure Description
[0075] Figure 1 This is a flowchart of the present invention;
[0076] Figure 2(a) is the original color image used in the embodiment of the present invention;
[0077] Figure 2(b) is an observed image of a color image with a sampling rate of 15% in an embodiment of the present invention;
[0078] Figure 2(c) shows when Transform Φ is a discrete Fourier transform. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the figure.
[0079] Figure 2(d) shows when The transformation Φ is a discrete cosine transform. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the figure.
[0080] Figure 2(e) is when Transformation Φ is a data-based transformation. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the image.
[0081] Figure 2(f) shows when Transform Φ is a discrete Fourier transform. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the figure.
[0082] Figure 2(g) shows when The transformation Φ is a discrete cosine transform. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the figure.
[0083] Figure 2(h) is when Transformation Φ is a data-based transformation. The restoration result of a color image with a sampling rate of 15% in this embodiment of the invention is shown in the figure. Detailed Implementation
[0084] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0085] To overcome the drawbacks of convex substitution tensor kernel norms, this invention selects the Schatten-p norm of the transformed tensor to better study the low-rank property of the tensor.
[0086] Advantages of the Schatten-p norm of the transforming tensor: it penalizes larger singular values less severely than the tensor kernel norm; and it estimates the tensor rank more tightly than the tensor kernel norm.
[0087] Furthermore, by combining total variational regularization along the spatial and tube dimensions of the tensor with the Schatten-p norm of the transformed tensor, the local smoothness of the tensor along the spatial and tube dimensions can be better preserved while satisfying the global low-rank property, thereby improving the effect and performance of low-rank tensor completion.
[0088] Total variation has the ability to preserve the local smoothness of tensors, that is, to calculate the difference between adjacent elements along the spatial and tube dimensions of the tensor and then sum them. This total variation regularization is combined with the Schatten-p norm of the transformed tensor to construct a new mathematical model. Finally, it is transformed into a minimization problem of the model. The optimization solution is performed using the framework of the alternating direction multiplier method, and finally the completed tensor is obtained.
[0089] This invention provides a low-rank tensor completion method based on total variational regularization, the flowchart of which is shown below. Figure 1 As shown, the specific steps include the following:
[0090] Step 1: Characterize the low-rank property of the tensor using the Schatten-p norm of the transformation tensor;
[0091] To further preserve the local smoothness of the tensor along the spatial and tube dimensions while maintaining its low-rank property, total variational regularization and the Schatten-p norm of the transformation tensor are combined to construct a new low-rank tensor completion model. The construction process of the low-rank tensor completion model is as follows:
[0092]
[0093]
[0094] in, This represents the tensor that needs to be completed. Tensor representing observation, tensor to be completed and the tensor of observation The size of each tensor is n1×n2×n3 (the tensor to be completed is the tensor that needs to be solved to obtain the final completed tensor; the observed tensor is the tensor containing missing values, which is the tensor from the initial input to the solution method); Ω represents the observation set, Tensor to be completed and the tensor of observation The element values are equal in the observation set; It transforms the Schatten-p norm of the tensor to characterize the tensor to be completed. The low-rank nature; It is a total variational regularization term to protect the tensor to be supplemented. Local smoothness along the spatial and tubular dimensions; α is the regularization parameter.
[0095] According to the definition of the Schatten-p norm of the transformation tensor, the tensor to be completed can be... The Schatten-p norm of the transformation tensor is decomposed into the sum of the weights of the Schatten-p norms of the transformation tensors of multiple subtensors; simultaneously, according to the definition of total variation, the tensor to be completed is calculated. By summing the differences between adjacent elements along the three dimensions, formula (1) becomes formula (2):
[0096]
[0097]
[0098]
[0099] Here, F represents the total difference operator between adjacent elements, which can be decomposed into difference operators F along three different dimensions. h F v and F t (i.e., F = [F]) h ,F v ,F t ]); Tensor It can be solved into n subtensors by integral Φ. i = 1, 2, ..., n; 0 <p<1,p i >0 and
[0100] Step 2: To obtain the optimized solution of the model, three auxiliary variables are introduced. Using the Alternating Direction Method of Multipliers (ADMM) framework, the augmented Lagrangian function of the proposed low-rank tensor completion model is constructed. Multiple sub-problems are solved, iterating until the convergence condition is met, ultimately yielding the completed tensor. The implementation process is as follows:
[0101] Because the variables in the model in formula (2) are interdependent, three auxiliary variable tensors need to be introduced. and Formula (2) is transformed into Formula (3).
[0102]
[0103]
[0104]
[0105]
[0106]
[0107]
[0108] Where, F = [F h ,F v ,F t ],0 <p<1,p i >0 and
[0109] Using the framework of the alternating direction multiplier method, we optimize the solution of formula (3). First, we construct the augmented Lagrangian function of formula (3), which has the following form:
[0110]
[0111] in, and It is a Lagrange multiplier, and μ is the penalty parameter.
[0112] First update tensor The subproblem can be expressed as formula (5):
[0113]
[0114] in and k represents the kth iteration, and k+1 represents the (k+1)th iteration.
[0115] Based on the definition of the Φ product and the unitary invariance of the Frobenius norm, equation (5) can be transformed into equation (6):
[0116]
[0117] With tensor Each forward slice As variables, transform formula (6) into formula (7):
[0118]
[0119] From formula (7), we can solve for each matrix by matrix differentiation. Then the matrix Combining tensors along the tube dimension Then on the tensor Expanding to matrix U using the third mode, then multiplying the left side of matrix U by the transpose of the transformation matrix Φ (the size of the transformation matrix Φ is n³ × n³), and then performing the inverse operation of the third mode expansion of (ΦU) yields...
[0120] Second update tensor The subproblem can be expressed as formula (8):
[0121]
[0122] According to the definition of the Schatten-p norm of the transformation tensor, equation (8) can be transformed into equation (9):
[0123]
[0124] Formula (9) is based on tensors Each forward slice The research object is [the subject of study]. The solution is obtained according to formula (10).
[0125]
[0126] Among them, prox λf (·) is an approximation operator, defined as follows:
[0127] The matrix obtained by solving according to formula (10) Combining tensors along the tube dimension Then on the tensor Expanded into a matrix according to the third mode Then in the matrix Multiply the left side by the transpose of the transformation matrix Φ (the size of the transformation matrix Φ is n³ × n³), then... The inverse operation of the third mode expansion yields
[0128] The third update tensor The subproblem can be expressed as formula (11):
[0129]
[0130] Let tensor The tensor can be solved using formula (12).
[0131]
[0132] Where sign(·) is the sign function, |·| is the absolute value of the elements, and ° represents the product between elements.
[0133] Fourth update tensor The subproblem can be expressed as formula (13):
[0134]
[0135] Based on the properties of the Frobenius norm, equation (13) can be solved by differentiation. Formula (14) is the differentiated form:
[0136]
[0137] Among them, F * Denotes the adjoint operator of F, The solution can be obtained using FFT3 (3D Fourier Transform) and iFFT3 (Inverse 3D Fourier Transform):
[0138]
[0139] in, This represents a tensor whose elements are all 1.
[0140] Fifth update tensor The subproblem of (i.e., the tensor to be completed) can be expressed as formula (16):
[0141]
[0142] Based on the properties of the Frobenius norm, we can first differentiate equation (16), and then calculate the final recovered tensor to be completed according to equation (17).
[0143]
[0144] Among them, Ω ⊥ Let represent the complement of the set Ω.
[0145] Update the Lagrange multipliers according to formulas (18), (19), (20), and (21). and
[0146]
[0147]
[0148]
[0149]
[0150] Each iteration updates every variable. Iteration stops when the maximum number of iterations is reached or the convergence condition of formula (22) is met, and the final completed tensor is output.
[0151] max(Con1,Con2,Con3,Con4)<∈ (22)
[0152] in,
[0153]
[0154] ∈ is the set threshold.
[0155] The following uses classic color images to illustrate the effectiveness of the low-rank tensor completion method based on total variational regularization for color images containing missing values:
[0156] The data for this experiment comes from classic color images, with a size of 300×300×3, which can be regarded as a third-order tensor. In the experiment, according to the sampling rate, the pixel values at certain positions in each channel of the color image are randomly set to 0, thus forming a tensor containing missing values. The experimental task is to complete tensors containing missing values using observable pixel values at a sampling rate of 15%.
[0157] Figure 2(a) shows the original color image (processed to display grayscale), and Figure 2(b) shows a color image with missing values at a sampling rate of 15% (processed to display grayscale). Figures 2(c) to 2(h) The results are shown for the restoration by the method of the present invention at a sampling rate of 15%. Comparing the color image containing missing values with the restored sampled image, the color image containing missing values before restoration has a peak signal-to-noise ratio of 3.8947 dB and a structural similarity of 0.0161, while the restored color image containing missing values... When transformed into a discrete Fourier transform, the peak signal-to-noise ratio is 27.2272 dB, and the structural similarity is 0.7925; when When transformed into a discrete cosine transform, the peak signal-to-noise ratio is 27.313 dB, and the structural similarity is 0.8143; when When transformed into a data-based transformation, the peak signal-to-noise ratio is 27.7915 dB, and the structural similarity is 0.8222; when When transformed into a discrete Fourier transform, the peak signal-to-noise ratio is 27.2433 dB, and the structural similarity is 0.7932; when When transformed into a discrete cosine transform, the peak signal-to-noise ratio is 27.2947 dB, and the structural similarity is 0.8136; when When transformed into a data-based transformation, the peak signal-to-noise ratio is 27.7913 dB, and the structural similarity is 0.8227. Furthermore, in terms of visual restoration, the method of this invention restores the overall structure, main information, and some details of the color image. Comparison before and after restoration verifies the effectiveness of the method.
[0158] In summary, the method of the present invention has a better recovery effect on color images with missing values at low sampling rates.
Claims
1. A method for low-rank tensor completion based on total variation regularization, which utilizes observed pixel values to complete a tensor with missing values characterized in that Includes the following steps: S1, for color images containing missing values, utilize the transform tensor Schatten- Norms characterize the low-rank property of tensors, and combine total variational regularization and transformation tensors with Schatten- By combining norms, a low-rank tensor completion model is constructed; S2, by introducing auxiliary variables and using the framework of the alternating direction multiplier method, an augmented Lagrangian function for the low-rank tensor completion model is constructed. Multiple sub-problems are solved separately, and the process is iterated until the convergence condition is met, finally obtaining the completed tensor. The completed tensor is then converted into an image to obtain the restored color image. In step S1, the total variation regularization and the transformation tensor Schatten- Combining norms, the low-rank tensor completion model expression is as follows: ， ， in, This represents the tensor that needs to be completed. Tensor representing observation, tensor to be completed and the tensor of observation The size is ; Represents the observation set, Tensor to be completed and the tensor of observation The element values are equal in the observation set; It is a transformation tensor Schatten- Norms, used to characterize the tensor to be filled The low-rank nature; It is a total variational regularization term; It is a regularization parameter; Simultaneously, based on the definition of total variation, the tensor to be completed is calculated. Summing the differences between adjacent elements along each of the three dimensions: ， ， in, The total difference operator between adjacent elements is decomposed into difference operators along three different dimensions. and , ); Tensor to be completed pass The integral solution is Size tensor ; , and ; In step S2, the implementation process of constructing the augmented Lagrangian function of the low-rank tensor completion model is as follows: Introducing three auxiliary variable tensors , and Tensors to be completed The formula for summing the differences of adjacent elements along each of the three dimensions can be transformed into the following formula: ， ， ， ， ， ， in, ; The expression for the augmented Lagrange function is as follows: ， in, , , and It is a Lagrange multiplier. It is a penalty parameter; Tensors to be completed The subproblem is represented as: ， Based on the properties of the Frobenius norm, we first solve the subproblem by differentiation, and then calculate the final recovered tensor to be completed. : ， Update the Lagrange multipliers according to the following formula. : ， ， ， ， Each iteration updates every variable, and the iteration stops when the maximum number of iterations is reached. The output is the final completed tensor. 终 ; Alternatively, the iteration can stop and the final completed tensor can be output when the following condition is met. 终 : ， in, , ， , ; This is the set threshold; k represents the kth iteration, and k+1 represents the (k+1)th iteration; express The complement of a set.
2. The low-rank tensor completion method based on total variational regularization according to claim 1, characterized in that, First update tensor The subproblem can be represented as: ， in and ; according to With the definition of the product and the unitary invariance of the Frobenius norm, the above equation is further transformed into: ， With tensor Each forward slice As variables, Then we have: ， Next, solve each matrix using matrix differentiation. Then the matrix Combining tensors along the tube dimension Then, regarding the tensor... Expanded into a matrix according to the third mode Then multiply the left side of matrix U by the transformation matrix. Transpose, transformation matrix The size is ; then The inverse operation of the third mode expansion yields .
3. The low-rank tensor completion method based on total variational regularization according to claim 1, characterized in that, Second update tensor The subproblem is represented as: ， Based on the transformation tensor Schatten- The definition of a norm is as follows: ， Solve : ， in, It is an approximation operator, defined as follows: ; The matrix obtained by solving Combining tensors along the tube dimension , For tensor Each forward slice, Then, regarding the tensor... Expanded into a matrix according to the third mode Then in the matrix Multiply by the transformation matrix on the left Transpose, transformation matrix The size is ; then The inverse operation of the third mode expansion yields .
4. The low-rank tensor completion method based on total variational regularization according to claim 1, characterized in that, The third update tensor The subproblem is represented as: ， Let tensor Solve for the tensor : ， in, It is a symbolic function. It is the absolute value of the element. This represents the product between elements; k represents the k-th iteration, and k+1 represents the (k+1)-th iteration.
5. The low-rank tensor completion method based on total variational regularization according to claim 1, characterized in that, Fourth update tensor The subproblem is represented as: ， Based on the properties of the Frobenius norm, the solution can be obtained by differentiation. The derivative expression is as follows: ， in, express The adjoint operator, pass and Seeking a solution: ， in, This represents a tensor whose elements are all 1; k represents the k-th iteration, k+1 represents the (k+1)-th iteration; FFT3 represents the 3D Fourier transform, and iFFT3 represents the inverse 3D Fourier transform.