Visual Data Reconstruction Method and System Based on Direction-Aware Tensor Nuclear Norm

Abstract

The present invention discloses a visual data reconstruction method and system based on direction-aware tensor nuclear norm, which represents visual data as a multi-order tensor x; obtains the direction-aware tensor nuclear norm of the visual data, and establishes a low-rank tensor completion model based on the direction-aware tensor nuclear norm; optimizes the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data. The present invention proposes a new direction-aware tensor nuclear norm, which reconciles the influence of directions by transforming all modes of the tensor, and realizes the tensor integrity reconstruction of high-dimensional data through an efficient iterative algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-dimensional data reconstruction, and relates to a visual data reconstruction method and system based on direction-aware tensor nuclear norm. Background Art

[0002] Visual data is information captured by visual sensors (such as cameras, webcams, etc.) and represented in the form of images or videos. Images, videos, and derivative data all contain a large amount of high-dimensional data. Due to being damaged, affected by noise interference, distortion, etc., or limited by observation conditions, only partial data can be observed. Data reconstruction aims to recover the original data from a set of limited observations. Therefore, high-dimensional data reconstruction plays an important role in improving data quality. High-dimensional data can usually be represented in tensor form, and using low-rank constraints to recover partial observed data has attracted great research enthusiasm. Due to the characteristics of tensors themselves, many tensor nuclear norms have been defined to replace the rank of tensors. Existing definitions of nuclear norms either ignore the influence of direction and only perform transformations along one mode, or break the original structure of the tensor. In addition, each dimension of the data has its own characteristics that cannot be predicted in advance, which all reduce the performance of the model. Summary of the Invention

[0003] The purpose of the present invention is to provide a visual data reconstruction method and system based on direction-aware tensor nuclear norm, propose a new direction-aware tensor nuclear norm, reconcile the influence of direction by performing transformations on all modes of the tensor, and achieve tensor integrity reconstruction of high-dimensional data through an efficient iterative algorithm.

[0004] The technical solution to achieve the purpose of the present invention is as follows:

[0005] A visual data reconstruction method based on direction-aware tensor nuclear norm, comprising the following steps:

[0006] S01: Represent the visual data as a multi-order tensor ;

[0007] S02: Obtain the direction-aware tensor nuclear norm of the visual data, and establish a low-rank tensor completion model based on the direction-aware tensor nuclear norm;

[0008] S03: Optimize the low-rank tensor completion model through an optimization algorithm to obtain the complete reconstructed tensor data.

[0009] In a preferred technical solution, the direction-aware tensor nuclear norm in step S02 is:

[0010] ;

[0011] Where , is the dimension of the tensor; the parameter P is the dimension direction combination, , represents the d-order tensor nuclear norm based on the unitary transformation.

[0012] In the preferred technical solution, is defined as:

[0013] ;

[0014] Among them, is the unitary transformation of the tensor, represents the tensor nuclear norm, is different modes of.

[0015] In the preferred technical solution, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes:

[0016] Low-rank tensor completion is to re-infer the unobserved elements in the observed tensor from the limited known elements, and the model is expressed as:

[0017] ;

[0018] Among them, represents the rank function, is the observed tensor, is the mapping operator on the observation set , and:

[0019] ;

[0020] The tensor completion model based on the direction-aware tensor nuclear norm is:

[0021] ;

[0022] Use the Lagrangian function to restore the above model:

[0023] ;

[0024] According to the optimal solution write out the Lagrange multiplier , that is satisfies the KKT condition: , is to find the partial derivative.

[0025] In the preferred technical solution, the method for optimizing the low-rank tensor completion model through an optimization algorithm includes:

[0026] Introduce auxiliary variables , rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as:

[0027] ;

[0028] Construct the augmented Lagrangian function of the above formula as:

[0029] ;

[0030] is the regularization factor, is the Frobenius norm, and the solution is obtained by alternately iterating by solving each sub-problem.

[0031] In the preferred technical solution, the expression for obtaining the solution of each variable is:

[0032] The solution of the auxiliary variable is:

[0033] ;

[0034] where, represents the T-SVD optimal solution of the tensor;

[0035] to be solved is:

[0036] ;

[0037] where, represents the complement on the observation set .

[0038] Iteratively calculate the Lagrange multiplier as:

[0039] .

[0040] In the preferred technical solution, the method for solving the solution of each variable includes:

[0041] Set the initial value , multiplier term ;

[0042] Perform iterative calculation, and the iterative process is:

[0043] ;

[0044] Keep iterating, when , is the number of iterations, is the tolerance error or the number of iterations is greater than the maximum number of iterations, output the complete reconstructed tensor data.

[0045] The present invention also discloses a visual data reconstruction system based on the direction-aware tensor nuclear norm, including:

[0046] A tensor acquisition module that represents visual data as a multi-order tensor ;

[0047] A construction module based on the direction-aware tensor nuclear norm, which obtains the direction-aware tensor nuclear norm of visual data and establishes a low-rank tensor completion model based on the direction-aware tensor nuclear norm;

[0048] A reconstruction module that optimizes the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

[0049] In a preferred technical solution, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in the construction module based on the direction-aware tensor nuclear norm includes:

[0050] Low-rank tensor completion is to re-infer the unobserved elements in the observed tensor from a limited number of known elements The model is expressed as:

[0051] ;

[0052] Among them, represents the rank function, is the observed tensor, is the mapping operator on the region and:

[0053] ;

[0054] The tensor completion model based on the direction-aware tensor nuclear norm is:

[0055] ;

[0056] Use the Lagrangian function to restore the above model:

[0057] ;

[0058] Write out the Lagrange multiplier according to the optimal solution , that is satisfies the KKT condition: , is to find the partial derivative.

[0059] The present invention also discloses a computer storage medium, on which a computer program is stored, and when the computer program is executed, it implements the above-mentioned visual data reconstruction method based on the direction-aware tensor nuclear norm.

[0060] Compared with the prior art, the remarkable advantages of the present invention are as follows:

[0061] The present invention proposes a new direction-aware tensor nuclear norm, which harmonizes the influence of directions by transforming all modes of the tensor, and realizes the tensor completion reconstruction of high-dimensional data through an efficient iterative algorithm. Compared with the existing methods for reconstructing high-dimensional data based on low-rank tensor completion, the performance has been significantly improved. Description of the Drawings

[0062] Figure 1 is the flowchart of the visual data reconstruction method based on the direction-aware tensor nuclear norm for this embodiment;

[0063] Figure 2 are the reconstruction results of different methods at a 10% sampling rate of the color video;

[0064] Figure 3 are the reconstruction results of different methods at a 20% sampling rate of the color video;

[0065] Figure 4 are the reconstruction results of different methods at a 10% sampling rate of the light field image;

[0066] Figure 5 are the reconstruction results of different methods at a 20% sampling rate of the light field image. Detailed Embodiments

[0067] The principle of the present invention is as follows: The present invention defines a new direction-aware tensor nuclear norm (OA-TNN), and uses the modified ADMM optimization algorithm to optimize the model. Compared with the existing methods for reconstructing high-dimensional data based on low-rank tensor completion, the performance has been significantly improved.

[0068] Embodiment 1:

[0069] As Figure 1 shown, a visual data reconstruction method based on the direction-aware tensor nuclear norm includes the following steps:

[0070] S01: Represent the visual data as a multi-order tensor ;

[0071] S02: Obtain the direction-aware tensor nuclear norm of the visual data, and establish a low-rank tensor completion model based on the direction-aware tensor nuclear norm;

[0072] S03: Optimize the low-rank tensor completion model through an optimization algorithm to obtain the complete reconstructed tensor data.

[0073] In a preferred embodiment, the direction-aware tensor nuclear norm in step S02 is:

[0074] ;

[0075] Among them, , is the dimension of the tensor; the parameter P is the dimension direction combination, , represents the d-order tensor nuclear norm based on the unitary transformation.

[0076] In a preferred embodiment, is defined as:

[0077] ;

[0078] Where is the unitary transformation of the tensor, represents the tensor nuclear norm, is different modes.

[0079] In a preferred embodiment, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes:

[0080] Low-rank tensor completion is to re-infer the unobserved elements in the observed tensor from the limited known elements, and the model is expressed as:

[0081] ;

[0082] Where represents the rank function, is the observed tensor, is the mapping operator on the observation set and:

[0083] ;

[0084] The tensor completion model based on the direction-aware tensor nuclear norm is:

[0085] ;

[0086] Use the Lagrangian function to recover the above model:

[0087] ;

[0088] According to the optimal solution write out the Lagrange multiplier , that is satisfies the KKT condition: , is to find the partial derivative.

[0089] In a preferred embodiment, the method for optimizing the low-rank tensor completion model through an optimization algorithm includes:

[0090] Introduce an auxiliary variable , and rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as:

[0091] ;

[0092] Construct the augmented Lagrangian function of the above formula as:

[0093] ;

[0094] is the regularization factor, is the Frobenius norm, and the solution that meets the conditions is obtained by alternately iterating by solving each sub-problem.

[0095] In a preferred embodiment, the expression for obtaining the solution of each variable is:

[0096] The solution of the auxiliary variable is:

[0097] ;

[0098] Among them, represents the T-SVD optimized solution of the tensor.

[0099] The to be solved is:

[0100] ;

[0101] Among them, represents the complement set on the observation set .

[0102] The Lagrange multiplier is calculated iteratively as:

[0103] .

[0104] In a preferred embodiment, the method for solving the solution of each variable includes:

[0105] Set the initial value , the multiplier term ;

[0106] Perform iterative calculation, and the iterative process is:

[0107] ;

[0108] Keep iterating, when , is the number of iterations is the allowable error or the number of iterations When it is greater than the maximum number of iterations, the complete reconstructed tensor data is output.

[0109] In another embodiment, a computer storage medium stores a computer program, and when the computer program is executed, the above-mentioned visual data reconstruction method based on the direction-aware tensor nuclear norm is implemented. The above reconstruction method is adopted here and will not be elaborated further.

[0110] In another embodiment, a visual data reconstruction system based on the direction-aware tensor nuclear norm includes:

[0111] a tensor acquisition module that represents visual data as a multi-order tensor ;

[0112] a direction-aware tensor nuclear norm construction module that obtains the direction-aware tensor nuclear norm of visual data and establishes a low-rank tensor completion model based on the direction-aware tensor nuclear norm;

[0113] a reconstruction module that optimizes the low-rank tensor completion model through an optimization algorithm to obtain the complete reconstructed tensor data.

[0114] Specifically, taking a preferred embodiment as an example below, the working process of the visual data reconstruction system based on the direction-aware tensor nuclear norm is described as follows:

[0115] First, determine the TNN differences in different directions of the given tensor.

[0116] Use a third-order tensor to represent a video that has frames, and the size of each frame is . By permuting the three dimensions, different tensors can be obtained, namely: , or . Before data reconstruction, it is impossible to predict in advance which dimension direction each tensor element belongs to. Therefore, the TNN values of these directions are usually different, that is

[0117] ;

[0118] where represents the tensor nuclear norm.

[0119] For determining the case of high-order tensors, there are multiple dimensional direction orientations. For example, for a fourth-order tensor, it is more complex. Therefore, it is very important to enumerate all directions and define a TNN that considers all directions. According to theoretical analysis and derivation, if is a d-order tensor, d≥3, then as long as mode 1 and mode 2 are fixed, the TNNs of different combinations of X are equivalent, that is, the number of TNNs of different dimensional direction slices The combination. Therefore, through the above analysis and processing, the influencing factors of the number of tensor modes in different dimensional directions are simplified.

[0120] Secondly, define the tensor norm OA-TNN.

[0121] Given a tensor , d≥3, then The direction-aware tensor nuclear norm of is defined as:

[0122] ;

[0123] where , each n is the respective dimension of the tensor; the parameter P is combinations, and , represents the d-order tensor nuclear norm based on unitary transformation, defined as:

[0124] ;

[0125] where, is the unitary transformation of the tensor, is different modes.

[0126] Thirdly, establish a low-rank tensor completion model based on OA-TNN.

[0127] Assume that on the observation set (region) the observed tensor , satisfies:

[0128] ;

[0129] is the observed tensor, the direction dimension . The low-rank tensor completion problem aims to re-infer the unobserved elements in from the finite known elements. Usually, the model is expressed as:

[0130] (1)

[0131] where is the rank of the tensor, is the observed tensor, is the mapping operator on, and:

[0132] ;

[0133] Equation (1) is an NP problem. The tensor completion model based on OA-TNN is:

[0134] (2)

[0135] Restore the above model using the Lagrangian function:

[0136] ;

[0137] Then, according to the optimal solution write out the Lagrange multipliers , that is satisfying the KKT conditions (Karush - Kuhn - Tucker): , is to find the partial derivative. After mathematical proof, we can find satisfying the above formula, and the optimal solution holds.

[0138] Fourth, establish the optimization solution process under the ADMM method framework.

[0139] Introduce auxiliary variables ,

[0140] Therefore, the original formula (2) is rewritten as:

[0141] .

[0142] So the augmented Lagrangian function of the above formula is:

[0143] ;

[0144] is the regularization factor, is the Frobenius norm. By solving each sub - problem, we alternately iterate until a solution that meets the conditions is found. Here, the expressions for the solutions of each variable involved are directly listed.

[0145] The solution of the auxiliary variable is:

[0146] (3)

[0147] Among them, represents the T - SVD optimization solution of the tensor;

[0148] Under the transformation, the SVD decomposition is: , where is the left singular matrix, is the diagonal matrix, represents the transpose of the tensor, represents the tensor product under the denote , where .

[0149] to be determined is:[[]]

[0150] (4)

[0151] where denotes the complement set on the observation set .

[0152] The Lagrange multiplier is iteratively calculated as:[[]]

[0153] (5)

[0154] The specific steps of the above solution process are as follows:[[]]

[0155] Input: Sampling set , observed incomplete data , regularization parameter , auxiliary variable , multiplier , iterate on , maximum number of iterations , tolerance .

[0156] 1: Initialization, , multiplier term . ;

[0157] 2: for k = 1;

[0158] 3: Update according to equation (3);

[0159] 4: Update to according to equation (4);

[0160] 5: Update according to equation (5);

[0161] 6: k = k + 1;

[0162] 7: Calculate according to equation (4);

[0163] 8: or , end k the loop;

[0164] Output: Complete reconstructed tensor data.

[0165] Fifth, to verify the effectiveness of the present invention, experiments were conducted on a standard dataset, and the PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) metrics were used to compare with other mainstream image reconstruction methods. They are defined as follows:

[0166] (6)

[0167] where are the data dimension sizes respectively, is the reconstructed value of denotes the infinity norm.

[0168] (7)

[0169] where , respectively represent , the means of is the standard deviation of is the standard deviation of is the covariance between and , are constants.

[0170] Experimental results

[0171] 1. YaleFace Dataset B data

[0172] The data includes 38 instance images, and each instance image has 9 different poses with a light flux of 64. The size of each image is , so the data size for each pose is . In the experiment, pixels were randomly sampled at sampling rates of 10% and 20% respectively, and compared with different methods such as OTNN, TCTV, CHTNN, MTTD, and TCTN. The reconstruction results are shown in Table 1.

[0173] Table 1

[0174]

[0175] 2. Color video

[0176] Another important application of tensor completion is video restoration. The YUV video dataset was used as the processing object. The size of each frame of the image is , 100 frames were selected for testing. The sampling rates were 10% and 20% respectively, and the results are as Figure 2 , Figure 3 shown.

[0177] 3. Light field images

[0178] Six light field images were selected. Each image contains 81 views, and the view grid size is , and the image size is . Therefore, the size of the light field image tensor dataset is . The sampling rates are 10% and 20% respectively. Figure 4 , Figure 5 are the effects after reconstruction.

[0179] The above embodiments are the preferred embodiments of the present invention. However, the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A visual data reconstruction method based on direction-aware tensor nuclear norm, characterized in that It includes the following steps: S01: Represent the visual data as a multi - order tensor ; S02: Obtain the direction-aware tensor nuclear norm of the visual data, and establish a low-rank tensor completion model based on the direction-aware tensor nuclear norm; the direction-aware tensor nuclear norm is: ， Among them, , where n is the dimension of the tensor; the parameter P is all combinations in the dimension direction, , represents the nuclear norm of a d-order tensor based on unitary transformation; is defined as: ， Among them, is the unitary transformation of the tensor, represents the tensor nuclear norm, is different modes of S03: Introduction Introduce an auxiliary variable, and optimize the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

2. The visual data reconstruction method based on direction-aware tensor nuclear norm according to claim 1, characterized in that The method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes: Low-rank tensor completion is to infer the unobserved elements in the observed tensor from the limited known elements, and the model is expressed as: in which ， Among them, represents the rank function, is the observation tensor, is the observation set is the mapping operator on, and: ， The tensor completion model based on the direction-aware tensor nuclear norm is: ， Use the Lagrangian function to recover the above model: ， According to the optimal solution Write out the Lagrange multipliers , that is Satisfy the KKT conditions: , To find the partial derivative.

3. The visual data reconstruction method based on the direction-aware tensor nuclear norm according to claim 2, characterized in that The method for optimizing the low-rank tensor completion model through an optimization algorithm includes: Introduction auxiliary variable , rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as: ； Construct the augmented Lagrangian function of the above formula as: ， is a regularization factor, is the Frobenius norm, and the solution that meets the conditions is obtained by alternately iterating by solving each sub-problem until the solution is found.

4. The visual data reconstruction method based on the direction-aware tensor nuclear norm according to claim 3, wherein Finding the solution that satisfies the conditions includes obtaining the expression of the solution for each variable, including: The solution of the auxiliary variable is: ， Among them, represents the T-SVD optimal solution of the tensor; To be determined is ， Among them, represents the complement set on the observation set ; Iteratively calculate the Lagrange multiplier as: 。 5. The visual data reconstruction method based on the direction-aware tensor nuclear norm according to claim 4, wherein, Solve the expression of the solution for each variable obtained, including: Set an initial value , multiplier term ; Perform iterative calculation, and the iterative process is: ， Iterate continuously. When , is the number of iterations, is the tolerance error or the number of iterations is greater than the maximum number of iterations, output the complete reconstructed tensor data.

6. A visual data reconstruction system based on direction-aware tensor nuclear norm, characterized in that, It includes: Tensor acquisition module, representing visual data as a multi-order tensor ; A low-rank tensor completion model construction module, which obtains the direction-aware tensor nuclear norm of the visual data and establishes a low-rank tensor completion model based on the direction-aware tensor nuclear norm; the direction-aware tensor nuclear norm is: ， Among them, , where n is the dimension of the tensor; the parameter P is all combinations in the dimension direction, , represents the nuclear norm of a d-order tensor based on unitary transformation; is defined as: ， Among them, is the unitary transformation of the tensor, represents the tensor nuclear norm, is different modes of Reconstruction module, introducing auxiliary variables, and optimizing the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

7. The visual data reconstruction system based on the direction-aware tensor nuclear norm according to claim 6, wherein The method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in the low-rank tensor completion model construction module includes: Low-rank tensor completion is for inferring the unobserved elements in an observed tensor from a limited number of known elements, and the model is expressed as: in which ， Among them, represents the rank function, is the observation tensor, is the region is the mapping operator on, and: ， The tensor completion model based on the direction-aware tensor nuclear norm is: ， Use the Lagrangian function to recover the above model: ， According to the optimal solution Write out the Lagrange multipliers , that is Satisfy the KKT conditions: , For taking partial derivatives.

8. A computer storage medium, on which a computer program is stored, characterized in that, When the computer executes the computer program, it implements the visual data reconstruction method based on the direction-aware tensor nuclear norm described in any one of claims 1-5.

Images