Visual data reconstruction method and system based on direction perception tensor nuclear norm

Abstract

The invention discloses a visual data reconstruction method and system based on a direction perception tensor nuclear norm. Visual data is expressed as a multi-order tensor x; obtaining a direction perception tensor nuclear norm of the visual data, and establishing a low-rank tensor completion model based on the direction perception tensor nuclear norm; and optimizing the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data. According to the method, a new direction perception tensor nuclear norm is provided, the influence of the direction is reconciled by converting all modes of the tensor, and tensor integrity reconstruction of high-dimensional data is achieved through an efficient iterative algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-dimensional data reconstruction, and the present invention relates to a visual data reconstruction method and system based on a direction-aware tensor nuclear norm. Background Art

[0002] Visual data is information captured by visual sensors (such as cameras, etc.) and represented in the form of images or videos. Images, videos, and derived data all contain a large amount of high-dimensional data. Due to damage, noise interference, distortion, or limited observation conditions, only part of the data can be observed. Data reconstruction aims to recover the original data from a limited set of observations, so high-dimensional data reconstruction plays an important role in improving data quality. High-dimensional data can usually be represented in the form of tensors, and the use of low-rank constraints to recover part of the observed data has aroused great research enthusiasm. Due to the characteristics of tensors themselves, many tensor nuclear norms have been defined to replace the rank of tensors. The existing nuclear norm definitions either ignore the influence of direction and only transform along one mode, or break the original structure of the tensor. In addition, each dimension of the data has its own characteristics that cannot be predicted in advance, which reduces the performance of the model. Summary of the invention

[0003] The purpose of the present invention is to provide a visual data reconstruction method and system based on the direction-aware tensor nuclear norm, and propose a new direction-aware tensor nuclear norm, which reconciles the influence of direction by transforming all modes of the tensor and realizes tensor integrity reconstruction of high-dimensional data through an efficient iterative algorithm.

[0004] The technical solution to achieve the purpose of the present invention is: A visual data reconstruction method based on a direction-aware tensor nuclear norm comprises the following steps: S01: Representing visual data as multi-order tensors ; S02: Obtaining the direction-aware tensor nuclear norm of the visual data, and establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm; S03: Optimize the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

[0005] In the preferred technical solution, the direction-aware tensor nuclear norm in step S02 is: ;

[0006] in, , is the dimension of the tensor; the parameter P is the dimension direction combination, , Represents the nuclear norm of a d-order tensor based on a unitary transformation.

[0007] In the preferred technical solution, Defined as: ;

[0008] in, is the unitary transformation of the tensor, represents the tensor nuclear norm, for different modes.

[0009] In the preferred technical solution, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes: Low-rank tensor completion is to re-infer the observed tensor from a limited number of known elements. For the unobserved elements in , the model is expressed as: ;

[0010] in, represents the rank function, is the observation tensor, For the observation set A mapping operator on , and: ;

[0011] Direction-aware tensor nuclear norm The tensor completion model is: ;

[0012] Restore the above model using the Lagrangian function: ;

[0013] According to the optimal solution Write down the Lagrange multipliers ,Right now Meet KKT conditions: , In order to seek partial guidance.

[0014] In the preferred technical solution, the method for optimizing the low-rank tensor completion model by using an optimization algorithm includes: Introduction Auxiliary variables , rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as follows: ; The augmented Lagrangian function of the above formula is: ;

[0015] is the regularization factor, is the Frobenius norm, and the algorithm iterates alternately by solving each subproblem until a solution that satisfies the conditions is found.

[0016] In the preferred technical solution, the expression for obtaining the solution of each variable is: The solution for the auxiliary variables is: ;

[0017] in, represents the T-SVD optimized solution of a tensor; Waiting for for: ;

[0018] in, Indicates that in the observation set The complement of .

[0019] The iterative calculation of the Lagrange multiplier is: .

[0020] In the preferred technical solution, the method for solving each variable includes: Setting initial values , multiplier term ; Perform iterative calculation, the iterative process is: ; Continuously iterate, when , is the number of iterations, is the tolerance or number of iterations When the number of iterations is greater than the maximum, the complete reconstruction tensor data is output.

[0021] The present invention also discloses a visual data reconstruction system based on a direction-aware tensor nuclear norm, comprising: Tensor acquisition module, which represents visual data as multi-order tensors ; Based on the direction-aware tensor nuclear norm, a module is constructed to obtain the direction-aware tensor nuclear norm of visual data, and a low-rank tensor completion model is established based on the direction-aware tensor nuclear norm. The reconstruction module optimizes the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

[0022] In the preferred technical solution, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in the direction-aware tensor nuclear norm construction module includes: Low-rank tensor completion is to re-infer the observed tensor from a limited number of known elements. The unobserved elements in the model are expressed as: ;

[0023] in, represents the rank function, is the observation tensor, For Region A mapping operator on , and: ;

[0024] Direction-aware tensor nuclear norm The tensor completion model is: ;

[0025] Restore the above model using the Lagrangian function: ;

[0026] According to the optimal solution Write down the Lagrange multipliers ,Right now Meet KKT conditions: , In order to seek partial guidance.

[0027] The present invention further discloses a computer storage medium on which a computer program is stored. When the computer program is executed, the visual data reconstruction method based on the direction-aware tensor nuclear norm is implemented.

[0028] Compared with the prior art, the present invention has the following significant advantages: This paper proposes a new direction-aware tensor nuclear norm, which reconciles the influence of direction by transforming all modes of the tensor, and achieves tensor integrity reconstruction of high-dimensional data through an efficient iterative algorithm. Compared with the existing high-dimensional data reconstruction method based on low-rank tensor integrity, the performance is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of a visual data reconstruction method based on a direction-aware tensor nuclear norm of this embodiment; Figure 2 The reconstruction results of different methods for color video with 10% sampling rate; Figure 3 The reconstruction results of different methods for color video with 20% sampling rate; Figure 4 The reconstruction results of different methods for light field images with 10% sampling rate; Figure 5Reconstruction results of different methods for light field images with 20% sampling rate. DETAILED DESCRIPTION

[0030] The principle of the present invention is as follows: the present invention defines a new direction-aware tensor nuclear norm (OA-TNN), optimizes the model using a modified ADMM optimization algorithm, and significantly improves the performance compared with the existing high-dimensional data reconstruction method based on low-rank tensor integrity.

[0031] Embodiment 1:

[0032] like Figure 1 As shown, a visual data reconstruction method based on a direction-aware tensor nuclear norm comprises the following steps: S01: Representing visual data as multi-order tensors ; S02: Obtaining the direction-aware tensor nuclear norm of the visual data, and establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm; S03: Optimize the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

[0033] In a preferred embodiment, the direction-aware tensor nuclear norm in step S02 is: ;

[0034] in, , is the dimension of the tensor; the parameter P is the dimension direction combination, , Represents the nuclear norm of a d-order tensor based on a unitary transformation.

[0035] In a preferred embodiment, Defined as: ;

[0036] in is the unitary transformation of the tensor, represents the tensor nuclear norm, for different modes.

[0037] In a preferred embodiment, the method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes: Low-rank tensor completion is to re-infer the observed tensor from a limited number of known elements. For the unobserved elements in , the model is expressed as: ;

[0038] in, represents the rank function, is the observation tensor, For the observation set A mapping operator on , and: ;

[0039] Direction-aware tensor nuclear norm The tensor completion model is: ;

[0040] Restore the above model using the Lagrangian function: ;

[0041] According to the optimal solution Write down the Lagrange multipliers ,Right now Meet KKT conditions: , In order to seek partial guidance.

[0042] In a preferred embodiment, the method for optimizing the low-rank tensor completion model by using an optimization algorithm includes: Introduction Auxiliary variables , rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as follows: ; The augmented Lagrangian function of the above formula is: ;

[0043] is the regularization factor, is the Frobenius norm, and the algorithm iterates alternately by solving each subproblem until a solution that satisfies the conditions is found.

[0044] In a preferred embodiment, the expression for obtaining the solution of each variable is: The solution for the auxiliary variables is: ; in, Represents the T-SVD optimized solution of a tensor.

[0045] Waiting for for: ;

[0046] in, In the observation set The complement of .

[0047] The iterative calculation of the Lagrange multiplier is: .

[0048] In a preferred embodiment, the method for obtaining the solution of each variable includes: Setting initial values , multiplier term ; Perform iterative calculation, the iterative process is: ; Continuously iterate, when , is the number of iterations is the tolerance or number of iterations When the number of iterations is greater than the maximum, the complete reconstruction tensor data is output.

[0049] In another embodiment, a computer storage medium stores a computer program, which, when executed, implements the above-mentioned visual data reconstruction method based on the direction-aware tensor nuclear norm. The above reconstruction method is adopted and will not be described in detail here.

[0050] In another embodiment, a visual data reconstruction system based on a direction-aware tensor nuclear norm includes: Tensor acquisition module, which represents visual data as multi-order tensors ; Based on the direction-aware tensor nuclear norm, a module is constructed to obtain the direction-aware tensor nuclear norm of visual data, and a low-rank tensor completion model is established based on the direction-aware tensor nuclear norm. The reconstruction module optimizes the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

[0051] Specifically, the workflow of the visual data reconstruction system based on the direction-aware tensor nuclear norm is described below by taking a preferred embodiment as an example: First, the TNN differences in different directions of a given tensor are determined.

[0052] Using a third-order tensor Represents a video that has Frames, each frame size is By permuting the three dimensions, we can get different tensors, namely: ,or Before data reconstruction, it is impossible to predict in advance which dimension each tensor element belongs to. Therefore, the TNN values ​​in these directions are usually different, that is, ; in Represents the tensor nuclear norm.

[0053] It is more complicated to determine the orientation of multiple dimensions for higher-order tensors, such as fourth-order tensors. Therefore, it is very important to enumerate all directions and define a TNN that considers all directions. According to theoretical analysis and derivation, if is a d-order tensor, d ≥ 3, then as long as mode 1 and mode 2 are fixed, the TNNs of different combinations of X are equivalent, that is, the number of TNNs of slices in different dimensional directions is Therefore, through the above analysis and processing, the factors affecting the number of tensor modes in different dimensional directions are simplified.

[0054] Secondly, the tensor norm OA-TNN is defined.

[0055] Given a tensor , d ≥ 3, then The direction-aware tensor nuclear norm of is defined as: ;

[0056] in , each n is the dimension of the tensor; the parameter P is combinations, and , represents the d-order tensor nuclear norm based on unitary transformation, defined as: ;

[0057] in, is the unitary transformation of the tensor, for different modes.

[0058] Third, a low-rank tensor completion model based on OA-TNN is established.

[0059] Assume that in the observation set (region) Upper observation tensor ,satisfy: ; is the observed tensor, with the directional dimension The low-rank tensor completion problem aims to re-infer from a limited number of known elements. For unobserved elements in , the model is usually expressed as: (1) in is the rank of the tensor, is the observation tensor, for A mapping operator on , and: ;

[0060] (1) is an NP problem. The tensor completion model based on OA-TNN is: (2) Restore the above model using the Lagrangian function: ;

[0061] Then according to the optimal solution Write down the Lagrange multipliers ,Right now Satisfies the KKT condition (Karush-Kuhn-Tucker): , To find the partial derivative. After mathematical proof, we can find Satisfying the above formula, The optimal solution of is established.

[0062] Fourth, establish the optimization solution process under the framework of ADMM method.

[0063] Introduction Auxiliary variables , Therefore, the original formula (2) is rewritten as: .

[0064] So the augmented Lagrangian function of the above formula is: ;

[0065] is the regularization factor, is the Frobenius norm. By solving each subproblem, the iteration is repeated until a solution that satisfies the conditions is found. The expression of the solution for each variable involved is directly listed here.

[0066] The solution for the auxiliary variables is: (3) in, represents the T-SVD optimized solution of a tensor; exist The SVD decomposition under transformation is: ,in is a left singular matrix, is a diagonal matrix, represents the transpose of a tensor, express Transform the tensor product. Then: express , in .

[0067] Waiting for for: (4) in, Indicates that in the observation set The complement of .

[0068] The iterative calculation of the Lagrange multiplier is: (5) The specific steps of the above solution process are as follows: Input: Sample set , observed incomplete data , the regularization parameter , auxiliary variables , multiplier ,right Iterate, maximum number of iterations , allowable error .

[0069] 1: Initialization, , multiplier term . ; 2: for k =1; 3: Update according to formula (3) ;

[0070] 4: Update according to formula (4) for ;

[0071] 5: Update according to formula (5) ;

[0072] 6: k=k+1; 7: Calculate according to formula (4) ;

[0073] 8: or ,Finish k cycle; Output: Complete reconstructed tensor data.

[0074] Fifth, in order to verify the effectiveness of the present invention, experiments were conducted on a standard dataset, and the PSNR (PeakSignal-to-Noise Ratio) and SSIM (Structural Similarity) indicators were used to compare with other mainstream image reconstruction methods. They are defined as follows: (6) in are the data dimension sizes, for The reconstruction value of express Infinity norm.

[0075] (7) in , Respectively , The mean of for The standard deviation of for The standard deviation of for and The covariance of , is a constant.

[0076] Experimental Results 1. YaleFace Dataset B The data includes 38 instance images, each with 9 different poses and a light lumens of 64. The size of each image is , so the data size for each posture is In the experiment, pixels are randomly sampled with sampling rates of 10% and 20% respectively, and compared with different methods such as OTNN, TCTV, CHTNN, MTTD, and TCTN. The reconstruction results are shown in Table 1.

[0077] Table 1

[0078] 2. Color video Another important application of tensor integrity is video restoration. The YUV video dataset is used as the processing object. The size of each frame is , 100 frames were selected for testing. The sampling rates were 10% and 20% respectively. The results are as follows Figure 2 , Figure 3 shown.

[0079] 3. Light Field Images Six light field images are selected, each image contains 81 views, and the view grid size is , the image size is , so the size of the light field image tensor dataset is The sampling rates are 10% and 20% respectively. Figure 4 , Figure 5 This is the effect after reconstruction.

[0080] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. A visual data reconstruction method based on direction-aware tensor nuclear norm, characterized in that: The following steps are involved: S01: Representing visual data as multi-order tensors ; S02: Obtaining the direction-aware tensor nuclear norm of the visual data, and establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm; S03: Optimize the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

2. The visual data reconstruction method based on the direction-aware tensor nuclear norm according to claim 1 is characterized in that: The direction-aware tensor nuclear norm in step S02 is: ， in, , is the dimension of the tensor; the parameter P is the dimension direction combination, , Represents the nuclear norm of a d-order tensor based on a unitary transformation.

3. The visual data reconstruction method based on the direction-aware tensor nuclear norm according to claim 2 is characterized in that: Defined as: ， in, is the unitary transformation of the tensor, represents the tensor nuclear norm, for different modes.

4. The method for visual data reconstruction based on direction-aware tensor nuclear norm according to claim 3, characterized in that: The method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in step S02 includes: Low-rank tensor completion is to re-infer the observed tensor from a limited number of known elements. The unobserved elements in the model are expressed as: ， in, represents the rank function, is the observation tensor, For the observation set A mapping operator on , and: ， Direction-aware tensor nuclear norm The tensor completion model is: ， Restore the above model using the Lagrangian function: ， According to the optimal solution Write down the Lagrange multipliers ,Right now Meet KKT conditions: , In order to seek partial guidance.

5. The method for visual data reconstruction based on direction-aware tensor nuclear norm according to claim 4, characterized in that: Methods for optimizing low-rank tensor completion models through optimization algorithms include: Introduction Auxiliary variables , rewrite the low-rank tensor completion model based on the direction-aware tensor nuclear norm as follows: ； The augmented Lagrangian function of the above formula is: ， is the regularization factor, is the Frobenius norm, and the algorithm iterates alternately by solving each subproblem until a solution that satisfies the conditions is found.

6. The method for visual data reconstruction based on direction-aware tensor nuclear norm according to claim 5, characterized in that: The expression for the solution of each variable is: The solution for the auxiliary variables is: ， in, represents the T-SVD optimized solution of a tensor; Waiting for for: ， in, Indicates that in the observation set The complement of ; The iterative calculation of the Lagrange multiplier is: 。 7. The method for visual data reconstruction based on direction-aware tensor nuclear norm according to claim 6, characterized in that: Methods for finding solutions for each variable include: Setting initial values , multiplier term ; Perform iterative calculation, the iterative process is: ， Continuously iterate, when , is the number of iterations, is the tolerance or number of iterations When the number of iterations is greater than the maximum, the complete reconstruction tensor data is output.

8. A visual data reconstruction system based on direction-aware tensor nuclear norm, characterized in that: include: Tensor acquisition module, which represents visual data as multi-order tensors ; Based on the direction-aware tensor nuclear norm, a module is constructed to obtain the direction-aware tensor nuclear norm of visual data, and a low-rank tensor completion model is established based on the direction-aware tensor nuclear norm. The reconstruction module optimizes the low-rank tensor completion model through an optimization algorithm to obtain complete reconstructed tensor data.

9. The visual data reconstruction system based on direction-aware tensor nuclear norm according to claim 8, characterized in that: The method for establishing a low-rank tensor completion model based on the direction-aware tensor nuclear norm in the direction-aware tensor nuclear norm building module includes: Low-rank tensor completion is to re-infer the observed tensor from a limited number of known elements. The unobserved elements in the model are expressed as: ， in, represents the rank function, is the observation tensor, For Region A mapping operator on , and: ， Direction-aware tensor nuclear norm The tensor completion model is: ， Restore the above model using the Lagrangian function: ， According to the optimal solution Write down the Lagrange multipliers ,Right now Meet KKT conditions: , In order to seek partial guidance.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the visual data reconstruction method based on the direction-aware tensor nuclear norm described in any one of claims 1 to 7 is implemented.

Images