Image reconstruction method, system and storage medium based on tensor nuclear norm model

Abstract

The present invention discloses an image reconstruction method, system and storage medium based on a tensor nuclear norm model, which represents an image as a third-order tensor, wherein the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and performs a tensor model expansion; based on a weighted truncated tensor nuclear norm without direction dependence, an optimization model for recovering an incomplete observation tensor is obtained; the optimization model for recovering an incomplete observation tensor is optimized by an optimization algorithm to obtain complete reconstructed tensor data. The present invention proposes a new weighted truncated tensor nuclear norm without direction dependence, and optimizes the model using a modified ADMM optimization algorithm to achieve efficient and accurate reconstruction of undersampled images and video data.

Description

Technical Field

[0001] The present invention belongs to the technical field of image reconstruction, and the present invention relates to an image reconstruction method, system and storage medium based on a tensor nuclear norm model. Background Art

[0002] In the field of digital image processing, image reconstruction is an important technology that aims to restore images that are damaged, noisy, distorted, etc. through relevant algorithms and methods. Image reconstruction plays an important role in improving image quality and clarity. Images, videos, and derived data all contain a large amount of high-dimensional data. Due to the spatial complexity of high-dimensional data, high-dimensional data processing is often converted into matrix or vector form, which undoubtedly destroys the structural characteristics of the data itself and does not fully utilize the internal potential structural characteristics. Tensors are the general form of vectors and matrices that are expanded from low dimensions to high dimensions in organizational structure, and can maintain the internal structure. Using tensors to express image data does not require separating channels, time, and space dimensions, and can be fully expressed at one time. Although tensor data can express the structural characteristics of high-dimensional data, it is usually low-rank or can be approximated as low-rank, so the low-rank matrix integrity method can be extended to the tensor case for image data recovery. However, in existing methods, either all the singular values ​​of the matrix of interest are shrunk, or the defined tensor nuclear norm has direction dependence, resulting in a decrease in recovery performance.

[0003] The Tensor Nuclear Norm Minimization method uses the tensor nuclear norm as a regularization term for low-rank tensors and restores low-rank tensors by minimizing the tensor nuclear norm. The tensor nuclear norm has a direction-dependent characteristic, so methods such as tensor product (T-product) and tensor TNN (Truncate nuclear norm) have deviations in the calculation of tensor rank, resulting in poor results. Summary of the invention

[0004] The purpose of the present invention is to provide an image reconstruction method, system and storage medium based on a tensor nuclear norm model, and propose a new weighted direction-independent truncated tensor nuclear norm to achieve efficient and accurate reconstruction of undersampled images and video data.

[0005] The technical solution to achieve the purpose of the present invention is:

[0006] An image reconstruction method based on a tensor nuclear norm model comprises the following steps:

[0007] S01: Represent the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and perform tensor model expansion;

[0008] S02: Based on the weighted non-directionally dependent truncated tensor nuclear norm, an optimization model for incomplete observation tensor recovery is obtained;

[0009] S03: Optimizing the optimization model for recovering the incomplete observation tensor through an optimization algorithm to obtain complete reconstructed tensor data.

[0010] In the preferred technical solution, in step S01, the tensor Mode n Expand for:

[0011]

[0012] For tensors The pattern n expansion, inverse operation is the tensor recovery of mode n, represents the real number space of dimension N, Represents each dimension, .

[0013] In the preferred technical solution, the weighted direction-independent truncated tensor nuclear norm in step S02 is for:

[0014]

[0015] In the formula , , , for The SVD decomposition of the orthogonal matrix, represents the matrix transpose, is a diagonal matrix, is the nuclear norm, is the identity matrix.

[0016] In the preferred technical solution, the optimization model for incomplete observation tensor recovery obtained in step S02 is:

[0017]

[0018] in, For the sampling set The mapping operator on , , for A subset of .

[0019] The preferred technical solution also includes deforming the optimization model for incomplete observation tensor recovery, introducing dimensions and Same auxiliary variables , the optimization model is rewritten as:

[0020]

[0021] Its Lagrangian augmented form is:

[0022]

[0023] in, is the regularization parameter, for The Frobenius norm of , Is the multiplier.

[0024] In the preferred technical solution, the model is optimized using a modified ADMM optimization algorithm, and the specific steps include:

[0025] Setting initial values ;

[0026] After a series of derivations, the iterative process is:

[0027]

[0028] in, is the number of iterations;

[0029] Update auxiliary variables , tensor , multiplier ;

[0030] when , When the allowable error or the number of iterations k is greater than the maximum number of iterations, the complete reconstructed tensor data is output.

[0031] The present invention also discloses an image reconstruction system based on a tensor nuclear norm model, comprising:

[0032] The tensor model expansion module represents the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and performs tensor model expansion;

[0033] The optimization model building module obtains the optimization model for incomplete observation tensor recovery based on the weighted non-directionally dependent truncated tensor nuclear norm;

[0034] The optimization module optimizes the optimization model of the incomplete observation tensor recovery through the optimization algorithm to obtain the complete reconstructed tensor data.

[0035] In the preferred technical solution, the optimization model for incomplete observation tensor recovery obtained in the optimization model construction module is:

[0036]

[0037] in, For the sampling set The mapping operator on , , for A subset of .

[0038] The preferred technical solution also includes deforming the optimization model for incomplete observation tensor recovery, introducing dimensions and Same auxiliary variables , the optimization model is rewritten as:

[0039]

[0040] Its Lagrangian augmented form is:

[0041]

[0042] in, is the regularization parameter, for The Frobenius norm of , Is the multiplier.

[0043] The present invention further discloses a computer storage medium on which a computer program is stored. When the computer program is executed, the image reconstruction method based on the tensor nuclear norm model is implemented.

[0044] Compared with the prior art, the present invention has the following significant advantages:

[0045] The present invention proposes a new weighted truncated tensor nuclear norm without direction dependence, and optimizes the model using the modified ADMM optimization algorithm to achieve efficient and accurate reconstruction of undersampled images and video data. Without direction dependence, there will be no deviation in the calculation of tensor rank, and the effect is better. By comparing with the current image reconstruction method based on low-rank tensors, a significant reconstruction effect is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the image reconstruction method based on the tensor nuclear norm model of this embodiment;

[0047] Figure 2 The restoration effects of different methods at different sampling rates;

[0048] Figure 3 This is the local magnification effect when the sampling rate is 40%. DETAILED DESCRIPTION

[0049] The principle of the present invention is: the present invention proposes a new weighted direction-independent truncated tensor nuclear norm, optimizes the model using a modified ADMM optimization algorithm, and realizes efficient and accurate reconstruction of undersampled image and video data.

[0050] Embodiment 1:

[0051] like Figure 1 As shown, an image reconstruction method based on a tensor nuclear norm model comprises the following steps:

[0052] S01: Represent the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and perform tensor model expansion;

[0053] S02: Based on the weighted non-directionally dependent truncated tensor nuclear norm, an optimization model for incomplete observation tensor recovery is obtained;

[0054] S03: Optimizing the optimization model for recovering the incomplete observation tensor through an optimization algorithm to obtain complete reconstructed tensor data.

[0055] In a preferred embodiment, in step S01, the tensor Mode n Expand for:

[0056]

[0057] For tensors The pattern n expansion, inverse operation is the tensor recovery of mode n, represents the real number space of dimension N, Represents each dimension, .

[0058] In a preferred embodiment, the weighted direction-independent truncated tensor nuclear norm in step S02 is for:

[0059]

[0060] In the formula , , , for The SVD decomposition of the orthogonal matrix, represents the matrix transpose, is a diagonal matrix, is the nuclear norm, is the identity matrix.

[0061] In a preferred embodiment, the optimization model for recovering the incomplete observation tensor obtained in step S02 is:

[0062]

[0063] in, For the sampling set The mapping operator on , , for A subset of .

[0064] In a preferred embodiment, the optimization model for recovering incomplete observation tensors is deformed to introduce dimensions and Same auxiliary variables , the optimization model is rewritten as:

[0065]

[0066] Its Lagrangian augmented form is:

[0067]

[0068] in, is the regularization parameter, for The Frobenius norm of , Is the multiplier.

[0069] In a preferred embodiment, the model is optimized using a modified ADMM optimization algorithm, and the specific steps include:

[0070] Setting initial values ;

[0071] After a series of derivations, the iterative process is:

[0072]

[0073] in, is the number of iterations;

[0074] Update auxiliary variables , tensor , multiplier ;

[0075] when , When the allowable error or the number of iterations k is greater than the maximum number of iterations, the complete reconstructed tensor data is output.

[0076] In another embodiment, a computer storage medium stores a computer program, and when the computer program is executed, the image reconstruction method based on the tensor nuclear norm model is implemented. The above image reconstruction method is adopted and will not be described in detail here.

[0077] In another embodiment, an image reconstruction system based on a tensor nuclear norm model includes:

[0078] The tensor model expansion module represents the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and performs tensor model expansion;

[0079] The optimization model building module obtains the optimization model for incomplete observation tensor recovery based on the weighted non-directionally dependent truncated tensor nuclear norm;

[0080] The optimization module optimizes the optimization model of the incomplete observation tensor recovery through the optimization algorithm to obtain the complete reconstructed tensor data.

[0081] Specifically, the workflow of the image reconstruction system based on the tensor nuclear norm model is described below by taking a preferred embodiment as an example:

[0082] From a subset of partial observations Recovering the tensor The mathematical basis of is a low-rank optimization problem:

[0083] (1)

[0084] in represents the real number space of dimension N, Represents each dimension, ; represents the rank function, For the mapping operator:

[0085]

[0086] Formula (1) is an NP problem. In order to make the low-rank tensor integrity problem solvable, various tensor nuclear norms have been proposed as alternative rank functions based on different tensor decomposition methods. We propose a method for optimizing the calculation of reconstructed image data based on the tensor nuclear norm model to achieve efficient and accurate reconstruction of undersampled images and video data. A new weighted orientation-independent truncated tensor nuclear norm (WOITNN) is defined, and the model is optimized using the modified ADMM (Alternating Direction Method of Multipliers iteration) optimization algorithm. By comparing with the current image reconstruction methods based on low-rank tensors, significant reconstruction effects are achieved.

[0087] The specific implementation method is as follows:

[0088] First, the image is represented as a third-order tensor. The first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and the tensor model is expanded. Let the tensor Mode n Expand for:

[0089] (2)

[0090] definition For tensors The pattern n expansion, inverse operation Recover a tensor of mode n.

[0091] Secondly, the weighted orientation-independent truncated tensor nuclear norm (WOITNN) is defined.

[0092] The Tensor Nuclear Norm Minimization method uses the tensor nuclear norm as a regularization term for low-rank tensors and restores low-rank tensors by minimizing the tensor nuclear norm. The tensor nuclear norm has a direction-dependent characteristic, so methods such as tensor product (T-product) and tensor TNN (Truncate nuclear norm) have deviations in the calculation of tensor rank and result in poor results. The proposed WOITNN is:

[0093] (3)

[0094] In the formula , , , for The SVD decomposition of the orthogonal matrix, represents the matrix transpose, It is a diagonal matrix.

[0095] According to the defined tensor nuclear norm, the optimization model for incomplete observation tensor recovery is obtained: (4)

[0096] in For collection A mapping operator on , and:

[0097]

[0098] It is difficult to directly optimize equation (4), so we introduce the dimension Same auxiliary variables , formula (4) can be changed to:

[0099] (5)

[0100] Its Lagrangian augmented form is:

[0101] (6)

[0102] in is the regularization parameter. Set the initial value . After a series of derivations, the iterative process can be obtained:

[0103] (7)

[0104] (8)

[0106] (9)

[0108] Third, the modified ADMM (Alternating Direction Method of Multipliersiteration) optimization algorithm is used to optimize the above model. The specific steps are as follows:

[0109] (1) Input parameters: sampling set , observed incomplete data , the regularization parameter , auxiliary variables , multiplier , . Maximum number of iterations Iter, allowable error .

[0110] (2) Initialization: , ;

[0111] (3) for

[0112] Update the auxiliary variables according to formula (7) ;

[0113] Update according to formula (8) ;

[0114] Update according to formula (9) ;

[0115] K=k+1;

[0116] if , then end cycle.

[0117] (4) Output the complete reconstructed tensor data.

[0118] Fourth, in order to verify the effectiveness of the method proposed in this paper, experiments were conducted on multiple standard image datasets, and the PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) indicators were used to compare with other mainstream image reconstruction methods. They are defined as follows:

[0119] (10)

[0120] in are the three dimensions of the image, for The recovery value.

[0121] (11)

[0122] in , Respectively , The mean of for The standard deviation of for The standard deviation of for and The covariance of , is a constant.

[0123] Experimental Results

[0124] 1. Image data

[0125] Color images contain red, green, and blue (RGB), which can be represented by a third-order tensor. Due to problems such as image corruption and partial sampling, the image cannot obtain complete observation data. We use PSNR and SSIM to evaluate the image reconstruction performance. We use natural landscape, plant, sky, mountain and other image examples 1-8 for performance comparison. The size of each example image is In the experiments, pixels were randomly sampled with sampling rates ranging from 10% to 90%.

[0126] Table I. PSNR and SSIM when the sampling rate is 10% to 90%

[0127]

[0128] Table II: PSNR and SSIM values ​​when the sampling rate is 10% to 90%

[0129]

[0130] Tables I and II show the comparison of different image restoration performance.

[0131] Figure 2 The image restoration results under different sampling rates are given. The first row: the original images with sampling rates of 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80% and 90%. The second to sixth rows are the restoration results of LTT, ELTR, FCTN, MSCN and TCFN respectively. The last row is the result obtained by our method.

[0132] Figure 3 A part of the selected image is enlarged, and the restoration effect is compared by observing the restored details. From the results, it can be seen that the restoration performance and effect of our proposed method have obvious advantages.

[0133] 2. Video image data

[0134] Video restoration is another important application of tensor integrity. We use the classic Akiyo video as the processing object. The size of each frame is ,The first ten frames are selected for testing. The sampling rates are 20%, 40% and 80% respectively. Table III shows the performance comparison of different methods.

[0135] Table III: PSNR and SSIM values ​​when the sampling rate is 20%, 40%, and 80% (video image)

[0136]

[0137] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement modes and shall be included in the protection scope of the present invention.

Claims

1. An image reconstruction method based on a tensor nuclear norm model, characterized in that: The following steps are involved: S01: Represent the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and perform tensor model expansion; S02: Based on the weighted non-directional truncated tensor nuclear norm, an optimization model for incomplete observation tensor recovery is obtained; the weighted non-directional truncated tensor nuclear norm for: ， In the formula , , , for The SVD decomposition of the orthogonal matrix, represents the matrix transpose, is a diagonal matrix, is the nuclear norm, is the identity matrix; S03: Optimizing the optimization model for recovering the incomplete observation tensor to obtain complete reconstructed tensor data.

2. The image reconstruction method based on the tensor nuclear norm model according to claim 1, characterized in that: Tensor in step S01 Mode n Expand for: ， For tensors The pattern n expansion, inverse operation is the tensor recovery of mode n, represents the real number space of dimension N, Represents each dimension, .

3. The image reconstruction method based on the tensor nuclear norm model according to claim 1, characterized in that: The optimization model for incomplete observation tensor recovery obtained in step S02 is: ， in, For the sampling set The mapping operator on , , for A subset of .

4. The image reconstruction method based on the tensor nuclear norm model according to claim 3, characterized in that: It also includes deformation of the optimization model for incomplete observation tensor recovery, introducing dimensionality and Same auxiliary variables , the optimization model is rewritten as: ， Its Lagrangian augmented form is: ， in, is the regularization parameter, for The Frobenius norm of , Is the multiplier.

5. The image reconstruction method based on the tensor nuclear norm model according to claim 4, characterized in that: The model is optimized using the modified ADMM optimization algorithm. The specific steps include: Setting initial values ; After a series of derivations, the iterative process is: ， in, is the number of iterations; Update auxiliary variables , tensor , multiplier ; when , When the allowable error or the number of iterations k is greater than the maximum number of iterations, the complete reconstructed tensor data is output.

6. An image reconstruction system based on a tensor nuclear norm model, characterized in that: include: The tensor model expansion module represents the image as a third-order tensor, where the first dimension represents the image height, the second dimension represents the image width, and the third dimension represents the number of image channels, and performs tensor model expansion; The optimization model building module obtains the optimization model for incomplete observation tensor recovery based on the weighted non-directional truncated tensor nuclear norm; the weighted non-directional truncated tensor nuclear norm for: ， In the formula , , , for The SVD decomposition of the orthogonal matrix, represents the matrix transpose, is a diagonal matrix, is the nuclear norm, is the identity matrix; The optimization module optimizes the optimization model of the incomplete observation tensor recovery through the optimization algorithm to obtain the complete reconstructed tensor data.

7. The image reconstruction system based on the tensor nuclear norm model according to claim 6, characterized in that: The optimization model for incomplete observation tensor recovery obtained in the optimization model building module is: ， in, For the sampling set The mapping operator on , , for A subset of .

8. The image reconstruction system based on the tensor nuclear norm model according to claim 7, characterized in that: It also includes deformation of the optimization model for incomplete observation tensor recovery, introducing dimensionality and Same auxiliary variables , the optimization model is rewritten as: ， Its Lagrangian augmented form is: ， in, is the regularization parameter, for The Frobenius norm of , Is the multiplier.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the image reconstruction method based on the tensor nuclear norm model described in any one of claims 1 to 5 is implemented.

Images