Color image restoration method based on adaptive tensor rank completion

Abstract

The invention belongs to the technical field of image restoration, and particularly relates to a color image restoration method based on adaptive tensor rank completion. The method comprises the following steps: firstly, processing a to-be-restored image to generate a reference image and a to-be-complemented high-order tensor; generating a plurality of candidate initial tensor ranks for the to-be-completed high-order tensor based on the Sobel sequence; thirdly, complementing the high-order tensor to be complemented based on the candidate initial tensor rank, and selecting the candidate initial tensor rank corresponding to the reconstructed image with the highest structural similarity with the reference image as an initial tensor rank; then, generating a plurality of first-order candidate tensor ranks, determining a first-order basic tensor rank, and performing variation on the first-order basic tensor rank to obtain a first-order variation tensor rank; reconstructing the high-order tensor to be complemented based on the first-order variation tensor rank to obtain a first-order optimal reconstructed high-order tensor; the to-be-completed high-order tensor is completed according to the first-order optimal reconstruction high-order tensor, and first iteration of the to-be-restored image is completed; and finally, performing repeated iteration on the to-be-restored image, complementing the to-be-complemented high-order tensor based on each order of optimal candidate tensor rank, and taking the reconstructed image with the highest structural similarity as the optimal restored image. According to the method, the low-rank property of the image can be better mined, the recovery effect is improved, and meanwhile, the overfitting problem caused by the unbalanced initial rank is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image restoration, and in particular is a color image restoration method based on adaptive tensor rank completion. Background Art

[0002] With the advancement of science and technology, digital images have been widely used in many fields such as medicine, industrial production, life and entertainment. In the process of image acquisition, transmission and storage, it is inevitable that various interferences will occur, resulting in partial data loss. In order to solve this problem, image restoration is needed. In particular, color images are essentially in the form of tensors, so color image restoration is essentially a tensor completion problem.

[0003] The tensor completion problem has been effectively solved under the low-rank constraint. According to the different ways of characterizing the tensor rank, that is, the different ways of tensor decomposition, tensor completion can be divided into methods based on Tucker decomposition, tensor train (TT) decomposition and tensor ring (TR) decomposition. Different decomposition models have differences in capturing the global correlation of tensors. Tucker decomposition decomposes high-dimensional tensors into a core tensor and a corresponding dimensional factor matrix. The corresponding expansion matrix is ​​a form in which a certain dimension corresponds to all the remaining dimensions. This unbalanced expansion method limits the Tucker model's ability to capture the correlation of high-dimensional tensors. TT decomposition decomposes high-dimensional tensors into multiple second-order or third-order tensors and connects them through TT ranks. Unlike Tucker decomposition, TT decomposition is expanded along the dimension to obtain a more balanced expansion matrix. Especially after using Ket enhancement to increase the dimension of low-dimensional tensors, it is better than the Tucker model in capturing the global correlation of high-dimensional tensors. However, Ket enhancement technology destroys the original data structure and causes artifact blocks, which affects the visual effect of the image and reduces the accuracy of image restoration. Based on TT, the TR model realizes the connection of all adjacent dimensions and adopts a circular expansion method to obtain the most balanced expansion matrix, which greatly enhances the model's ability to capture correlations and achieves outstanding results in the problem of tensor completion.

[0004] At present, tensor completion methods based on the Tucker model need to make convergence judgments during the iteration process, so as to adaptively determine the Tucker rank of the tensor, which may easily lead to overfitting or underfitting of the reconstructed image. Methods based on the TT model and the TR model need to obtain the rank information of the complete image data in advance, and continuously adjust the parameters according to the experimental results of the comparison with the original image in order to achieve the best reconstruction effect. However, in practical applications, the data of the image to be reconstructed is often incomplete, and it is difficult to obtain the necessary rank information, which limits the practical application of the TT and TR methods. Recently, the adaptive reconstruction method based on backtracking search uses the reference image to guide the TT rank search, which solves the need for the original rank information of the existing TT decomposition method. However, the unbalanced initial rank will lead to overfitting of the reconstructed image and cause the backtracking search to fall into the local optimum, thus limiting the image restoration effect. Summary of the invention

[0005] In view of the deficiencies in the prior art, the technical problem to be solved by the present invention is to propose a color image restoration method based on adaptive tensor rank completion.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] A color image restoration method based on adaptive tensor rank completion, characterized in that the method comprises the following steps:

[0008] Step 1: Process the image to be restored to generate a reference image and a high-order tensor to be completed; generate multiple candidate initial tensor ranks for the high-order tensor to be completed based on the Sobel sequence;

[0009] Step 2: Completing the high-order tensors to be completed according to the ranks of the candidate initial tensors respectively, and obtaining the completed tensors corresponding to the ranks of the candidate initial tensors; converting the completed tensors into images of the same size as the image to be restored, and obtaining the reconstructed images corresponding to the ranks of the candidate initial tensors; calculating the structural similarity between each reconstructed image and the reference image, and taking the candidate initial tensor rank corresponding to the reconstructed image with the highest structural similarity as the initial tensor rank;

[0010] Step 3: According to the first-order tensor rank of the initial tensor rank, generate a first-order candidate tensor rank list consisting of multiple candidate tensor ranks; reconstruct the high-order tensor to be completed according to the candidate tensor rank to obtain multiple reconstructed tensors; convert each reconstructed tensor into an image of the same size as the image to be restored to obtain multiple three-dimensional reconstructed images; calculate the structural similarity between each three-dimensional reconstructed image and the reference image, and take the candidate tensor rank corresponding to the highest structural similarity as the first-order basic tensor rank; mutate the first-order basic tensor rank to obtain the first-order mutated tensor rank; according to the first-order mutated tensor rank, Reconstruct and convert the reconstructed tensor into an image of the same size as the image to be restored, and obtain the mutated reconstructed image; compare the structural similarity between the reconstructed images before and after the mutation and the reference image, and use the tensor rank corresponding to the reconstructed image with the highest structural similarity as the first-order optimal candidate tensor rank, use the first-order optimal candidate tensor rank to update the initial tensor rank, and use the reconstructed tensor corresponding to the first-order optimal candidate tensor rank as the first-order optimal reconstructed high-order tensor; according to the first-order optimal reconstructed high-order tensor, use the random assignment method to complete the high-order tensor to be completed, and complete the first iterative recovery of the image to be restored;

[0011] Step 4: Repeat step 3, and perform adaptive rank selection on the remaining tensor ranks of the initial tensor rank in turn to obtain the optimal candidate tensor ranks of each order, and form the optimal candidate tensor rank Pareto set; according to the optimal candidate tensor rank Pareto set, the high-order tensors to be completed are completed in turn, and the completed tensors are converted into images of the same size as the image to be restored, and the structural similarity between the reconstructed image and the reference image is calculated, and the reconstructed image with the highest structural similarity is selected as the optimal restored image, thereby completing the restoration of the color image.

[0012] Furthermore, in step 3, the improved adaptive T mutation algorithm is used to mutate the first-order basic tensor rank, and the mutation rule of the improved adaptive T mutation algorithm is:

[0013]

[0014] In the formula, is the first-order variation tensor rank, r 1, …,r N is the tensor rank of the first-order basic tensor rank, N is the rank; T(iter,r Max ) represents the upper bound r on the tensor rank Max Under the constraint, the T distribution with the number of iterations iter as the degree of freedom parameter.

[0015] Furthermore, in step 1, the k-th order tensor rank of the t-th candidate initial tensor rank is Calculated by the following formula:

[0016]

[0017] In the formula, represents the upper limit of the k-th tensor rank of the t-th candidate initial tensor rank; λ s Represents the Sobel sequence, which is randomly and uniformly distributed in the interval (0,1].

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] First, by using the assignment dimensionality increase method to convert the low-dimensional tensor into a high-dimensional tensor, the prior information can be further enhanced by combining the use of known area data to increase redundant information and fill in missing area data, which can better explore the low rank of the image and improve the restoration effect.

[0020] Second, the Sobel sequence is used to generate a list of candidate initial tensor ranks, which provides a more reasonable initialization scheme for adaptive rank backtracking search and solves the reconstruction overfitting problem caused by unbalanced initial rank schemes. Adaptive T mutation is used to avoid backtracking search from falling into local optimality. Under the multi-order tensor rank optimization task, the optimal candidate tensor rank Pareto set is constructed to obtain the optimal tensor rank and restored image.

[0021] Third, the image restoration method based on adaptive tensor rank completion of the present invention avoids the over-fitting and under-fitting problems caused by the rank estimation algorithm using convergence judgment, as well as the dependence on the complete information of the original image. At the same time, it is generalizable and is suitable for tensor completion methods of models such as Tucker, TT and TR. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the overall flow chart of the present invention;

[0023] Figure 2 The image restoration effect comparison between the improved adaptive rank estimation method of the present invention and the existing method is shown;

[0024] Figure 3 The image restoration effect of the improved method of the present invention is compared with that of the existing original image-based method. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is described in detail below with reference to specific drawings and embodiments, which are not intended to limit the protection scope of the present application.

[0026] The present invention provides a color image restoration method based on adaptive tensor rank completion (hereinafter referred to as method, see Figures 1 to 3 ), including the following steps:

[0027] Step 1: Process the image to be restored to generate a reference image and a high-order tensor to be completed; generate a candidate initial tensor rank list d based on the Sobel sequenceinit ;

[0028] 1.1) With size 2 p ×2 p ×3 images to be restored For example, is a real number domain, p is a positive integer; the restored image is treated by interpolation method Complete the image to get the reference image containing complete image information

[0029] 1.2) Using tensor enhancement method to restore the image The data is rearranged to restore the image without changing the data structure. Convert to a higher-order tensor For high-order tensors according to Repeated superposition is performed in the form of The tensor rank is expressed as is the kth tensor rank of the high-order tensor to be completed, and N=p+2 is the high-order tensor to be completed The order of

[0030] 1.3) Completing high-order tensors based on Sobel sequences Generate M candidate initial tensor ranks and obtain the candidate initial tensor rank list d init =[d 1, …,d t ,…,d M ];in, represents the tth candidate initial tensor rank, represents the k-th tensor rank of the t-th candidate initial tensor rank, which is calculated by the following formula:

[0031]

[0032] In the formula, represents the upper limit of the k-th tensor rank of the t-th candidate initial tensor rank; λ s is a Sobel sequence, randomly and uniformly distributed in the interval (0,1].

[0033] Step 2: Get the initial tensor rank;

[0034] Based on the candidate initial tensor rank list d init According to the tth candidate initial tensor rank, the parallel matrix decomposition model of formula (2) is used to complete the high-order tensor Complete and get the completed tensor corresponding to the rank of the t-th candidate initial tensor

[0035]

[0036] In the formula, α k is the weighting factor, X [k] Represents a completed tensor The k-th order expansion matrix of and Represents the expanded matrix X [k] The decomposition matrix, r k Represents the expanded matrix X [k] The estimated rank of I j Represents the high-order tensor to be completed The size of the jth dimension; Represents a completed tensor and the high-order tensor to be completed The elements in the known data set Ω remain equal;

[0037] The completion tensor corresponding to the rank of the t-th candidate initial tensor Perform the inverse operation of structured addressing according to the index and convert it into the image to be restored. For images of the same size, the reconstructed image corresponding to the tth candidate initial tensor rank is obtained Similarly, the reconstructed images corresponding to the remaining candidate initial tensor ranks are obtained;

[0038] Calculate each reconstructed image and the reference image separately The structural similarity between them is used to select the candidate initial tensor rank corresponding to the reconstructed image with the highest structural similarity as the initial tensor rank in, is the initial tensor rank r init The k-th tensor rank of .

[0039] Step 3: According to the initial tensor rank r init The first-order tensor rank of , generates a list of first-order candidate tensor ranks Determine the first-order basic tensor rank, perform mutation operation on the first-order basic tensor rank based on the improved adaptive T mutation algorithm, and obtain the first-order variation tensor rank; reconstruct the high-order tensor to be completed based on the first-order variation tensor rank, and obtain the first-order optimal reconstructed high-order tensor; complete the high-order tensor to be completed according to the first-order optimal reconstructed high-order tensor, and obtain the high-order tensor to be completed after the first assignment and dimensionality increase, and complete the first iterative recovery of the image to be restored;

[0040] 3.1) For the initial tensor rank The first-order tensor rank r 1 initAdd different offsets to generate v candidate tensor ranks, which constitute the first-order candidate tensor rank list in, is the rank of the u-th candidate tensor, δ u is the offset list δ = [-2 i , -2 i-1 , …, 2 0 , …, 2 i -1 , 2 i ];

[0041] 3.2) List of first-order candidate tensor ranks Minimize the rank of each candidate tensor in and use formula (2) to complete the high-order tensor Reconstruct and obtain v reconstruction tensors; perform the inverse operation weighting of the tensor enhancement method in step 1.2) on the reconstruction tensors to convert each reconstruction tensor into a tensor with the image to be restored. Images of the same size to obtain a 3D reconstructed image Calculate the 3D reconstruction images and the reference images The structural similarity between them is based on the first-order candidate tensor rank list Determine the candidate tensor rank corresponding to the highest structural similarity through backtracking search Take the candidate tensor rank as the first-order base tensor rank Right now

[0042] Based on the improved adaptive T mutation algorithm for the first order basic tensor rank Perform mutation operation to obtain the first-order mutation tensor rank Recorded as Among them, AdpT(·) represents the improved adaptive T mutation function. The improved adaptive T mutation perturbs the tensor rank so that the algorithm has better global search ability. The mutation rule of the improved adaptive T mutation algorithm is:

[0043]

[0044] In the formula, r 1, …,r N is the first-order basis tensor rank The tensor rank of each order; T(iter,r Max ) represents the upper bound r on the tensor rank Max Under the constraint, the T distribution with the number of iterations iter as the degree of freedom parameter;

[0045] Based on the first-order variation tensor rank Treating Completion of High-Order Tensors Reconstruct and convert the reconstructed tensor into the image to be restored The images of the same size are used to obtain the reconstructed image after mutation; the structural similarity between the reconstructed images before and after mutation and the reference image is compared, and the tensor rank corresponding to the reconstructed image with the highest structural similarity is taken as the first-order optimal candidate tensor rank Use the first-order best candidate tensor rank Update the initial tensor rank r init ; Rank the first-order optimal candidate tensor The corresponding reconstruction tensor is used as the first-order optimal reconstruction high-order tensor

[0046] 3.3) Reconstructing high-order tensors based on first-order optimality Use random assignment to complete high-order tensors Perform completion to complete the first iterative restoration of the image to be restored;

[0047] Specifically, in the high-order tensor to be completed The missing region Ω c Randomly select 50% of the locations as the assignment area Ω Ra , using the first-order optimal reconstruction of high-order tensors For the assignment region Ω Ra Complete the task and obtain the high-order tensor to be completed after the first dimension increase. The filling process is recorded as Represents the high-order tensor to be completed and optimally reconstruct high-order tensors In the assignment region Ω Ra The elements in remain equal;

[0048] The valued dimensionality raising technique proposed in the present invention uses the redundant information of the data itself to perform dimensionality raising in step 1.2), and at the same time, assigns values ​​to the high-order tensor to be completed based on the reconstructed data, thereby enhancing the low-rank correlation of the image by combining known and unknown information.

[0049] Step 4: Repeat the operation in step 3, and perform adaptive rank selection on the remaining tensor ranks of the initial tensor rank in turn to obtain the optimal candidate tensor rank of each order. Then we get the optimal candidate tensor rank Pareto set According to the optimal candidate tensor rank Pareto set d P Treat the completed high-order tensors one by one The image is completed and the structural similarity between each reconstructed image and the reference image is calculated. The reconstructed image with the highest structural similarity is selected as the optimal restored image, thus completing the restoration of the color image.

[0050] In order to verify the effectiveness of the method of the present invention, the method of the present invention is extended to Tucker, TT and TR decomposition methods. Figure 2 and Figure 3 The improvement method of the present invention is compared with the existing method in restoring color images. Figure 2 It can be seen that compared with the Tucker adaptive rank estimation method using convergence judgment and the TT adaptive rank estimation method using only backtracking search, the improved rank estimation method proposed in the present invention has achieved the best recovery effect in all missing cases. On the one hand, due to the use of the Sobel sequence to generate a list of candidate initial tensor ranks, a more balanced candidate initial tensor rank is obtained, thereby avoiding the underfitting effect of image reconstruction caused by the backtracking search due to the unbalanced initial tensor rank that is too low or too high. On the other hand, the improved adaptive T mutation is used to mutate the tensor rank, which enhances the global search capability of the backtracking search and prevents the algorithm from falling into the local optimum. The optimal candidate tensor rank Pareto set can also ensure the selection of the optimal tensor rank and the restoration of the image. Figure 3 It can be seen that compared with the TT and TR methods that use the original complete image to obtain the tensor rank, the method of the present invention obtains the optimal structural similarity performance without any prior complete rank information. This is because while constructing the high-order tensor to be completed, the redundant information of the data itself is used to increase the dimension, and the high-order tensor to be completed is randomly assigned and completed based on the reconstructed data. By combining known data with randomly assigned unknown data, the low-rank correlation of the image is enhanced.

[0051] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A color image restoration method based on adaptive tensor rank completion, characterized in that: The method comprises the following steps: Step 1: Process the image to be restored to generate a reference image and a high-order tensor to be completed; generate multiple candidate initial tensor ranks for the high-order tensor to be completed based on the Sobel sequence; Step 2: Completing the high-order tensors to be completed according to the ranks of the candidate initial tensors respectively, and obtaining the completed tensors corresponding to the ranks of the candidate initial tensors; converting the completed tensors into images of the same size as the image to be restored, and obtaining the reconstructed images corresponding to the ranks of the candidate initial tensors; calculating the structural similarity between each reconstructed image and the reference image, and taking the candidate initial tensor rank corresponding to the reconstructed image with the highest structural similarity as the initial tensor rank; Step 3: According to the first-order tensor rank of the initial tensor rank, generate a first-order candidate tensor rank list consisting of multiple candidate tensor ranks; reconstruct the high-order tensor to be completed according to the candidate tensor rank to obtain multiple reconstructed tensors; convert each reconstructed tensor into an image of the same size as the image to be restored to obtain multiple three-dimensional reconstructed images; calculate the structural similarity between each three-dimensional reconstructed image and the reference image, and take the candidate tensor rank corresponding to the highest structural similarity as the first-order basic tensor rank; mutate the first-order basic tensor rank to obtain the first-order mutated tensor rank; according to the first-order mutated tensor rank, Reconstruct and convert the reconstructed tensor into an image of the same size as the image to be restored, and obtain the mutated reconstructed image; compare the structural similarity between the reconstructed images before and after the mutation and the reference image, and use the tensor rank corresponding to the reconstructed image with the highest structural similarity as the first-order optimal candidate tensor rank, use the first-order optimal candidate tensor rank to update the initial tensor rank, and use the reconstructed tensor corresponding to the first-order optimal candidate tensor rank as the first-order optimal reconstructed high-order tensor; according to the first-order optimal reconstructed high-order tensor, use the random assignment method to complete the high-order tensor to be completed, and complete the first iterative recovery of the image to be restored; Step 4: Repeat step 3, and perform adaptive rank selection on the remaining tensor ranks of the initial tensor rank in turn to obtain the optimal candidate tensor ranks of each order, and form the optimal candidate tensor rank Pareto set; according to the optimal candidate tensor rank Pareto set, the high-order tensors to be completed are completed in turn, and the completed tensors are converted into images of the same size as the image to be restored, and the structural similarity between the reconstructed image and the reference image is calculated, and the reconstructed image with the highest structural similarity is selected as the optimal restored image, thereby completing the restoration of the color image.

2. The color image restoration method based on adaptive tensor rank completion according to claim 1, characterized in that: In step 3, the improved adaptive T mutation algorithm is used to mutate the first-order basic tensor rank. The mutation rule of the improved adaptive T mutation algorithm is: In the formula, is the first-order variation tensor rank, r1,…,r N is the tensor rank of the first-order basic tensor rank, N is the rank; T(iter,r Max ) represents the upper bound r on the tensor rank Max Under the constraint, the T distribution with the number of iterations iter as the degree of freedom parameter.

3. The color image restoration method based on adaptive tensor rank completion according to claim 1 or 2, characterized in that: In step 1, the k-th tensor rank of the t-th candidate initial tensor rank is Calculated by the following formula: In the formula, represents the k-th tensor rank upper limit of the t-th candidate initial tensor rank; λ s Represents the Sobel sequence, which is randomly and uniformly distributed in the interval (0,1].

Images