Hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition

Abstract

The invention discloses a hyperspectral super-resolution reconstruction method based on subspace tensor decomposition, and the method specifically comprises the following steps: firstly, employing a singular value decomposition technology, extracting a spectrum subspace from a low-resolution hyperspectral image, and obtaining a hyperspectral subspace; and through a continuous difference regularization method, spectral representation which is closer to a real spectrum and has stronger continuity and higher smoothness is obtained. Then, learning is carried out on the multispectral image with the high spatial resolution, so that a tensor coefficient is obtained; specifically, block clustering is carried out on a multispectral image, coefficients in the same class group are integrated into a three-dimensional tensor, and low-rank tensor prior and row sparsity constraint are utilized to estimate the tensor coefficients. And finally, generating a super-resolution reconstruction result based on the obtained spectrum subspace dictionary and the subspace coefficient.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral image super-resolution reconstruction, and particularly to a hyperspectral super-resolution reconstruction method based on subspace tensor decomposition. Background Art

[0002] Due to its unique multi-dimensional data characteristics, hyperspectral images occupy an important position in the field of image processing. Their rich spectral information enables them to be widely applied in multiple fields such as land use mapping, agricultural monitoring, and water resource management. However, there is an inherent mutual restriction relationship between the spectral resolution and the spatial resolution of hyperspectral images, which hinders their further popularization in practical applications. Therefore, improving the spatial resolution of hyperspectral images has become an urgent technical problem to be solved currently.

[0003] To solve the above problems, an effective technical approach is to fuse a high-spatial-resolution multispectral image with a low-spatial-resolution hyperspectral image to generate a hyperspectral image with high spatial resolution. This fusion method can make full use of the spatial resolution advantage of the multispectral image and the spectral resolution advantage of the hyperspectral image, thereby achieving the improvement of the quality of hyperspectral data.

[0004] In recent years, tensor decomposition technology has received extensive attention and application in the field of hyperspectral imaging. This technology effectively captures the multi-dimensional characteristics of data by modeling and decomposing the high-order data structure of hyperspectral images. However, there are still many challenges in existing tensor decomposition-based algorithms. The most prominent one is the insufficient joint utilization of spectral information and spatial information of hyperspectral images, which limits the performance improvement of the algorithms in hyperspectral image processing. Summary of the Invention

[0005] According to the problems existing in the prior art, the present invention discloses a hyperspectral super-resolution reconstruction method based on subspace tensor decomposition, which specifically includes the following steps:

[0006] Step 1: Degrade a high-resolution hyperspectral image to generate a low-resolution hyperspectral image and a high-resolution multispectral image;

[0007] Step 2: Use singular value decomposition technology to extract the spectral subspace from the low-resolution hyperspectral image;

[0008] Step 3: Perform continuous difference regularization processing on the extracted spectral subspace to obtain a spectral representation that is closer to the real spectrum and has strong continuity and smoothness;

[0009] Step 4: Perform block clustering processing on the high-resolution multispectral image, and integrate the coefficients within the same clustering group into a three-dimensional tensor;

[0010] Step 5: Estimate the coefficients of the three-dimensional tensor based on the low-rank tensor prior and row sparsity constraint;

[0011] Step 6: Use the extracted spectral subspace dictionary and the estimated tensor coefficients for image fusion to generate a high-spatial-resolution hyperspectral image.

[0012] Further, the specific content of Step 1 is as follows:

[0013] Apply the spatial mean downsampling function to the high-spatial-resolution hyperspectral image to generate a low-spatial-resolution hyperspectral image. The high-spatial-resolution multispectral image is obtained by performing spectral dimensionality reduction on the high-spatial-resolution hyperspectral image using a spectral dimensionality reduction matrix.

[0014] Further, the specific content of Step 2 is as follows:

[0015] Step 2-1: The low-spatial-resolution hyperspectral image preserves the vast majority of spectral information. The spectral vectors of these two images exist in the same spectral subspace. In this paper, the low-resolution hyperspectral image X is decomposed to obtain an estimate of the low-dimensional subspace D. Estimate the low-dimensional spectral subspace D from the LR-HSI through singular value decomposition, that is:

[0016] X = U1Σ1V1 T

[0017] where Σ1 is a diagonal matrix containing the singular values of X and arranged in non-decreasing order; U1 and V1 are the left singular vector and right singular vector matrices respectively, and they are column-orthogonal.

[0018] Step 2-2: Usually, the number of large singular values in the singular value matrix is small, but they account for the vast majority of the sum of the singular values. Therefore, we retain the largest L singular values and their corresponding singular vectors to approximate X, obtaining:

[0019]

[0020] where, The preliminary estimate of the subspace D is as follows:

[0021]

[0022] Further, the specific content of Step 3 is as follows:

[0023] Applying a spectral smoothing constraint to the sub-tensor dictionary can ensure that the reconstructed spectrum has sufficient smoothness. By introducing a first-order difference operator M, multiplying the spectral dictionary by this operator to measure the smoothness of the spectrum, and performing regularization on it, the spectral smoothing constraint term is obtained:

[0024]

[0025] This regularization term in the model helps to maintain the smooth characteristics of the spectrum, improve the quality of the reconstruction result, and obtain the subspace D after spectral smoothing.

[0026] Further, the specific steps of step 4 are as follows:

[0027] Step 4-1: There are many similar blocks in the entire spatial domain. The high spatial resolution multispectral image retains most of the spatial information, so we can learn the non-local similarity from it. First, divide the high spatial resolution multispectral image space into several blocks with a spatial size of and a spectral size of s.

[0028] Step 4-2: Use the efficient k-means++ method to cluster the divided blocks into k groups. This method can make the initial centroids more dispersed, thus more likely to avoid falling into local optimal solutions and improving the efficiency at the same time.

[0029] Further, the specific steps of step 5 are as follows:

[0030] Step 5-1: In order to make full use of the non-local similarity of the image to impose a low tensor multi-rank constraint on the grouped coefficients, estimate the coefficients by solving the following problem:

[0031]

[0032] where is the tensor multi-rank regularization term, and λ is the regularization parameter.

[0033] Step 5-2: By using the l 1,1,2 norm to characterize the sparsity of the sub-tensor coefficients, the loss of image information is avoided. Therefore, we define the row sparsity term:

[0034]

[0035] where w k represents the weight coefficient, and ⊙ represents the dot product operation. Add the sparsity term to the tensor coefficients in each iteration to obtain the tensor coefficients with spectral smoothing properties.

[0036] Step 5-3: Use ADMM to solve the tensor coefficients. Stop the algorithm when the maximum number of iterations is reached. We set the number of iterations to 100.

[0037] Further, the specific steps of step 6 are as follows:

[0038] After obtaining the final subspace D and the final tensor coefficients C, we can get:

[0039] Z = DC

[0040] Thus, the reconstruction of high spatial resolution hyperspectral images is achieved.

[0041] A hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition disclosed by the present invention has the following beneficial effects. Through singular value decomposition (SVD) and low-rank constraint, the low-dimensional subspace of the low-resolution hyperspectral image can be effectively estimated, and combined with the information of the high-resolution multispectral image, the spatial resolution of the image can be significantly improved. By using block clustering and non-local similarity constraints, the ability to recover image details is further enhanced. The method introduces a spectral difference operator and spectral smoothing constraints, which can effectively reduce the noise and discontinuity in the spectral dimension and enhance the smoothness of spectral information. Through regularization processing, it is ensured that the reconstructed hyperspectral image has higher fidelity in the spectral dimension. The method also adopts low tensor multi-rank constraints and row sparse constraints, which can make full use of the non-local similarity of the image and avoid the loss of image detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0043] Figure 1 It is a tensor decomposition diagram of HR-HSI;

[0044] Figure 2 It is a flowchart of the method of the present invention;

[0045] Figure 3 It is a residual diagram and reconstruction result diagram of the balloon dataset in the CAVE dataset;

[0046] Figure 4 It is a residual diagram and reconstruction result diagram of the oil painting dataset in the CAVE dataset;

[0047] Figure 5 It is a PSNR result diagram of each band of the balloon dataset in the CAVE dataset;

[0048] Figure 6 It is a PSNR result diagram of each band of the oil painting dataset in the CAVE dataset. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] To make the technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention:

[0050] Such as Figure 2The hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition shown below specifically includes the following steps:

[0051] S1: Determine the hyperspectral dataset. During the initialization of the simulation experiment, the dataset used in the experiment comes from the CAVE database. Each type of scene image in the CAVE database contains 31 spectral bands, and the spectral coverage range is 400nm - 700nm. Two datasets of balloon and oil painting selected from this database have a spatial size of 512×512 and contain 262,144 pixels in total.

[0052] S1-1: Use the original high-resolution hyperspectral image in this dataset as the ground truth of the ground object, and then use the spatial mean downsampling function to act on the high-spatial-resolution hyperspectral image to generate a low-spatial-resolution hyperspectral image.

[0053] S1-2: The high-spatial-resolution hyperspectral image is obtained by performing spectral dimensionality reduction on the high-spatial-resolution hyperspectral image using the spectral dimensionality reduction matrix provided by the Nikon700 camera.

[0054] S2: As Figure 1 shown, decompose the low-resolution hyperspectral image X to obtain an estimate of the low-dimensional subspace. Estimate the low-dimensional spectral subspace from the LR-HSI through singular value decomposition (SVD), that is:

[0055] X = U1Σ1V1 T

[0056] where Σ1 is a diagonal matrix containing the singular values of X and arranged in non-decreasing order; U1 and V1 are the left singular vector and right singular vector matrices respectively, and they are column-orthogonal.

[0057] S3: Usually, the number of large singular values in the singular value matrix is small, but they account for the vast majority of the sum of the singular values. Therefore, we retain the largest L singular values and their corresponding singular vectors to approximate X, and obtain:

[0058]

[0059] where The preliminary estimate of the subspace D is as follows:

[0060]

[0061] S4: Apply the spectral difference operator to the learned spectral subspace to enhance the smoothness of the spectral information;

[0062] For the sub-tensor dictionary, impose a spectral smoothness constraint. Then, the spectrum reconstructed according to the spectrally smoothed sub-tensor dictionary will also be smooth enough.

[0063] By introducing a first-order difference operator M, multiplying the spectral dictionary by the first-order difference operator to obtain a measure of spectral smoothness, and regularizing the spectral smoothness to obtain a spectral smoothing term:

[0064]

[0065] Thus, the subspace D after spectral smoothing is obtained.

[0066] S5: Perform block clustering on the multi-spectral image;

[0067] Specifically, adopt the following method: Divide the HR-MSI space into several blocks with a spatial size of and a spectral size of s, and then cluster the blocks into k groups using the efficient k-means++ method;

[0068] S5: Impose low tensor multi-rank and row sparse constraints on the grouped coefficients;

[0069] S5-1: To make full use of the non-local similarity of the image, we impose low tensor multi-rank on the grouped coefficients and estimate the coefficients by solving the following problem:

[0070]

[0071] S5-2: By using the l 1,1,2 norm to characterize the sparsity of the sub-tensor coefficients and avoid the loss of image information, we thus define the row sparse term:

[0072]

[0073] where w k represents the weight coefficient, and ⊙ represents the dot product operation. Add the sparse term to the tensor coefficients in each iteration to obtain the tensor coefficients with spectral smoothing properties.

[0074] S5-3: Use ADMM to solve the tensor coefficients and stop the algorithm when the maximum number of iterations is reached. We set the number of iterations to 100.

[0075] S6: After obtaining the final subspace D and the final tensor coefficients C, we can get:

[0076] Z = DC

[0077] Thus, the super-resolution reconstruction of the high spatial resolution hyperspectral image is realized.

[0078] The present invention proposes a hyperspectral image super-resolution reconstruction method based on subspace tensor learning and conducts experimental verification on two subsets of "balloon" and "oil painting" in the CAVE dataset. During the super-resolution reconstruction process, through subjective evaluation (such as Figure 3 andFigure 4 As shown, it presents the false color composite images generated by five comparison algorithms and this method, as well as their corresponding error images, in the seventh, fifteenth, and twentieth bands. The experimental results show that this method performs excellently in terms of the reconstruction results and minimizes the reconstruction error. To further quantify the performance of this method, five widely used hyperspectral super-resolution reconstruction quality evaluation metrics were introduced in the experiment, including peak signal-to-noise ratio (PSNR), spectral angle mapper (SAM), structural similarity (SSIM), root mean square error (RMSE), and error relative global dimension synthesis (ERGAS). Among them, the larger the PSNR and SSIM values, and the smaller the RMSE, SAM, and ERGAS values, the better the reconstruction effect. The relevant experimental results are shown in Tables 1 and 2. In addition, Figure 5 and Figure 6 respectively show the PSNR values of each band in the two datasets of "balloon" and "oil painting". The results show that this method achieves the optimal performance in most bands.

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] A hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition disclosed by the present invention. By combining spectral smoothing, low-rank constraint, and sparsity constraint, this method can significantly improve the quality of the reconstructed image, making it closer to the real high-resolution hyperspectral image. The reconstructed image has high precision in both spatial resolution and spectral resolution, and is applicable to fields such as remote sensing, medical imaging, environmental monitoring, etc. Through low-dimensional subspace estimation and tensor decomposition, this method can effectively compress the data volume of hyperspectral images, reduce storage and transmission costs, and reduce the dependence on hardware resources while ensuring the image quality. Therefore, this method can be widely applied to fields such as remote sensing image processing, medical image analysis, environmental monitoring, agricultural assessment, etc., providing an efficient and reliable solution for the super-resolution reconstruction of hyperspectral images. In the field of remote sensing, it can improve the accuracy of ground object classification and target recognition; in the medical field, it can enhance the detection ability of lesion areas.

[0084] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent replacements or changes, shall be covered by the protection scope of the present invention.

Claims

1. A hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition, characterized in that Including: Select a hyperspectral image dataset. Use the original high-resolution hyperspectral image as the ground truth of the ground object. Generate a low-resolution hyperspectral image through spatial mean downsampling. Perform spectral dimensionality reduction on the high-resolution hyperspectral image based on a spectral dimensionality reduction matrix to obtain a high-resolution multispectral image; Perform singular value decomposition on the low-resolution hyperspectral image to obtain a preliminary estimate of the low-dimensional subspace. Retain the first L largest singular values and their corresponding singular vectors to approximately represent the low-resolution hyperspectral image, and extract the spectral subspace of the low-resolution hyperspectral image; Apply a spectral smoothing constraint to the preliminarily estimated subspace. By introducing a first-order difference operator M, calculate the spectral smoothing term, and regularize the subspace to obtain the spectrally smoothed subspace D; Divide the high-resolution multispectral image into multiple blocks. Use the k-means++ method to cluster the blocks to obtain k groups of clustering results. Integrate the coefficients within the same clustering group into a three-dimensional tensor; Estimate the coefficients of a three-dimensional tensor based on a low-rank tensor prior and row sparsity constraints, impose a low tensor multi-rank constraint on the grouped coefficients, and estimate the coefficients through an optimization problem while using the norm l 1,1,2 Impose a row sparsity constraint on the coefficients; Use the ADMM algorithm to solve the tensor coefficients, set the maximum number of iterations to obtain the final tensor coefficients C, and perform image fusion according to the final subspace D and tensor coefficients C to generate a high-spatial-resolution hyperspectral image.

2. The hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition according to claim 1, wherein: The spectral subspace of the low-resolution hyperspectral image is obtained in the following manner: Represent the low-spatial-resolution hyperspectral image as X, and decompose X through singular value decomposition into the following formula: X = U1Σ1V1 T where, Σ1 is a diagonal matrix containing the singular values of X, and is arranged in non-decreasing order. U1 and V1 are the left singular vector and right singular vector matrices respectively; Obtain the diagonal matrix, the left singular vector and right singular vector matrices, retain the largest L singular values and their corresponding singular vectors to approximate X, and obtain: Among them, are the left singular vector matrix and the right singular vector matrix that respectively retain the largest L singular values; Obtain a preliminary estimate of the sub-spectral space D: Introduce a first-order difference operator M to the preliminary estimate of the spectral subspace D. Multiply the spectral dictionary by the first-order difference operator to obtain a measure of spectral smoothness, and perform regularization processing on the spectral smoothness to obtain the spectral smoothing term: Thus, obtain the spectrally smoothed spectral subspace D.

3. A hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition according to claim 1, characterized in that: The solution process of the tensor coefficients C is as follows: Divide the multi-spectral image space into several blocks with a spatial size of and a spectral size of s. Cluster multiple blocks into k groups using the k-means++ method, Apply a low-tensor multi-rank constraint to the grouped coefficients using the non-local similarity of the image, and estimate the coefficients by solving the following problem: Among them, is the tensor multi-rank regularization term, λ is the regularization parameter, and the ADMM algorithm is used to solve the above formula. By introducing the variable the augmented Lagrangian function is obtained: where μ is the penalty parameter, is the Lagrange multiplier, and the Lagrangian function is solved by solving the following problem: Updated Lagrange multiplier Solve to obtain the tensor coefficients C.

4. A hyperspectral image super-resolution reconstruction method based on subspace tensor decomposition according to claim 1, characterized in that: The process of imposing row sparsity constraints on the coefficients of the three-dimensional tensor is: By using the norm l 1,1,2 to characterize the sparsity of sub-tensor coefficients, define the row-sparse term: where w k represents a weight coefficient, and ⊙ represents a dot product operation Add a sparse term to the tensor coefficients for constraint in each iteration, thereby obtaining tensor coefficients with spectral smoothing properties.

Images