A tensor singular spectrum analysis method for three-dimensional feature extraction of hyperspectral imagery

Abstract

The present application relates to remote sensing image processing technical field, specifically a kind of tensor singular spectrum analysis method for hyperspectral image three-dimensional feature extraction, including based on spatial self-similarity adaptive embedding, decomposition and low rank representation based on t-SVD, feature image and classification, compared with prior art, through adaptive embedding and t-SVD process, a new adaptive embedding operation, utilize the spatial similarity feature of HSI, jointly utilize target pixel and non-local similar pixel, combine with re-projection operation, keep target inter-class difference while enhanced intra-class similarity, by designing a trajectory tensor, combined with t-SVD, jointly represent the global low rank feature of HSI, the arrangement of similar pixel in trajectory tensor makes it have low rank feature, and further extract low rank feature by truncated t-SVD, realize the feature extraction of three-dimensional hyperspectral image, and then improve the class separability in hyperspectral image classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, in particular to a tensor singular spectrum analysis method for three-dimensional feature extraction of hyperspectral images. BACKGROUND

[0002] The rapid development of satellite platforms and sensor technology has made hyperspectral images widely used in environmental monitoring, land resource investigation and evaluation, urban planning and other fields. The classification effect of hyperspectral images greatly determines the effect of their application. However, the original hyperspectral images are often affected by sensor noise and environmental conditions, which leads to changes in the spectral characteristics of ground objects, resulting in poor classification performance. Therefore, effective feature extraction is crucial to improve the class separability in hyperspectral image classification.

[0003] In the past few decades, a series of feature extraction methods have been developed. Among them, linear transformation models such as principal component analysis (PCA) and linear discriminant analysis (LDA) have been widely used for spectral feature extraction of HSI. In addition, some manifold learning methods have been further developed to analyze the intrinsic features of HSI and improve the separability of spectral pixels. However, these methods only consider spectral information and ignore the potential role of spatial information. In recent years, spectral-spatial joint feature extraction methods have received extensive attention. In most spectral-spatial methods, spectral transformation methods are used to extract spectral features, and spatial methods are further used to extract spatial features to generate joint spectral-spatial features. In addition, some dual-branch networks have been proposed to extract features in the spectral domain and the spatial domain respectively, and to fuse the features of the two branches to improve the classification performance. In fact, the spectral and spatial processing of these methods is an independent process, and the final spectral-spatial features are obtained by simple feature fusion. However, these methods often ignore the joint dependence of the spectral continuity and spatial similarity specific to hyperspectral images.

[0004] To solve the above problems, some three-dimensional spectral space feature extraction methods are developed due to their potential to more effectively extract intrinsic features in high-dimensional data, which can be further divided into three categories: 3D filter-based methods, tensor-based methods and deep learning-based methods, in which a three-dimensional filter or operator is usually used to extract spectral-spatial features simultaneously, such as 3D Gabor, etc., but the dimension of the three-dimensional features obtained by these methods is usually huge, and the distinguishability of each module is different, in addition, the processing unit of these methods is a small three-dimensional block, which can only represent the local structure of hyperspectral image. Tensor is used for data analysis because it can effectively explore its low rank, in which tensor singular value decomposition (t-SVD) has been proposed for image inpainting and denoising, but there are relatively few studies on using it for feature extraction. Deep learning methods, such as the classic three-dimensional convolutional neural network (3DCNN), can extract spectral and local spatial features, but still face all the problems existing in deep learning, such as including the need for a large number of training samples, a large number of hyperparameters and the lack of model interpretability.

[0005] Recently, singular spectral analysis (SSA), a technique for time series analysis, has proven its ability in hyperspectral feature extraction. SSA acts on the spectral domain by embedding local and global spectral features of pixels. Similarly, its two-dimensional version (2DSSA) is able to fuse local and global features of a given band image through a two-dimensional embedding window, well preserving the global correlation. This makes it feasible to perform a global processing on the hyperspectral cube. However, neither SSA nor 2DSSA can extract spectral and spatial features simultaneously, although the state-of-the-art further proposes a spectral-spatial SSA (1.5DSSA), which only considers local spectral and spatial information, failing to describe the global correlation of HSI.

[0006] Therefore, it is necessary to design a tensor singular spectral analysis method for three-dimensional feature extraction of hyperspectral images, which can not only extract spectral and spatial features, but also consider the global information of the cube. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide a tensor singular spectral analysis method for three-dimensional feature extraction of hyperspectral images, which can not only extract spectral and spatial features, but also consider the global information of the cube. By constructing an effective trajectory tensor and through tensor decomposition and reconstruction, the low-rank intrinsic features of hyperspectral image can be obtained, solving the problems of low classification accuracy of traditional SSA, inability to extract 3-D spectral spatial features, and spectral variability, and deeply mining the intrinsic features of the image.

[0008] In order to achieve the above purpose, the present application provides a tensor singular spectral analysis method for three-dimensional feature extraction of hyperspectral images, comprising the following steps:

[0009] S1: Adaptive embedding based on spatial self-similarity

[0010] S101: For any one spectral pixel, determine a search region centered on it;

[0011] S102: In the search region, determine the pixel similarity between the neighborhood pixels and the center pixel according to the Euclidean distance;

[0012] S103: Select a certain number of neighborhood pixels in S102 and the similar pixels in the center pixel to form a matrix, and arrange all the pixel corresponding matrices into a trajectory tensor;

[0013] S2: Decomposition and low-rank representation based on t-SVD, t-SVD is tensor singular value decomposition

[0014] S201: Perform t-SVD decomposition on the obtained trajectory matrix to obtain the corresponding left singular tensor, right singular tensor and tube singular value tensor;

[0015] S202: By selecting the reconstruction number, the corresponding singular tensor is truncated to reconstruct a low-rank trajectory tensor that retains the main intrinsic features;

[0016] S3: Feature image and classification

[0017] S301: Re-project the low-rank trajectory tensor to obtain a feature image with the same size as the original image;

[0018] S302: In the feature image, a certain number of samples of each type of feature are randomly selected as training, the remaining samples are used as test, and a support vector machine (SVM) classifier is used for precision evaluation.

[0019] The trajectory tensor in S103 is a pseudo-Hankel matrix corresponding to each band image in the spatial direction, and retains the original spectral information in the spectral direction.

[0020] The selection method of similar pixels in S103 is to determine a local search region, and within the search region, a certain number of pixels with the smallest Euclidean distance are selected as similar pixels according to the Euclidean distance.

[0021] The decomposition in S201 applies t-SVD to the trajectory tensor and performs decomposition in the Fourier transform domain.

[0022] The re-projection method in S301 is to project the high spectral image in the reverse embedding manner.

[0023] Compared with existing technologies, this invention achieves three-dimensional extraction of spectral and spatial features through adaptive embedding and t-SVD processes. By employing a novel adaptive embedding operation, it utilizes the spatial similarity features of HSI, jointly leverages target pixels and non-locally similar pixels, and combines them with corresponding reprojection operations. This enhances intra-class similarity while maintaining inter-class differences. By designing a trajectory tensor and combining it with t-SVD, the global low-rank features of HSI are jointly characterized. The arrangement of similar pixels in the trajectory tensor gives it low-rank features, and further extraction of low-rank features is achieved by truncating t-SVD. This enables feature extraction of three-dimensional hyperspectral images, thereby improving the class separability in hyperspectral image classification.

[0024] This invention improves upon the basic steps of traditional SSA in several aspects, primarily in the following areas: adaptive window construction, construction of low-rank trajectory tensors, and the appropriate application of t-SVD. In the adaptive window, a two-dimensional matrix of pixel number × band number is obtained based on the center pixel and its Euclidean distance to similar pixels. Compared to the traditional 2DSSA embedding method, this method includes more similar pixels within the window. Regarding trajectory tensor construction, this tensor simultaneously contains spectral and spatial information and exhibits good low-rank properties in all three modal directions. In the application of t-SVD, this is the first time t-SVD has been applied to the SSA method. It enables simultaneous spectral and spatial decomposition and reconstruction, and through truncation, effectively preserves the main low-rank intrinsic characteristics, which is beneficial for the preservation and purification of key information. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0026] Figure 2 A schematic diagram illustrating the adaptive search for similar pixels within a window;

[0027] Figure 3 A classification comparison chart of TSSA with 2DSSA, 3DGabor, 3DCNN and TRPCA;

[0028] Figure 4 This is a schematic diagram comparing the classification accuracy after feature extraction according to the present invention. Detailed Implementation

[0029] The present invention will now be further described with reference to the accompanying drawings.

[0030] This invention provides a tensor singular spectrum analysis method for three-dimensional feature extraction from hyperspectral images:

[0031] like Figures 1-4 As shown, it includes the following steps:

[0032] S1: Adaptive embedding based on spatial self-similarity

[0033] S101: For any one spectral pixel, determine a search region centered on it;

[0034] S102: In the search region, determine the pixel similarity between the neighborhood pixels and the center pixel according to the Euclidean distance;

[0035] S103: Select a certain number of neighborhood pixels in S102 and the similar pixels in the center pixel to form a matrix, and arrange all the pixel corresponding matrices into a trajectory tensor;

[0036] S2: Decomposition and low-rank representation based on t-SVD, t-SVD is tensor singular value decomposition

[0037] S201: Perform t-SVD decomposition on the obtained trajectory matrix to obtain the corresponding left singular tensor, right singular tensor and tube singular value tensor;

[0038] S202: By selecting the reconstruction number, the corresponding singular tensor is truncated to reconstruct a low-rank trajectory tensor that retains the main intrinsic features;

[0039] S3: Feature image and classification

[0040] S301: Re-project the low-rank trajectory tensor to obtain a feature image with the same size as the original image;

[0041] S302: In the feature image, a certain number of samples of each type of feature are randomly selected as training, the remaining samples are used as testing, and a support vector machine (SVM) classifier is used for precision evaluation.

[0042] The trajectory tensor in S103 is a pseudo-Hankel matrix corresponding to each band image in the spatial direction, and retains the original spectral information in the spectral direction.

[0043] The selection method of similar pixels in S103 is to determine a local search region, and within the search region, a certain number of pixels with the smallest Euclidean distance are selected as similar pixels according to the Euclidean distance.

[0044] The decomposition in S201 applies t-SVD to the trajectory tensor and performs decomposition in the Fourier transform domain.

[0045] The re-projection method in S301 is to project the high spectral image in the reverse embedding manner.

[0046] Embodiment:

[0047] The present application provides a tensor singular spectrum analysis method for three-dimensional feature extraction of hyperspectral images. The method is used for processing of hyperspectral images of Indian Pine area acquired by an AVIRIS (Airborne Visible Infrared Imaging Spectrometer) sensor. The spatial resolution and spectral resolution of the AVIRIS sensor are 10 meters and 10 nanometers respectively, the spectral range is 400-2400 nm, and there are 224 spectral bands in total. Four bands with a value of 0 are removed. In addition, the values of bands [104-108], [150-163] and 220 of the Indian Pine image are affected by water absorption bands, so these bands are also removed in this case. Therefore, the Indian Pine image used in this case has 200 bands, the image size is 145x145 pixels, and there are 16 types of ground objects (mainly different crops), and the remaining pixels are considered as background.

[0048] As shown in Figure 1 The present application provides a tensor singular spectrum analysis method for three-dimensional feature extraction of hyperspectral images, which comprises the following detailed steps:

[0049] 1. For each pixel of a given hyperspectral image, a search window with a size of w x w is first determined with the pixel as the center. For pixels at the edge, the pixels at the outer circle of the image are mirrored to supplement the pixels, so that each pixel has a search area with the same size. In the present application, the search window is set to have a size of 11x11.

[0050] 2. In the search area, the spectral similarity between the center pixel and the remaining pixels is determined, and the Euclidean distance is selected as the criterion. The obtained Euclidean distances are sorted in ascending order, and the first L similar pixels including the center pixel are selected, where L < w x w.

[0051] 3. The L spectral pixels corresponding to any pixel are arranged into a matrix where B is the number of bands of the hyperspectral image. Each matrix is flipped and arranged in the form of a tensor, as shown below:

[0052]

[0053] where T is the obtained trajectory tensor, and N is the number of pixels of the hyperspectral image. The trajectory tensor T has the characteristics of low rank, mainly reflected in the spatial column direction of the upper pixel similarity and the spectral direction of the band height correlation.

[0054] 4. For the obtained trajectory tensor, use t-SVD to perform tensor decomposition. First, perform Fourier transform on the trajectory tensor T to obtain the tensor T in the transform domain F . Then, perform SVD decomposition on each forward face of the tensor T F , to obtain the corresponding singular vectors and singular values, as follows:

[0055] [U F , S F , V F ] = SVD(T F )

[0056] where U F and V F are the obtained left singular tensor and right singular tensor, respectively, and S F is the tube singular tensor, whose size is and respectively. Finally, use the inverse Fourier transform to transform the obtained three singular tensors to the real number domain, to obtain and

[0057] 5. Determine the ideal reconstruction component number r, and use it as the truncated rank of the tensor, to perform truncation on the three singular tensors obtained in the previous step, as follows:

[0058] U' = U(:, 1:r, :)

[0059] S' = S(1:r, 1:r, :)

[0060] V' = V(:, 1:r, :)

[0061] where U', V' and S' are the truncated left singular tensor, right singular tensor and tube singular tensor, respectively. Then use the truncated tensors to perform trajectory tensor reconstruction, as follows:

[0062] T r = U' * S' * V' T

[0063] where T r represents the reconstructed trajectory tensor, which retains the main intrinsic low-rank characteristics of the original trajectory tensor and removes part of the influence components.

[0064] 6. Project the obtained reconstructed trajectory tensor back to the size of the original hyperspectral image, and the projection method is to project T rEach of the lateral slice matrix is disassembled into a plurality of pixels, and the pixels are returned to the original spatial position. For some pixels in certain positions, the pixels can be averaged to be within a normal numerical range. After processing, a feature image with the same size as the original hyperspectral image can be obtained.

[0065] 7. A certain number of pixel samples are randomly selected from the obtained feature image as training, and the remaining samples are used as testing. An SVM classifier is used to evaluate the accuracy of the features, and the image classification result is obtained.

[0066] In order to verify the classification effect of spatial feature extraction, 2% of the samples in the image are randomly selected as training, and the remaining samples are used for testing. The sampling method is stratified sampling, and the samples in the training set and the test set do not overlap. The accuracy evaluation index includes three: overall accuracy (OA), average accuracy (AA) and kappa coefficient. In addition, in order to avoid systematic errors and reduce random bias, the average classification accuracy and standard deviation of 10 independent training and testing are statistically analyzed.

[0067] Figure 4 The overall classification accuracy of the method of the present application and 2DSSA, 3DGabor, 3DCNN and TRPCA methods for Indian Pine image classification is given. As shown in Figure 4 compared with other 2-D and 3-D methods, the method of the present application can achieve better classification accuracy, and the accuracy standard deviation under different random samples is low, which reflects the robustness of the method classification. The classification result corresponding to the classification map is shown in Figure 3 As can be seen from the classification result of TSSA, the salt and pepper noise phenomenon existing in the ground object is eliminated, and the edge information of the ground object is better preserved, 2DSSA and 3DGabor change the original shape of the ground object, and 3DCNN and TRPCA exist classification spots.

[0068] Overall, the method of the present application has better feature extraction and classification ability.

[0069] The above describes the embodiments of the present application, but the scope of the present application is not limited to this. Users can make various changes without departing from the spirit of the present application, and the changes are included in the protection scope of the present patent.

[0070] The present application solves the problems that the prior art cannot characterize the local structure of hyperspectrum, needs a large number of training samples, a large number of hyperparameters and a model lacking interpretability, cannot simultaneously extract spectral and spatial features, only considers local spectral and spatial information, cannot describe the global correlation of HSI and the like, through constructing an effective trajectory tensor, and through tensor decomposition and reconstruction, obtaining the low-rank intrinsic feature of hyperspectrum, solving the problems of low classification precision of the traditional SSA, inability to extract 3-D spectral spatial features and spectral variability and the like, deeply mining the intrinsic feature of the image, having high processing efficiency, simultaneously processing the spatial and spectral dimensions, and being capable of reasonably evaluating the effectiveness of the extracted feature.

Claims

1. A tensor singular spectrum analysis method for hyperspectral image three-dimensional feature extraction, characterized in that, The method comprises the following steps: S1: adaptive embedding based on spatial self-similarity; S101: for any one spectral pixel, determine a search region centered on it; S102: in the search region, determine the pixel similarity between the neighborhood pixels and the center pixel according to the Euclidean distance; S103: select a certain number of neighborhood pixels in S102 to combine with the similar pixels in the center pixel as a matrix, and arrange the matrix corresponding to all pixels as a trajectory tensor; S2: decomposition and low-rank representation based on t-SVD (tensor singular value decomposition); S201: perform t-SVD decomposition on the obtained trajectory matrix to obtain corresponding left singular tensor, right singular tensor and tube singular value tensor; S202: by selecting a reconstruction number, the main intrinsic characteristic trajectory tensor is reconstructed by truncating the corresponding singular tensor; S3: feature image and classification: S301: re-project the low-rank trajectory tensor to obtain a feature image with the same size as the original image; S302: in the feature image, a certain number of samples of each type of feature are randomly selected as training samples, the remaining samples are used as test samples, and a support vector machine classifier is used for precision evaluation; The trajectory tensor in S103 is a pseudo-Hankel matrix corresponding to each band image in the spatial direction, and the original spectral information is preserved in the spectral direction; The selection method of similar pixels in S103 is to determine a local search region, and within the search region, a certain number of pixels with the smallest Euclidean distance are selected as similar pixels according to the Euclidean distance; The decomposition in S201 applies t-SVD to the trajectory tensor and performs decomposition in the Fourier transform domain; The re-projection method in S301 is to project the high spectral image in the reverse embedding manner; The left singular tensor, right singular tensor and tube singular value tensor in S201 are as follows: first, perform Fourier transform on the trajectory tensor T to obtain the tensor TF in the transform domain, then perform SVD decomposition on each forward section of the tensor TF to obtain the corresponding singular vector and singular value, as shown below: ； where UFand VFare the left and right singular value tensors obtained, respectively, and SFis the tube singular value tensor, whose size is , and ; finally, the inverse Fourier transform is used to transform the three singular tensors obtained into the real number domain, respectively, to obtain , and .

Images