Unsupervised multivariate feature extraction method for hyperspectral image classification

Abstract

The invention discloses an unsupervised multivariate feature extraction method for hyperspectral image classification, and relates to the technical field of spatial-spectral joint feature extraction, and the method comprises the steps: carrying out the smooth denoising of a spectral curve through employing an MSMC, and capturing multi-scale spectral features through employing a cascade residual mode; gaussian smooth features of the hyperspectral remote sensing image are obtained by using MRCD, deep and shallow layer spatial features are extracted, and the deep and shallow layer spatial features are fused to obtain final spatial features; using M3FuNet to fuse the multi-scale spectral features and the final spatial features to obtain spatial-spectral joint features; the three features are input into a classifier to be summarized for decision fusion, and a spectrum decision, a space decision and a fusion decision are obtained; and obtaining a final classification decision of a spectrum decision, a space decision and a fusion decision through a majority voting mode. According to the method, spectral and spatial multiple scales of the hyperspectral remote sensing image are mined, characterization contributions of spectral features and spatial features are reserved, and single-class feature confrontation is relieved.

Description

Technical Field

[0001] The invention relates to the technical field of space-spectrum joint feature extraction, and in particular to an unsupervised multivariate feature extraction method for hyperspectral image classification. Background Art

[0002] Hyperspectral remote sensing images (HSI) have high spectral resolution, high feature dimension, and rich and continuous spectral curves of objects, and have the imaging characteristics of "image-spectrum integration". In the actual application of hyperspectral remote sensing images, there is a lot of redundant information in the high-dimensional feature data, and as the feature dimension increases, the classification accuracy will show the Hughes phenomenon of "increasing first and then decreasing". Therefore, the original data needs to be quickly reduced in dimension before application. Feature selection and feature extraction are currently two commonly used dimensionality reduction methods. Due to the high cost of prior manual labeling of hyperspectral remote sensing images and the small amount of labeled sample data, unsupervised feature extraction methods are usually used.

[0003] Traditional unsupervised learning methods include principal component analysis (PCA) and independent component analysis (ICA). PCA projects the original data into a low-dimensional subspace by finding an orthogonal transformation that maximizes the total variance of the projected data, but a large amount of local feature information in non-principal component directions will be lost in the application; ICA finds independent components by maximizing the statistical independence of estimated components, but cannot model nonlinear relationships in complex data.

[0004] Traditional machine learning-based feature extraction methods cannot express the nonlinear characteristics of hyperspectral remote sensing images well, while deep learning methods can better process nonlinear data, and the feature extraction of hyperspectral remote sensing images is better than traditional machine learning-based feature extraction methods to a certain extent. Chen et al. used convolutional neural networks (CNNs) to extract effective local spatial features from hyperspectral images. However, the above classical neural networks require a large number of prior labels, insufficient data information utilization, and low efficiency of hyperparameter iterative training. Hong et al. proposed a spectral Transformer (SpectralFormer) that can learn spectral local sequence information from adjacent frequency bands of hyperspectral images. Xu et al. proposed a random patch network (RPNet), which extracts random patches as convolution kernels from hyperspectral images without any pre-training and has high classification efficiency. However, the random patch convolution method in RPNet relies on deep and shallow layer features and randomness. While it can take into account global spatial information, it also brings the problem of unstable representation ability.

[0005] On the other hand, the classification performance of spectral features and spatial features in different classification networks or classifiers is different, and the retained feature information also has different emphases. In order to improve the classification accuracy, spatial-spectral joint features are usually used. The three-dimensional convolutional neural network model (3D-CNN) performs convolution operations on the hyperspectral cube to extract deep spatial-spectral features, but it still does not fully utilize the spectral information, and is prone to over-smoothing, resulting in the misclassification of small objects. Xu et al. proposed a joint classification method that integrates the spectral-spatial feature extraction branch and the classifier, and transfers the training error of the classifier to the feature extraction branch through back propagation, making the learned features more discriminative. Although the above-mentioned spatial-spectral joint classification method can extract multi-scale features in the spatial domain and the spectral domain when extracting features, it does not consider the spatial-spectral dependency, resulting in the fusion of features. Single-class feature confrontation is prone to occur, that is, the accuracy of some single classes of fused features decreases instead of increases. In the classification module, it relies too much on the representation ability of the fused features, and the spatial-spectral feature retention is low, which will lose some key feature information. Summary of the invention

[0006] The present invention provides an unsupervised multivariate feature extraction method for hyperspectral image classification, so as to solve the problem that the spatial-spectral joint classification method in the prior art does not consider the spatial-spectral dependency when extracting features, resulting in single-class feature confrontation in feature fusion and loss of some key feature information.

[0007] The present invention specifically provides the following technical solution: an unsupervised multivariate feature extraction method for hyperspectral image classification, comprising the following steps:

[0008] Acquire several hyperspectral remote sensing images;

[0009] The spectral feature extraction method MSMC is used to smooth and denoise the spectral curve of the hyperspectral remote sensing image, and the cascade residual method is used to capture the multi-scale spectral features of the smooth and denoised spectral curve;

[0010] The spatial feature extraction method MRCD is used to obtain Gaussian smoothing features of the hyperspectral remote sensing image, and deep and shallow spatial features are extracted based on the Gaussian smoothing features, and the deep and shallow spatial features are fused to obtain the final spatial features;

[0011] Using M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain spatial-spectral joint features;

[0012] The multi-scale spectral features, the final spatial features and the spatial-spectral joint features are input into the classifier for aggregation and decision fusion to obtain spectral decisions, spatial decisions and fusion decisions;

[0013] Obtaining the final classification decision of the spectral decision, spatial decision and fusion decision by majority voting;

[0014] The high light remote sensing image is processed through the final classification decision to obtain the final classification result.

[0015] Preferably, constructing the spectral curve based on the superpixel spectrum set and performing superpixel principal component analysis on the spectral curve comprises the following steps:

[0016] The ERS image segmentation algorithm is used to generate s non-overlapping homogeneous regions in the hyperspectral remote sensing image, that is, s super-pixels are obtained;

[0017] The graph G = (V, A) in the hyperspectral remote sensing image is divided into connected subgraphs, and the entropy rate term H(A) and the balance term B(A) are introduced into the objective function of the ERS image segmentation algorithm to optimize the superpixel segmentation.

[0018]

[0019] Among them, ω is used to balance the contribution of the entropy rate term H(A) and the balance term B(A), and is a weight greater than 0. E is the weight function v→R + ∪(0} is the image edge set consisting of pairwise similarities;

[0020] Construct hyperspectral cube data X∈R by imaging B continuous spectral bands M×N×B , and transform the three-dimensional hyperspectral cube data X∈R M×N×B Reconstructed into a two-dimensional matrix X = [x1, x2, ..., x B ]∈R L×B ;

[0021] From the high-dimensional space X∈R L×B The target dimension b is specified in the linear mapping in , and the first b eigenvectors are retained from large to small according to the eigenvalues ​​to form a projection vector matrix W, that is, y i =W T x i ; Where L = M × N;

[0022] The linear transformation matrix is ​​obtained by solving the objective function;

[0023] W * = argmaxTr(W T Cov(X)W),W T W=I

[0024] Where W T Cov(X) is the covariance matrix of X, Tr(X) is the square matrix of the trace of X, and I is the identity matrix;

[0025] The first principal component p1 of the hyperspectral data is obtained by superpixel principal component analysis, and the first principal component p1 is converted into a two-dimensional structure as the segmentation information standard pattern p1∈R of ERS M×N ;

[0026] Perform ESR segmentation on p1 to obtain superpixel segmentation label pattern I ERS ∈R M×N ;

[0027]

[0028] in, is the i-th superpixel;

[0029] Wherein, the ERS is empirically segmented by the spatial resolution of the HSI to calculate s;

[0030]

[0031] in is a round-down operation, res is the spatial resolution of HSI;

[0032] Repeat extracting the same number of principal components P for each homogeneous region, and finally concatenating all regional vectors to obtain a new hyperspectral matrix X∈R L×P .

[0033] Preferably, the method of using the spectral feature extraction method MSMC to smooth and denoise the spectral curve of the hyperspectral remote sensing image comprises the following steps:

[0034] The supervector block vectors corresponding to each band of the hyperspectral cube are concatenated into a matrix as the supervector matrix Among them, M s is the number of pixels in the homogeneous region, N s is the number of bands (N s =B);

[0035] The median absolute deviation algorithm MAD is used to smooth and denoise the supervector matrix.

[0036] Preferably, the M s Each row of data maintains similar eigenvalues, including the following steps:

[0037] Calculate W s The median of each row of data W median And all elements with W median The absolute bias vector bias;

[0038] Pick The median of the signal deviates from the critical value m ad ;

[0039]

[0040] Ms i =[m i , m2,…,m B ] is W s The i-th row vector, yes the median;

[0041] Fixed parameter α adjusts m ad The decision-making proportion, the data correction formula is as follows:

[0042]

[0043] W s Approximate mean constraints are added to each row of data to further smooth the supervector matrix;

[0044] Calculate the distribution principal number M of each row of the supervector matrix ad , directly use the M of this band ad Replace the eigenvalues ​​of these supervectors and correct the supervector blocks whose deviation is greater than a threshold;

[0045]

[0046]

[0047] Wherein MAD′(·) is the MAD correction algorithm;

[0048] Corrected W s Then the supervector and superpixel structure are restored, and all corrected superpixels are stitched together to restore the hyperspectral cube structure.

[0049] Preferably, the method of capturing multi-scale spectral features of the smoothed denoised spectral curve by using a cascaded residual method comprises the following steps:

[0050] Use 3×3 convolution, fill the border with mirror image, and set the radius r to 1 to calculate the Gaussian smoothing map Gau;

[0051] The standard pattern pixel minus the Gaussian smoothing image Gau obtains the difference value Δ, that is, the first principal component p1 is obtained;

[0052] If the absolute value of Δ is greater than the set threshold, the point is improved and sharpened;

[0053] Δ=p1(i,j)-Gau(i,j)

[0054]

[0055] Among them, Dau It is the sharpened image, abs(Δ) is the absolute value of Δ, and β adjusts the decision weight of Δ;

[0056] The number of superpixels is determined by a set of approximately smooth curve values, and the superpixel segmentation is performed. ERS Divide into 2c+1 homogeneous region images;

[0057]

[0058] Among them, c represents the range of the scale and the cth division method;

[0059] After inputting c feature samples into the classifier, the optimal decision result is obtained by decision fusion; the decision formula is:

[0060]

[0061] Where l is the label of one of the C categories, j is the number of the classifier, l j is the result of predicting the c-th segmentation scale feature; Nc i is the number of times the i-th class is predicted in the classifier, α j is the voting strength of the j-th classifier; where j∈{1, 2, …2c+1};

[0062] The cascade residual structure is introduced at different division scales, the first five principal components are retained, and the features of different scales are cascaded through the residual module to obtain the spectral feature X spectral ∈R L×p ;

[0063]

[0064] The obtained spectral feature samples are input into SVM to complete image classification.

[0065] Preferably, the method of using the spatial feature extraction method MRCD to obtain the Gaussian smoothing features of the hyperspectral remote sensing image comprises the following steps:

[0066] A random projection theory is introduced in RPNet to randomly transform the two-dimensional hyperspectral data X∈R L×B Projection to k-dimensional subspace X R ∈R L×k middle;

[0067] Keep the first P principal components of the original two-dimensional hyperspectral data, X P ∈R L×P , and then whiten it so that the data has similar variances in different dimensions;

[0068] Note Xp The covariance matrix of p , using singular value decomposition to obtain W P The left singular value vector V P and the singular value matrix U P ;

[0069]

[0070] Among them, X W ∈R L×P It is the low-dimensional feature after whitening, λ i is the i-th singular value of the singular value matrix;

[0071] The hyperspectral space is divided into c spatial sub-blocks with the same length and width as the convolution kernel Cov = [cov1, cov2, ..., cov c ], respectively for the whitened data X W Convolution is performed, the window movement step is fixed to 1, and the i-th convolution feature yes:

[0072]

[0073] in is a two-dimensional convolution operation, is the j-th dimension feature of the whitened data, is the j-th dimension feature of the i-th convolution kernel;

[0074] Combine all the convolutional features to get the c-dimensional spatial feature X spatial ∈R L×c .

[0075] Preferably, extracting deep and shallow spatial features based on the Gaussian smoothing feature, and fusing the deep and shallow spatial features to obtain the final spatial features comprises the following steps:

[0076] Use Gaussian smoothing on the i-th dimension of the spatial features Processing, set the blur radius to 1, that is, take the target center point h and the surrounding 8 points, and according to the specified weight parameter sigma, get the total weight W of these 9 points c , and then calculate the weighted average W of these 9 points cp ;

[0077] Use the weighted average W of these 9 points cp Divide by W c , and get the final weight matrix W gau ;

[0078]

[0079] Where ρ represents the weight parameter, Wgau As a Gaussian convolution kernel, update the eigenvalue of the center point h;

[0080] For spatial feature X spatiai ∈R L×c The features of each dimension repeat this convolution operation, so the i-th convolution feature Xcon i For:

[0081]

[0082] Where W gau It can be regarded as a two-dimensional Gaussian function W gau (x, y):

[0083]

[0084] Combine all the obtained Gaussian smooth features to obtain the c-dimensional spatial feature X spatial ∈R L×c ;

[0085] By cascading the spatial feature X spatial ∈R L×c The shallow features and deep features in are cascaded and fused to obtain the final feature set:

[0086]

[0087] Where n is the number of layers of random block convolution, that is, n = c-lay;

[0088] Finally, the obtained spatial feature samples are input into SVM to complete image classification.

[0089] Preferably, the use of M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain spatial-spectral joint features, including the following steps:

[0090] Through the M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain the spatial-spectral joint features.

[0091]

[0092] in, is the fusion feature, K is its spectral dimension;

[0093] Will Convert to 2D space Perform standardization operation:

[0094]

[0095] in, is the feature average value, is the characteristic standard deviation.

[0096] Preferably, the multi-scale spectral features, the final spatial features and the spatial-spectral joint features are input into a classifier for aggregation and decision fusion to obtain spectral decisions, spatial decisions and fusion decisions, including the following steps:

[0097] The extracted spectral features, spatial features and fusion features are input into the classifier in parallel to obtain the spectrum R a , spatial decision R b and the fusion decision R s ;

[0098] The three decisions are voted by majority to get the final classification decision R l .

[0099] Preferably, the three decisions are voted by majority to obtain the final classification decision R l , including the following steps:

[0100] Let i be a, b, s, i.e., the three feature flags, and n be p, nc, K, i.e., the dimensions of the three features, to obtain the final decision formula:

[0101]

[0102]

[0103] Among them, l i is its predicted i-th classifier decision, γ i is the decision strength of the i-th feature, and R represents the comparison with R a , R b , R s Three decisions.

[0104] Compared with the prior art, the present invention has the following significant advantages:

[0105] The present invention uses a spectral feature extraction method MSMC to perform smooth denoising on the spectral curve of the hyperspectral remote sensing image, adopts a cascade residual method to capture multi-scale spectral features of the smooth denoising spectral curve, realizes the alignment of spectral curves of adjacent bands, enhances the sparsity of the features of the homogeneous region, strengthens the spatial-spectral dependency, uses a spatial feature extraction method MRCD to obtain the Gaussian smoothing features of the hyperspectral remote sensing image, extracts deep and shallow spatial features based on the Gaussian smoothing features, fuses the deep and shallow spatial features to obtain the final spatial features, enhances the smoothness of the edge of the feature, improves the robustness of the feature space, and uses M 3FuNet mines the spectral and spatial multi-scale, deep and shallow features of hyperspectral remote sensing images, retains the representation contribution of spectral and spatial features, uses multi-decision fusion to achieve feature verification, and alleviates single-class feature confrontation. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 (a) is the spectrum curve of three different ground objects with different distributions. Figure 1 (b) is the local law of supervector block deviation from supervector (SvBD) diagram;

[0107] Figure 2 It is a flow chart of the SMC algorithm of the present invention;

[0108] Figure 3 Spectral classification diagram using the proposed MSMC in the present invention;

[0109] Figure 4 The spatial classification diagram for the present invention using the proposed MRCD;

[0110] Figure 5 It is a process diagram of Gaussian smoothing of the present invention;

[0111] Figure 6 M of the present invention 3 FuNet framework diagram;

[0112] Figure 7 The proposed M of the present invention 3 Architecture diagram of FuNet;

[0113] Figure 8 Pseudo-color images and ground truth maps of the Indian pine dataset of the present invention;

[0114] Fig. 9 Pseudo-color images and ground truth maps of the University of Pavia dataset of the present invention;

[0115] Fig.10 False color images and ground truth maps of the Salinas dataset of the present invention;

[0116] Fig.11 It is a visualization diagram of supervectors and supervector matrices corresponding to the three experimental data sets of the present invention;

[0117] Fig.12 It is a performance diagram of the cascade fusion of the ultra-low dimensional features of the present invention in three experimental data sets of different scales;

[0118] Fig.13 The first dimension feature map of the first layer of RPNet is extracted in 5 layers of Gaussian smoothing for three experimental data sets in this invention;

[0119] Fig.14 A graph comparing the performance of including Gaussian smoothing (curve yes) and excluding Gaussian smoothing (curve no) using three experimental data sets in multiple experiments for the present invention;

[0120] Fig.15 This is a diagram of parameter adjustment of M3FuNet using three experimental data sets of the present invention;

[0121] Fig.16 The classification diagram obtained by different methods on the Indian pine dataset of the present invention;

[0122] Fig.17 The classification diagram obtained by different methods on the University of Pavia dataset of the present invention;

[0123] Fig.18 The classification diagrams obtained by different methods on the Salinas dataset of the present invention;

[0124] Fig.19 The overall classification accuracy diagram of different methods and different numbers of training samples per category for three experimental data of the present invention ((a) Indian Pines, (b) Pavia University, (c) Salinas). DETAILED DESCRIPTION

[0125] The following is a clear and complete description of the technical solutions of the embodiments of the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0126] In order to overcome the limitation of supervector block shift on spectral feature interpretation, alleviate the instability of random convolution method representation and the single-class feature confrontation in space-spectrum joint classification, new spectral feature extraction methods MSMC and spatial feature extraction methods MRCD are proposed. In order to alleviate the single-class feature confrontation of feature fusion, MSMC and MRCD try not to consider spectral information or spatial information alone, so that the extracted features have similar representation degrees on the classifier, and improve the representation ability of joint features. In the spectral feature extraction method MSMC, the SuperPCA of multi-scale cascade residuals is introduced as a dimensionality reduction method to capture the spectral features of multi-scale homogeneous regions. Before dimensionality reduction, in order to overcome the supervector block shift, the superpixel unit of the neighboring band is corrected by the supervector matrix correction method SMC to achieve smooth denoising of the overall spectral curve, so that the offset spectral bands in the homogeneous region are aligned with the overall spectral curve, and the interpretability of the spectral curve in the homogeneous region is enhanced. Because spectral features will disperse the numerical values ​​of ground objects and increase the distance between classes, and the distance within a class needs to maintain the spatial-spectral dependency, random convolution and Gaussian smoothing are integrated into a new method in MRCD to extract deep and shallow spatial features under a multi-scale structure and improve the robustness of the feature space, thereby strengthening the spatial-spectral dependency of spatial-spectral feature fusion. 3 FuNet is used for hyperspectral remote sensing image classification. It adopts an end-to-end structure, uses MSMC and MRCD to extract spectral features and spatial features respectively, cascades the extracted spectral and spatial features, and uses the majority voting decision fusion method to obtain the final classification result, which improves the preservation of spatial-spectral features and expands the multi-feature decision strength of the classifier.

[0127] The following specifically describes an unsupervised multivariate feature extraction method for hyperspectral image classification provided by the present invention, which includes the following steps:

[0128] Step S1: Acquire several hyperspectral remote sensing images.

[0129] Step S2: Use the spectral feature extraction method MSMC to smooth and denoise the spectral curve of the hyperspectral remote sensing image, and use the cascade residual method to capture the multi-scale spectral features of the smoothed denoised spectral curve.

[0130] Superpixel principal component analysis (SuperPCA): In order to extract strong spatial-spectral dependencies, the spatial texture is divided into multiple homogeneous regions, and each homogeneous region learns a different principal component projection matrix. Including:

[0131] The ERS image segmentation algorithm is used to generate s non-overlapping homogeneous regions in the hyperspectral remote sensing image HSI, that is, s super-pixels are obtained. The graph G = (V, A) in the hyperspectral remote sensing image is divided into connected subgraphs, and the entropy rate term H(A) and the balance term B(A) are introduced into the objective function of the ERS image segmentation algorithm to optimize the superpixel segmentation.

[0132]

[0133] Among them, ω is used to balance the contribution of the entropy rate term H(A) and the balance term B(A), and is a weight greater than 0. E is the weight function v→R + The image edge set is composed of pairwise similarities of ∪{0}. The hyperspectral cube data X∈R is constructed by imaging B continuous spectral bands. M×N×B , the image rows corresponding to each band are M, and the columns are N. Before reducing the dimensionality of the spectral band, the three-dimensional hyperspectral cube data X∈R M×N×B Reconstructed into a two-dimensional matrix X∈R M×N×B Reconstructed into a two-dimensional matrix X = [x1, x2, ..., x B ]∈R L×B (L=M×N); thus each column represents all pixels in a band, namely, a pixel vector.

[0134] PCA from the original high-dimensional space X∈R L×B Find a linear mapping in the linear mapping, specify the target dimension b (b < B), retain the first b eigenvectors from large to small according to the eigenvalues ​​and combine them into a projection vector matrix W, that is, y i =W T x i From a mathematical point of view, the linear transformation matrix is ​​obtained by solving the objective function.

[0135] W * = argmaxTr(W T Cov(X)W),W T W=I

[0136] W * = argmaxTr(W T Cov(X)W),W T W=I

[0137] Where W T Cov(X) is the covariance matrix of X, Tr(X) is the square matrix of the trace of X, and I is the unit matrix. The first principal component p1, p1 of the hyperspectral data is obtained by PCA superpixel principal component analysis, and the first principal component p1 is converted into a two-dimensional structure as the segmentation information standard pattern p1∈R of ERS. M×N ; Perform ESR segmentation on p1 to obtain superpixel segmentation mark pattern I ERS ∈R M×N .

[0138]

[0139] in, is the i-th superpixel. Then there is no overlap between the s superpixels (homogeneous regions). Generally speaking, ERS can be empirically segmented by the spatial resolution of HSI to calculate s.

[0140]

[0141] in For the rounding operation, Fes is the spatial resolution of HSI. Repeatedly extract the same number of principal components P for each homogeneous region, and finally concatenate all regional vectors to obtain a new hyperspectral matrix X∈R L×P Super Vector Matrix Correction (SMC): Most hyperspectral remote sensing images have spectral heterogeneity. The spectral curves of similar objects are approximately consistent, but the spectral characteristic values ​​of each band of the imaging spectral curve will produce different degrees of distortion, such as Figure 1 (a) As shown. From the perspective of superpixels, the spectral curve distortion is manifested as the spectrum of some bands deviates significantly from the overall spectral curve (supervector), namely, supervector block deviation (SvBD), which destroys the spectral smoothness of the supervector, as shown in Figure 1 (b) as shown.

[0142] In order to alleviate the restriction of supervector block offset on supervector spectrum smoothness, the present invention proposes a supervector matrix correction method SMC. The algorithm flow is as follows: Figure 2 In this step, the spectral feature extraction method MSMC is used to smooth and denoise the spectral curve of the hyperspectral remote sensing image, including the following steps:

[0143] Since the number of pixels in all bands of the same homogeneous region of the hyperspectral cube is the same, the supervector block corresponding to each band is stretched into a one-dimensional vector (supervector block vector) of the same length. For ease of processing, the supervector block vectors corresponding to each band of the hyperspectral cube are concatenated into a matrix as the supervector matrix Among them, M s is the number of pixels in the homogeneous region, N s is the number of bands (N s =B). The median absolute deviation algorithm MAD for processing electrical signals is used to smooth and denoise the supervector matrix. s Each row of data maintains similar eigenvalues, including the following steps:

[0144] Calculate W s The median of each row of data W median And all elements with W median The absolute deviation vector Pick The median of the signal deviates from the critical value m ad .

[0145]

[0146] Ms i =[m1, m2, …, m B ] is W s The i-th row vector, yes The median of . Fixed parameter α adjusts m ad The decision-making proportion, the data correction formula is as follows:

[0147]

[0148] On this basis, W s Add approximate mean constraints to each row of data and smooth the supervector matrix. Correct the supervector blocks with large deviations and calculate the distribution principal number M of each row of the supervector matrix. ad , using the M ad Replace the eigenvalues ​​of these supervectors and correct the supervector blocks whose deviation is greater than a threshold;

[0149]

[0150]

[0151] Where MAD′(·) is the MAD correction algorithm. The corrected W s Then restore the supervector and superpixel structure, and then stitch all the corrected superpixels to restore the hyperspectral cube structure. The complete MSMC spectral feature extraction process is as follows: Figure 3 As shown. Before superpixel division, the sharpening algorithm contrast enhancement filter (USM) is used to enhance the contrast of the division standard pattern and strengthen the edge decision of the ERS algorithm on the image texture. The cascade residual method is used to capture the multi-scale spectral features of the smooth denoised spectral curve, including the following steps:

[0152] Use 3×3 convolution, fill the border with mirror image, and set the radius r to 1 to calculate the Gaussian smoothing map Gau, which is a low-pass filter. Subtract the Gaussian smoothing map Gau from the standard pattern pixel to get the difference value Δ, and get the first principal component p1. If the absolute value of Δ is greater than the threshold threshold, the point is improved and sharpened;

[0153] Δ=p1(i,j)-Gau(i,j),

[0154]

[0155] Among them, Dau It is the image after sharpening, abs(Δ) is the absolute value of Δ, and β adjusts the decision weight of Δ. This allows the ERS algorithm to better distinguish the edges of objects under a smaller number of segmentation settings, and reduce the interference of heterogeneous objects at the edges of the divided homogeneous regions. For the segmentation of superpixels, the number of superpixels of different scales is set to obtain spatially homogeneous regions of multiple scales. The spectral features obtained by dimensionality reduction also have the advantage of multiple scales. The number of superpixels is determined by a set of approximate smooth curve values. The segmentation of superpixels is divided into I ERS Divide into 2c+1 homogeneous region images.

[0156]

[0157] Among them, c is the scale range and the cth division method; in order to better utilize the advantages of multi-scale features, c feature samples are input into the classifier and decision fusion is used to obtain the optimal decision result; the decision formula is:

[0158] Where l is the label of one of the C categories, j is the number of the classifier, l j is the result of predicting the c-th segmentation scale feature; Nc i is the number of times the i-th class is predicted in the classifier, α j is the voting strength of the j-th classifier; Where j∈{1, 2,…2c+1}.

[0159] However, doing so generally brings a high time cost. For images with high spatial resolution, as the number of superpixels and feature decisions increase, the obtained multiple features have a certain degree of redundancy, and the generalization effect of a single feature on the classifier is not good. In order to solve this problem, a cascade residual structure is introduced at different division scales, and the first 5 principal components (ultra-low dimensional subspace) are retained to avoid excessive redundancy of multiple features. The features of different scales are cascaded through the residual module, so that the identification information retained by each can be complementary to compensate for the feature loss caused by the ultra-low dimensional subspace. In this way, after SMC and multi-scale cascade residual SuperPCA, the spectral feature X is obtained. spectral ∈R L×p .

[0160]

[0161] The obtained spectral feature samples are input into SVM to complete image classification.

[0162] Step S3: Use the spatial feature extraction method MRCD to obtain the Gaussian smoothing features of the hyperspectral remote sensing image, extract the deep and shallow spatial features based on the Gaussian smoothing features, and fuse the deep and shallow spatial features to obtain the final spatial features.

[0163] This step specifically includes the following steps:

[0164] A random projection theory is introduced in the random pattern network RPNet to randomly transform the two-dimensional hyperspectral data X∈R L×B Projection to k-dimensional subspace X R ∈R L×k In. Keep the first P principal components of the original two-dimensional hyperspectral data, X P ∈R L×P , and then whiten it so that the data has similar variances in different dimensions. P The covariance matrix of P , using singular value decomposition to obtain W p The left singular value vector V P and the singular value matrix U P The whitening formula is:

[0165]

[0166] Among them, X W ∈R L×P It is the low-dimensional feature after whitening, λ i is the ith singular value of the singular value matrix (arranged from large to small). The hyperspectral space is divided into c spatial sub-blocks with the same length and width as the convolution kernel Cov = [cov1, cov2, ..., cov c ], respectively for the whitened data X W Convolution is performed, the window movement step is fixed to 1, and the i-th convolution feature yes:

[0167]

[0168] in is a two-dimensional convolution operation, is the j-th dimension feature of the whitened data, is the j-th dimension feature of the ith convolution kernel. Due to the random characteristics of the random convolution method in RPNet, the overall mapping quality of the random convolution kernel determines the representation ability of the extracted features. The differences in the generated random patches will lead to unstable classification results. For this reason, Gaussian smoothing is introduced to disperse the convolution features. Since the distribution of various types of objects is unknown in advance, the convolution window often mixes heterogeneous data and abnormal data, interfering with the edge features of the objects. Gaussian smoothing can slow down the gradient of changes in spatial objects by dispersing data to a certain extent, improve the smoothness of the distribution of similar objects, better connect contextual information, reduce the representation instability caused by random convolution, and improve classification accuracy. The complete MRCD spatial feature extraction process is as follows: Figure 4 shown.

[0169] In this step, the deep and shallow spatial features are fused to obtain the final spatial features, including the following steps:

[0170] Gaussian smoothing uses a specified convolution window to take the weighted average of surrounding pixels for each pixel. The Gaussian smoothing process is as follows: Figure 5 Gaussian smoothing is a numerical dispersion method obtained by using the value of the Gaussian distribution graph Gau. Gaussian smoothing is used to calculate the i-th dimension of the spatial feature. Processing, set the blur radius to 1, that is, take the target center point h and the surrounding 8 points, and according to the specified weight parameter sigma, get the total weight W of these 9 points c , and then calculate the weighted average W of these 9 points cp ; At the same time, the sum of their weights needs to be equal to 1, using the weighted average W of these 9 points cp Divide by W c , and get the final weight matrix W gau .

[0171]

[0172] Where ρ represents the weight parameter. In this process, W cp =mean(W c ). gau As the Gaussian convolution kernel, update the feature value of the center point h, the Gaussian blur value of this pixel. spatial ∈R L×c The features of each dimension repeat this convolution operation, so the i-th convolution feature Xcon i for:

[0173]

[0174] Where W gau It can be regarded as a two-dimensional Gaussian function W gau (x, y):

[0175]

[0176] Combine all the obtained Gaussian smooth features to obtain the c-dimensional spatial feature X spatial ∈R L×c ; In this process, the spatial receptive field is also set unchanged to maintain the spatial context information of the feature.

[0177] Another advantage of the convolution method is that it can obtain deep features through different numbers of convolution layers. The random pattern Gaussian smoothing process is used as an independent convolution layer, and the number of random block convolution layers c-lay and the number of Gaussian blur layers g-lay are set as two independent parameters. Different levels of features are obtained according to the difference of the two parameters. The spatial feature X is cascaded spatial ∈R L×c The shallow features and deep features in the cascade fusion are combined to obtain spatial features with the advantages of shallow and deep layers. The final feature set is:

[0178]

[0179] Among them, n is the number of layers of random block convolution, that is, n = c-lay; the obtained spatial feature samples are input into SVM to complete image classification. The spectral feature extraction method MSMC is guided by the spatial superpixel distribution. The obtained spectral feature has obvious intra-class aggregation effect, enhances intra-class sparsity, focuses on correcting spectral heterogeneity, aggregates multi-scale features, and obtains the final spectral feature X spectral The spatial feature extraction method MRCD obtains deep and shallow convolution features by processing random patch convolution while maintaining the receptive field. It further applies Gaussian smoothing to the convolution features to guide the numerical dispersion. Under the condition that the inter-class distance is large enough, it alleviates the problem of unstable representation caused by random convolution. While extracting the global feature differences, it improves the smoothness of the edge of the feature and obtains the final spatial feature X spatial .

[0180] Step S4: Use M 3 FuNet fuses multi-scale spectral features and final spatial features to obtain spatial-spectral joint features. 3 The overall architecture of FuNet is as follows Figure 6 As shown. It includes the following steps:

[0181] By M 3 FuNet fuses multi-scale spectral features and final spatial features to obtain spatial-spectral joint features. The classification module inputs the three features into the classifier for decision fusion. Different network architectures have different focuses on feature extraction. The improved spectral feature extraction branch MSMC and the spatial feature extraction branch MRCD are fused to obtain complementary fusion features of spatial-spectral representation. The traditional spatial-spectral fusion framework uses three conventional fusion schemes: additive, multiplicative, and cascade. The dimensions of spatial features and spectral features are often inconsistent. In order to avoid the discussion of feature dimension issues caused by the discussion of consistent spatial-spectral dimensional fusion methods (additive, multiplicative), a more efficient cascade fusion method is used. This method has the advantage of maintaining the optimal feature space:

[0182]

[0183] in, is the fusion feature, and K is its spectral dimension. Before the extracted features are input into the classifier, in order to avoid the problems of singular values ​​and gradient explosion in the classifier, Convert to 2D space Perform standardization operation:

[0184]

[0185] in is the feature average value, is the characteristic standard deviation.

[0186] Step S5: Input the multi-scale spectral features, the final spatial features and the spatial-spectral joint features into the classifier for decision fusion to obtain spectral decision, spatial decision and fusion decision. It includes the following steps:

[0187] Different from the traditional feature fusion network framework, M 3 FuNet retains the feature contributions of the spectral branch MSMC and the spatial branch MRCD, and inputs the extracted spectral features, spatial features, and fusion features into the classifier in parallel to obtain the spectral decision R a , spatial decision R b and the fusion decision R s The three decisions are voted by majority to get the final classification decision R l .

[0188] Step S6: Obtain the final classification decision of spectral decision, spatial decision and fusion decision by majority voting. It includes the following steps:

[0189] Let i be a, b, s, i.e., the three feature flags, and n be p, nc, K, i.e., the dimensions of the three features. The decision formula is:

[0190]

[0191]

[0192] Among them, l i is its predicted i-th classifier decision, γ i is the decision strength of the i-th feature, and R represents the comparison with R a , R b , R s Three decisions. In order to objectively retain the discriminative information of all features, here γ i The value of M is always set to 1. 3 The specific process of FuNet is as follows Figure 7 shown.

[0193] Step S7: Process the high light remote sensing image through the final classification decision to obtain the final classification result.

[0194] Experimental results and analysis: Three commonly used hyperspectral remote sensing image datasets, Indian Pines, Pavia University and Salinas, were used to analyze the MSMC, MRCD and M 3 The effectiveness of the FuNet method in hyperspectral remote sensing image classification is verified, and it is compared with some classic methods and more advanced unsupervised feature extraction methods. 3 The influence of important parameters of FuNet on classification performance. In order to verify the effectiveness of MSMC in alleviating supervector offset and MRCD in alleviating the instability of random convolution method representation, the experimental results of the two methods were analyzed separately. Table 1 shows the relevant information of the three data sets, and Tables 2-4 show the number of divisions of the training set and test set of the three data sets in quantitative experiments. All control methods are divided according to this standard. Figure 8 , Fig. 9 and Fig.10 The fake color images and true labels of the objects in the three datasets are shown.

[0195] Experimental parameters: 1) Implementation Details: Implementation and experiments were carried out based on matlab2021a, Intel Core i5-3470CPU platform. In order to more accurately compare the classification performance of each method, the training samples are set to the same confidence level under the condition of unified division, and the spectral features, spatial features, and fusion features are input into SVM for classification. In order to evaluate the classification performance of the proposed method, the four evaluation indicators of the present invention are overall accuracy (OA), average accuracy (AA), producer accuracy (PA) and Kappa coefficient. In the proposed spectral branch MSMC, the retained dimension is fixed to 5 at each segmentation scale of all data sets, and the scale range c is fixed to ±4, that is, 2c+1=9 division scales. The first 7 scales are taken for the Indian Pines dataset, and the first 4 superpixel segmentation scales are taken for the Pavia University dataset. The first 3 superpixel segmentation scales are taken for the Salinas dataset. The number of superpixels of the three data sets are:

[0196]

[0197]

[0198]

[0199] In the proposed spatial branch MRCD, the sample dimension of all datasets obtained by PCA is fixed to 20, the blur radius sigma is set to 1 based on experience, the convolution window size is set to 3×3, and mirror padding is used when performing convolution operations on image edges. The parameters c-lay and g-lay are set to 5, 2 for the Indian Pines dataset. The parameters c-lay and g-lay are set to 3, 3 for the Pavia University dataset. The parameters c-lay and g-lay are set to 4, 3 for the Salinas dataset. Other parameter settings are discussed separately.

[0200] 2) Comparison With Classic and Advanced Methods: In order to comprehensively evaluate the performance of the proposed method from multiple perspectives, two machine learning classification algorithms were selected, including support vector machine (SVM) and random forest (RF), a relatively novel discriminant marginalized least squares (DMLSR) based on linear regression and sparseness was selected, and three feature extraction methods were selected, including superpixel principal component analysis (SuperPCA), random pattern network (RPNet), and superpixel adaptive singular spectrum analysis (SpaSSA). Parameter settings for the comparison method: the number of decision trees in RF was set to 200; the two regularization parameters λ1 and λ2 and the number of neighbors k in DMLSR were set to (1e-4, 1e-1, 1), (1e-4, 1e-1, 1), and (1e-4, 1e-1, 3) for the three data sets respectively; the number of superpixels divided in SuperPCA for the three data sets was 100, 20, and 100 respectively. The number of dimensionality reduction targets per layer of RPNet was set to 3, the convolution window size was 20x20, and the number of convolution layers was set to 5. The number of SpaSSA superpixels is fixed to 50, the two thresholds T1 and T2 are fixed to 3 and 10, and the one-dimensional embedding window is fixed to 10.

[0201] In order to ensure the generalization of the experimental classification ability, all methods were subjected to classification experiments 30 times, and all indicators were averaged. It is worth noting that except for the large fluctuation of the classification results of RPNet, the fluctuation of the classification results of other methods is small and almost negligible.

[0202] Table 1 Information about the dataset

[0203]

[0204] Table 2. Training and testing sample division of Indian Pines dataset

[0205]

[0206] Table 3. Training and testing sample division of Pavia University dataset

[0207] category Feature Name train test total 1 Asphaltl 67 6564 6631 2 Meadows 187 18462 18649 3 Gravel 21 2078 2099 4 Tress 31 3033 3064 5 Metal sheets 14 1331 1345 6 Bare Soil 51 4978 5029 7 Bitumen 14 1316 1330 8 Bricks 37 3645 3682 9 Shadows 10 937 947 total - 432 42344 42776

[0208] Table 4. Training and testing sample division of the Salinas dataset

[0209] category Feature Name train test total 1 Brocoli_green_weeds_1 21 1988 2009 2 Brocoli_green_weeds_2 38 3688 3726 3 Fallow 20 1956 1976 4 Fallow_rough_plow 14 1380 1394 5 Fallow_smooth 27 2651 2678 6 Stubble 40 3919 3959 7 Celerr 36 3543 3579 8 Grapes_untrained 113 11158 11271 9 Soil_vinyard_develop 63 6140 6203 10 Corn_senesced_green_weeds 33 3245 3278 11 Lettuce_romaine_4wk 11 1057 1068 12 Lettuce_romaine_5wk 20 1907 1927 13 Lettuce_romaine_6wk 10 906 916 14 Lettuce_romaine_7wk 11 1059 1070 15 Vinyard_untrained 73 7195 7268 16 Vinyard_vertical_tre11is 19 1788 1807 total - 549 53580 54129

[0210] Analysis of MSMC: First, we evaluate the effectiveness of the supervector matrix correction SMC proposed in the MSMC method for supervector block offset. SMC converts all homogeneous regions into supervector matrices, and then corrects the spectral units of homogeneous regions; select a supervector from each of the three data sets, and after SMC correction, visualize the supervector results before and after correction. Fig.11 . Fig.11 Each small figure is specifically: (a) Indian Pines before SMC, (b) Indian Pines after SMC, (c) Pavia University before SMC, (d) Pavia University after SMC, (e) Salinas before SMC, (f) Salinas after SMC. The supervector blocks represented by some bands of the supervector before correction will obviously deviate from the overall supervector curve. The overall spectral energy is highly irregular and the spectral variability is high. The two-dimensional supervector matrix shows that the spectral energy of the objects between bands is similar, and the spectral energy similarity of the objects within the band (spectral arrangement of the same object) is low. From the perspective of the spectrum between bands and the homogeneous area within a single band, it explains the problems of high band correlation and large distance within the object class. The spectral energy between each band of the corrected supervector is smoother, and some abnormal supervector blocks with too high offset are eliminated. The corrected supervector matrix shows that SMC strengthens the similarity of the energy of objects in homogeneous regions, shrinks the intra-class distance while maintaining the edge of the homogeneous region, and improves the distinguishability of spectra of heterogeneous objects.

[0211] In order to illustrate the generality of the effect of SMC, the classification results of the data before correction, i.e. RAW, are compared intuitively with the classification results of the 9 segmentation scales after correction. SVM is selected as the classifier. Tables 5-7 list the classification results of the three data sets before and after correction. No other feature extraction is performed on the HSI samples, and the spectral dimension is consistent with the original data. The Indian Pines data set performs best at the 7th scale, with OA 13%-15% higher than RAW, AA 21%-25% higher than RAW, and Kappa 15%-19% higher than RAW. The Pavia University data set performs best at the 1st scale, with OA 7%-10% higher than RAW, AA 5%-9% higher than RAW, and Kappa 9%-13% higher than RAW. The Salinas data set performs best at the 1st scale, with OA 9%-11% higher than RAW, AA 5%-9% higher than RAW, and Kappa 9%-12% higher than RAW.

[0212] Under the premise of using the SMC corrected data, the acceptance capacity of the superpixel dimensionality reduction method based on the residual structure in the ultra-low-dimensional subspace for the contribution of each scale feature is further analyzed. Fig.12 As the number of cascaded feature layers increases, extreme points of the accuracy curve appear in all three data sets. One extreme point of accuracy for the Indian Pines data set appears at the 7th scale, two extreme points of accuracy for the Pavia University data set appear at the 4th and 6th scales, and one extreme point of accuracy for the Salinas data set appears at the 3rd scale. This is also the basis for selecting the segmentation scale in the quantitative comparison experiment. In the case of cascaded residual superpixels, it can not only retain more identification information, but also effectively reduce data redundancy and thus improve the classifier interpretation speed. It is worth noting that since the MSMC method is a feature extraction method based on superpixel guidance and has the advantage of multi-scale features, it is a spatially guided spectral feature extraction that can interpret strong spatial-spectral dependencies. When such spectral features and spatial features are fused, the confrontation of single-class features is effectively alleviated.

[0213] Table 5 Indian Pines

[0214] - RAW 1 2 3 4 5 6 7 8 9 OA (%) 79.89 93.70 95.00 94.82 95.41 96.19 96.77 96.99 96.33 96.44 AA(%) 70.76 92.39 94.36 94.46 95.14 95.45 96.25 96.88 96.67 96.17 Kappa (%) 77.03 92.81 94.30 94.09 94.77 95.66 96.32 96.56 96.16 95.94

[0215] Table 6 Pavia University

[0216] - RAW 1 2 3 4 5 6 7 8 9 OA (%) 87.69 97.21 95.90 96.50 95.43 95.50 95.82 94.94 95.06 94.11 AA(%) 85.11 94.78 92.88 92.33 91.59 90.75 91.17 90.71 90.97 91.96 Kappa (%) 83.63 96.28 94.54 95.33 93.91 94.01 94.43 93.27 93.42 92.20

[0217] Table 7 Salinas

[0218] - RAW 1 2 3 4 5 6 7 8 9 OA (%) 87.99 98.96 98.96 97.84 97.83 97.39 97.25 97.13 95.77 96.62 AA(%) 92.87 98.75 98.94 98.65 98.65 98.39 98.33 98.31 97.65 98.12 Kappa (%) 86.63 98.84 98.84 97.60 97.58 97.09 96.94 96.80 95.28 96.24

[0219] Analysis of MRCD: The effect of Gaussian smoothing on alleviating the instability of random convolution feature representation is evaluated. In the experiment, the blur radius sigma of Gaussian smoothing is fixed to 1 and the edge mirror convolution method of 3×3 window size is used. First, the processing effect of 5-layer Gaussian smoothing on random pattern convolution features is visualized, and the first dimension of each layer feature is selected for display, such as Fig.13 As shown, Fig.13 (a) is the Indian Pines dataset, Fig.13 (b) Pavia University and Fig.13 (c) is the Salinas dataset. It is obvious that after each layer of Gaussian smoothing, the features of the objects are more prominent and the edges of the objects are smoother. This is based on the large inter-class distance of the random pattern convolution feature. Therefore, with enough information for identifying objects, Gaussian smoothing highlights the features of the objects and smoothes the easily confused parts of the edges of the objects, making the classifier more inclined to use deep features for comprehensive decision-making, thereby obtaining more robust concatenated deep features, which can alleviate the problem of unstable representation ability caused by random pattern convolution. In addition, in order to ensure that spatial information is not lost to the greatest extent, compared with traditional convolutional neural networks (CNNs), the experiment removes the pooling operation to keep the receptive field of the feature consistent with the original spatial dimension and prevent the loss of image edge details.

[0220] In order to verify the effectiveness of Gaussian smoothing on the convolutional features of random patches in multiple experiments, such as Fig.14 In 15 repeated parallel experiments, the accuracy of the experiment fluctuates more than that of the general classification method, but it can be seen that the curve after Gaussian smoothing (yes) is more stable than the curve of ordinary random pattern convolution (no), and the classification accuracy is higher in most cases. Fig.14 (a) is the Indian Pines dataset, Fig.14 (b) Pavia University and Fig.14 (c) is the Salinas dataset. The average accuracy curves of the three datasets after Gaussian smoothing are 98.23%, 94.15%, and 98.81%, and the variances are 0.035, 0.365, and 0.079, respectively. The average accuracy curves of ordinary random patch convolution are 97.97%, 93.01%, and 98.57%, and the variances are 0.098, 0.881, and 0.148, respectively. This shows that the introduction of RPNet under Gaussian smoothing obtains more robust spatial features, and after feature fusion, it can stably contribute to the identification information of spatial features and enhance spatial-spectral dependence.

[0221] M 3FuNet's quantitative experimental results: Due to the high dimensionality and robustness of spectral representation, in the multivariate fusion module, based on the maximum point of the precision curve of the spectral cascade residual fusion in the MSMC analysis, the spectral scale fusion retention of the Indian Pines dataset is fixed to 7, the spectral scale retention of the Pavia University dataset is fixed to 4, and the spectral scale retention of the Salinas dataset is fixed to 3. 3 FuNet network has a combined sensitivity (OA) to two parameters in the MRCD method: the number of Gaussian smoothing layers g-lay and the number of random convolution layers c-lay. The alternative variation values ​​are g-lay = [1, 2, 3, 4, 5, 6] and c-lay = [1, 2, 3, 4, 5, 6]. Fig.15 , you can see Fig.15 (a) Indian Pines dataset, Fig.15 (b) In the Pavia University dataset, the parameter c-lay is the dominant factor in performance. After 3 layers, the performance tends to be stable. When the c-lay value is low, the g-lay value has a greater impact. When the c-lay value is 1, g-lay almost dominates. When the c-lay value is large, the influence of g-lay decreases significantly. Fig.15 (c) The influence of the two parameters in the Salinas dataset is almost the same. Experiments show that the optimal parameter combinations (g-lay, c-lay) for the three datasets are (5, 2), (3, 3), and (4, 3), respectively, indicating that the range of alternative change values ​​mentioned has a certain degree of generalization, reflecting that M 3 The performance of FuNet reaches the theoretical upper limit and enters the local optimum of parameter combination after meeting certain parameter values.

[0222] Tables 8-10 show the classification results of different methods in OA, AA, PA, and Kappa in quantitative experiments on Indian Pines, Pavia University, and Salinas datasets. SVM, RF, and DMLSR achieved similar classification results on the Salinas dataset. SVM was similar to DMLSR and better than RF on the Indian Pines dataset. The effect on the Pavia University dataset was much better than RF and DMLSR, which proves that SVM has a stronger generalization ability among the classic classification methods and a higher upper limit for feature interpretation. The more advanced feature extraction methods SuperPCA, SpaSSA, RPNet, and the MSMC, MRCD, and M proposed in this paper are more suitable for classification. 3 FuNet uses SVM classifiers for fair comparison. In order to compare the representation change process of multi-feature fusion more carefully, and to intuitively reflect the M 3The FuNet network alleviates the SCFC phenomenon and combines the spatial and spectral dual feature decision fusion results M 2 Fu added contrast.

[0223] The performance of SuperPCA, SpaSSA and RPNet are better than traditional classifiers (SVM and RF), and the overall index is higher than that of the linear regression method based on sparseness (DMLSR). Among them, the classification accuracy of SuperPCA in the 9th category in the Indian Pines dataset is 0, indicating that the category is completely misclassified. It performs poorly in the face of spectral heterogeneity. The PCA dimension reduction in the segmented homogeneous region still has a classification bottleneck. SpaSSA and RPNet also focus on the extraction of local spatial information. The accuracy can reach the level of mainstream methods in recent years. This is not surprising, because these two methods can deeply capture spectral information on the basis of spatial information extraction and realize spatial-spectral joint feature extraction. It is worth noting that SpaSSA adopts a spatial domain adaptive method, and the feature mapping space is more robust, while RPNet is based on a random subspace idea and uses a random block convolution method to cascade multiple features. Although this method breaks through the accuracy bottleneck of the convolutional network, it also leads to the problem of poor feature robustness.

[0224] The MSMC, MRCD and M 3 FuNet has better classification performance than the above methods. Compared with RPNet, MRCD has achieved improved stability, and the obtained feature space is more robust, and its performance is also better than RPNet. It is obvious that the classification results of MSMC and MRCD on the three data sets are similar, indicating that the characterization effects obtained from the perspective of spectrum and space are consistent. In the previous theoretical analysis, such feature fusion results can alleviate the single-class feature confrontation phenomenon. The dual fusion feature results M on the three data sets are 2 Fu's OA, AA, and Kappa are all higher than the corresponding MSMC and MRCD, which verifies the effectiveness of multi-feature fusion in this case and achieves the effect of mutual verification between the spectral domain and the spatial domain. 3 FuNet performs better than M in the Indian Pines dataset 2 Fu has been slightly improved again, which is more obvious in the single-category accuracy. This shows that the feature verification of multi-feature decision fusion can solve the problem of partial information loss in dual feature fusion to a certain extent, and maintain the decision contribution of spectral features and spatial features. In the Pavia University and Salinas data sets, the accuracy of the two can be regarded as equal, proving that the fused features have reached the upper limit of representation, the difference of the triple features is small, and the balance of various feature samples is high.

[0225] Table 8 Indian Pines

[0226]

[0227] Table 9 Pavia University

[0228]

[0229] Table 10 Salinas

[0230]

[0231]

[0232] Table 11 shows the execution time of all methods (from feature extraction to prediction). The running time of the proposed method is generally better than SpaSSA, M 3 FuNet takes more time than a single feature extraction method because the multi-feature fusion module takes up more time; 3 FuNet is a serial experiment conducted on this experimental equipment, that is, MSMC and MRCD are executed sequentially, and the time spent is almost the same as MSMC.

[0233] For visual comparison, different classification methods are compared in the form of classification maps ((a) ground truth map, (b) training sample map, (c) SVM map, (d) RF map, (e) DMLSR map, (f) SuperPCA, (g) RPNet map, (h) SpaSSA map, (i) MRCD map, (j) MSMC map, (k) M 2 Fu graph and (1)M 3 FuNet Figure) for intuitive comparison, such as Fig.16 , Fig.17 and Fig.18 . Pixel-level classification models (RF and SVM) generate a lot of salt and pepper noise in the classification graph, while methods that consider local domain information (RPNet and SpaSSA) can significantly reduce this phenomenon, indicating that the extraction of spatial domain information plays a key role in visualization. Under the guidance of superpixel segmentation, methods that act on homogeneous regions (SuperPCA and MHMC) have different degrees of mitigation of the impact of salt and pepper noise. 3 FuNet and M 2 Fu has a certain decision correction ability under the fusion of multiple features. The obtained classification map is smoother than MSMC and has a better effect in suppressing noise.

[0234] Test M 3FuNet’s sensitivity to changes in the number of training samples. The number of training samples for each class ranges from 20 to 200, with an interval of 20, to test the changes in OA caused by continuous changes in training samples, such as Fig.19 As shown in Figure 2, OA increases with the increase of training scale. Compared with other methods, M 3 FuNet has the best performance.

[0235] Table 11 Execution time of different methods

[0236] Classifier Indian Pines Pavia University Salinas SVM 2.66 1.11 6.65 RF 5.34 8.46 17.39 DMLSR 11.97 1.88 4.60 SuperPCA 12.27 6.92 14.56 RPNet 3.66 7.4l 9.83 SpaSSA 27.48 267.18 294.26 MRCD 5.50 50.21 13.90 MSMC 62.01 112.28 60.29 <![CDATA[M 3 FuNet]]> 68.89 175.74 88.31

[0237] The above content is a further detailed description of the present invention in combination with a specific preferred embodiment. For technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. An unsupervised multivariate feature extraction method for hyperspectral image classification, characterized in that: The steps include: Acquire several hyperspectral remote sensing images; The spectral feature extraction method MSMC is used to smooth and denoise the spectral curve of the hyperspectral remote sensing image, and the cascade residual method is used to capture the multi-scale spectral features of the smooth and denoised spectral curve; The spatial feature extraction method MRCD is used to obtain Gaussian smoothing features of the hyperspectral remote sensing image, and deep and shallow spatial features are extracted based on the Gaussian smoothing features, and the deep and shallow spatial features are fused to obtain the final spatial features; Using M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain spatial-spectral joint features; The multi-scale spectral features, the final spatial features and the spatial-spectral joint features are input into the classifier for aggregation and decision fusion to obtain spectral decisions, spatial decisions and fusion decisions; The final classification decision of spectral decision, spatial decision and fusion decision is obtained by majority voting; The high light remote sensing image is processed through the final classification decision to obtain the final classification result.

2. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 1, characterized in that: The spectral curve is constructed based on the superpixel spectral set, and superpixel principal component analysis is performed on the spectral curve, including the following steps: The ERS image segmentation algorithm is used to generate s non-overlapping homogeneous regions in the hyperspectral remote sensing image, that is, s super-pixels are obtained; The graph G=(V,A) in the hyperspectral remote sensing image is divided into connected subgraphs, and the entropy rate term H(A) and the balance term B(A) are introduced into the objective function of the ERS image segmentation algorithm to optimize the superpixel segmentation. Among them, ω is used to balance the contribution of the entropy rate term H(A) and the balance term B(A), and is a weight greater than 0. E is the weight function v→R + The image edge set consisting of pairwise similarities of ∪{0}; Construct hyperspectral cube data X∈R by imaging B continuous spectral bands M×N×B , and transform the three-dimensional hyperspectral cube data X∈R M×N×B Reconstructed into a two-dimensional matrix X = [x1, x2, ..., x B ]∈R L×B ; From the high-dimensional space X∈R L×B The target dimension b is specified in the linear mapping in , and the first b eigenvectors are retained from large to small according to the eigenvalues ​​to form a projection vector matrix W, that is, y i =W T x i ; Where L = M × N; The linear transformation matrix is ​​obtained by solving the objective function; IN * =argmaxTr(W T Cov(X)W),W T In=I Where W T Cov(X) is the covariance matrix of X, Tr(X) is the square matrix of the trace of X, and I is the identity matrix; The first principal component p1 of the hyperspectral data is obtained by superpixel principal component analysis, and the first principal component p1 is converted into a two-dimensional structure as the segmentation information standard pattern p1∈R of ERS M×N ; Perform ESR segmentation on p1 to obtain superpixel segmentation label pattern I ERS ∈R M×N ; Among them, H i is the i-th superpixel; ERS obtains s by empirical segmentation through HSI spatial resolution; in is a round-down operation, res is the spatial resolution of HSI; Repeat extracting the same number of principal components P for each homogeneous region, and finally concatenating all regional vectors to obtain a new hyperspectral matrix X∈R L×P .

3. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 1, characterized in that: The method of using the spectral feature extraction method MSMC to smooth and denoise the spectral curve of the hyperspectral remote sensing image comprises the following steps: The supervector block vectors corresponding to each band of the hyperspectral cube are concatenated into a matrix as the supervector matrix Among them, M s is the number of pixels in the homogeneous region, N s is the number of bands (N s =B); The median absolute deviation algorithm MAD is used to smooth and denoise the supervector matrix.

4. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 3, characterized in that: The M s Each row of data maintains similar eigenvalues, including the following steps: Calculate W s The median of each row of data W median And all elements with W median The absolute bias vector bias; Pick The median of the signal deviates from the critical value m ad ; Ms i =[m1,m2,…,m B ] is W s The i-th row vector, yes the median; Fixed parameter α adjusts m ad The decision-making proportion, the data correction formula is as follows: W s Approximate mean constraints are added to each row of data to further smooth the supervector matrix; Calculate the distribution principal number M of each row of the supervector matrix ad , directly use the M of this band ad Replace the eigenvalues ​​of these supervectors and correct the supervector blocks whose deviation is greater than a threshold; Where MAD'(·) is the MAD correction algorithm; the corrected W s Then the supervector and superpixel structure are restored, and all corrected superpixels are stitched together to restore the hyperspectral cube structure.

5. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 1, characterized in that: The method of capturing multi-scale spectral features of the smoothed denoised spectral curve by using a cascaded residual method comprises the following steps: Use 3×3 convolution, fill the border with mirror image, and set the radius r to 1 to calculate the Gaussian smoothing map Gau; The standard pattern pixel minus the Gaussian smoothing image Gau obtains the difference value Δ, that is, the first principal component p1 is obtained; If the absolute value of Δ is greater than the set threshold, the point is improved and sharpened; Δ=p1(i,j)-Gau(i,j) Among them, D au It is the sharpened image, abs(Δ) is the absolute value of Δ, and β adjusts the decision weight of Δ; The number of superpixels is determined by a set of approximately smooth curve values, and the superpixel segmentation is performed. ERS Divide into 2c+1 homogeneous region images; Among them, c represents the range of the scale and the cth division method; After inputting c feature samples into the classifier, the optimal decision result is obtained by decision fusion; the decision formula is: Where l is the label of one of the C categories, j is the number of the classifier, l j is the result of predicting the c-th segmentation scale feature; Nc i is the number of times the i-th class is predicted in the classifier, α j is the voting strength of the j-th classifier; where j∈{1,2,…2c+1}; The cascade residual structure is introduced at different division scales, the first five principal components are retained, and the features of different scales are cascaded through the residual module to obtain the spectral feature X spectral ∈R L×p ; The obtained spectral feature samples are input into SVM to complete image classification.

6. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 1, characterized in that: The method of using the spatial feature extraction method MRCD to obtain the Gaussian smoothing features of the hyperspectral remote sensing image comprises the following steps: A random projection theory is introduced in RPNet to randomly transform the two-dimensional hyperspectral data X∈R L×B Projection to k-dimensional subspace X R ∈R L×k middle; Retain the first P principal components of the original two-dimensional hyperspectral data, X P ∈R L×P , and then whiten it so that the data has similar variances in different dimensions; Note X P The covariance matrix of P , using singular value decomposition to obtain W P The left singular value vector V P and the singular value matrix U P ; Among them, X W ∈R L×P It is the low-dimensional feature after whitening, λ i is the i-th singular value of the singular value matrix; The hyperspectral space is divided into c spatial sub-blocks with the same length and width as the convolution kernel Cov = [cov1, cov2, ..., cov c ], respectively for the whitened data X W Perform convolution, the i-th convolution feature yes: in is a two-dimensional convolution operation, is the j-th dimension feature of the whitened data, is the j-th dimension feature of the i-th convolution kernel; Combine all the convolutional features to get the c-dimensional spatial feature X spatial ∈R L×c .

7. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 6, characterized in that: Extracting deep and shallow spatial features based on the Gaussian smoothing feature, and fusing the deep and shallow spatial features to obtain the final spatial features, includes the following steps: Use Gaussian smoothing on the i-th dimension of the spatial features Processing, set the blur radius to 1, that is, take the target center point h and the surrounding 8 points, and according to the specified weight parameter sigma, get the total weight W of these 9 points c , and then calculate the weighted average W of these 9 points cp ; Use the weighted average W of these 9 points cp Divide by W c , and get the final weight matrix W gau ; Where ρ represents the weight parameter, W gau As a Gaussian convolution kernel, update the eigenvalue of the center point h; For spatial feature X spatial ∈R L×c The features of each dimension repeat this convolution operation, so the i-th convolution feature Xcon i for: Where W gau It can be regarded as a two-dimensional Gaussian function W gau (x,y): Combine all the obtained Gaussian smooth features to obtain the c-dimensional spatial feature X spatial ∈R L×c ; By cascading the spatial feature X spatial ∈R L×c The shallow features and deep features in are cascaded and fused to obtain the final feature set: Where n is the number of layers of random block convolution, that is, n = c-lay; Finally, the obtained spatial feature samples are input into SVM to complete image classification.

8. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 1, characterized in that: The use of M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain the spatial-spectral joint features. The steps include: Through the M 3 FuNet fuses the multi-scale spectral features and the final spatial features to obtain the spatial-spectral joint features. Will Convert to 2D space Perform standardization operation: in, is the fusion feature, K is its spectral dimension; is the feature average value, is the characteristic standard deviation.

9. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 8, characterized in that: The multi-scale spectral features, the final spatial features and the spatial-spectral joint features are input into the classifier for aggregation and decision fusion to obtain spectral decisions, spatial decisions and fusion decisions, including the following steps: The extracted spectral features, spatial features and fusion features are input into the classifier in parallel to obtain the spectral decision R a , spatial decision R b and the fusion decision R s ; The three decisions are voted by majority to get the final classification decision R l .

10. The unsupervised multivariate feature extraction method for hyperspectral image classification according to claim 9, characterized in that: The three decisions are obtained by majority voting to obtain the final classification decision R l , including the following steps: Let i represent the three feature flags a, b, s, and n represent the dimensions p, nc, K of the three features, and obtain the final decision formula: Among them, l i is its predicted i-th classifier decision, γ i is the decision strength of the i-th feature, and R represents the comparison with R a , R b , R s Three decisions.

Images