A small sample HSI classification method based on a new GCN and CNN network

Abstract

This paper proposes a small-sample HSI classification method based on a novel GCN and CNN network, comprising the following steps: S1, performing linear projection mapping on HSI data and converting pixel features into superpixel features; S2, feeding the resulting superpixel features into a graph encoder, then extracting features in the form of HSI graph data, and decoding the graph data features through a decoder; S3, performing nonlinear feature processing on the graph data features to obtain Feature 1; S4, performing spectral cleaning convolution on the HSI data and simultaneously performing dimensionality reduction on the high-dimensional HSI data to generate a reduced-dimensionality feature map; S5, processing the reduced-dimensionality feature map through an attention mechanism and information interaction, and then convolving it with spatial features through another layer of attention mechanism to obtain Feature 2; S6, performing ConCat multi-feature fusion on Feature 1 obtained in step S3 and Feature 2 obtained in step S5, and outputting the classification result. This method solves problems such as small-sample classification, high noise, and loss of spectral-spatial semantic position information.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral image processing, and in particular to a small sample HSI classification method based on a novel GCN and CNN network. Background Art

[0002] With the rapid development of hyperspectral remote sensing technology, its application in Earth observation is becoming increasingly widespread. Hyperspectral image (HSI) classification is a key method for hyperspectral remote sensing image analysis and has garnered widespread attention in areas related to the national economy, such as natural resource assessment, urban land use planning, precision agriculture and forestry, and environmental protection.

[0003] HSI provides very rich spectral and spatial information, and can provide accurate research data when characterizing land object categories. However, the data collected by current hyperspectral image sensors often contain a large amount of redundant information, so the samples of land object features that can be manually labeled are often very limited. Therefore, it is very necessary to design a method that can also efficiently classify when the labeled samples are limited. The main challenges of hyperspectral image classification come from: (1) the joint utilization rate of spectrum and space is not high; (2) the hyperspectral image model used for classification is too complex; (3) the classification effect achieved when the training samples are limited is not ideal. Although the current hyperspectral image classification methods under small samples have alleviated the difficulties of small sample learning to a certain extent, most methods still have the problem of overly complex models. Summary of the Invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a small sample HSI classification method based on a new GCN and CNN network.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a small sample HSI classification method based on a novel GCN and CNN network, comprising the following steps:

[0006] S1, linear discriminant analysis LDA is used to perform linear projection mapping on HSI data, and then the pixel features are converted to superpixel features through the SLIC segmentation algorithm;

[0007] S2: The superpixel features are fed into the graph encoder to form graph data. The ARMA graph neural network is then used to extract the features of the HSI graph data. The decoder then decodes the graph data features.

[0008] S3, performs nonlinear feature processing on the graph data features to obtain feature 1; through nonlinear feature processing, the decoded graph data features are made more stable and the network robustness is improved.

[0009] S4, denoises the HSI data through spectral cleaning convolution, and performs dimensionality reduction processing on the high-dimensional HSI data to generate a reduced-dimensionality feature map with a size of C×H×W;

[0010] S5: The dimensionality reduction feature map is used to locate the spectral spatial position information through the attention mechanism, and then the spectral spatial features after positioning are processed through spatial feature convolution, so that the features extracted by spatial convolution tend to be diversified. After that, another layer of attention mechanism and spatial feature convolution is performed to finally obtain feature 2;

[0011] S6, performing ConCat multi-feature fusion on the feature 1 obtained in step S3 and the feature 2 obtained in step S5, and outputting the classification result.

[0012] Furthermore, during the conversion of pixel features to superpixel features by the SLIC segmentation algorithm, a correlation matrix C∈R is generated. HW×T To record the clustering of pixels into superpixels, where H and W represent the height and width of the correlation matrix respectively, and T represents the number of superpixels divided into; the relationship between pixels and superpixels is defined as:

[0013]

[0014] C i,j It represents the specific value of the matrix C at the spatial position (i, j);

[0015] Represents the i-th pixel at spatial position;

[0016] S j The jth spatial position is represented by a superpixel;

[0017] represents the normalized X;

[0018] X represents HSI pixel features;

[0019] Flatten(.) means flattening the pixels in the HSI data space;

[0020] Then, the graph encoder is used to encode the correlation matrix C and the HSI pixel feature X to generate a new feature vector:

[0021]

[0022] Encoder(X; C) means encoding operation on X and C;

[0023] is a matrix The transpose of It represents the normalization of the correlation matrix C;

[0024] Then, the features in the graph nodes are extracted through graph convolution with ARMA filters, and then the features in the graph nodes are projected into the spatial dimension using the graph solver:

[0025]

[0026] Decoder(V; C) represents the decoding operation of V and C;

[0027] Reshape(.) means restoring the flattened HSI data;

[0028] V represents the node features in ARMA graph convolution.

[0029] Finally, through LDA projection dimensionality reduction and superpixel segmentation algorithm SLIC, undirected graph data G = (V, E) is generated, where E represents the edge of the graph data.

[0030] Furthermore, the feature extraction of HSI graph data is performed by ARMA graph neural network, and the graph node features obtained are:

[0031]

[0032] Represents the graph node features obtained through the ARMA graph neural network;

[0033] K represents the image signal after superposition of K first-order filters ARMA1;

[0034] M represents M adjacency matrices that need to be superimposed;

[0035] a k Represents the correlation coefficient a after superposition of k first-order filters ARMA1;

[0036] b k Represents the correlation coefficient b after superposition of k first-order filters ARMA1;

[0037] μ m represents the eigenvalue of the mth adjacent matrix;

[0038] u m Represents the eigenvalue of the mth adjacent matrix after superposition of K 1st-order filters ARMA1;

[0039] for u m The matrix transpose of .

[0040] Furthermore, the ARMA1 filter is iteratively expressed as a new graph convolution layer:

[0041]

[0042] Represents the graph signal at time t-1 after passing through the ARMA graph neural network filter;

[0043] Represents the graph signal at time t after passing through the ARMA graph neural network filter;

[0044] σ represents the activation function;

[0045] represents the corrected Laplace matrix;

[0046] W represents the weight after ARMA graph neural network filtering;

[0047] V represents the graph node features;

[0048] U represents the unfiltered weights that need to be learned during the training process.

[0049] Furthermore, the S3 includes:

[0050] HSI feature information of the i-th channel position After mapping the features through nonlinear feature transformation, it is expressed as:

[0051]

[0052] Represents the position information of the i-th position neuron in the l-th layer of the neural network;

[0053] Represents the position information of the i-th position neuron in the l-1 layer of the neural network;

[0054] The weight coefficient of the neuron at position i in the neural network of layer l-1;

[0055] Indicates the size of the noise at the i-th position neuron of the l-1 layer neural network;

[0056] ψ represents the activation function LeakyReLU.

[0057] Furthermore, the HSI data is convolved through spectral cleaning and is expressed as:

[0058]

[0059] in Represents the HSI feature map after spectral cleaning convolution;

[0060] ψ represents the activation function LeakyReLU;

[0061] Represents the weight coefficient of the cth channel of the input l-1 layer at the spatial position (i, j);

[0062] Represents the HSI feature map with a height of h, a width of w, and a number of bands b for the input of the l-1 layer;

[0063] Indicates that the height of the l-1 layer input is h, the width is w, and the c-th channel offset size.

[0064] l represents the convolution of the lth layer of the neural network;

[0065] i, j represent spatial positions;

[0066] b represents the number of bands;

[0067] c represents a channel.

[0068] In the article It represents the normalization of the HSI feature map X, which is the batchNomal in the convolutional network, where h, w, and b represent the height, width, and number of bands of the HSI data respectively.

[0069] The spectral cleaning convolution layer in this stage is used to process the noise and redundant band information contained in the original HSI. It also facilitates the subsequent coordinate attention mechanism to extract HSI feature information in space.

[0070] The spectrum cleaning convolution adopts two layers of two-dimensional convolution with a convolution kernel of 1×1.

[0071] Furthermore, locating the spectral spatial position information by the attention mechanism includes the following steps:

[0072] For the HSI spectral space feature map X∈R that is input to the attention mechanism H×W×C , use a pooling kernel of size (H×1) to encode each channel along the horizontal coordinate of the feature map, H, W, C represent the input feature map height H, width W, and number of channels C respectively; then the feature map of the cth channel with a height of h after encoding is:

[0073]

[0074] Represents the feature map of the cth channel with the encoded height h;

[0075] W represents the width of the input HSI spectral spatial feature map;

[0076] x c (h,i) represents the spectral feature map of the cth channel at the spatial position (h,i), i represents the horizontal position, and h represents the height;

[0077] Similarly, a pooling kernel of size (1×W) is used to encode each channel along the vertical coordinate, and the new output size is generated:

[0078]

[0079] in Represents the feature map of the cth channel with a width of w after encoding;

[0080] x c (j,w) represents the spectral feature map of the cth channel at the spatial position (j,w), j represents the vertical position, and w represents the width;

[0081] After nonlinear transformation along the horizontal and vertical directions, a new feature map of the perception of a pair of directions is generated. and The concatenate operation is performed, and then the convolution transformation function F1 with a convolution kernel of 1×1 is used to transform it, and then the nonlinear activation function δ is used to activate it to obtain the feature map f of the intermediate transition between the horizontal and vertical directions of the spectral spatial position encoding.

[0082]

[0083] F1 is the convolution transformation function;

[0084] [z h ,z w ] indicates and Perform concatenate operation;

[0085] The transition feature map is then decomposed into two separate vectors f along the spatial dimension that do not interfere with each other h ∈R C / r×H With f w ∈R C / r×W , where r represents the channel reduction ratio factor; again using the 1×1 convolution transformation function F h 、F w Transform the two vectors to have the same number of channels as the spectral feature map X:

[0086] g h =σ(F h (f h )) (17)

[0087] g w=σ(F w (f w ))

[0088] where g h 、g w Respectively represent the spectral feature maps encoded in the vertical and horizontal directions after convolution transformation;

[0089] σ is the sigmoid activation function;

[0090] F h 、F w Represent the convolution transformation functions for vertical and horizontal coding respectively;

[0091] f h 、f w Feature maps representing the intermediate transitions of vertical and horizontal encoding respectively;

[0092] Finally, the output g h 、g w Perform weighted aggregation operation with the spectral feature map X to obtain the final spatial position output y c (i,j):

[0093]

[0094] x c (i, j) represents the spectral feature map of the cth channel at the spatial position (i, j);

[0095] Represents the feature map of the cth channel in the horizontal direction i after weight aggregation;

[0096] Represents the feature map of the cth channel in the vertical direction j after weight aggregation.

[0097] Furthermore, after the spatial feature convolution is used to perform information interaction processing on the spectral spatial features after positioning, the feature map X is expressed as:

[0098]

[0099] It represents the output after separable convolution feature extraction;

[0100] ψ represents the activation function LeakyReLU;

[0101] It represents the weight coefficient of the depth-wise separable convolutional layer;

[0102] Represents the input HSI feature map;

[0103] Indicates the size of the HSI feature map noise.

[0104] This part of the convolution not only enables information interaction between channels, but also further saves computational overhead.

[0105] In summary, by employing the aforementioned technical solutions, the present method combines ARMA graph neural networks with convolutional neural networks to form a novel framework suitable for HSI image classification. This method leverages the excellent performance of ARMA graph convolution at superpixels and depthwise separable convolution at pixels for HSI image classification, addressing issues such as small sample size, high noise, and loss of spectral spatial semantic position information in hyperspectral image classification. Furthermore, to better capture spectral node information, this paper adds a Coordinate Attention mechanism to the convolutional neural network (CNN).

[0106] Furthermore, convolutional networks offer suboptimal classification results when training samples are limited. While ARMA graph neural networks perform well in this case, their classification accuracy is further limited by superpixel segmentation. The present invention, by combining the ARMA graph neural network with the CNN's sensitivity to subtle pixel-level features, complements these results. Therefore, the present invention can effectively perform classification tasks even when training samples are insufficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0108] Figure 1 This is a schematic diagram of converting pixel-level features into superpixel features in the present invention.

[0109] Figure 2 It is the classification results of the IP dataset obtained by different methods, as well as the pseudo-color map and the real ground feature map.

[0110] Figure 3 This is a schematic diagram of the classification results obtained by different methods for the BS dataset, as well as the pseudo-color map and the true labels of the objects.

[0111] Figure 4 This is a schematic diagram of the classification results, pseudo-color images, and true labels of the objects obtained using different methods for the WHU dataset.

[0112] Figure 5 This is the structural diagram of the MAGC algorithm of the present invention. DETAILED DESCRIPTION

[0113] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0114] The present invention proposes a MAGC algorithm, the structure diagram of the MAGC algorithm is as follows: Figure 5 The relevant parameter settings are shown in Table 1. The specific workflow is as follows: HSI data is denoised through spectral cleaning convolution, which simultaneously reduces the dimensionality of the high-dimensional HSI data. After spectral cleaning convolution, a (C×H×W) dimensionality reduction feature map is generated. The spectral spatial position information is then located through the Coordinate Attention mechanism in the lower branch. This information is then fed into the spatial feature convolution for interactive processing of the located spectral spatial features, diversifying the features extracted by the spatial convolution. This is followed by another layer of Coordinate Attention and spatial feature convolution. The HSI data input from the upper branch is subjected to LDA linear projection mapping and then to the SLIC algorithm for pixel-to-superpixel feature conversion. The resulting superpixel features are fed into the graph encoder to form graph data. The ARMA graph neural network is then used to extract features from the HSI graph data. The decoder then decodes the graph data features. A nonlinear feature conversion layer stabilizes the decoded graph data features. Finally, the features extracted from the upper and lower branches are subjected to ConCat multi-feature fusion to output the classification result.

[0115] Among them, the ARMA filter graph neural network can not only suppress noise well but also capture the characteristics of the global graph structure. At the same time, the lightweight depth-separable convolution has excellent spatiotemporal characteristics.

[0116] Table 1 Settings of MAGC related parameters

[0117]

[0118] 1. First, Linear Discriminant Analysis is used to project the high-dimensional HSI data onto a straight line, thereby converting it into low-dimensional data for processing. Then, the SLIC segmentation algorithm is used to cluster the HSI pixels in space using a simple and efficient clustering method to generate superpixels as the input of the graph convolutional network. The specific implementation process is as follows: Figure 1 As shown, during the conversion process, an association matrix C∈R is generated HW×T To record the clustering of pixels into superpixels, where T represents the number of superpixels divided. We define the relationship between pixels and superpixels:

[0119]

[0120] C i,j It represents the specific value of the matrix C at the spatial position (i, j), and Flatten(.) means flattening the pixels in the HSI data space. It represents the pixel at the spatial position i, and then uses the graph encoder to encode the correlation matrix C and the HSI pixel feature X to generate a new feature vector.

[0121]

[0122] It represents the normalization of the correlation matrix C.

[0123] Then, the features in the graph nodes are extracted through graph convolution with ARMA filters, and then the features in the graph nodes are projected into the spatial dimension using the graph solver:

[0124]

[0125] Reshape(.) restores the flattened HSI data, and V represents the node features in the ARMA graph convolution. Finally, through LDA projection dimensionality reduction and superpixel segmentation algorithm SLIC, an undirected graph G = (V, E) is generated, where E represents the edges of the graph.

[0126] 2. Extract graph node features from the encoded HSI data, and convert the undirected graph data signal V t A At time t it is expressed as:

[0127]

[0128] where φ i Represents the weight coefficient of the transformed HSI image data feature;

[0129] V t-i Represents the HSI graph data features at time ti;

[0130] χ j Represents the weight coefficient of the noise of the converted HSI image data;

[0131] α and β represent autoregressive moving average polynomials; one is in the graph neural network and the other is in the bias.

[0132] ε t-j Indicates the bias size at time tj;

[0133] ε tIt represents the bias size at time t.

[0134] During the graph signal propagation process, the graph node feature V (0) ,V (1) ,V (2) ,…,V (k) …for graph convolution node features, they are converted into filtered information Expressed as: This is considered to be the graph signal that the ARMA graph neural network did not extract the features of the graph dataset. This item is represented as the graph signal after ARMA graph neural network filtering, and the final graph node feature can be expressed as:

[0135]

[0136] Where I represents the identity matrix;

[0137] K indicates that there are K HSI signals in total;

[0138] φ k Represents the weight coefficients of the k-order Laplace matrix before the filter;

[0139] L k represents the k-th order Laplace matrix (filter response of the spectral signal).

[0140] χ k Represents the weight coefficient of the k-order Laplace matrix after the filter;

[0141] V represents the HSI graph node features.

[0142] Since the filtered graph node signal filter is just a common polynomial graph filter, and to capture more graph node features, a higher-order polynomial must be used. However, high-order polynomial filters are prone to overfitting the graph signal, resulting in higher computational costs.

[0143] The polynomial graph filter generates a dense matrix in matrix calculation, which hinders the implementation of matrix sparse multiplication GCN. In order to implement sparse matrix multiplication of GCN, an iterative first-order recursion is used to approximate the ARMA method. The first-order recursive ARMA1 filter is expressed as:

[0144]

[0145] Represents the HSI graph node features extracted by the first-order recursive ARMA1 filter at time t+1;

[0146] a and b represent the correlation coefficients of the first-order recursive filter ARMA1; one is in the graph neural network and the other is in the bias.

[0147] represents the HSI graph node features extracted by the first-order recursive ARMA1 filter at time t;

[0148] V represents the node features of the HSI graph;

[0149] in,

[0150]

[0151] M represents the size of the adjacency matrix;

[0152] λ max ,λ min Represent the maximum eigenvalue and minimum eigenvalue of the Laplace matrix respectively;

[0153] I represents the identity matrix.

[0154] From the convergence of formula (3) and formula (4), the response of the ARMA1 filter can be obtained as follows:

[0155]

[0156] represents the response of the ARMA1 filter;

[0157] t represents time t;

[0158] Represents the graph signal at the initial moment of passing through the ARMA graph neural network filter;

[0159] τ represents the intermediate variable, and there are a total of τ signals after filtering;

[0160] V represents the graph node features;

[0161] i means there are i signals before filtering;

[0162] a, b represent the correlation coefficients of the first-order recursive filter ARMA1;

[0163] For |a|<1,μ m ∈[-1,1], when t→∞, we can get from the properties of power function The term converges to 0. From formula (4), we can see that the eigenvector size of the adjacency matrix M is consistent with that of the Laplace matrix L. Let μ m for

[0164] μ m =(λ max -λ min ) / 2 (6)

[0165] μ mrepresents unknown variables;

[0166] λ max ,λ min Represent the maximum eigenvalue and minimum eigenvalue of the Laplace matrix respectively;

[0167] From formula (4)(5)(6), we can know that

[0168]

[0169] According to the convergence of power series, we can get

[0170]

[0171] Formula (8) can be written as the response of the ARMA1 filter:

[0172]

[0173] represents the response of the ARMA1 filter signal;

[0174] μ m represents unknown variables;

[0175] Then, k 1st-order ARMA1 filters are superimposed and summed, and formula (2) can be reduced to ARMA k The analytical form of the filter is:

[0176]

[0177] K represents the image signal after superposition of K first-order filters ARMA1;

[0178] M represents M adjacency matrices that need to be superimposed;

[0179] a k Represents the correlation coefficient a after superposition of k first-order filters ARMA1;

[0180] b k Represents the correlation coefficient b after superposition of k first-order filters ARMA1;

[0181] μ m represents the eigenvalue of the mth adjacent matrix;

[0182] u m Represents the eigenvalue of the mth adjacent matrix after superposition of K 1st-order filters ARMA1;

[0183] for u m The matrix transpose of ;

[0184] in It is proved that the first-order filter ARMA1 can be transformed into the K-order filter ARMA k Iterative transformation.

[0185] Finally, the ARMA1 filter is iterated to obtain a new graph convolution layer:

[0186]

[0187] represents the corrected Laplace matrix;

[0188] W represents the weight after ARMA graph neural network filtering;

[0189] U represents the unfiltered weight that needs to be learned during the training process;

[0190] σ represents the activation function;

[0191] V represents the graph node features.

[0192] 3. Through nonlinear feature processing, the network robustness is improved, so that the HSI feature information of the i-th channel position The features mapped by nonlinear feature transformation are:

[0193]

[0194] in Represents the position information of the i-th position neuron in the l-th layer of the neural network;

[0195] Represents the position information of the i-th position neuron in the l-1 layer of the neural network;

[0196] The weight coefficient of the neuron at position i in the neural network of layer l-1;

[0197] Indicates the size of the noise at the i-th position neuron of the l-1 layer neural network;

[0198] ψ represents the activation function LeakyReLU.

[0199] 4. HSI Data Cleaning: The spectral cleaning CNN begins with two layers of 1×1 two-dimensional convolution. This spectral cleaning convolution layer primarily removes noise and redundant band information from the raw HSI data. It also facilitates the subsequent coordinate attention mechanism to extract HSI feature information in space. Specifically, the HSI features X input to this convolution layer are:

[0200]

[0201] Where ψ represents the activation function LeakyReLU;

[0202] Represents the weight coefficient of the cth channel of the input l-1 layer at the spatial position (i, j);

[0203] Represents the HSI feature map with a height of h, a width of w, and a number of bands b for the input of the l-1 layer;

[0204] Indicates that the height of the l-1 layer input is h, the width is w, and the c-th channel offset size.

[0205] l represents the convolution of the lth layer of the neural network;

[0206] i, j represent spatial positions;

[0207] b represents the number of bands;

[0208] c represents a channel.

[0209] In the article It represents the normalization of the HSI feature map X, which is the batchNomal in the convolutional network, where h, w, and b represent the height, width, and number of bands of the HSI data respectively.

[0210] 5. Coordinate Attention locates the spectral spatial position information: For the HSI spectral spatial feature map X∈R input to the attention mechanism H×W×C , use a pooling kernel of size (H×1) to encode each channel along the horizontal coordinate of the feature map, H, W, C represent the input feature map height H, width W, and number of channels C respectively; then the feature map of the cth channel with a height of h after encoding is:

[0211]

[0212] Represents the feature map of the cth channel with the encoded height h;

[0213] W represents the width of the input HSI spectral spatial feature map;

[0214] x c (h,i) represents the spectral feature map of the cth channel at the spatial position (h,i), i represents the horizontal position, and h represents the height;

[0215] Similarly, a pooling kernel of size (1×W) is used to encode each channel along the vertical coordinate, and the new output size is generated:

[0216]

[0217] in Represents the feature map of the cth channel with a width of w after encoding;

[0218] w represents width;

[0219] c represents the cth channel;

[0220] x c (j,w) represents the spectral feature map of the cth channel at the spatial position (j,w), j represents the vertical position, and w represents the width;

[0221] i, j represent spatial positions;

[0222] z represents the size of the encoded output.

[0223] After nonlinear transformation along the horizontal and vertical directions, a pair of new feature maps of direction perception will be generated. and The concatenate operation is performed, and then the convolution transformation function F1 with a convolution kernel of 1×1 is used to transform it, and then the nonlinear activation function δ is used to activate it to obtain the feature map f of the intermediate transition between the horizontal and vertical directions of the spectral spatial position encoding.

[0224]

[0225] F1 is the convolution transformation function;

[0226] [z h ,z w ] indicates and Perform concatenate operation;

[0227] The transition feature map is then decomposed into two separate vectors f along the spatial dimension that do not interfere with each other h ∈R C / r×H With f w ∈R C / r×W , where r represents the channel reduction ratio factor. Again, the 1×1 convolution transformation function F h 、F w Transform the two vectors to have the same number of channels as the spectral feature map X:

[0228] g h =σ(F h (f h )) (17)

[0229] g w =σ(Fw (f w ))

[0230] where g h 、g w Respectively represent the spectral feature maps encoded in the vertical and horizontal directions after convolution transformation;

[0231] σ is the sigmoid activation function;

[0232] F h 、F w Represent the convolution transformation functions for vertical and horizontal coding respectively;

[0233] f h 、f w Feature maps representing the intermediate transitions of vertical and horizontal encoding respectively;

[0234] Finally, the output g h 、g w Perform weighted aggregation operation with the spectral feature map X to obtain the final spatial position output y c (i,j).

[0235]

[0236] x c (i, j) represents the spectral feature map of the cth channel at the spatial position (i, j);

[0237] Represents the feature map of the cth channel in the horizontal direction i after weight aggregation;

[0238] Represents the feature map of the cth channel in the vertical direction j after weight aggregation.

[0239] 6. Spatial feature extraction: This part of the convolution not only enables information interaction between channels, but also further reduces computational overhead. The feature map X input to the spatial feature extraction convolution is:

[0240]

[0241] It represents the output after separable convolution feature extraction;

[0242] ψ represents the activation function LeakyReLU;

[0243] It represents the weight coefficient of the depth-wise separable convolutional layer;

[0244] Represents the input HSI feature map;

[0245] Indicates the size of the HSI feature map noise.

[0246] Finally, the feature maps extracted from the upper branch and the lower branch are concatenated to output the classification results.

[0247] To validate the performance of the MAGC algorithm, the experiment compared it with seven other state-of-the-art hyperspectral image classification methods. These included two methods based on convolutional neural networks (HamidaEtAl and HybridsN), three algorithms based on graph convolutional networks (GAN, GAT, and ARMA), and two state-of-the-art algorithms based on a hybrid of graph convolution and convolutional neural networks (CEGCN and WFCG).

[0248] from Figure 2 Comparing the effects of different methods, the results achieved by the HamidaEtAlNet and HybridSN methods are not ideal. The main reason is that the convolutional neural network loses serious spatial information features for categories with fewer training samples. Although traditional GCN and GAT have achieved better classification results than CNN, the superpixel segmentation has limited the classification accuracy. The ARMA autoregressive translation graph neural network is more comprehensive than GCN and GAT in extracting spectral spatial information superpixel node information. Due to the use of weighted feature fusion, WFCG has achieved an overall accuracy of 94.73%. The effect of the GACP method of the present invention is closer to the real object label, and the smoothing effect is more obvious.

[0249] from Figure 3 Comparing the results of the classification methods, the HamidaEtAlNet method has the lowest classification effect because it only uses 3DCNN to build the model. HybridSN uses a hybrid of 2DCNN and 3DCNN to extract spectral spatial features and achieved a classification result of 57.71%. Judging from the classification results of the three graph neural networks, the autoregressive translation filter ARMA graph neural network has better noise suppression ability in HSI data, so it achieved better results than the GCN and GAT methods. The GACP method proposed in this invention is significantly better than the prediction graphs of the other seven methods in terms of both boundary clarity and smoothing effect.

[0250] from Figure 4Comparing the performance of the classification methods, WFCG, CEGCN, and GACP showed that GACP achieved 4.28% and 2.55% higher overall accuracy than WFCG and CEGCN, respectively. HamidaEtAl and HybridSN had more misclassified points in their classification prediction graphs, resulting in lower classification accuracy than the three graph neural networks (ARMA, GCN, and GAT). Even for vegetables with fewer sample sizes, such as romaine lettuce, the GACP method produced clearer classification results than the other seven classification methods.

[0251] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A small sample HSI classification method based on a new GCN and CNN network, characterized by: The following steps are involved: S1, linear discriminant analysis LDA is used to perform linear projection mapping on HSI data, and then the pixel features are converted to superpixel features through the SLIC segmentation algorithm; S2: The superpixel features are fed into the graph encoder to form graph data. The ARMA graph neural network is then used to extract the features of the HSI graph data. The decoder then decodes the graph data features. S3, perform nonlinear feature analysis on graph data features deal with, Get feature one, including: HSI feature information of the i-th channel position After mapping the features through nonlinear feature transformation, it is expressed as: Represents the position information of the i-th position neuron in the l-th layer of the neural network; Represents the position information of the i-th position neuron in the l-1 layer of the neural network; The weight coefficient of the neuron at position i in the neural network of layer l-1; Indicates the size of the noise at the i-th position neuron of the l-1 layer neural network; ψ represents the activation function LeakyReLU; S4, denoises the HSI data through spectral cleaning convolution, and performs dimensionality reduction processing on the high-dimensional HSI data to generate a dimensionality reduction feature map; S5: The dimensionality reduction feature map is used to locate the spectral spatial position information through the attention mechanism, and then the spectral spatial features after positioning are processed through spatial feature convolution. After that, another layer of attention mechanism and spatial feature convolution is performed to finally obtain feature 2; S6, performing ConCat multi-feature fusion on the feature 1 obtained in step S3 and the feature 2 obtained in step S5, and outputting the classification result.

2. The small sample HSI classification method based on the new GCN and CNN network according to claim 1 is characterized in that: In the process of converting pixel features to superpixel features in the SLIC segmentation algorithm, a correlation matrix C∈R is generated. HW ×T To record the clustering of pixels into superpixels, where H and W represent the height and width of the correlation matrix respectively, and T represents the number of superpixels divided into; the relationship between pixels and superpixels is defined as: C i,j It represents the specific value of the matrix C at the spatial position (i, j); Represents the i-th pixel at spatial position; S j The jth spatial position is represented by a superpixel; represents the normalized X; X represents HSI pixel features; Flatten(.) means flattening the pixels in the HSI data space; Then, the graph encoder is used to encode the correlation matrix C and the HSI pixel feature X to generate a new feature vector: Encoder(X; C) means encoding operation on X and C; is a matrix The transpose of It represents the normalization of the correlation matrix C; Then, the features in the graph nodes are extracted through graph convolution with ARMA filters, and then the features in the graph nodes are projected into the spatial dimension using the graph solver: Decoder(V; C) represents the decoding operation of V and C; Reshape(.) means restoring the flattened HSI data; V represents the node features in ARMA graph convolution; Finally, through LDA projection dimensionality reduction and superpixel segmentation algorithm SLIC, undirected graph data G = (V, E) is generated, where E represents the edge of the graph data.

3. The small sample HSI classification method based on the new GCN and CNN network according to claim 1 is characterized in that: The feature extraction of HSI graph data is performed by ARMA graph neural network, and the graph node features obtained are: Represents the graph node features obtained through the ARMA graph neural network; K represents the image signal after superposition of K first-order filters ARMA1; M represents M adjacency matrices that need to be superimposed; a k Represents the correlation coefficient a after superposition of k first-order filters ARMA1; b k Represents the correlation coefficient b after superposition of k first-order filters ARMA1; μ m represents the eigenvalue of the mth adjacent matrix; u m Represents the eigenvalue of the mth adjacent matrix after superposition of K 1st-order filters ARMA1; for u m The matrix transpose of .

4. The small sample HSI classification method based on the new GCN and CNN network according to claim 3 is characterized in that: The ARMA1 filter is iteratively expressed as a new graph convolution layer: Represents the graph signal at time t-1 after passing through the ARMA graph neural network filter; Represents the graph signal at time t after passing through the ARMA graph neural network filter; σ represents the activation function; represents the corrected Laplace matrix; W represents the weight after ARMA graph neural network filtering; V represents the graph node features; U represents the unfiltered weights that need to be learned during the training process.

5. The small sample HSI classification method based on the new GCN and CNN network according to claim 1 is characterized in that: After the HSI data is spectrally cleaned and convolved, it is expressed as: in Represents the HSI feature map after spectral cleaning convolution; ψ represents the activation function LeakyReLU; Represents the weight coefficient of the cth channel of the input l-1 layer at the spatial position (i, j); Represents the HSI feature map with a height of h, a width of w, and a number of bands b for the input of the l-1 layer; Indicates that the height of the l-1 layer input is h, the width is w, and the c-th channel offset size; l represents the convolution of the lth layer of the neural network; i, j represent spatial positions; b represents the number of bands; c represents a channel.

6. The small sample HSI classification method based on the new GCN and CNN network according to claim 1 is characterized in that: The method of locating the spectral spatial position information by the attention mechanism includes the following steps: For the HSI spectral space feature map X∈R that is input to the attention mechanism H×W×C , use a pooling kernel of size (H×1) to encode each channel along the horizontal coordinate of the feature map, H, W, C represent the input feature map height H, width W, and number of channels C respectively; then the feature map of the cth channel with a height of h after encoding is: Represents the feature map of the cth channel with the encoded height h; W represents the width of the input HSI spectral spatial feature map; x c (h,i) represents the spectral feature map of the cth channel at the spatial position (h,i), i represents the horizontal position, and h represents the height; Similarly, a pooling kernel of size (1×W) is used to encode each channel along the vertical coordinate, and the new output size is generated: in Represents the feature map of the cth channel with a width of w after encoding; x c (j,w) represents the spectral feature map of the cth channel at the spatial position (j,w), j represents the vertical position, and w represents the width; Then and Perform the concatenate operation, then use the convolution transformation function F1 with a convolution kernel of 1×1 to transform it, and then use the nonlinear activation function δ to activate it, and obtain the feature map f of the intermediate transition between the horizontal and vertical encoding of the spectral spatial position: f=δ(F1([z h ,z w ]))(16) F1 is the convolution transformation function; [z h ,z w ] indicates and Perform concatenate operation; The transition feature map is then decomposed into two separate vectors f along the spatial dimension that do not interfere with each other h ∈R C / r×H With f w ∈R C / r×W , where r represents the channel reduction ratio factor; again using the 1×1 convolution transformation function F h 、F w Transform the two vectors to have the same number of channels as the spectral feature map X: g h =σ(F h (f h ))(17) g w =σ(F w (f w )) where g h 、g w Respectively represent the spectral feature maps encoded in the vertical and horizontal directions after convolution transformation; σ is the sigmoid activation function; F h 、F w Represent the convolution transformation functions for vertical and horizontal coding respectively; f h 、f w Feature maps representing the intermediate transitions of vertical and horizontal encoding respectively; Finally, the output g h 、g w Perform weighted aggregation operation with the spectral feature map X to obtain the final spatial position output y c (i,j): x c (i, j) represents the spectral feature map of the cth channel at the spatial position (i, j); Represents the feature map of the cth channel in the horizontal direction i after weight aggregation; Represents the feature map of the cth channel in the vertical direction j after weight aggregation.

7. The small sample HSI classification method based on the new GCN and CNN network according to claim 1 is characterized in that: After the spatial feature convolution is used to perform information interaction processing on the spectral spatial features after positioning, the feature map X is expressed as: It represents the output after separable convolution feature extraction; ψ represents the activation function LeakyReLU; It represents the weight coefficient of the depth-wise separable convolutional layer; Represents the input HSI feature map; Indicates the size of the HSI feature map noise.

Images