Hyperspectral image classification method under limited training samples based on hybrid cnn-gcn network

Abstract

The application relates to a hyperspectral image classification method based on a hybrid CNN-GCN network under limited training samples, and relates to a hyperspectral image classification method.The application aims to solve the problems that existing marked samples are insufficient, an existing graph convolution network does not sufficiently utilize edge relations, network performance is limited, and the classification accuracy of a hyperspectral image is low.The specific process of the hyperspectral image classification method based on the hybrid CNN-GCN network under limited training samples is as follows: I. establishing a hybrid network model HCGN of a convolutional neural network and a graph convolution network;obtaining the trained hybrid network model HCGN of the convolutional neural network and the graph convolution network; II. inputting a to-be-detected hyperspectral image into the trained hybrid network model HCGN of the convolutional neural network and the graph convolution network, and completing the classification of the to-be-detected hyperspectral image.The application is used in the field of hyperspectral image classification.

Description

Hyperspectral image classification method based on limited training samples of hybrid CNN-GCN network Technical Field

[0001] This invention relates to a hyperspectral image classification method. Background Technology

[0002] Hyperspectral images (HSIs) typically have hundreds of bands, each containing information about spectral properties [1]. Based on the material composition and structural characteristics reflected by HSIs, the spectral information of HSIs can be extracted to more effectively distinguish objects of interest, demonstrating strong material identification capabilities. Currently, HSIs have been widely applied in agriculture [2], forestry [3], urban planning [4], geological exploration [5], and other fields. In these applications, hyperspectral image classification is a common technique. However, hyperspectral mixing, spectral variability, and complex noise effects make it difficult to extract image identification information. In addition, labeled HSI samples are scarce. Therefore, hyperspectral image classification remains a challenging topic in the field of hyperspectral imaging.

[0003] In recent years, thanks to advancements in equipment and the expansion of accessible image data, deep learning (DL) has made groundbreaking progress in computer vision [6]-

[12] and natural language processing

[13] -

[18] . DL can automatically extract more abstract and distinctive features, avoiding artificial engineering and the need for prior knowledge. Similarly, DL is also very popular in hyperspectral image classification tasks. In hyperspectral image classification, the more popular DL frameworks mainly include autoencoders (AEs), convolutional neural networks (CNNs), generative adversarial networks (GANs), recurrent neural networks (RNNs), capsule networks (CapsNet), Transformer networks, and graph convolutional networks (GCNs). Chen et al.

[19] first used principal component analysis (PCA)

[20] to reduce the dimensionality of HSI, and then used the designed stacked autoencoder for image feature extraction. In

[21] , Hang et al. designed a hyperspectral image classification network based on the sequentiality of adjacent spectral bands using cascaded RNNs. GAN networks mainly consist of two main parts

[22] : a generator and a discriminator. The generator generates pseudo images by learning the distribution of real images, and the discriminator is used to distinguish between real images and pseudo images generated by the generator. In

[23] , Odena et al. designed an auxiliary classifier using GAN and used it to handle multi-classification tasks. In addition, Radford et al.

[24] combined GAN and CNN frameworks to construct a deep convolutional GAN ​​(DCGAN), which has been widely used.

[0004] It is worth noting that CNN is one of the mainstream frameworks of DL, and many excellent works have emerged. According to the characteristics of feature extraction, this series of CNN-based methods can be divided into three categories: spectral feature-based methods, spatial feature-based methods, and joint methods based on spectral-spatial joint

[25] . In

[26] , Hu et al. used 1-DCNN to directly extract features from the HSI spectral domain and obtained better performance than support vector machine and traditional deep learning-based methods. Chen et al.

[27] used 2-DCNN to extract features of nonlinearity, discriminativeness and invariance of images. In addition, they also proposed a finite element model based on 3-DCNN with regularization to extract effective spectral spatial features of hyperspectral images and provided competitive results. Hamida et al.

[28] divided HSI into multiple 3D cubes and constructed 3D-CNN to extract spectral-spatial joint features of images. Roy et al.

[29] combined 3-DCNN and 2-DCNN to design a hybrid CNN classification network. In

[29] , Yu et al. proposed a spatially spectral-dense CNN framework with feedback attention, which solved the problems of high complexity, information redundancy and inefficient description of previous networks. However, as CNN networks become deeper, they become more complex. While deeper CNNs can acquire higher-order semantic features, they also bring more challenges. For example, gradient vanishing or exploding

[30] , overfitting

[31] , computational complexity

[32] , and interpretability

[33] . To address these issues, Zhong et al.

[34] designed a Spectral-spatial residual network (SSRN), which uses residual structures for modeling and is used to extract spatial-spectral features of images. Although the above work has shown good performance, the training of the network depends on sufficient labeled samples. However, labeling HSI samples is time-consuming and labor-intensive

[35] . Therefore, it is necessary to design classification networks with small training samples. Recently, some classification networks based on small training samples have been proposed, and their classification performance is quite satisfactory. For example, Ma et al.

[36] proposed a double-branch multi-attention (DBMA) network and verified that the model can perform well with limited training samples. In

[37] , Li et al. improved DBMA and designed a double-branch dual-attention (DBDA) network, and verified the effectiveness of the attention mechanism.In addition, Roy et al.

[38] proposed an attention-based adaptive spectral–spatial kernel improved residual network (A2S2K-ResNet) from the perspective of receptive field, and achieved good classification performance with small training samples. Similarly, we also proposed a feedback expansion convolution network (FECNet)

[39] with small training samples, and further improved the classification accuracy.

[0005] In the past two years, deep learning methods based on the Transformer framework have emerged one after another. Among them, the Vision Transformer (ViT)

[40] is a pioneering work in the field of computer vision. In

[41] , He et al. combined transfer learning and ViT to propose a Spatial-Spectral Transformer (SST) network, which was successfully applied to the hyperspectral image classification task. In addition, Qing et al.

[42] used attention mechanism and ViT modeling to effectively capture continuous spectral relationships. Hong et al.

[43] proposed a Spectral Transformer (SF) network from the perspective of spectral sequence. In order to overcome the constraint of the fixed geometric structure characteristics of the convolution kernel, Zhong et al.

[44] proposed a new Spectral-Spatial Transformer network (SSTN). In

[45] , Sun et al. proposed a Spectral-Spatial feature tokenization Transformer (SSFTT) which captures rich spectral spatial features and high-level semantic features. To address the difficulty of CNNs in representing long-range dependencies, Song et al.

[46] proposed a bottleneck spatial-spectral Transformer (BS2T) network. Although Transformers have strong capabilities in representing long-range dependencies, their classification performance still needs further improvement.

[0006] In contrast, deep learning frameworks based on graph convolution are not as popular as CNN and Transformer, but there are still some excellent works. HSI usually contains complex features and is irregularly distributed. Unlike CNN, GCN is able to aggregate the features of nodes in non-Euclidean

[47] space. Kipf and Welling et al.

[48] proposed the properties of graph convolution (GConv) through graph learning theory. According to the definition of graph convolution, Qin et al.

[49] proposed a graph convolution class network and used it to extract the spatial spectral features of HSI. In

[50] , Mou et al. proposed a nonlocal GCN. The network takes the entire image data as the input of the network, but this inevitably brings a large number of computational parameters. In order to alleviate the problem of computational explosion, Hong et al.

[51] proposed a new Minibatch GCN (mini GCN). In addition, Yang et al.

[52] used GraphSAGE to limit the size of the input graph. In order to reduce memory consumption, Wan et al.

[53] applied superpixels to the GCN structure and used them as graph nodes of graph convolution. In

[54] , Liu et al. proposed a dual-branch semi-supervised classification network that can extract pixel-level and superpixel-level features simultaneously. However, using superpixel technology to segment HSI will occupy a large amount of memory, which limits its application to some extent. Therefore, a fast dynamic graph convolutional network and CNN (Fast dynamic graph convolutional network and CNN, FDGC) parallel network

[55] were proposed to alleviate the problem of high model complexity. However, insufficient labeled samples still limit the classification performance of the above networks. In addition, the spatial structure based on superpixels cannot take into account the individual features of pixels. In contrast, CNN can extract more pixel-level spatial spectral features by sliding a square window. It is worth noting that CNN has difficulty learning long-distance spatial structures. Summary of the Invention

[0007] The purpose of this invention is to address the problems of insufficient labeled samples and the fact that existing graph convolutional networks do not fully utilize adjacent relationships, resulting in limited network performance and low accuracy in hyperspectral image classification. Therefore, this invention proposes a hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples.

[0008] The specific process of the hyperspectral image classification method based on limited training samples of a hybrid CNN-GCN network is as follows:

[0009] I. Establish a hybrid network model HCGN, combining convolutional neural networks and graph convolutional networks;

[0010] Obtain a well-trained hybrid network model HCGN, which combines a convolutional neural network and a graph convolutional network.

[0011] 2. Input the hyperspectral image to be tested into the trained network model HCGN, which is a hybrid of convolutional neural network and graph convolutional network, to complete the classification of the hyperspectral image to be tested.

[0012] The beneficial effects of this invention are as follows:

[0013] This invention proposes a hybrid CNN-GCN network (HCGN) for HSI classification using limited samples, designed to capture pixel-level fine-grained information and superpixel-level long-range structural information of images. First, LDA dimensionality reduction and superpixel segmentation are introduced. Next, a multi-scale graph edge enhanced module (MS-GEEM) is proposed to obtain multi-level structural features of the graph through long-range spatial relationships. Second, a multi-scale cross fusion module (MS-CFM) is proposed to extract multi-level fine-grained features. Finally, the extracted multi-level superpixel-level features and multi-level pixel-level features are concatenated. This invention proposes a graph edge enhanced module that enhances the representational power of the edge set and uses a multi-scale graph structure to capture rich multi-level graph features. This invention also designs a multi-scale cross fusion module to learn small-scale regular regions to obtain multi-level fine-grained features at the pixel level of the image. The proposed HCGN network for limited samples combines the CNN and GCN frameworks. By learning the features of small-scale regular regions and large-scale irregular regions, it extracts multi-level superpixel-level features and multi-level pixel-level features. Extensive experiments on four common datasets quantitatively validate the performance of the proposed HCGN method under limited samples. This solves the problems of insufficient labeled samples and the fact that existing graph convolutional networks do not fully utilize the adjacent relationships, which limits network performance and leads to low accuracy in hyperspectral image classification. Attached Figure Description

[0014] Figure 1 shows the overall framework of HCGN. GPM (Graph Projection Module) is the graph mapping module, LDA is the linear discriminant analysis algorithm, SLIC is the linear iterative clustering algorithm, and n1, n2, and n3 are the segmentation scale factors. For three types of undirected graphs, the Multi-scale graph edge enhanced module (MS-GEEM) is a multi-scale graph edge enhancement module. eLet X be a matrix, A be the adjacency matrix, GCN be a graph convolutional network, Shallow feature extraction be a shallow feature extraction module, Multi-scale cross fusion module (MS-CFM) be a multi-scale cross fusion module, Classification layer be a classification layer, FC be a classification layer, X be the input image of the model, and Y be the predicted result.

[0015] Figure 2 shows a comparison between ordinary graph convolutional structures and graph edge enhancement structures. (a) is an ordinary graph convolutional block, and (b) is a graph edge enhancement block. G(H) l Let A be a graph structure constructed based on adjacency relation A, and let G(H) be a graph structure constructed based on adjacency relation A. l A e ) is based on the adjacency relationship A e The constructed graph structure, W l H represents the weights corresponding to the graph convolution in the l-th layer. l+1 W1 represents the node features output by the l-th layer. l The weights of the graph convolution in the first branch of GEEM. The input node features are the graph convolutional features of the first branch in GEEM. The weights of the graph convolution in the second branch of GEEM. The input node features are the graph convolutional features of the second branch in GEEM;

[0016] Figure 3 shows the graph for obtaining the adjacency matrix A. e The relationship between A and the adjacency matrix A is A = A e +I, a1, a2, a3, a4, a5, a6 are superpixels, and I is the identity matrix; Figure 4 is the structure diagram of MS-GEEN, and the || operation represents the cascading of features; Figure 5 is the structure diagram of the shallow feature extraction CNN module; Figure 6 is the structure diagram of MS-CFM;

[0017] Figure 7 shows the classification accuracy of MS-GEEM on four datasets (including OA, AA, and Kappa) using different combinations of superpixel scale factors λ. The horizontal axis represents the four different combinations of scale factors λ, and the vertical axis represents the classification accuracy.

[0018] Figure 8 shows the classification results of all methods on the IndianPines dataset. (a) is the ground cover map, (b) is the false color map, and (c)-(l) are the classification results of 3DCNN (65.81%), DBDA (95.92%), Hybrid-SN (71.27%), A2S2KRsNet (95.60%), FECNet (96.09%), SSTN (95.14%), BS2T (95.25%), CEGCN (94.93%), FDGCN (92.15%), and HCGN (96.32%), respectively.

[0019] Figure 9 shows the classification results of all methods on the Pavia dataset. (a) is the ground cover map, (b) is the false color map, and (c)-(l) are the classification results of 3DCNN (80.32%), DBDA (97.80%), Hybrid-SN (87.38%), A2S2KRsNet (95.29%), FECNet (98.12%), SSTN (94.30%), BS2T (95.96%), CEGCN (98.34%), FDGCN (91.22%) and HCGN (98.62%), respectively.

[0020] Figure 10 shows the classification results of all methods on the Salinas data. (a) is the ground cover map, (b) is the false color map, and (c)-(l) are 3DCNN (86.17%), DBDA (96.30%), Hybrid-SN (94.45%), A2S2KRsNet (97.84%), FECNet (98.44%), SSTN (94.67%), BS2T (96.68%), CEGCN (98.31%), FDGCN (97.24%) and HCGN (99.18%), respectively.

[0021] Figure 11 shows the classification results of all methods on the WHU-Hi-LongKou data. (a) is the ground cover map, (b) is the false color map, and (c)-(l) are the classification results of 3DCNN (89.73%), DBDA (97.86%), Hybrid-SN (93.94%), A2S2KRsNet (96.73%), FECNet (98.08%), SSTN (94.75%), BS2T (98.01%), CEGCN (97.84%), FDGCN (94.62%) and HCGN (98.40%), respectively.

[0022] Figure 12 shows the visual effects of different methods using t-SNE for feature visualization on the IndianPines dataset: (a) SSTN, (b) FECNet, (c) CEGCN, and (d) HCGN.

[0023] Figure 13 shows the visual effects of different methods using t-SNE for feature visualization on Pavia data: (a) SSTN, (b) FECNet, (c) CEGCN, and (d) HCGN.

[0024] Figure 14 shows the visual effects of different methods using t-SNE for feature visualization on Salinas data: (a) SSTN, (b) FECNet, (c) CEGCN, and (d) HCGN.

[0025] Figure 15 shows the visual effects of different methods using t-SNE for feature visualization on the WHU-Hi-LongKou data: (a) SSTN, (b) FECNet, (c) CEGCN, and (d) HCGN.

[0026] Figure 16 shows the output feature visualization results of graph convolution operations constructed using three different superpixel scale factors on the Pavia dataset: (a) λ = 50, (b) λ = 100 and (c) λ = 150.

[0027] Figure 17 is a comparison of the traditional CNN module and MS-CFM, (a) traditional multi-scale CNN module, (b) MS-CFM;

[0028] Figure 18 shows a comparison of the classification results of the traditional CNN module and MS-CFM on the IndianPines dataset. MS-CFM is the multi-scale cross-fusion module proposed in this invention, while CNN is a general module.

[0029] Figure 19 shows a comparison of the overall classification accuracy (OA) of all methods using different training sample ratios on four datasets: (a) Indian Pines dataset, (b) Pavia dataset, (c) Salinas dataset, and (d) WHU-Hi-LongKou dataset. The horizontal axis represents the training sample ratio, and the vertical axis represents the overall accuracy (OA) value. Detailed Implementation

[0030] Specific Implementation Method 1: The specific process of this implementation method for hyperspectral image classification based on limited training samples of a hybrid CNN-GCN network is as follows:

[0031] This invention develops a hybrid CNN-GCN network for hyperspectral image classification. First, to improve the representational power of the edge set, we enhance the edge information of the adjacency matrix by designing an edge enhancement module, enabling the superpixel feature information of surrounding nodes to be fully learned. To obtain rich contextual graph information, we propose a multi-scale graph edge enhancement module based on the edge enhancement module. However, since the superpixel-based GCN spatial structure cannot consider the individual features of pixels, we propose a multi-scale cross-fusion module to learn small-scale regular regions to obtain multi-level fine features at the pixel level. Then, the extracted superpixel-level and pixel-level features are concatenated. Finally, extensive experiments demonstrate that, compared with other state-of-the-art CNNs and Transformers methods, the proposed method achieves better classification performance with limited training samples. This invention combines multi-scale CNNs, GCNs, and superpixel segmentation to achieve excellent hyperspectral image classification performance with a moderate number of training parameters. In recent years, Convolutional Neural Networks (CNNs) have impressed with their excellent feature representation capabilities. Unlike CNNs, Graph Convolutional Networks (GCNs) are well-suited for handling the intrinsic popular structures of hyperspectral images and have also garnered significant attention. However, existing GCN-based methods do not fully utilize edge relationships, limiting their performance. Furthermore, insufficient labeled samples are a common problem in hyperspectral image processing. Therefore, this invention proposes a Hybrid CNN-GCN network (HCGN) for hyperspectral image classification with limited training samples. First, a Graph Edge Enhancement Module (GEEM) is designed to enhance the superpixel-level features of graph edge nodes, improving the spatial discrimination ability of ground objects. Specifically, since rich multi-scale information is complementary, a Multi-scale Graph Edge Enhancement Module (MS-GEEM) is proposed based on GEEM to fully utilize texture structures of different sizes. Then, to enhance the pixel-level multi-level fine feature representation of the image, a Multi-scale Cross Fusion Module (MS-CFM) is proposed based on the CNN framework. Finally, the extracted pixel-level features and superpixel-level features are concatenated. A series of experiments demonstrate that, compared to some state-of-the-art methods, HCGN combines the advantages of CNN and GCN frameworks, exhibiting superior classification performance with limited training samples, showcasing the excellence and immense potential of HCGN.The overall framework of HCGN is shown in Figure 1. As can be seen from the figure, HCGN comprises five parts: a graph mapping module, a multi-scale graph edge enhancement module, a shallow feature extraction module, a multi-scale cross-fusion module, and a classification layer. This invention represents the original HSI data as follows. Where H and W represent the spatial size of HSI, and B represents the spectral band. First, the input image undergoes superpixel segmentation and multi-scale edge enhancement to obtain superpixel-level features. Second, the input image undergoes shallow feature extraction and multi-scale cross-fusion to obtain pixel-level features. Finally, the two types of features are concatenated and sent to a classification layer for classification, resulting in the image's label set. N is the maximum label value of a pixel.

[0032] 1. Establish a hybrid network model HCGN (Hybrid CNN-GCN network) that combines convolutional neural networks (CNNs) and graph convolutional networks (GCNs); obtain a trained hybrid network model HCGN.

[0033] 2. Input the hyperspectral image to be tested into the trained network model HCGN, which is a hybrid of convolutional neural network and graph convolutional network, to complete the classification of the hyperspectral image to be tested.

[0034] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that: In Specific Implementation Method One, a hybrid network model HCGN (Hybrid CNN-GCN network, HCGN) is established, which combines Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs).

[0035] The process of obtaining a hybrid network model HCGN, which combines a trained convolutional neural network and a graph convolutional network, is as follows:

[0036] Step 1: Randomly sample hyperspectral images (HSI);

[0037] The hyperspectral image training set is represented as Tag is

[0038] Where H, W, and B represent the height, width, and number of spectral bands of the hyperspectral image HSI, respectively. Represents real numbers;

[0039] The training set is randomly selected from each category according to a preset ratio;

[0040] Step 2: Construct a hybrid network model HCGN (Hybrid CNN-GCN network) that combines convolutional neural networks (CNNs) and graph convolutional networks (GCNs).

[0041] Obtain a well-trained network model HCGN, which is a hybrid of a convolutional neural network and a graph convolutional network.

[0042] The other steps and parameters are the same as in Specific Implementation Method 1.

[0043] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that step 2 involves constructing a network model HCGN that combines convolutional neural networks and graph convolutional networks;

[0044] The process of obtaining a hybrid network model HCGN, which combines a trained convolutional neural network and a graph convolutional network, is as follows:

[0045] The hybrid network model HCGN, which combines convolutional neural networks and graph convolutional networks, consists of five parts: a graph mapping module, a multi-scale graph edge enhancement module, a shallow feature extraction CNN module, a multi-scale cross-fusion module, and a classification layer.

[0046] Step 2: 1. Use the hyperspectral image training set X to input the Graph Projection Module (GPM) to complete the mapping of the image from pixels to superpixels, converting each superpixel into a node to obtain an undirected graph;

[0047] Step 22: Input the node features corresponding to the nodes in the undirected graph and the connection relationship matrix between the nodes into the Multi-scale graph edge enhanced module (MS-GEEM), and obtain the multi-level structural features of the graph through the long-distance spatial relationship of the graph;

[0048] Steps 2 and 3: Input the hyperspectral image training set X into the output features of the shallow feature extraction CNN module;

[0049] Step 24: Input the output features of the shallow feature extraction CNN module into the multi-scale cross fusion module MS-CFM to extract multi-level fine features of the hyperspectral image;

[0050] Step 25: Concatenate the multi-level structural features of the graph obtained in Step 22 with the features of the hyperspectral image extracted in Step 24, and then feed the concatenated features into the FC classification layer for classification; obtain the trained network model HCGN, a hybrid of convolutional neural network and graph convolutional network. Other steps and parameters are the same as in Implementation Method 1 or 2.

[0051] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that: in step two, the hyperspectral image training set X input graph projection module (GPM) completes the mapping of the image from pixels to superpixels, converting each superpixel into a node to obtain an undirected graph; the specific process is as follows:

[0052] For the GCN model structure, an accurate graph plays an important role in the expressiveness of features. However, using image pixels as graph nodes will bring a huge computational burden. Fortunately, dividing pixels into different regions to construct graph nodes has been proven to effectively reduce computational costs in GCN-based models

[53] . Therefore, in order to make the constructed graph nodes as accurate as possible and alleviate the computational burden, this paper uses Simple lineariterative clustering (SLIC)

[56] to perform super-pixel segmentation to complete the mapping of the image from pixels to superpixels. The GPM structure is shown in the upper left part of Figure 1.

[0053] Step 2.11: To make the dimensionality-reduced data more discriminative and contain more information, the Linear Discriminant Analysis (LDA) algorithm is used to perform supervised dimensionality reduction on the hyperspectral image training set X, resulting in the dimensionality-reduced data. Where L is the number of spectral bands in the dimensionality-reduced data;

[0054] Step 2.12: To divide superpixel regions of different sizes, a segmentation scale factor λ is introduced. The total number of superpixels k obtained by using different segmentation scale factors λ for each superpixel segmentation is different, as shown below.

[0055] k = (H × W) / λ, 1 ≤ λ

[0056] Where H×W represents the total number of pixels in the image;

[0057] To complete the transformation of an image from pixels to superpixels, the dimensionality-reduced data X is... LDA The input linear iterative clustering (SLIC) is used for the first superpixel segmentation, dividing the image into multiple spatially connected and spectrally similar superpixel images. The superpixel set obtained from the first superpixel segmentation is:

[0058] To complete the transformation of an image from pixels to superpixels, the dimensionality-reduced data X is... LDA The input linear iterative clustering (SLIC) is used for a second superpixel segmentation, dividing the image into multiple spatially connected and spectrally similar superpixel images. The superpixel set obtained from the second superpixel segmentation is:

[0059] To complete the transformation of an image from pixels to superpixels, the dimensionality-reduced data X is... LDA The input linear iterative clustering (SLIC) is used for a third superpixel segmentation, dividing the image into multiple spatially connected and spectrally similar superpixel images. The superpixel set obtained from the third superpixel segmentation is: Among them, S i Represents the data X after dimensionality reduction LDA The i-th superpixel after superpixel segmentation, and S represents i The first in 1 pixel, Z i For S i The total number of pixels in;

[0060] The superpixels should satisfy the following conditions

[0061]

[0062] Among them, S j Represents the data X after dimensionality reduction LDA The j-th superpixel after superpixel segmentation, and Represents the empty set;

[0063] Each pixel exists only in any partitioned superpixel S i In the image training set X, the sum of pixels within all superpixels is the same as the sum of pixels in the superspectral image training set X.

[0064] It is worth noting that the total number of superpixels in an image varies depending on the segmentation scale factor λ, which means that the image information contained therein is different.

[0065] Steps 2-3: By constructing adjacency relationships between superpixels, the centroid of each superpixel in the superpixel set obtained from the first superpixel segmentation is taken as a node. After converting the centroids of all superpixels in the superpixel set into nodes, a node set is obtained, which constitutes an undirected graph.

[0066] Using the centroid of each superpixel in the superpixel set obtained from the second superpixel segmentation as a node, and converting the centroids of all superpixels in the superpixel set into nodes, we obtain a node set, which constitutes an undirected graph.

[0067] Using the centroid of each superpixel in the superpixel set obtained from the third superpixel segmentation as a node, and converting the centroids of all superpixels in the superpixel set into nodes, we obtain a node set, which constitutes an undirected graph.

[0068] in, Let ε and ε represent the nodes and edges of an undirected graph, respectively.

[0069] Therefore, in order to construct graphs at different neighborhood scales to capture multi-scale feature information, it is assumed that different scale factors λ = {λ i |n1,n2,n3}, HSI is transformed into the corresponding three undirected graphs.

[0070] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0071] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that: in step two, the node features corresponding to the nodes in the undirected graph and the connection relationship matrix between the nodes are input into the Multi-scale graph edge enhanced module (MS-GEEM) to obtain the multi-level structural features of the graph; the specific process is as follows:

[0072] Step 221: Obtain the adjacency matrix A based on the connection relationships between the segmented superpixels. e Based on adjacency matrix A e The connection matrix A is obtained by combining the identity matrix I;

[0073] The superpixel feature V is a matrix of nodes in an undirected graph. In an undirected graph, each node is the centroid of a superpixel. Let be the k-th node in an undirected graph; T represents the transpose of the graph.

[0074] Based on undirected graph structure Construction diagram structure G1(V) l A) G2(V l A), G3(V l A);

[0075] Step 2.2.2. Graph Structure G1(V) l When A in (A) is the connectivity matrix A, the graph structure G1(V) is... l A) Inputting into a fully connected FC layer yields feature V1. l ;Graph structure G1(V l In A), A is taken as the adjacency matrix A eAt that time, the graph structure G1(V) l A) Input the fully connected (FC) layer to obtain features.

[0076] Features and characteristics Feature fusion is performed to obtain feature V. l+1 ;

[0077] V l and V l+1 These represent the input and output of the l-th layer, respectively; the output V of the l-th layer... l+1 As input to the (l+1)th layer;

[0078] Feature V l+1 Replace the graph structure G1(V) l V in A) l The graph structure G1(V) is obtained. l+1 A);

[0079] Graph structure G1(V) l+1 When A in (A) is the connectivity matrix A, the graph structure G1(V) is... l+1 A) Input the fully connected (FC) layer to obtain features. Graph structure G1(V) l+1 In A), A is taken as the adjacency matrix A e At that time, the graph structure G1(V) l+1 A) Input the fully connected (FC) layer to obtain features.

[0080] Features and characteristics Feature fusion is performed to obtain feature V. l+2 ;

[0081] Steps 2-3, Graph Structure G2(V) l When A in A) takes the connection matrix A, the graph structure G2(V) l A) Input the fully connected (FC) layer to obtain features. Graph structure G2(V) l In A), A is taken as the adjacency matrix A e At that time, the graph structure G2(V) l A) Input the fully connected (FC) layer to obtain features.

[0082] Features and characteristics Feature fusion is performed to obtain feature V. l+3 ;

[0083] Feature V l+3 Replace the graph structure G2(V)l V in A) l The graph structure G2(V) is obtained. l+3 A);

[0084] Graph structure G2(V) l+3 When A in A) takes the connection matrix A, the graph structure G2(V) l+3 A) Input the fully connected (FC) layer to obtain features. Graph structure G2(V) l+3 In A), A is taken as the adjacency matrix A e At that time, the graph structure G2(V) l+3 A) Input the fully connected (FC) layer to obtain features.

[0085] Features and characteristics Feature fusion is performed to obtain feature V. l+4 ;

[0086] Step 224, Graph Structure G3 (V) l When A in (A) is the connectivity matrix A, the graph structure G3(V) is... l A) Input the fully connected (FC) layer to obtain features. Graph structure G3(V) l In A), A is taken as the adjacency matrix A e At that time, the graph structure G3(V) l A) Input the fully connected (FC) layer to obtain features.

[0087] Features and characteristics Feature fusion is performed to obtain feature V. l+5 ;

[0088] Feature V l+5 Replace the graph structure G3(V) l V in A) l The graph structure G3(V) is obtained. l+5 A);

[0089] Graph structure G3(V) l+5 When A in (A) is the connectivity matrix A, the graph structure G3(V) is... l+5 A) Input the fully connected (FC) layer to obtain features. Graph structure G3(V) l+5 In A), A is taken as the adjacency matrix A e At that time, the graph structure G3(V) l+5 A) Input the fully connected (FC) layer to obtain features.

[0090] Features and characteristics Feature fusion is performed to obtain feature V. l+6 ;

[0091] Step 225: Apply the feature V obtained in Step 222. l+2 The features V obtained in steps two and three l+4 The feature V obtained in step two is... l+6 Features are concatenated and used as the output of the Multi-scale Graph Edge Enhanced Module (MS-GEEM).

[0092] Multi-scale graph edge enhanced module (MS-GEEM)

[0093] The effectiveness of multi-scale information in information extraction tasks for hyperspectral image classification has been widely proven

[57]

[58] . This is because the land cover structure contained in HSI is intricate, and the context information extracted at different scales can enrich the regional features of the image. In the method designed in this invention, the three undirected graphs provided are used to capture multi-scale image spectral spatial information. In addition, considering that different nodes in the graph contribute differently to feature extraction, in order to enhance the representation ability of effective adjacent nodes, inspired by edge convolution (EdgeConv)

[59] , we propose a multi-scale graph edge enhanced module (MS-GEEM). Next, we will introduce the proposed GEEM and MS-GEEM in detail.

[0094] Typically, diagrams Encoding is performed using a node matrix H and an adjacency matrix A. The i-th row of H represents the i-th node, and A... i,j Let represent the edge weight between the i-th node and the j-th node. The Laplace operator

[60] can be defined as L = DA. By further standardizing it, the graph Laplace can be defined as

[0095] In the above formula, I is the identity matrix. Assume the graph signal for each node is x∈R. N According to Defferrard et al., graph convolution is defined as...

[0096] Where, x∈R N Let g be the node signal of the graph. θ This represents a spectral filter, * represents the spectral convolution operation, and K represents the spectral filter g. θThe order, T k For Chebyshev polynomials, λ max express The largest eigenvalue, θ∈R K This represents the K-order vector of Chebyshev coefficients.

[0097] Typically, for ease of application, Kipf and Welling

[48] set K in Equation 5 to 1, and λ to 1. max Taking 2, formula 5 is simplified to:

[0098] Formula 6 is normalized, and the convolution calculation is further obtained as follows:

[0099] In the above formula, and In multidimensional image processing, the graph signal can be generalized to X∈R N ×C C represents the channel dimension of the image. Therefore, the propagation rule of the graph convolution model is defined as follows:

[0100] Among them, H l and H l+1 Let W represent the input and output of the l-th layer, respectively, where σ represents the activation function and W is the input and output of the l-th layer. (l) This represents the weights of the l-th layer. Typically, the graph convolutional network used contains two layers, and the output of the second layer uses a softmax classifier. Finally, the convolution of the graph convolutional network is defined as...

[0101] in, f(·,·) is the graph convolution function. The structure of the graph convolution network is shown in Figure 2(a).

[0102] Through the above analysis, it can be seen that GCN utilizes the connectivity between graph nodes to pass information about the current layer itself and its neighboring nodes to the next layer. However, this connectivity is obtained by constructing an adjacency matrix. In other words, besides the node itself, the information of its surrounding nodes is also very important. Taking a 6-node graph as an example, the process of obtaining the adjacency matrix A is shown in Figure 3. Matrix A... e The adjacency matrix A is a matrix in which the self-information is removed. Therefore, considering the neighborhood information of surrounding nodes, we propose a graph edge enhanced module (GEEM), which enhances the transmission of self- and neighborhood information, and its structure is shown in Figure 2(b). Specifically, assume H l and H l+1 Let A and A' represent the input and output of the l-th layer, respectively. eThe adjacency matrices of the two branches are used respectively to fully extract the graph information of the image. Typically, GCN uses two layers of graph convolution for feature extraction; in this invention, this corresponds to two GEEM structures. The entire feature extraction process can be represented as follows:

[0103] in, This indicates the fusion of two features. W i This represents the weights of the graph convolution at the i-th layer.

[0104] It is worth noting that HSI contains complex land cover structures, and contextual information extracted at different scales can enrich the regional features of the image. Therefore, in order to further capture multi-scale image spectral spatial information and enhance the representational power of features, we adopted three different superpixel scale factors to obtain three different graph structures, which were then applied to the proposed GEEM to obtain a new module, MS-GEEM. Finally, the multi-scale graph features obtained from the three different graph structures were concatenated, and the structure is shown in Figure 4.

[0105] According to Formula 11, G1, G2, and G3 are input into the MS-GEEM module to perform graph convolution operations, and the results are fused; expressed as F = f1(G1, A, A e )||f3(G2,A,A e )f3||(G3,A,A e )

[0106] Where G1, G2, and G3 represent the module's inputs, and F represents the output. i (·,·,·) represents the graph convolution function of MS-GEEM. Other steps and parameters are the same as in specific implementation methods one to four to one.

[0107] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that: in steps two and three, the hyperspectral image training set X is input into the shallow feature extraction CNN module to extract features from the hyperspectral image. Specifically, in recent years, Convolutional Neural Networks (CNNs) have been widely used in hyperspectral image classification tasks due to their superior feature learning capabilities. Typically, CNN modules are used to reduce redundant information contained in HSI images and enhance the robustness of the module. Similarly, before the proposed Multi-scale Cross Fusion Module (MS-CFM), a CNN module is used for shallow feature learning and to reduce redundant information contained in the HSI, as shown in Figure 5.

[0108] The shallow feature extraction CNN module consists of a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 1×1 convolutional layer.

[0109] The formula for calculating a two-dimensional convolution module is as follows:

[0110] Among them, f cnn (·) represents a two-dimensional convolution function (including normalized BN layers, ReLU activation layers, and convolutional layers), where H and W represent the height and width of the input image, respectively. Let b represent the weight parameters of the d-th feature map at position (i,j). i,j This is a bias term; other steps and parameters are the same as those in specific implementation methods one to five to one.

[0111] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the entire process of the shallow feature extraction CNN module is represented as y = f cnn (f cnn (x))

[0112] In the formula, x and y represent the input and output of the image, respectively, and f cnn (·) represents a two-dimensional convolution function, which includes a normalized BN layer, a ReLU activation layer, and a convolutional layer in sequence. Other steps and parameters are the same as in specific implementation methods one to six to one.

[0113] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that: in step two-four, the output features of the shallow feature extraction CNN module are input into the multi-scale cross fusion module MS-CFM to extract multi-level features of the hyperspectral image; the specific process is as follows:

[0114] The multi-scale cross-fusion module MS-CFM includes, in sequence, a first convolutional block, a second convolutional block, a third convolutional block, and a fourth convolutional block;

[0115] The first convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 1×1 convolutional layer;

[0116] The second convolutional block consists of a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 5×5 convolutional layer.

[0117] The third convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 3×3 convolutional layer;

[0118] The fourth convolutional block includes, in sequence, a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 7×7 convolutional layer;

[0119] The shallow feature extraction CNN module outputs features that are input into the first convolutional block, and the first convolutional block outputs feature 1;

[0120] The shallow feature extraction CNN module outputs features which are input into the second convolutional block, and the second convolutional block outputs feature 2.

[0121] Feature 1 and Feature 2 are fused and then fed into the third convolutional block, which outputs Feature 3.

[0122] Feature 1 and Feature 2 are fused and then fed into the fourth convolutional block, which outputs Feature 4.

[0123] The feature obtained by fusing feature 3 and feature 4 is used as the output of the multi-scale cross-fusion module MS-CFM.

[0124] The MS-CFM structure is shown in Figure 6. Among them, This represents feature fusion. As shown in the figure, MS-CFM mainly consists of multiple BN, ReLU, and convolutions. Notably, the convolutional layers use different kernel sizes, and the feature extraction branches are cross-connected. This multi-scale structure can effectively extract multi-level features of the image and improve the diversity of spatial spectral features. Other steps and parameters are the same as in one of the specific implementation methods one to seven.

[0125] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that the entire calculation process of the multi-scale cross-fusion module MS-CFM is represented as follows:

[0126]

[0127] Where x and y represent the input and output of the image, respectively, f cnn1 The convolution function for the first convolutional block (x passes through a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 1×1 convolutional layer in sequence), f cnn2 The convolution function for the second convolutional block (x passes through a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 5×5 convolutional layer in sequence), f cnn3 The convolution function for the third convolutional block (x passes through a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 3×3 convolutional layer in sequence), f cnn4The convolution function for the 4th convolutional block (x passes through a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 7×7 convolutional layer in sequence). The fusion of features is indicated. Other steps and parameters are the same as in any of the specific implementation methods one through eight.

[0128] The beneficial effects of the present invention are verified using the following embodiments:

[0129] Example 1:

[0130] To better explain the Hyperspectral Image Classification Network (HCGN), the implementation process of the network is presented.

[0131] Taking the Indian Pines dataset as an example, the input image is Label First, the output after LDA dimensionality reduction is flattened into... The SLIC operation is used to convert pixels to superpixels, resulting in the image. and the correlation matrix Where the number of superpixels k = [H×W] / λ. Taking λ = 10, 100, and 150, we can obtain three images with different numbers of superpixels. And three correlation matrices. Next, the three types of graphs containing graph features V are respectively sent to the three branches of the MS-GEEM module for graph feature extraction to obtain superpixel multi-level features. Next, according to Equations 13 and 14, the input X is then processed by the shallow feature extraction module and MS-CFM for spatial spectral feature extraction, and the input and output sizes remain unchanged during the extraction process. Finally, the extracted superpixel features and pixel features are concatenated and sent to the softmax classifier for classification.

[0132] Table I. Implementation Process of HCGN

[0133]

[0134] Example 2:

[0135] Extensive experiments were conducted on the HCGN and comparison method proposed in this invention, and all experimental results are presented.

[0136] A. Dataset

[0137] The experiments used four common HSI datasets: Indian Pines, Pavia, Salinas, and WHU-Hi-LongKou. We will briefly introduce the category information and training sample size of these four datasets below. The Indian Pines dataset was captured in India in 1992 by the AVIRIS imaging spectrometer. The image size is 145×145 pixels and contains 16 categories, primarily farm-related, including corn, grass, soybean, and woods. The image contains 220 spectral bands; after removing the water absorption bands, 200 bands were used for the experiments. Furthermore, the spatial resolution of the image is 20m, and the wavelength range is 0.4–2.5μm. The Pavia dataset was captured in Italy in 2003 by the ROSIS-03 imaging spectrometer. The image spatial resolution is 1.3m, the spatial size is 610×340 pixels, and it contains 103 usable bands. The main land cover categories included are trees and asphalt roads. The Salinas dataset was captured in the United States by the AVIRIS imaging spectrometer. It has a spatial size of 512×217, 224 spectral bands, and contains 16 categories. Notably, the WHU-Hi-LongKou dataset was captured in 2018 in Longkou Town, Hubei Province, China, by a DJI Motrice 600 Pro drone platform. It has a spatial size of 550×400, a spatial resolution of approximately 0.463m, a wavelength range of 0.4µm-1µm, and contains 270 usable spectral bands. Furthermore, this image depicts a simple farm, with main categories including corn, cotton, and sesame. Detailed category information and the number of training samples for the above four datasets are shown in Table II. As can be seen, we selected a limited number of samples for each category as the training set, with the remainder used as the test set.

[0138] Table II lists the category names and sample sizes for the four datasets.

[0139]

[0140]

[0141] B. Experimental Setup

[0142] 1) Platform Details: To ensure the accuracy of the experiments, all experiments were conducted on the same platform. The hardware used consisted of an Intel(R) Core(TM) i9-9900K CPU with 128GB of memory and an NVIDIA GeForce RTX 3090 GPU with 24GB of RAM. Furthermore, the PyTorch framework was used, and all final experimental results were the average of thirty experiments. 2) Evaluation Metrics: To more effectively evaluate the model's superiority, this paper uses four commonly used performance metrics: classification accuracy per class, overall accuracy (OA), average accuracy (AA), and Kappa coefficient. 3) Comparison Algorithms: In order to evaluate the classification performance of the proposed HCGN, this paper selects some advanced deep learning-based hyperspectral image classification networks, including five CNN-based methods: 3DCNN

[28] , DBDA

[37] , Hybrid-SN

[29] , A2S2KResNet

[38] and FECNet

[39] , two Transformer-based methods: SSTN

[44] and BS2T

[46] , and two GCN-based methods: CEGCN

[54] and FDGCN

[55] .

[0143] C. Sensitivity analysis of parameters:

[0144] In the proposed HCGN network, some hyperparameters affect the network's classification performance. For example, to obtain the graph topological representation of HSI, a scale factor λ is introduced during superpixel segmentation. Different λ values ​​result in different regions being partitioned and different graph representations being constructed, which will affect the amount of information exchange between nodes. Therefore, selecting the optimal λ is essential for different λ values. This invention explores various combinations of λ on four datasets, and the experimental results are shown in Figure 7. As can be seen from Figure 7, the horizontal axis represents four different combinations of scale factors λ, and the vertical axis represents the classification accuracy. The choice of different λ combination sets varies depending on the spatial resolution of different datasets. Specifically, in the Indian Pines dataset, the different λ combination sets are λ = {10, 50, 100, 150}, and in the Pavia, Salinas, and WHU-Hi-LongKou datasets, the different λ combination sets are λ = {50, 100, 150, 200}. All four combinations extract three elements from the combination set sequentially and insuffically. Furthermore, the yellow bars represent OA values, the red bars represent AA values, and the green bars represent Kappa values. It's easy to see that the optimal results across the four datasets are found in case 3, case 1, case 1, and case 1, with corresponding λ combinations of {10,100,150}, {50,100,150}, {50,100,150}, and {50,100,150}, respectively. Additionally, the obtained combinations show that smaller segmentation scales often yield better classification performance. This is because smaller segmentation scales capture more nodes, resulting in more refined node information and richer node interactions. However, this inevitably leads to more parameters, making network training more difficult. Therefore, the optimal segmentation scale factor λ combinations selected by the network proposed in this invention across the four datasets are {10,100,150}, {50,100,150}, {50,100,150}, and {50,100,150}, respectively.

[0145] D. Numerical and visual comparisons with other algorithms

[0146] 1) Quantitative Evaluation: To verify the superiority of HCGN, this invention selected several advanced deep learning networks for comparison. The quantitative results of experiments on the Indian Pines, Pavia, Salinas, and WHU-Hi-LongKou datasets are recorded in Tables III-VI. These results include classification accuracy, overall accuracy, Kappa coefficient, required network parameters, training time, and testing time for each category. Tables III-VI show that our method, combining GCN and CNN, achieves superior classification accuracy compared to CNN- and Transformer-based methods with small training samples. This is because HCGN focuses more on global feature smoothing rather than local smoothing. This not only improves intra-class similarity but also makes inter-class features more discriminative. Compared to GCN-based methods, the classification accuracy obtained by our method also has a significant advantage. This is inferred because our invention not only utilizes the interaction of multi-scale spatial information to aid graph convolution operations but also employs multi-scale CNN blocks to obtain richer high-level image features. Furthermore, as shown in Tables III-VI, the required parameters for our method are moderate, while the required training and testing times are satisfactory. It is worth noting that, with parameters and computation time within our capabilities, the proposed method HCGN achieves optimal classification accuracy. Specifically, on the IndianPines dataset, the OA values ​​obtained by the proposed method HCGN are 30.51%, 0.40%, 25.05%, 0.72%, 0.23%, 1.18%, 1.07%, 1.39%, and 4.17% higher than those of methods 3DCNN, DBDA, Hybrid-SN, A2S2KResNet, FECNet, SSTN, BS2T, CEGCN, and FDGCN, respectively. This verifies that the proposed method has a greater classification advantage than methods based on Transformer and GCN. On the Pavia dataset, the OA values ​​obtained by HCGN are 18.30%, 0.82%, 11.24%, 3.33%, 0.50%, 4.32%, 2.66%, 0.28%, and 7.40% higher than those of methods 3DCNN, DBDA, Hybrid-SN, A2S2KResNet, FECNet, SSTN, BS2T, CEGCN, and FDGCN, respectively. On the Salinas dataset, the OA, AA, and Kappa values ​​obtained by HCGN are 99.18%, 99.21%, and 99.09%, respectively, all of which are optimal values ​​and all exceed 99%.On the Salinas dataset, the OA values ​​obtained by the HCGN method are 8.67%, 0.54%, 4.46%, 1.67%, 0.32%, 3.65%, 0.39%, 0.56%, and 3.78% higher than those of the methods 3DCNN, DBDA, Hybrid-SN, A2S2KResNet, FECNet, SSTN, BS2T, CEGCN, and FDGCN, respectively. Therefore, experimental comparisons fully verify that the proposed method has more advanced classification accuracy. Furthermore, by comparing Transformer-based and CNN-based methods, this invention demonstrates that an ensemble network of GCN and CNN focusing on global feature smoothing and multi-scale features exhibits better classification performance.

[0147] 2) Visual Evaluation: This invention also presents the classification results of all methods on four datasets, as shown in Figures 8-11. As can be seen from the figures, some CNN- and Transformer-based methods exhibit significant salt-and-pepper noise. This is because CNN- and Transformer-based methods do not adequately consider the spatial relationships between samples, which greatly limits classification performance. Although GCN-based methods CEGCN and FDGCN consider spatial relationships, the resulting classification maps still contain misclassifications and noise. To improve classification performance, this invention combines GCN and CNN, acquiring rich multi-scale features and contextual information. The resulting classification maps not only suppress misclassifications and noise but also effectively preserve detailed information.

[0148] Table III shows the classification accuracy (%), running time (s), and parameters of all methods on the IndianPines dataset.

[0149]

[0150]

[0151] Table IV shows the classification accuracy (%), running time (s), and parameters for all methods on the Pavia dataset.

[0152]

[0153]

[0154] Table V shows the classification accuracy (%), running time (s), and parameters for all methods on the Salinas dataset.

[0155]

[0156] Table VI shows the classification accuracy (%), running time (s), and parameters of all methods on the WHU-Hi-LongKou dataset.

[0157]

[0158]

[0159] To clearly visualize the feature distribution, this invention also uses t-SNE on four datasets to verify whether the features extracted by HCGN are beneficial to the feature clustering of network training. The visualization results are shown in Figures 12-15. Specifically, this invention selects SSTN (based on the Transformer framework), FECNet (based on the CNN framework), and HCGN (based on the GCN framework) as comparison methods. As shown in Figure 12, the feature visualizations obtained by SSTN and FECNet show significant confusion between different categories. The feature visualizations obtained by CEGCN and HCGN (based on the GCN framework) are superior to those of SSTN and FECNet. Compared to CEGCN, HCGN maintains a larger inter-class distance. Similar to the results on the IndianPines dataset, in the feature visualizations obtained on the Pavia, Salinas, and WHU-Hi-LongKou datasets, as shown in Figures 13-15, CEGCN and HCGN significantly outperform SSTN and FECNet. However, compared to CEGCN, HCGN, due to its multi-scale feature and contextual information approach, visualizes a distribution with larger inter-class and smaller intra-class distances. Therefore, compared with other methods, HCGN combines CNN and GCN, focusing more on global feature smoothing, improving intra-class similarity, and making inter-class features more discriminative. In addition to the provided classification result image and feature distribution visualization, to analyze the impact of different superpixel scale factors on feature extraction, this invention also conducted graph convolution output feature visualization experiments on the Pavia dataset with scale factors λ of 50, 100, and 150, as shown in Figure 16. It can be observed that when the scale factor is small, the extracted features focus more on the fine features of the image, as shown in Figure 16(a). However, as the scale factor gradually increases, the extracted features focus more on global features, as shown in Figure 16(c). This fully demonstrates that using different scale factors can enrich the hierarchical features of the image and promote the improvement of classification performance.

[0160] E. Ablation Experiments: The proposed HCGN model mainly includes three modules: GPM, MS-CFM, and MS-GEEM. To verify the effectiveness of multi-scale graph convolution, this invention also observed the contribution of the single-branch structure GEEM of MS-GEEM to the classification accuracy OA. The experimental results are shown in Table VII. As can be seen from the table, MS-CFM can significantly improve the model's classification accuracy. Next, GEEM fully considers the spatial relationships of the graph and models it using the GCN framework, effectively improving the OA value. Considering the complex land cover structure contained in HSI, the contextual information extracted at different scales can enrich the regional features of the image. Based on GEEM, the MS-GEEM module was proposed, and with limited training samples, the OA values ​​of the Indian Pines, Pavia, Salinas, and WHU-Hi-LongKou datasets were improved to 96.32%, 98.62%, 99.18%, and 98.40%, respectively. Compared with GEEM, MS-GEEM improved the OA values ​​by 0.29%, 0.27%, 0.50%, and 0.11% on the four datasets. Therefore, ablation experiments demonstrate that the proposed MS-CFM and MS-GEEM can effectively improve accuracy and have strong generalization ability.

[0161] Table VII shows the contribution (%) of different modules in the proposed network to the classification accuracy (OA).

[0162]

[0163] Furthermore, to further verify the effectiveness of the proposed module, this invention compares the proposed MS-CFM with a typical CNN module, as shown in Figures 17(a) and (b). As can be seen from Figures 17(a) and (b), the typical CNN module directly connects only four convolutional blocks, while MS-CFM cross-connects four types of convolutional blocks. The experimental results are shown in Figure 18, with the OA, AA, and Kappa values ​​of the two modules represented by different colors. Figure 18 shows that MS-CFM has higher OA, AA, and Kappa values, which verifies that using a cross-fusion structure is more effective in extracting the spatial spectral features of HSI.

[0164] F. Impact of Training Sample Quantity: The impact of different training sample ratios on the classification accuracy (OA) of HCGN and nine contrastive networks was investigated. Due to significant differences in the number of labeled samples across different datasets, the labeled sample ratios selected for the Indian Pines, Pavia, Salinas, and WHU-Hi-LongKou datasets also varied. Specifically, the training sample ratios for the Indian Pines dataset were {1%, 3%, 5%, 10%}, for the Pavia dataset {0.5%, 1%, 3%, 5%}, for the Salinas dataset {0.1%, 0.5%, 1%, 3%}, and for the WHU-Hi-LongKou dataset {0.1%, 0.5%, 1%, 2%}. Figure 19 records the experimental results for the four datasets. In Figure 19, the horizontal axis represents the training sample ratio, and the vertical axis represents the overall accuracy (OA). Different colored curves represent different methods. The experimental results show that the method proposed in this invention can still maintain good competitiveness even with a relatively limited number of training samples. This also proves that the method of this invention can still have strong feature learning ability when the number of samples is very insufficient. This is because we fully consider the graph spatial relationship of HSI, effectively extract multi-scale graph spatial spectral features, enrich feature diversity, and improve the decision-making ability of high-level features.

[0165] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

[0166] References

[0167] [1] S.Li, W.Song, L.Fang, Y.Chen, P.Ghamisi, andJ.A.Benediktsson, "Deeplearning for hyperspectral image classification: An overview," IEEETrans.Geosci.Remote Sens., vol.57, no.9, pp.6690–6709, Sep.2019.

[0168] [2]A.Bannari,A.Pacheco,K.Staenz,H.McNairn,andK.Omari,“Estimating andmapping crop residues cover on agricultural lands using hyperspectral andIKONOS data,”Remote Sens.Environ.,vol.104,no.4,pp.447–459,2006.

[0169] [3]Zhang,C.,Du,L.,&Wang,X.(2020).Hyperspectral remote sensingtechnology and its applications in agriculture:Areview.Journal of IntegrativeAgriculture,19(3),631-647.

[0170] [4]Du,Q.,Wang,Y.,&Li,J.(2018).Urban land-use mapping using multi-temporal high-resolution hyperspectral imagery.ISPRS International Journal ofGeo-Information,7(4),152.

[0171] [5]B.Fang,Y.Li,H.Zhang,andJ.C.-W.Chan,“Collaborative learning oflightweight convolutional neural network and deep clustering forhyperspectral image semi-supervised classification with limited trainingsamples,”ISPRS Journal of Photogrammetry and Remote Sensing,vol.161,pp.164–178,2020.

[0172] [6]C.Szegedy et al.,“Going deeper with convolutions,”in Proc.IEEEConf.Comput.Vis.Pattern Recognit.(CVPR),Jun.2015,pp.1–9.

[0173] [7]K.He,X.Zhang,S.Ren,andJ.Sun,“Deep residual learning for imagerecognition,”in Proc.IEEE Conf.Comput.Vis.Pattern Recognit.(CVPR),Jun.2016,pp.770–778.

[0174] [8]R.Girshick,“FastR-CNN,”in Proc.IEEE Int.Conf.Comput.Vis.(ICCV),Dec.2015,pp.1440–1448.

[0175] [9]J.Redmon,S.Divvala,R.Girshick,and A.Farhadi,“You only look once:Unified,real-time object detection,”in Proc.IEEE Conf.Comput.Vis.PatternRecognit.(CVPR),Jun.2016,pp.779–788.

[0176]

[10] O.Ronneberger,P.Fischer,and T.Brox,“U-Net:Convolutional networksfor biomedical image segmentation,”in Proc.Int.Conf.Med.Image Comput.Comput.-Assist.Intervent.,2015,pp.234–241.

[0177]

[11] Z.-Q.Zhao,P.Zheng,S.-T.Xu,and X.Wu,“Object detection with deeplearning:A review,”IEEE Trans.Pattern Anal.Mach.Intell.,vol.30,no.11,pp.3212–3232,Nov.2019.

[0178]

[12] D.Banesh et al.,“An image-based framework for ocean featuredetection and analysis,”J.Geovisualization Spatial Anal.,vol.5,no.2,pp.1–21,Dec.2021.

[0179]

[13] B.Shi,X.Bai,and C.Yao,“An end-to-end trainable neural network forimage-based sequence recognition and its application to scene textrecognition,”IEEE Trans.Pattern Anal.Mach.Intell.,vol.39,no.11,pp.2298–2304,Nov.2016.

[0180]

[14] F.Yin,Y.-C.Wu,X.-Y.Zhang,and C.-L.Liu,“Scene text recognitionwith sliding convolutional character models,”2017,arXiv:1709.01727.

[0181]

[15] D.W.Otter,J.R.Medina,and J.K.Kalita,“A survey of the usages ofdeep learning for natural language processing,”IEEE Trans.NeuralNetw.Learn.Syst.,vol.32,no.2,pp.604–624,Feb.2021.

[0182]

[16]

[26] A.Galassi,M.Lippi,and P.Torroni,“Attention in naturallanguage processing,”IEEE Trans.Neural Netw.Learn.Syst.,vol.32,no.10,pp.4291–4308,Oct.2021.

[0183]

[17] S.Sakhavi,C.Guan,and S.Yan,“Learning temporal information forbrain-computer interface using convolutional neural networks,”IEEETrans.Neural Netw.Learn.Syst.,vol.29,no.11,pp.5619–5629,Nov.2018.

[0184]

[18] Z.Tan,J.Chen,Q.Kang,M.Zhou,A.Abusorrah,and K.Sedraoui,“Dynamicembedding projection-gated convolutional neural networks for textclassification,”IEEE Trans.Neural Netw.Learn.Syst.,vol.33,no.3,pp.973–982,Mar.2021.

[0185]

[19] Y.Chen,Z.Lin,Z.Xing et al.,“Deep Learning-Based Classification ofHyperspectral Data,”IEEE J.Sel.Topics Appl.Earth Observ.Remote Sens.,vol.7,no.6,pp.2094-2107,2017.

[0186]

[20] M.D.Farrell and R.M.Mersereau,“On the impact of PCA dimensionreduction for hyperspectral detection of difficult targets,”IEEEGeosci.Remote Sens.Lett.,vol.2,no.2,pp.192–195,2005.

[0187]

[21] R.Hang,Q.Liu,D.Hong,and P.Ghamisi,“Cascaded recurrent neuralnetworks for hyperspectral image classification,”IEEE Trans.Geosci.RemoteSens.,vol.57,no.8,pp.5384–5394,Aug.2019.

[0188]

[22] I.J.Goodfellow,J.Pouget-Abadie,M.Mirza et al.,“GenerativeAdversarial Networks,”in Proc.NIPS,2014,pp.2672–2680.

[0189]

[23] A.Odena,C.Olah,and J.Shlens,“Conditional Image Synthesis WithAuxiliary Classifier GANs,”2016.

[0190]

[24] A.Radford,L.Metz,and S.Chintala,“Unsupervised RepresentationLearning with Deep Convolutional Generative Adversarial Networks,”inProc.ICLR,Jan.2016,pp.1–16.

[0191]

[25] S.T.Li,W.W.Song,L.Y.Fang et al.,“Deep Learning for HyperspectralImage Classification:An Overview,”IEEE Trans.Geosci.Remote Sens.,vol.57,no.9,pp.6690-6709,Sep,2019.

[0192]

[26] W.Hu,Y.Y.Huang,L.Wei et al.,“Deep Convolutional Neural Networksfor Hyperspectral Image Classification,”J.Sensors,2015.

[0193]

[27] Y.Chen,H.Jiang,C.Li et al.,“Deep Feature Extraction andClassification of Hyperspectral Images Based on Convolutional NeuralNetworks,”IEEE Trans.Geosci.Remote Sens.,vol.54,no.10,pp.6232-6251,2016.

[0194]

[28] A.B.Hamida,A.Benoit,P.Lambert,and C.B.Amar,“3-d deep learningapproach for remote sensing image classification,”IEEE Transactions ongeoscience and remote sensing,vol.56,no.8,pp.4420–4434,2018.

[0195]

[29] C.Yu,R.Han,M.Song et al.,“Feedback Attention-Based Dense CNN forHyperspectral Image Classification,”IEEE Trans.Geosci.Remote Sens.,vol.PP,no.99,pp.1-16,2021.

[0196]

[30] Li,G.,Li,W.,Zhang,L.,&Shi,T.(2019).Deep hyperspectral imageclassification with gradient boosting networks.IEEE Transactions onGeoscience and Remote Sensing,57(10),7929-7942.

[0197]

[31] Kang,W.,Zhang,H.,&Wu,Y.(2020).A new method of hyperspectral imageclassification based on deep learning with dropout regularization.RemoteSensing,12(4),706.

[0198]

[32] Zhang,L.,Li,W.,&Shi,T.(2020).Deep neural network pruning forhyperspectral image classification.IEEE Transactions on Geoscience and RemoteSensing,58(8),5667-5680.

[0199]

[33] Li,D.,Wu,Y.,Chen,J.,&Hu,J.(2020).Hyperspectral imageclassification with attention-based deep feature fusion network.IEEETransactions on Geoscience and Remote Sensing,58(4),2548-2560.

[0200]

[34] Z.Zhong,J.Li,Z.Luo,and M.Chapman,“Spectral–spatial residualnetwork for hyperspectral image classification:A 3-D deep learningframework,”IEEE Trans.Geosci.Remote Sens.,vol.56,no.2,pp.847–858,Feb.2018.

[0201]

[35] Chen,X.,&Zhang,L.(2020).Active learning for hyperspectral imageclassification with minimum class variance.IEEE Transactions on Geoscienceand Remote Sensing,59(5),3865-3877.

[0202]

[36] W.Ma,Q.Yang,Y.Wu,W.Zhao,and X.Zhang,“Double-branch multiattentionmechanism network for hyperspectral image classification,”Remote Sens.,vol.11,no.11,p.1307,Jun.2019.

[0203]

[37] R.Li,S.Zheng,C.Duan,Y.Yang,and X.Wang,“Classification ofhyperspectral image based on double-branch dual-attention mechanism network,”Remote Sens.,vol.12,no.3,p.582,Feb.2020.

[0204]

[38] Roy S K,Manna S,Song T,et al.Attention-Based Adaptive Spectral-Spatial Kernel ResNet for Hyperspectral Image Classification[J].IEEETransactions on Geoscience and Remote Sensing,2020:1-13.

[0205]

[39] C.Shi,D.Liao,T.Zhang and L.Wang,"Hyperspectral ImageClassification Based on Expansion Convolution Network,"in IEEE Transactionson Geoscience and Remote Sensing,vol.60,pp.1-16,2022.

[0206]

[40] A.Dosovitskiy et al.,“An image is worth 16×16 words:Transformersfor image recognition at scale,”2020,arXiv:2010.11929.

[0207]

[41] X.He,Y.Chen,and Z.Lin,"Spatial-spectral transformer forhyperspectral image classification,"Remote Sensing,vol.13,no.3,p.498,2021.

[0208]

[42] Y.Qing,W.Liu,L.Feng,and W.Gao,"Improved transformer net forhyperspectral image classification,"Remote Sensing,vol.13,no.11,p.2216,2021.

[0209]

[43] D.Hong et al.,"SpectralFormer:Rethinking Hyperspectral ImageClassification With Transformers,"in IEEE Transactions on Geoscience andRemote Sensing,vol.60,pp.1-15,2022.

[0210]

[44] Z.Zhong,Y.Li,L.Ma,J.Li and W.-S.Zheng,"Spectral–SpatialTransformer Network for Hyperspectral Image Classification:A FactorizedArchitecture Search Framework,"in IEEE Transactions on Geoscience and RemoteSensing,vol.60,pp.1-15,2022.

[0211]

[45] L.Sun,G.Zhao,Y.Zheng and Z.Wu,"Spectral–Spatial FeatureTokenization Transformer for Hyperspectral Image Classification,"in IEEETransactions on Geoscience and Remote Sensing,vol.60,pp.1-14,2022.

[0212]

[46] R.Song,Y.Feng,W.Cheng,Z.Mu and X.Wang,"BS2T:Bottleneck Spatial–Spectral Transformer for Hyperspectral Image Classification,"in IEEETransactions on Geoscience and Remote Sensing,vol.60,pp.1-17,2022.

[0213]

[47] W.Guo,G.Xu,W.Liu,B.Liu,and Y.Wang,“CNN-combined graph residualnetwork with multilevel feature fusion for hyperspectral imageclassification,”IET Comput.Vis.,vol.15,no.8,pp.592–607,2021.

[0214]

[48] T.N.Kipf and M.Welling,“Semi-supervised classification with graphconvolutional networks,”in Proc.Int.Conf.Learn.Represent.(ICLR),2017,pp.1–14.

[0215]

[49] A.Qin,Z.Shang,J.Tian,Y.Wang,T.Zhang,and Y.Y.Tang,“Spectral–spatial graph convolutional networks for semisupervised hyperspectral imageclassification,”IEEE Geosci.Remote Sens.Lett.,vol.16,no.2,pp.241–245,Feb.2019.

[0216]

[50] L.Mou,X.Lu,X.Li,and X.X.Zhu,“Nonlocal graph convolutionalnetworks for hyperspectral image classification,”IEEE Trans.Geosci.RemoteSens.,vol.58,no.12,pp.8246–8257,Dec.2020.

[0217]

[51] D.Hong,L.Gao,J.Yao,B.Zhang,A.Plaza,and J.Chanussot,“Graphconvolutional networks for hyperspectral image classification,”IEEETrans.Geosci.Remote Sens.,vol.59,no.7,pp.5966–5978,Jul.2021.

[0218]

[52] P.Yang,L.Tong,B.Qian,Z.Gao,J.Yu,and C.Xiao,“Hyperspectral imageclassification with spectral and spatial graph using inductive representationlearning network,”IEEE J.Sel.Topics Appl.Earth Observ.Remote Sens.,vol.14,pp.791–800,2021

[0219]

[53] S.Wan,C.Gong,P.Zhong,B.Du,L.Zhang,and J.Yang,“Multiscale dynamicgraph convolutional network for hyperspectral image classification,”IEEETrans.Geosci.Remote Sens.,vol.58,no.5,pp.3162–3177,May2020.

[0220]

[54] Q.Liu,L.Xiao,J.Yang,and Z.Wei,“CNN-enhanced graph convolutionalnetwork with pixel-and superpixel-level feature fusion for hyperspectralimage classification,”IEEE Trans.Geosci.Remote Sens.,vol.59,no.10,pp.8657–8671,Oct.2020.

[0221]

[55] Q.Liu,Y.Dong,Y.Zhang and H.Luo,"A Fast Dynamic GraphConvolutional Network and CNN Parallel Network for Hyperspectral ImageClassification,"in IEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-15,2022.

[0222]

[56] R.Achanta,A.Shaji,K.Smith,A.Lucchi,P.Fua,and S.Süsstrunk,“SLICsuperpixels compared to state-of-the-art superpixel methods,”IEEETrans.Pattern Anal.Mach.Intell.,vol.34,no.11,pp.2274–2282,Nov.2012.

[0223]

[57] N.Srivastava,G.Hinton,A.Krizhevsky,I.Sutskever,andR.Salakhutdinov,“Dropout:Asimple way to prevent neural networks fromoverfitting,”J.Mach.Learn.Res.,vol.15,no.1,pp.1929–1958,2014.

[0224]

[58] S.Zhang and S.Li,“Spectral-spatial classification ofhyperspectral images via multiscale superpixels based sparse representation,”in Proc.IEEE IGARSS,Jul.2016,pp.2423–2426.

[0225]

[59] Y.Wang,Y.Sun,Z.Liu,S.E.Sarma,M.M.Bronstein,and J.M.Solomon,“Dynamic graph CNN for learning on point clouds,”ACM Trans.Graph.,vol.38,no.5,pp.1–12,Nov.2019.

[0226]

[60] J.Bai,B.Ding,Z.Xiao,L.Jiao,H.Chen and A.C.Regan,"HyperspectralImage Classification Based on Deep Attention Graph Convolutional Network,"inIEEE Transactions on Geoscience and Remote Sensing,vol.60,pp.1-16,2022,Artno.5504316.

Claims

1. A hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples, characterized in that: The specific process of the method is as follows:

1. Establish a hybrid network model HCGN (Convolutional Neural Network and Graph Convolutional Network); obtain a trained hybrid network model HCGN; 2. Input the hyperspectral image to be tested into the trained hybrid network model HCGN to complete the classification of the hyperspectral image to be tested; The specific process of establishing the hybrid network model HCGN in step 1 and obtaining the trained hybrid network model HCGN is as follows: Step 1: Randomly sample hyperspectral images; the hyperspectral image training set is represented as... Tag is Where H, W, and B represent the height, width, and number of spectral bands of the hyperspectral image HSI, respectively. Represents a real number; the training set is randomly selected for each category according to a preset ratio; Step 2: Construct a hybrid network model HCGN of convolutional neural network and graph convolutional network; obtain the trained hybrid network model HCGN of convolutional neural network and graph convolutional network; Step 2: Construct a hybrid network model HCGN of convolutional neural network and graph convolutional network; obtain the trained hybrid network model HCGN of convolutional neural network and graph convolutional network; The specific process is as follows: The hybrid network model HCGN of convolutional neural network and graph convolutional network includes five parts, namely, graph mapping module, multi-scale graph edge enhancement module, shallow feature extraction CNN module, multi-scale cross-fusion module and classification layer; Step 2: Input the hyperspectral image training set X into the graph The mapping module completes the mapping of the image from pixels to superpixels, converting each superpixel into a node to obtain an undirected graph. Step 22: The node features corresponding to the nodes in the undirected graph and the connection matrix between the nodes are input into the multi-scale graph edge enhancement module to obtain the graph features. Step 23: The hyperspectral image training set X is input into the shallow feature extraction CNN module to output the features. Step 24: The output features of the shallow feature extraction CNN module are input into the multi-scale cross-fusion module MS-CFM to extract the features of the hyperspectral image. Step 25: The graph features obtained in Step 22 and the hyperspectral image features extracted in Step 24 are concatenated and then fed into the FC classification layer for classification. The trained network model HCGN, a hybrid of convolutional neural network and graph convolutional network, is obtained.

2. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 1, characterized in that: In step two, the hyperspectral image training set X input graph mapping module completes the mapping of the image from pixels to superpixels, converting each superpixel into a node to obtain an undirected graph. The specific process is as follows: Step 211: Supervised dimensionality reduction of the hyperspectral image training set X is performed using the Linear Discriminant Analysis (LDA) algorithm to obtain the dimensionality-reduced data. Where L is the number of spectral bands in the dimensionality-reduced data; Step 2: The total number of superpixels k obtained by using different segmentation scale factors λ for each superpixel segmentation is different, expressed as follows: k = (H × W) / λ, 1 ≤ λ, where H × W represents the total number of image pixels; The dimensionality-reduced data X LDA The first superpixel segmentation is performed using linear iterative clustering (SLIC). The superpixel set obtained from the first superpixel segmentation is: Dimensionally reduced data X LDA The input is linear iterative clustering (SLIC) for a second superpixel segmentation. The superpixel set obtained from the second superpixel segmentation is: Dimensionally reduced data X LDA The input is linear iterative clustering (SLIC) for a third superpixel segmentation. The superpixel set obtained from the third superpixel segmentation is: Among them, S i Represents the data X after dimensionality reduction LDA The i-th superpixel after superpixel segmentation, and S represents i The first in 1 pixel, Z i For S i The total number of pixels in the superpixel array; the superpixels should satisfy the following conditions. Among them, S j Represents the data X after dimensionality reduction LDA The j-th superpixel after superpixel segmentation, and Represent the empty set; Steps 2, 1, and 3: Using the centroid of each superpixel in the superpixel set obtained from the first superpixel segmentation as a node, after converting the centroids of all superpixels in the superpixel set into nodes, a node set is obtained, which constitutes an undirected graph. Using the centroid of each superpixel in the superpixel set obtained from the second superpixel segmentation as a node, and converting the centroids of all superpixels in the superpixel set into nodes, we obtain a node set, which constitutes an undirected graph. Using the centroid of each superpixel in the superpixel set obtained from the third superpixel segmentation as a node, and converting the centroids of all superpixels in the superpixel set into nodes, we obtain a node set, which constitutes an undirected graph. in, Let ε and ε represent the nodes and edges of an undirected graph, respectively.

3. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 2, characterized in that: In step two, the node features corresponding to the nodes in the undirected graph and the connection relationship matrix between the nodes are input into the multi-scale graph edge enhancement module to obtain the graph features; The specific process is as follows: Step 221: Obtain the adjacency matrix A based on the connection relationship between the segmented superpixels. e Based on adjacency matrix A e The connection matrix A is obtained by combining the identity matrix I; The superpixel feature V is a matrix of nodes in an undirected graph. In an undirected graph, each node is the centroid of a superpixel. Let be the k-th node in an undirected graph; T is the transpose of the graph; based on the undirected graph structure. Construction diagram structure G1(V) l A) G2(V l A), G3(V l A); Step 2, Graph Structure G1(V) l When A in (A) is the connectivity matrix A, the graph structure G1(V) is... l A) Inputting into a fully connected FC layer yields feature V1. l ;Graph structure G1(V l In A), A is taken as the adjacency matrix A e At that time, the graph structure G1(V) l A) Input the fully connected (FC) layer to obtain features. Feature V1 l and characteristics Feature fusion is performed to obtain feature V. l+1 V l and V l+1 These represent the input and output of the l-th layer, respectively; the output V of the l-th layer... l+1 As the input to the (l+1)th layer; the feature V l+1 Replace the graph structure G1(V) l V in A) l The graph structure G1(V) is obtained. l+1 ,A); Graph structure G1(V l+1 When A in (A) is the connectivity matrix A, the graph structure G1(V) is... l+1 A) Input the fully connected (FC) layer to obtain features. Graph structure G1(V) l+1 In A), A is taken as the adjacency matrix A e At that time, the graph structure G1(V) l+1 A) Input the fully connected (FC) layer to obtain features. Features and characteristics Feature fusion is performed to obtain feature V. l+2 Steps 2-3, Graph Structure G2(V) l When A in A) takes the connection matrix A, the graph structure G2(V) l A) Input the fully connected (FC) layer to obtain features. Graph structure G2(V) l In A), A is taken as the adjacency matrix A e At that time, the graph structure G2(V) l A) Input the fully connected (FC) layer to obtain features. Features and characteristics Feature fusion is performed to obtain feature V. l+3 ; feature V l+3 Replace the graph structure G2(V) l V in A) l The graph structure G2(V) is obtained. l+3 ,A); Graph structure G2(V l+3 When A in A) takes the connection matrix A, the graph structure G2(V) l+3 A) Input the fully connected (FC) layer to obtain features. Graph structure G2(V) l+3 In A), A is taken as the adjacency matrix A e At that time, the graph structure G2(V) l+3 A) Input the fully connected (FC) layer to obtain features. Features and characteristics Feature fusion is performed to obtain feature V. l+4 Step 224, Graph Structure G3 (V) l When A in (A) is the connectivity matrix A, the graph structure G3(V) is... l A) Input the fully connected (FC) layer to obtain features. Graph structure G3(V) l In A), A is taken as the adjacency matrix A e At that time, the graph structure G3(V) l A) Input the fully connected (FC) layer to obtain features. Features and characteristics Feature fusion is performed to obtain feature V. l+5 ; feature V l+5 Replace the graph structure G3(V) l V in A) l The graph structure G3(V) is obtained. l+5 ,A); Graph structure G3(V l+5 When A in (A) is the connectivity matrix A, the graph structure G3(V) is... l+5 A) Input the fully connected (FC) layer to obtain features. Graph structure G3(V) l+5 In A), A is taken as the adjacency matrix A e At that time, the graph structure G3(V) l+5 A) Input the fully connected (FC) layer to obtain features. Features and characteristics Feature fusion is performed to obtain feature V. l+6 Step 225: Apply the feature V obtained in Step 222. l+2 The features V obtained in steps two and three l+4 The feature V obtained in step two is... l+6 Feature concatenation is performed and used as the output of the multi-scale map edge enhancement module.

4. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 3, characterized in that: In steps two and three, the hyperspectral image training set X is input into the shallow feature extraction CNN module to extract the features of the hyperspectral image. The specific process is as follows: The shallow feature extraction CNN module includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 1×1 convolutional layer in sequence.

5. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 4, characterized in that: The entire process of the shallow feature extraction CNN module can be represented as y = f cnn (f cnn In the above formula (x), x and y represent the input and output of the image, respectively, f cnn (·) represents a two-dimensional convolution function, which includes a normalized BN layer, an activation layer ReLU, and a convolutional layer in sequence.

6. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 5, characterized in that: In step two and four, the output features of the shallow feature extraction CNN module are input into the multi-scale cross-fusion module MS-CFM to extract features from the hyperspectral image. Specifically, the multi-scale cross-fusion module MS-CFM sequentially includes a first convolutional block, a second convolutional block, a third convolutional block, and a fourth convolutional block. The first convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 1×1 convolutional layer. The second convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 5×5 convolutional layer. The third convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 3×3 convolutional layer. The fourth convolutional block sequentially includes a normalized BN layer, a ReLU layer, a 1×1 convolutional layer, a normalized BN layer, a ReLU layer, and a 7×7 convolutional layer. The shallow feature extraction CNN module outputs features that are input into the first convolutional block, and the first convolutional block outputs feature 1; The shallow feature extraction CNN module outputs features and inputs them into the second convolutional block, which outputs feature 2. Feature 1 and feature 2 are fused and input into the third convolutional block, which outputs feature 3. Feature 1 and feature 2 are fused and input into the fourth convolutional block, which outputs feature 4. The feature obtained by fusing feature 3 and feature 4 is used as the output of the multi-scale cross-fusion module MS-CFM.

7. The hyperspectral image classification method based on a hybrid CNN-GCN network with limited training samples according to claim 6, characterized in that: The entire calculation process of the multi-scale cross-fusion module MS-CFM is represented as follows: Where x and y represent the input and output of the image, respectively, f cnn1 f is the convolution function for the first convolutional block. cnn2 f is the convolution function for the second convolutional block. cnn3 f is the convolution function for the third convolutional block. cnn4 The convolution function for the 4th convolutional block. This indicates the fusion of features.

Images