Dual-branch hyperspectral image classification method based on graph convolutional neural network and attention mechanism

Abstract

The application relates to a dual-branch hyperspectral image classification method based on a graph convolutional neural network and an attention mechanism, which comprises the following steps: inputting a hyperspectral original image to data for pretreatment; inputting the data after dimension reduction into a CNN attention mechanism branch, and obtaining processed branch features through an attention mechanism module, a data processing module, an attention mechanism module and convolution operation; inputting the data after dimension reduction into a GCN branch, performing simple linear iterative clustering, segmenting into k superpixels, encoding the k superpixels, obtaining superpixel graph nodes, inputting the graph nodes into a GCN module for feature extraction, decoding the extracted superpixel node features into pixel-level branch features; performing feature fusion on the two branches, using a cross-entropy function as a loss function to train the model, and using a softmax function to obtain a class label probability. The application takes limited hyperspectral images as research objects, improves high classification precision, and ensures classification speed.

Description

Technical Field

[0001] This invention relates to the fields of image processing and hyperspectral remote sensing image classification, specifically to a two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms. Background Technology

[0002] Hyperspectral images contain multiple spectral bands, each corresponding to a specific spectral reflectance. This results in hyperspectral images possessing rich spatial and spectral features. In remote sensing images, the spectral characteristics of different ground targets can be obtained by analyzing the spectral reflectance of each band, thereby enabling the classification of ground objects. Hyperspectral remote sensing is widely used in various fields, such as environmental monitoring, military target detection, and medical diagnosis.

[0003] There are many techniques that can be applied to hyperspectral image classification. Traditional machine learning algorithms, such as Support Vector Machines (SVM) and Random Forest, can train classification models through feature extraction, associating spectral features in the image with actual classification information to classify ground features in hyperspectral images. However, these methods require manual feature extraction, and some spectral feature information is lost during the feature association process.

[0004] With the development and improvement of deep learning technology, convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have been increasingly applied to hyperspectral image classification tasks. CNNs are a commonly used deep learning data processing method in image processing. In hyperspectral image classification, they can simultaneously extract spectral and spatial information and automatically learn and optimize the parameters within the CNN. Building residual structures through CNNs allows for deeper convolutional layers, enabling information fusion between layers and improving classification accuracy. However, CNNs also have limitations. For example, they may ignore some inter-pixel information during feature learning, and deeper layers can lead to more parameters, increasing the risk of overfitting and computational costs. Graph convolutional neural networks (GCNs) apply the principles of graph neural networks to CNNs. By transforming pixel information and inter-pixel relationships into point and line operations, they simplify computation, avoid multiple matrix operations, and better learn local structures in graphs, thus preserving irregular class boundary information in spectral images. While GCNs perform well in classification tasks, deeper network models significantly reduce classification effectiveness and have poor scalability. Summary of the Invention

[0005] This invention aims to address the technical problems of low accuracy, insufficient feature extraction, and complex models in current hyperspectral image classification models. It provides a two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms, which ensures high classification accuracy and fewer computational parameters with a small number of training samples.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0007] A two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms includes the following steps:

[0008] S1. Input the original hyperspectral image and preprocess the data;

[0009] Set the original input data of the hyperspectral image to X∈R H×W×C Where H is the height of the hyperspectral image, W is the width of the hyperspectral image, and C is the dimension of the hyperspectral image, principal component analysis is applied to reduce the dimension of the original image, filtering out useless noise and high-dimensional features. The dimension-reduced data is represented as follows:

[0010] S2. Input the dimensionality-reduced data from step S1 into the CNN attention mechanism branch. The data sequentially passes through the attention mechanism module CE_A, the data processing module CBL, the attention mechanism module CE_B, and the convolution operation, finally obtaining the processed branch feature F. CNN ;

[0011] S3. Input the dimensionality-reduced data from step S1 into the GCN branch; first, perform simple linear iterative clustering on the data to divide the dimensionality-reduced data into k superpixels; second, encode the k superpixels to obtain superpixel graph nodes; third, input the graph nodes into the GCN module for feature extraction; finally, decode the extracted superpixel node features into pixel-level branch features F. GCN ;

[0012] S4. Perform feature fusion on the CNN feature branches and GCN feature branches, use the cross-entropy function as the loss function to train the model, and use the softmax function to obtain the probability of each category label to be classified.

[0013] In the above technical solution, the attention mechanism module CE_A and attention mechanism module CE_B in step S2 are both composed of CA attention mechanism module and ECA attention mechanism module; attention mechanism module CE_A adds spatial features to channel information, and the dimensionality-reduced data is input to CA attention mechanism module and ECA attention mechanism module respectively, and finally outputs fused features.

[0014] In the above technical solution, the CA attention mechanism in step S2 performs feature processing in two directions, X and Y. In the X direction, the data undergoes average pooling in the form of H×1×C1, where H is the input feature height, 1 is the input feature width, and C1 is the input feature dimension. Simultaneously, in the Y direction, the data undergoes average pooling in the form of 1×W×C1, where 1 is the input feature height, W is the input feature width, and C1 is the input feature dimension. Let the feature layers after the two average pooling operations be... and Performing a tensor concatenation operation on two feature layers is represented as follows: The concatenated features are sequentially passed through a 1×1 convolutional layer, a batch normalization layer, and a LeakyReLU activation function layer, where the batch normalization layer is abbreviated as BN layer. The processed features are then split again into feature tensors in the X and Y directions. Both tensors pass through a 1×1 convolutional layer and a sigmoid function layer, represented as follows: and Finally and With input features X c1 Perform feature multiplication to obtain the input data. Feature Mapping The value at position (i,j) Where i∈(0,h) represents the feature height as i, and j∈(0,w) represents the feature width as j. This indicates feature multiplication.

[0015] In the above technical solution, the input feature dimension of the ECA attention mechanism in step S2 is H×W×C1. First, a global average pooling operation is used to transform the input feature dimension into 1×1×C1. i Where i is the number of layers with feature depth; secondly, a 1×1 convolution kernel is applied to share the depth dimension information of each layer with the depth dimension information of adjacent layers; thirdly, after passing through a Sigmoid function layer, the processed feature tensor has a dimension of 1×1×C. s Where s represents the number of feature depth layers; finally, the feature is multiplied by the input feature to obtain a feature with dimension H×W×C1. Where i represents the feature height, i∈(0,h); j represents the feature width, j∈(0,w); This indicates feature multiplication.

[0016] In the above technical solution, the specific operation of the CE_A structure in step S2 is to perform a feature multiplication operation on the features processed by the CA attention mechanism module and the ECA attention mechanism module, retaining one input feature branch to obtain the fused feature F1; then F1 passes through the CBL module, which includes a Conv, BN layer and LeakyReLU activation function with a convolution kernel size of 3×3; the attention mechanism module CE_B represents the feature being extracted sequentially by the CA attention mechanism module and the ECA attention mechanism module, and after processing by CE_B, feature F3 is obtained, which is finally passed through a convolution kernel with a size of 3×3. × A convolutional layer of 3 is used to obtain the feature F extracted by the CNN branch. CNN =Conv(F3).

[0017] In the above technical solution, step S3's SLIC will cluster pixel-level features of dimension H×W×C1 into features of dimension H×W×C1. k The superpixel features, where k is the number of superpixels of approximately equal size after clustering;

[0018] The pixel-level features and superpixel-level features are flattened into one-dimensional feature matrices X and W, respectively, where X T = [x1,x2,x3...x n ] T W T =[w1,w2,w3...w k ] T Let n and k be the number of pixels and the number of superpixels, respectively; establish the correlation matrix Q∈R between X and W. HW×k Where HW is the height and width of the image feature, and k is the number of superpixels, when X i ∈W j Time Q i.j =1, otherwise, Q i.j =0, where i∈[1,n], j∈[1,k]; normalize Q to a one-dimensional matrix Q1 with dimension d×1, then the graph node feature matrix V is expressed as V=Encoder(X,Q1)=X T Q1, with a feature dimension of n×d, the relationship between graph nodes is represented by the edges in the constructed graph, which is expressed as an adjacency matrix E with a dimension of n×n.

[0019] In the above technical solution, step S3 specifically involves performing a weighted summation of the neighboring nodes of each node in the graph and multiplying it by the parameter matrix to obtain the node features of the new layer; let H l H is the feature matrix of the l-th layer. l+1 Let be the feature matrix of the (l+1)th layer, then Where ReLU represents a nonlinear activation function; E represents the adjacency matrix of the relationships between nodes in the graph, and I represents the identity matrix; for The degree matrix, i and j represent matrices The dimension; propagation between layers is achieved by calculating and iterating the features of each node and performing graph convolution operations, and finally the node features are updated;

[0020] After completing two layers of GCN operations, the features are decoded to obtain the features F of the GCN branch. GCN Specifically represented as F GCN =Reshape(QV), where Q represents the association matrix between pixels and superpixel nodes, V represents the graph node feature matrix, and Reshape means reshaping the matrix.

[0021] In the above technical solution, step S4 involves concatenating the two feature branches obtained in steps S2 and S3, specifically as follows:

[0022]

[0023] in, This represents the concatenation of spectral dimensions.

[0024] In the above technical solution, the loss function used in the model training process in step S4 is the cross-entropy function, specifically expressed as:

[0025]

[0026] Where T represents the correct category of the label; P represents the model's prediction for each pixel; N represents the total number of samples in the dataset; C represents the number of categories in the dataset; y i,c p represents the c-th element of the i-th label y. i,c This indicates that the i-th pixel belongs to the c-th class.

[0027] The present invention has the following beneficial effects:

[0028] The present invention provides a two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms. This method uses a limited number of hyperspectral images as the research object, improves classification accuracy, and ensures classification speed.

[0029] The attention mechanism structure in this invention can fully extract spatial and spectral information from images. The two designed CE_A and CE_B feature extraction modules can enhance the network's focus while minimizing computational parameters, and can more effectively apply computational resources to the core regions of the model, thereby enhancing the model's feature extraction capabilities.

[0030] The GCN branch in this invention effectively avoids multiple matrix operations while performing feature extraction, greatly improving model training speed and reducing model parameters. PCA and SLIC significantly reduce model computation through dimensionality reduction and clustering of data. GCN can effectively convert spectral information into graph nodes, extracting spectral and spatial information more fully and further improving model classification accuracy.

[0031] This invention fuses different features extracted from CNN and GCN branches, resulting in a more comprehensive feature representation of the input data, which greatly improves classification accuracy and overall model performance. Attached Figure Description

[0032] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0033] Figure 1 This is a schematic flowchart of the method of the present invention.

[0034] Figure 2 This is a schematic diagram of the overall structure of the present invention.

[0035] Figure 3 This is a schematic diagram of the attention mechanism module CE_A in the feature extraction branch of a CNN.

[0036] Figure 4 This is a schematic diagram of the attention mechanism module CE_B in the feature extraction branch of a CNN.

[0037] Figure 5 This is a schematic diagram of the encoding operation structure in the feature extraction branch of GCN. Detailed Implementation

[0038] The inventive concept of this invention is as follows: This invention combines the advantages of CNN and GCN to design a two-branch hyperspectral image classification method with high accuracy, few model parameters, and full utilization of spectral and spatial features.

[0039] The present invention will now be described in detail with reference to the accompanying drawings.

[0040] Figure 1 The flowchart of this invention illustrates the specific workflow of the dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms:

[0041] S1. First, input the original hyperspectral image and preprocess the data. Set the original input data of the hyperspectral image as X∈R. H×W×C Where H is the height of the hyperspectral image, W is the width of the hyperspectral image, and C is the dimension of the hyperspectral image. Figure 2This is a diagram of the overall structure of the invention, mainly divided into two feature extraction channels: a CNN branch and a GCN branch. The extracted features are then fused to complete the final classification task, as shown below. Figure 2 As shown, Principal Component Analysis (PCA) can be used to reduce the dimensionality of the original image. Dimensionality reduction can filter out useless noise and high-dimensional features, retaining only the necessary feature dimensions. The dimensionality-reduced data can be represented as follows:

[0042]

[0043] S2. The dimensionality-reduced data from S1 is input into the CNN attention mechanism branch. The data sequentially passes through the attention mechanism module CE_A, the data processing module CBL, the attention mechanism module CE_B, and convolution (Conv) operations, finally obtaining the processed branch features F. CNN Both CE_A and CE_B are composed of CA attention mechanism and ECA attention mechanism.

[0044] The structure diagram of the attention mechanism module CE_A is as follows: Figure 3 As shown, this module can effectively incorporate spatial features into channel information to obtain more comprehensive fusion features. It mainly consists of the attention mechanism CA (Coordinate Attention) and the attention mechanism ECA (Efficient Channel Attention). The dimensionality-reduced data is input into the CA attention mechanism module and the ECA attention mechanism module respectively, and finally outputs the fusion features.

[0045] The CA attention mechanism module structure is as follows: Figure 3 As shown by the dashed box in the CA attention mechanism, feature processing is divided into two directions, X and Y. In the X direction, the data undergoes average pooling in the form of H×1×C1, where H is the input feature height, 1 is the input feature width, and C1 is the input feature dimension. Simultaneously, in the Y direction, the data undergoes average pooling in the form of 1×W×C1, where 1 is the input feature height, W is the input feature width, and C1 is the input feature dimension. Let the feature layers after the two average pooling operations be... and The tensor concatenation (Concat) operation on two feature layers can be represented as follows: The concatenated features are sequentially processed through a 1×1 convolutional layer, a batch normalization (BN) layer, and a Leaky ReLU activation function layer. The BN layer can be abbreviated as BN. The processed features are then split into feature tensors in the X and Y directions. Both tensors are processed through a 1×1 convolutional layer and a sigmoid function layer, which can be represented as follows: and Will and Input features By performing feature multiplication, the input data can be obtained. Feature mapping Y CA The value at position (i,j) is:

[0046]

[0047] Where i represents the feature height, i∈(0,h); j represents the feature width, j∈(0,w); Indicates feature multiplication, Y CA The feature dimension is H×W×C1.

[0048] The ECA attention mechanism module structure is as follows: Figure 3 As shown in the dashed box of the ECA attention mechanism module, the input feature dimension is H×W×C1. First, a global average pooling operation is used to reduce the input feature dimension to 1×1×C. i Where i is the number of layers with feature depth; secondly, a 1×1 convolution kernel is applied to share the depth dimension information of each layer with the depth dimension information of adjacent layers; thirdly, after passing through a Sigmoid function layer, the processed feature tensor has a dimension of 1×1×C. s Where s represents the number of feature depth layers, the feature depth is obtained. Finally, multiplying the feature with the input feature yields the input data. Feature mapping Y ECA The value at position (i,j) is:

[0049]

[0050] Where i represents the feature height, i∈(0,h); j represents the feature width, j∈(0,w); Multiplying the features together, we can finally obtain Y. ECA The feature dimension is H×W×C1.

[0051] The attention mechanism module CE_A specifically performs a feature fusion operation on the features processed by the CA and ECA attention mechanism modules. This module can effectively incorporate spatial features into the channel information, obtaining more comprehensive fused features. Because Y CA and Y ECA Since each input feature involves multiplication, only one input feature branch is used for feature multiplication during the calculation, ultimately yielding the fused feature F1. F1 then passes through the CBL module, which is as follows: Figure 2 The CBL Model shown in the lower center is an example, where Conv is the convolution kernel size of 3. ×The convolution operation is 3; the Batch Normalization (BN) layer accelerates the training and convergence of the model and effectively suppresses gradient vanishing or exploding, preventing overfitting; LeakyReLU is a non-linear activation function that adds non-linear parameters to the model, increasing its expressiveness and enabling it to better fit the data. The resulting features can then be represented as:

[0052] F2 = LeakyReLU(BN(Conv(F1)))

[0053] Attention mechanism module CE_B, for example Figure 4 As shown, the CE_B module can further enhance the model's expressive power without increasing computational complexity, thereby improving the model's classification performance. Specifically, the F2 features are sequentially processed through the CA attention mechanism module and the ECA attention mechanism module for feature extraction. No fusion operation is performed with the input features during this process; only one input feature fusion is performed after feature processing is complete. After CE_B processing, feature F3 is obtained, and finally, it is processed by a convolutional kernel of size 3. × A convolutional layer of 3 can yield the feature F extracted by the CNN branch. CNN =Conv(F3).

[0054] S3. Input the dimensionality-reduced data from S1 into the GCN branch. First, perform Simple Linear Iterative Clustering (SLIC) on the data to divide the dimensionality-reduced data into k superpixels. Second, encode the k superpixels to obtain superpixel graph nodes. Third, input the graph nodes into the GCN module for feature extraction. Finally, decode the extracted superpixel node features into pixel-level branch features F. GCN ;

[0055] SLIC will cluster pixel-level features of dimension H×W×C1 into features of dimension H×W×C1. k The superpixel features are defined by k, where k is the number of superpixels of approximately equal size after clustering. SLIC can reduce the computational cost of the model while preserving image edge and texture features. Its specific steps are as follows: 1. Initialize the center points of the superpixels and divide the image evenly into k superpixel blocks; 2. Calculate the value of each pixel to the superpixel center point and assign all pixels to the superpixel block closest to that pixel; 3. Calculate the mean of all pixels in each superpixel block and use this mean as the new superpixel center point; 4. Repeat steps 2 and 3 until the center point remains stable.

[0056] At this stage, the feature data cannot be directly input into the model for feature extraction. It needs to be used to construct graph nodes through encoding operations. The structure diagram of the encoding operations is shown below. Figure 5As shown. Flattening the pixel-level features and superpixel-level features into one-dimensional feature matrices X and W respectively, they can be represented as:

[0057]

[0058] Where n and k are the number of pixels and the number of superpixels, respectively. Establish the correlation matrix Q∈R between X and W. HW×k Where HW represents the height and width of the image feature, and k is the number of superpixels, then:

[0059]

[0060] Among them, X i W represents the pixel-level feature matrix. j Let Q represent the superpixel-level feature matrix; i∈[1,n], j∈[1,k]. Normalizing Q to a one-dimensional matrix Q1 with dimension d×1, the graph node feature matrix V can be expressed as:

[0061] V = Encoder(X, Q1) = X T Q1

[0062] Where Q1 is the normalized matrix of Q; X T V is the transpose of matrix X. The eigendimensional dimension of V is n×d. The relationships between graph nodes, i.e., the edges in the constructed graph, can be represented as the adjacency matrix E, with a dimension of n×n.

[0063] Figure 2 GCN can effectively extract feature information from graph data and reduce the number of matrix operations, thus reducing computational cost. Specifically, it performs a weighted summation of the neighboring nodes of each node in the graph and multiplies this summation with the parameter matrix to obtain the node features of a new layer. Let H... l H is the feature matrix of the l-th layer. l+1 Let be the feature matrix of the (l+1)th layer, then we have:

[0064]

[0065] Where ReLU represents a nonlinear activation function; E represents the adjacency matrix of the relationships between nodes in the graph, and I represents the identity matrix; for The degree matrix, i and j represent matrices Dimensions.

[0066] Propagation between layers is achieved by iteratively calculating the features of each node and performing graph convolution operations, ultimately updating the node features. After completing two layers of GCN operations, the features need to be decoded to obtain the features F of the GCN branch.GCN Specifically, it can be expressed as:

[0067] F GCN =Reshape(QV)

[0068] Where Q represents the association matrix between pixels and superpixel nodes, V represents the graph node feature matrix, and Reshape represents reshaping the matrix.

[0069] S4. Fuse features from the CNN and GCN feature branches, train the model using the cross-entropy function as the loss function, and use the softmax function to obtain the probability of each class label. Then, use F... CNN and F GCN Feature concatenation can be specifically represented as:

[0070]

[0071] in, This represents the concatenation of spectral dimensions. The loss function used during model training is the cross-entropy function, which can be specifically expressed as:

[0072]

[0073] Where T represents the correct category of the label; P represents the model's prediction for each pixel; N represents the total number of samples in the dataset; C represents the number of categories in the dataset; y i,c p represents the c-th element of the i-th label y. i,c This indicates that the i-th pixel belongs to the c-th class. Finally, the softmax function is used to predict the probability of each class label, thus completing the hyperspectral image classification task.

[0074] The present invention provides a two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms. This method uses a limited number of hyperspectral images as the research object, improves classification accuracy, and ensures classification speed.

[0075] The attention mechanism structure in this invention can fully extract spatial and spectral information from images. The two designed CE_A and CE_B feature extraction modules can enhance the network's focus while minimizing computational parameters, and can more effectively apply computational resources to the core regions of the model, thereby enhancing the model's feature extraction capabilities.

[0076] The GCN branch in this invention effectively avoids multiple matrix operations while performing feature extraction, greatly improving model training speed and reducing model parameters. PCA and SLIC significantly reduce model computation through dimensionality reduction and clustering of data. GCN can effectively convert spectral information into graph nodes, extracting spectral and spatial information more fully and further improving model classification accuracy.

[0077] This invention fuses different features extracted from CNN and GCN branches, resulting in a more comprehensive feature representation of the input data, which greatly improves classification accuracy and overall model performance.

[0078] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A two-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms, characterized in that, Includes the following steps: S1. Input the original hyperspectral image and preprocess the data; Set the raw input data of the hyperspectral image as Where H is the height of the hyperspectral image, W is the width of the hyperspectral image, and C is the dimension of the hyperspectral image, principal component analysis is applied to reduce the dimension of the original image, filtering out useless noise and high-dimensional features. The dimension-reduced data is represented as follows: ; S2. Input the dimensionality-reduced data from step S1 into the CNN attention mechanism branch. The data sequentially passes through the attention mechanism module CE_A, the data processing module CBL, the attention mechanism module CE_B, and convolution operations to finally obtain the processed branch features. ; S3. Input the dimensionality-reduced data from step S1 into the GCN branch. First, perform simple linear iterative clustering on the data to divide the dimensionality-reduced data into k superpixels. Second, encode the k superpixels to obtain superpixel graph nodes. Third, input the graph nodes into the GCN module for feature extraction. Finally, decode the extracted superpixel node features into pixel-level branch features. ; Step S3 involves performing a weighted summation of the neighboring nodes of each node in the graph and multiplying it by the parameter matrix to obtain the node features of the new layer; let... Let l be the feature matrix of the l-th layer. Let be the feature matrix of the (l+1)th layer, then Where ReLU represents a nonlinear activation function; E represents the adjacency matrix of the relationships between nodes in the graph, and I represents the identity matrix; for The degree matrix, i and j represent matrices The dimension; propagation between layers is achieved by calculating and iterating the features of each node and performing graph convolution operations, and finally the node features are updated; After completing two layers of GCN operations, the features are decoded to obtain the features of the GCN branch. Specifically, it is expressed as , Where Q represents the association matrix between pixels and superpixel nodes, V represents the graph node feature matrix, and Reshape represents reshaping the matrix; S4. Perform feature fusion on the CNN feature branches and GCN feature branches, use the cross-entropy function as the loss function to train the model, and use the softmax function to obtain the probability of each category label to be classified.

2. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, In step S2, attention mechanism modules CE_A and CE_B are both composed of CA attention mechanism module and ECA attention mechanism module. Attention mechanism module CE_A adds spatial features to channel information. The dimensionality-reduced data is input to CA attention mechanism module and ECA attention mechanism module respectively, and finally outputs fused features.

3. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, The CA attention mechanism in step S2 divides feature processing into two directions, X and Y; in the X direction, the data is processed in a way that... The average pooling operation is performed in the form of H, where H is the input feature height, l is the input feature width, and 1 is the input feature width. Input feature dimension; Meanwhile, in the Y direction, the data is... The average pooling operation is performed in the form of , where 1 is the input feature height, W is the input feature width, and . Let the input feature dimensions be denoted as , and the feature layers after two average pooling operations be denoted as . and Perform a tensor concatenation operation on two feature layers, represented as: The concatenated features will be sequentially processed by convolution kernels of size [size missing]. The system consists of convolutional layers, batch normalization layers, and Leaky ReLU activation function layers, with the batch normalization layer abbreviated as BN layer. The processed features are then further divided into feature tensors in the X and Y directions, both of which are processed... The convolutional layer and the sigmoid function layer are represented as follows: and Finally and Input features Perform feature multiplication to obtain the input data. Feature Mapping The value at position (i, j) ,in, , indicating that the feature height is i, , indicating that the feature width is j, This indicates feature multiplication.

4. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, The input feature dimension of the ECA attention mechanism in step S2 is First, a global average pooling operation is used to transform the input feature dimension into... , where i is the number of layers with feature depth; secondly, a convolution kernel of size 1 is applied. The convolution with layer 1 shares weights between the depth dimension information of each layer and the depth dimension information of adjacent layers; then, after passing through the Sigmoid function layer, the processed feature tensor has a dimension of 1. Where s represents the number of feature depth layers; finally, the feature is multiplied by the input feature to obtain the dimension. Features Where i represents the feature height, j represents the feature width. ; This indicates feature multiplication.

5. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, The specific operation of the CE_A structure in step S2 is to perform a feature multiplication operation on the features processed by the CA attention mechanism module and the ECA attention mechanism module, retaining one input feature branch to obtain the fused features. ; then After passing through the CBL module, CBL includes convolutional kernels with a size of 3. The system uses Conv, BN layers, and LeakyReLU activation function; the attention mechanism module CE_B represents the feature extraction process, where features are sequentially processed by the CA attention mechanism module and the ECA attention mechanism module, and then processed by CE_B to obtain the final feature. Finally, it goes through a convolution kernel of size 3 A convolutional layer of 3 is used to obtain the features extracted by the CNN branch. .

6. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, In step S3, SLIC will convert the dimension to Pixel-level feature clustering is of dimension 1 The superpixel features, where k is the number of superpixels of equal size after clustering; The pixel-level features and superpixel-level features are flattened into one-dimensional feature matrices X and W, respectively. , Let n and k be the number of pixels and the number of superpixels, respectively; establish the correlation matrix between X and W. Where HW is the height and width of the image feature, and k is the number of superpixels. hour ,otherwise, ,in, , Normalize Q to a one-dimensional matrix. , dimension Then the feature matrix V of the graph nodes is represented as The feature dimension is The relationships between graph nodes are represented by edges in the constructed graph, denoted as an adjacency matrix E with dimension 1. .

7. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, In step S4, the two feature branches obtained in steps S2 and S3 are concatenated, specifically as follows: ， in, This represents the concatenation of spectral dimensions.

8. The dual-branch hyperspectral image classification method based on graph convolutional neural networks and attention mechanisms according to claim 1, characterized in that, In step S4, the loss function used during model training is the cross-entropy function, specifically expressed as: ， Where T represents the correct category of the label; P represents the model's prediction for each pixel; N represents the total number of samples in the dataset; and C represents the number of categories in the dataset. Let represent the c-th element of the i-th label y. This indicates that the i-th pixel belongs to the c-th class.

Images