Hyperspectral image classification method of cross-hop node interaction graph attention network

Abstract

The invention discloses a hyperspectral image classification method of a cross-hop node interaction graph attention network. The method comprises the following steps: step 1, constructing a DNIGAT-CFF overall model architecture; 2, differentiated features are extracted based on the coupled convolution blocks; step 3, cross-hop node interaction graph attention network enhanced spectrum-spatial feature learning; step 4, carrying out multi-scale cross guidance feature fusion CGFF; step 5, important fusion features are highlighted by a weighted attention mechanism; according to the method, the attention network of cross-hop node interaction is constructed, interaction between nodes with different hop counts is effectively utilized, and the extraction capability of spectrum and spatial features is enhanced; a multi-scale cross guide feature fusion module is adopted, complementarity and correlation between different scale features are fully considered, and effective fusion of the multi-scale features is achieved; and in combination with a weighted attention mechanism, important features in the fused multi-scale features are highlighted, so that the precision of hyperspectral image classification is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and in particular to a hyperspectral image classification method using a cross-hop node interaction graph attention network. Background Art

[0002] Hyperspectral image classification is of great significance in remote sensing applications. Graph convolutional neural networks (GCNNs) have emerged as a promising approach for hyperspectral image classification, performing a convolution operation by weighted summing the features of its neighboring nodes to generate a feature representation of the target node. However, this weighted summation operation assumes the independence of neighboring nodes, ignoring potential interactions between them and hindering the representation of important features of the target node. Furthermore, the multi-scale features of hyperspectral images are beneficial for object recognition, but most methods employ only simple fusion operations such as addition and summation for these multi-scale features, which limits their ability to distinguish object categories. Summary of the Invention

[0003] The purpose of the present invention is to provide a hyperspectral image classification method based on a cross-hop node interaction graph attention network to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a hyperspectral image classification method based on a cross-hop node interaction graph attention network, comprising the following steps:

[0005] Step 1: Construct the overall DNIGAT-CFF model architecture: Display the overall network architecture in the form of a network diagram;

[0006] Step 2: Extract differential features based on coupled convolution blocks: Given the complex ground coverage of hyperspectral images, extracting key features is crucial for classification accuracy. To extract differential features from hyperspectral images, a coupled convolution block is introduced. It consists of two convolutional layers with different convolution kernel sizes, which are combined in parallel and the features extracted by different convolutional layers are fused through addition. Subsequently, a convolutional layer is applied to integrate the differential features extracted by the coupled convolution block.

[0007] Step 3: Enhance spectral-spatial feature learning with the cross-hop node interaction graph attention network: Construct adjacency matrices of spectral and spatial node interactions from the spectral and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance spectral and spatial feature learning;

[0008] Step 4: Multi-scale cross-guided feature fusion (CGFF): In the multi-scale convolutional feature fusion sub-network, 2D pyramid convolution is used as the backbone network to extract multi-scale features. Convolution kernels of sizes 1×1, 3×3, and 5×5 are used to obtain feature maps of different scales. 、 and ;use Guide separately and , information interaction is achieved through element-by-element multiplication and dimension-by-dimension summation operations, fully integrating multi-scale features;

[0009] Step 5: Weighted attention mechanism highlights important fusion features: extract the multi-scale feature map Linear map to key tensor Sum tensor , directly As a query tensor , query tensor With key tensor Perform inner product operation and obtain the attention score of channel feature map by scaling by proportional factor, apply softmax operation to channel feature map to generate channel attention map, channel attention map and value tensor Perform multiplication operations and use the softmax function to generate the weights of the multi-scale feature maps to obtain the weighted channel attention map with important feature expressions.

[0010] Preferably, the step 1 is based on the Pavia University data cube and represents it as ,in Represents the size of the space, Indicates the size of the spectral channel; Random cropping is performed and fed into DNIGAT-CFF for pixel-by-pixel ground cover classification.

[0011] Preferably, the step 1 is specifically as follows: DNIGAT-CFF consists of a GAT-based subnetwork and a multi-scale convolutional feature fusion subnetwork; in the GAT-based subnetwork, two-layer coupled convolution blocks are used to extract differentiated spectral-spatial features; then, in order to enhance spectral and spatial feature learning, the attention network of the node interaction graph with different hops is operated in a one-shot manner; specifically, adjacency matrices of spectral and spatial node interactions are constructed from the spectral channels and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance spectral and spatial feature learning; in the multi-scale convolutional feature fusion subnetwork, considering the correlation and complementarity between features of different scales, a multi-scale cross-guided feature fusion module is designed to fully fuse multi-scale features; then, a weighted attention mechanism is applied to highlight the important features in the fused multi-scale features; finally, the output features of the multi-scale feature fusion subnetwork and the GAT subnetwork are fused through an adaptive feature fusion method to generate features for hyperspectral image classification. ; The specific operation is shown in formula (1.1):

[0012]

[0013] in, is a hyperparameter used to control the feature weights of the GAT sub-network, and Control the feature weights of the feature fusion sub-network, both of which can adaptively adjust their sizes during training; and Represent the features extracted by the GAT-based sub-network and the feature fusion sub-network, respectively.

[0014] Preferably, the mathematical expressions in step 2 are as shown in formula (1.2) and formula (1.3):

[0015]

[0016]

[0017] in, Indicates the The feature map of the layer, and is the convolution kernel, and is the bias term, Represents batch normalization, which helps accelerate the convergence of the entire network and alleviate overfitting; Mish represents the non-monotonic self-regularization activation function, which improves the classification accuracy of the model. The detailed mathematical expression of Mish is shown in formula (1.4):

[0018]

[0019] in, and Represent the output and input of the mish function respectively, represents the multiplication operation; the Mish function is infinite in the upper bound, which avoids the saturation phenomenon caused by the gradient approaching zero, thereby accelerating network training; in addition, the function is bounded in the lower bound, which helps to produce a stronger regularization effect.

[0020] Preferably, the given graph in step 3 ,in represents the binary adjacency matrix of the graph, Represents the feature matrix of the node, each node consists of a size of In addition, if , then it means the node With node There is a connection between them; remember the target node The neighborhood set of , indicating that all nodes directly connected nodes; in addition, nodes Itself is also included in its neighborhood set, forming an extended neighborhood set, denoted as ; To visually represent the nodes degrees, using the symbol To indicate that ; Accordingly, the expansion is expressed as ;

[0021] Models based on graph neural networks have received widespread attention in processing complex ground coverage scenarios in hyperspectral images. They learn the features of target nodes by aggregating the features of neighboring nodes. Among these models, the GAT-based model shows superior performance in feature extraction, as shown in formula (1.5):

[0022]

[0023] in, Indicates the target node In the The feature representation of the layer, is a learnable weight matrix that Perform linear transformation; Can be obtained from the initial feature matrix, Represents the comparable weight coefficients obtained by applying the attention mechanism and softmax function to the input node features. represents the activation function;

[0024] The above function is usually performed as a weighted sum, where As a neighbor node However, the limitation of this weighted summation is that it cannot model the information interaction between adjacent node representations; the information interaction between the target node and its neighboring nodes is conducive to the expression of important features of the target node;

[0025] To effectively enhance the feature representation of target nodes in spectral or spatial channels, a graph attention network with different hop counts of nodes is proposed. This network leverages the interactions between nodes with different hop counts, enabling the graph attention network to extract more critical spectral and spatial features from hyperspectral images. Unlike traditional multi-hop graph methods, a graph strategy with different hop counts is introduced. This strategy breaks the limitation of fixed hop counts by innovatively fusing two multi-hop structures with different hop counts, thereby achieving multi-level effective information interaction between nodes. It is worth noting that the use of adjacency matrices with different hop counts helps avoid the problem of node self-interaction. In addition, different hop counts contribute differently to the feature representation of target nodes. The detailed construction process of DNIGAT is as follows:

[0026] Two adjacency matrices with different hop counts are constructed according to the spatial or spectral channels of the hyperspectral image: Hop adjacency matrix and Hop adjacency matrix ,in represents the batch size, Represents the spatial dimension or spectral dimension of the feature graph; in order to promote the interaction of nodes with the same hop and different hops of the target node, the following operations are applied to achieve node interaction between different hops;

[0027] first, and By adding element by element, we can generate , which records the Jump and The information of the jump node is shown in formula (1.6):

[0028]

[0029] in, Represents a union operation;

[0030] Then, a window multi-head self-attention mechanism is adopted to promote the interaction between nodes on the same hop and different hops from a local perspective; subsequently, a sliding window multi-head self-attention block is introduced to further promote the interaction between nodes on the same hop and different hops from a global perspective.

[0031] Preferably, the step 3 specifically includes the following steps:

[0032] 3.1. Using WMS for local range node interaction:

[0033] matrix After the deformation operation, a non-overlapping window partitioning method is used for uniform segmentation; (The area size is ) is evenly divided into multiple non-overlapping windows ,in (j=1,2,…, D / ),and Indicates the spatial size of the local window; then, in each divided sub-window (j=1,2,…, D / ) to promote node interaction within each sub-window; the specific operations are as follows:

[0034] Will As input to MHSA; then, Input linear layer and construct query matrix respectively , bond matrix Sum Matrix ; The specific operation is shown in formula (1.7):

[0035]

[0036] in, , and is the linear projection matrix, , and represents the learnable parameter matrix; subsequently, the attention calculation is performed on these three constructed matrices; in addition, the position mask is added in each attention head To mark the location information; the specific operations are shown in formula (1.8) and formula (1.9):

[0037]

[0038]

[0039] in, represents the softmax function, express The dimension size, represents the number of heads in MHSA, Represents the multi-head self-attention mechanism;

[0040] The operation of using WMS to interact with local nodes is shown in formula (1.10):

[0041]

[0042] in, express The result of the interaction between the local scope nodes, Represents the window multi-head self-attention mechanism;

[0043] 3.2. Use SMS for global node interaction:

[0044] Considering that node interactions occur only in each Therefore, there is a lack of global node interaction between different windows; for this reason, Apply SMS operations to facilitate node interactions within the same hop count and between different hop counts from a global perspective;

[0045] Therefore, in A sliding window operation is applied on The interaction between windows is introduced; then, Apply MHSA to promote node interaction in different windows; specifically, Input MHSA to linearly construct the query matrix , bond matrix Sum Matrix ; Subsequently, the attention calculation is performed based on these three linearly constructed matrices, as shown in formula (1.11):

[0046]

[0047] in, express The local-global scope node interaction results;

[0048] These operations not only enable interactions between nodes within the same hop count, but also facilitate interactions between nodes with different hop counts;

[0049] 3.3 One-Shot DNIGAT Module

[0050] Based on the adjacency matrix of nodes with different hop counts, a DNIGAT model is designed to enhance the extraction of spectral and spatial features by utilizing the interactions between nodes with different hop counts. The detailed operation process of the graph attention layer based on the interactions between nodes with different hop counts is shown in formulas (1.12) and (1.13).

[0051]

[0052]

[0053] in, Representation node In the The feature representation of the layer, and Both represent learnable weight matrices, represents the u-th layer, represents the activation function, Represents GAT based on the interaction of nodes with different hop counts, represents the Leaky ReLU function, Represents the shared attention mechanism;

[0054] In order to further alleviate the over-smoothing problem caused by multiple stacked graph attention layers, a graph attention network GAT with multiple nodes interacting with different hop counts is adopted and constructed in a one-shot connection mode. The corresponding specific operations are shown in formula (1.14):

[0055]

[0056] in, Indicates channel-wise concatenation operation; here, Set to 3; finally, the dropout layer and ELU activation function are used to help alleviate overfitting and improve the network training process.

[0057] Preferably, the step 4 is specifically as follows: using 2D pyramid convolution as the backbone network to extract multi-scale features; CGFF uses a 1×1 convolution kernel to replace the 7×7 convolution kernel, while keeping the sizes of other convolution kernels unchanged, focusing on extracting finer features; specifically, the feature map It is input into 2D PConv for feature extraction, and then feature maps of different scales are obtained. , and , each feature map has a specific receptive field; at the same time, 2D batch normalization operation and mish activation function are introduced in each 2D convolution layer;

[0058] In order to fully integrate and utilize the extracted multi-scale features, CGFF first uses Guide separately and ; Then, and Combined, so as to maximize the complementarity and correlation between the features of each scale; the detailed fusion operation is shown in formula (1.15), formula (1.16) and formula (1.17):

[0059]

[0060]

[0061]

[0062] in, , and They are respectively for , and The constructed trainable transformation matrix;

[0063] Based on the above operations, use respectively and To guide, and realize information interaction through element-by-element multiplication and dimension-by-dimension summation operations; the detailed mathematical operations are shown in formula (1.18) and formula (1.19):

[0064]

[0065]

[0066] in, represents the sum operation by dimension, represents the matrix transformation operation, Represents a slicing operation acting on the spatial dimension, represents an element-wise multiplication operation; subsequently, and The combination is performed through element-by-element summation and processed sequentially through the Leaky ReLU (LReLU) function and the softmax() function, as shown in formula (1.20):

[0067]

[0068] Based on the above operations, and Integration is performed to achieve full fusion of each scale feature; the specific fusion operation is shown in formula (1.21):

[0069]

[0070] in, represents the ReLU activation function, express , and The result after fusion;

[0071] After being processed by the multi-scale feature fusion module, the complementarity and correlation between feature maps of different scales are fully exploited, and feature maps with different receptive fields are fully fused, thereby retaining the diversity of features; in addition, the full fusion of multi-scale features enables more accurate object recognition in scenarios with complex object coverage in hyperspectral images.

[0072] Preferably, the step 5 is specifically as follows: first, the extracted multi-scale feature map Linear map to key tensor Sum tensor At the same time, it directly As a query tensor , without performing a linear mapping operation; querying the tensor With key tensor Perform inner product operation and pass the scaling factor Scaling is performed to obtain the attention score of the channel feature map, where for Then, a softmax operation is applied to these channel feature maps to generate Channel attention map of Perform multiplication to obtain the channel attention map after multiplication; next, in order to calculate the normalized weight of the channel attention map , using the softmax function; the softmax function is used to generate the weights of the multi-scale feature map, which helps to enhance the expression of important features while suppressing unimportant features; then, the channel attention map is combined with Multiplication operations are performed to obtain a weighted channel attention map with important feature expressions; the weighted attention mechanism is used to fuse important features to improve classification accuracy.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] This paper proposes a hyperspectral image classification method based on a cross-hop node interaction graph attention network. By constructing a cross-hop node interaction graph attention network, this method effectively leverages the interactions between nodes of different hop counts, enhancing the extraction of spectral and spatial features. A multi-scale cross-guided feature fusion module fully considers the complementarity and correlation between features at different scales, achieving effective fusion of multi-scale features. Combined with a weighted attention mechanism, this method highlights important features within the fused multi-scale features, thereby improving the accuracy of hyperspectral image classification. Experimental results demonstrate that this method achieves competitive classification performance compared to other state-of-the-art methods on four widely used hyperspectral datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is the overall structure diagram of the DNIGAT-CFF model of the present invention;

[0076] Figure 2 It is the overall structure diagram of the coupled convolution block of the present invention;

[0077] Figure 3 It is a graph of the interaction between nodes of different hop counts in the local scope and the global scope of the present invention;

[0078] Figure 4 This is a graph attention network with a single-head attention mechanism of the present invention; in the figure, the graph attention network (GAT) is used as an example (left), and the blue solid line with a single arrow represents the attention operation; the diagram shows the interaction between node H1 and its adjacent nodes, and the orange solid line with double arrows in the figure represents the interaction between different nodes (right);

[0079] Figure 5 This is a flowchart of the attention network of the node interaction graph with different hop counts in the one-shot connection mode of the present invention; in the figure, Indicates splicing by channel;

[0080] Figure 6 1 is a diagram showing the overall structure of the model of the multi-scale cross-guided feature fusion module of the present invention;

[0081] Figure 7 This is the overall structure diagram of the weighted attention mechanism of the present invention;

[0082] Figure 8 is the classification result diagram of different methods of the present invention on the PU dataset;

[0083] Figure 9 is the classification result diagram of different methods of the present invention on the IP dataset;

[0084] Figure 10 : This is the classification result diagram of different methods of the present invention on the Honghu dataset;

[0085] Figure 11 : This is the classification result diagram of different methods of the present invention on the Houston dataset;

[0086] Figure 12 It is a feature visualization diagram of different methods of the present invention on the IP dataset;

[0087] Figure 13 This is the ablation experiment result diagram of the present invention on four data sets; in the figure, 1) M_1: model without CC module; 2) M_2: model without DNIGAT module; 3) M_3: model without GAT-based branch; 4) M_4: model without CGFF module; 5) M_5: model without WA module; 6) M_6: model without multi-scale branch; 7) M_7: complete model. DETAILED DESCRIPTION

[0088] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0089] See also Figure 1-13 The present invention provides a hyperspectral image classification method based on a cross-hop node interaction graph attention network, comprising the following steps:

[0090] Step 1: Construct the overall DNIGAT-CFF model architecture: Display the overall network architecture in the form of a network diagram; take the Pavia University data cube as an example and represent it as ,in Represents the size of the space, Indicates the size of the spectral channel; Perform random cropping and feed it into DNIGAT-CFF for pixel-by-pixel ground cover classification;

[0091] Step 2: Extract differential features based on coupled convolution blocks: Given the complex ground coverage of hyperspectral images, extracting key features is crucial for classification accuracy. To extract differential features from hyperspectral images, a coupled convolution block is introduced. It consists of two convolutional layers with different convolution kernel sizes, which are combined in parallel and the features extracted by different convolutional layers are fused through addition. Subsequently, a convolutional layer is applied to integrate the differential features extracted by the coupled convolution block.

[0092] Step 3: Enhance spectral-spatial feature learning with the cross-hop node interaction graph attention network: Construct adjacency matrices of spectral and spatial node interactions from the spectral and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance spectral and spatial feature learning;

[0093] Step 4: Multi-scale cross-guided feature fusion (CGFF): In the multi-scale convolutional feature fusion sub-network, 2D pyramid convolution is used as the backbone network to extract multi-scale features. Convolution kernels of sizes 1×1, 3×3, and 5×5 are used to obtain feature maps of different scales. 、 and ;use Guide separately and , information interaction is achieved through element-by-element multiplication and dimension-by-dimension summation operations, fully integrating multi-scale features;

[0094] Step 5: Weighted attention mechanism highlights important fusion features: extract the multi-scale feature map Linear map to key tensor Sum tensor , directly As a query tensor , query tensor With key tensor Perform inner product operation and obtain the attention score of channel feature map by scaling by proportional factor, apply softmax operation to channel feature map to generate channel attention map, channel attention map and value tensor Perform multiplication operations and use the softmax function to generate the weights of the multi-scale feature maps to obtain the weighted channel attention map with important feature expressions.

[0095] Step 1 is as follows: DNIGAT-CFF consists of a GAT-based subnetwork and a multi-scale convolutional feature fusion subnetwork; in the GAT-based subnetwork, two-layer coupled convolution blocks are used to extract differentiated spectral-spatial features; subsequently, in order to enhance the learning of spectral and spatial features, the attention network of the node interaction graph with different hops is operated in a one-shot manner; specifically, the adjacency matrices of spectral and spatial node interactions are constructed from the spectral channels and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance the learning of spectral and spatial features; in the multi-scale convolutional feature fusion subnetwork, considering the correlation and complementarity between features of different scales, a multi-scale cross-guided feature fusion module is designed to fully fuse multi-scale features; then, a weighted attention mechanism is applied to highlight the important features in the fused multi-scale features; finally, the output features of the multi-scale feature fusion subnetwork and the GAT-based subnetwork are fused through an adaptive feature fusion method to generate features for hyperspectral image classification. ; The specific operation is shown in formula (1.1):

[0096]

[0097] in, is a hyperparameter used to control the feature weights of the GAT sub-network, and Control the feature weights of the feature fusion sub-network, both of which can adaptively adjust their sizes during training; and Represent the features extracted by the GAT-based sub-network and the feature fusion sub-network, respectively.

[0098] The mathematical expressions in step 2 are shown in formulas (1.2) and (1.3):

[0099]

[0100]

[0101] in, Indicates the The feature map of the layer, and is the convolution kernel, and is the bias term, Represents batch normalization, which helps accelerate the convergence of the entire network and alleviate overfitting; Mish represents the non-monotonic self-regularization activation function, which improves the classification accuracy of the model. The detailed mathematical expression of Mish is shown in formula (1.4):

[0102]

[0103] in, and Represent the output and input of the mish function respectively, represents the multiplication operation; the Mish function is infinite in the upper bound, which avoids the saturation phenomenon caused by the gradient approaching zero, thereby accelerating network training; in addition, the function is bounded in the lower bound, which helps to produce a stronger regularization effect.

[0104] Given the graph in step 3 ,in represents the binary adjacency matrix of the graph, Represents the feature matrix of the node, each node consists of a size of vector description; in addition, if , then it means the node With node There is a connection between them; remember the target node The neighborhood set of , indicating that all nodes directly connected nodes; in addition, nodes Itself is also included in its neighborhood set, forming an extended neighborhood set, denoted as ; To visually represent the nodes degrees, using the symbol To indicate that ; Accordingly, the expansion is expressed as ;

[0105] Models based on graph neural networks have received widespread attention in processing complex ground coverage scenarios in hyperspectral images. They learn the features of target nodes by aggregating the features of neighboring nodes. Among these models, the GAT-based model shows superior performance in feature extraction, as shown in formula (1.5):

[0106]

[0107] in, Indicates the target node In the The feature representation of the layer, is a learnable weight matrix that Perform linear transformation; Can be obtained from the initial feature matrix, Represents the comparable weight coefficients obtained by applying the attention mechanism and softmax function to the input node features. represents the activation function;

[0108] The above function is usually performed as a weighted sum, where As a neighbor node However, the limitation of this weighted summation is that it cannot model the information interaction between adjacent node representations; the information interaction between the target node and its neighboring nodes is conducive to the expression of important features of the target node;

[0109] To effectively enhance the feature representation of target nodes in spectral or spatial channels, a graph attention network with different hop counts of nodes is proposed. This network leverages the interactions between nodes with different hop counts, enabling the graph attention network to extract more critical spectral and spatial features from hyperspectral images. Unlike traditional multi-hop graph methods, a graph strategy with different hop counts is introduced. This strategy breaks the limitation of fixed hop counts by innovatively fusing two multi-hop structures with different hop counts, thereby achieving multi-level effective information interaction between nodes. It is worth noting that the use of adjacency matrices with different hop counts helps avoid the problem of node self-interaction. In addition, different hop counts contribute differently to the feature representation of target nodes. The detailed construction process of DNIGAT is as follows:

[0110] Two adjacency matrices with different hop counts are constructed according to the spatial or spectral channels of the hyperspectral image: Hop adjacency matrix and Hop adjacency matrix ,in represents the batch size, Represents the spatial dimension or spectral dimension of the feature graph; in order to promote the interaction of nodes with the same hop and different hops of the target node, the following operations are applied to achieve node interaction between different hops;

[0111] first, and By adding element by element, we can generate , which records the Jump and The information of the jump node is shown in formula (1.6):

[0112]

[0113] in, Represents a union operation;

[0114] Then, a window multi-head self-attention mechanism is adopted to promote the interaction between nodes on the same hop and different hops from a local perspective; subsequently, a sliding window multi-head self-attention block is introduced to further promote the interaction between nodes on the same hop and different hops from a global perspective.

[0115] Step 3 specifically includes the following steps:

[0116] 3.1. Using WMS for local range node interaction:

[0117] matrix After the deformation operation, a non-overlapping window partitioning method is used for uniform segmentation; (The area size is ) is evenly divided into multiple non-overlapping windows ,in (j=1,2,…, D / ),and Indicates the spatial size of the local window; then, in each divided sub-window (j=1,2,…, D / ) to promote node interaction within each sub-window; the specific operations are as follows:

[0118] Will As input to MHSA; then, Input linear layer and construct query matrix respectively , bond matrix Sum Matrix ; The specific operation is shown in formula (1.7):

[0119]

[0120] in, , and is the linear projection matrix, , and represents the learnable parameter matrix; subsequently, the attention calculation is performed on these three constructed matrices; in addition, the position mask is added in each attention head To mark the location information; the specific operations are shown in formula (1.8) and formula (1.9):

[0121]

[0122]

[0123] in, represents the softmax function, express The dimension size, represents the number of heads in MHSA, Represents the multi-head self-attention mechanism;

[0124] The operation of using WMS to interact with local nodes is shown in formula (1.10):

[0125]

[0126] in, express The result of the interaction between the local scope nodes, Represents the window multi-head self-attention mechanism;

[0127] 3.2. Use SMS for global node interaction:

[0128] Considering that node interactions occur only in each Therefore, there is a lack of global node interaction between different windows; for this reason, Apply SMS operations to facilitate node interactions within the same hop count and between different hop counts from a global perspective;

[0129] Therefore, in A sliding window operation is applied on The interaction between windows is introduced; then, Apply MHSA to promote node interaction in different windows; specifically, Input MHSA to linearly construct the query matrix , bond matrix Sum Matrix ; Subsequently, the attention calculation is performed based on these three linearly constructed matrices, as shown in formula (1.11):

[0130]

[0131] in, express The local-global scope node interaction results;

[0132] These operations not only enable interactions between nodes within the same hop count, but also facilitate interactions between nodes with different hop counts;

[0133] 3.3 One-Shot DNIGAT Module

[0134] Based on the adjacency matrix of nodes with different hop counts, a DNIGAT model is designed to enhance the extraction of spectral and spatial features by utilizing the interactions between nodes with different hop counts. The detailed operation process of the graph attention layer based on the interactions between nodes with different hop counts is shown in formulas (1.12) and (1.13).

[0135]

[0136]

[0137] in, Representation node In the The feature representation of the layer, and Both represent learnable weight matrices, represents the u-th layer, represents the activation function, Represents GAT based on the interaction of nodes with different hop counts, represents the Leaky ReLU function, Represents the shared attention mechanism;

[0138] In order to further alleviate the over-smoothing problem caused by multiple stacked graph attention layers, a graph attention network GAT with multiple nodes interacting with different hop counts is adopted and constructed in a one-shot connection mode. The corresponding specific operations are shown in formula (1.14):

[0139]

[0140] in, Indicates channel-wise concatenation operation; here, Set to 3; finally, the dropout layer and ELU activation function are used to help alleviate overfitting and improve the network training process.

[0141] Step 4 is as follows: 2D pyramid convolution is used as the backbone network to extract multi-scale features; CGFF uses a 1×1 convolution kernel to replace the 7×7 convolution kernel, while keeping the sizes of other convolution kernels unchanged, focusing on extracting finer features; specifically, the feature map It is input into 2D PConv for feature extraction, and then feature maps of different scales are obtained. , and , each feature map has a specific receptive field; at the same time, 2D batch normalization operation and mish activation function are introduced in each 2D convolution layer;

[0142] In order to fully integrate and utilize the extracted multi-scale features, CGFF first uses Guide separately and ; Then, and Combined, so as to maximize the complementarity and correlation between the features of each scale; the detailed fusion operation is shown in formula (1.15), formula (1.16) and formula (1.17):

[0143]

[0144]

[0145]

[0146] in, , and They are respectively for , and The constructed trainable transformation matrix;

[0147] Based on the above operations, use respectively and To guide, and realize information interaction through element-by-element multiplication and dimension-by-dimension summation operations; the detailed mathematical operations are shown in formula (1.18) and formula (1.19):

[0148]

[0149]

[0150] in, represents the sum operation by dimension, represents the matrix transformation operation, Represents a slicing operation acting on the spatial dimension, represents an element-wise multiplication operation; subsequently, and The combination is performed through element-by-element summation and processed sequentially through the Leaky ReLU (LReLU) function and the softmax() function, as shown in formula (1.20):

[0151]

[0152] Based on the above operations, and Integration is performed to achieve full fusion of each scale feature; the specific fusion operation is shown in formula (1.21):

[0153]

[0154] in, represents the ReLU activation function, express , and The result after fusion;

[0155] After being processed by the multi-scale feature fusion module, the complementarity and correlation between feature maps of different scales are fully exploited, and feature maps with different receptive fields are fully fused, thereby retaining the diversity of features; in addition, the full fusion of multi-scale features enables more accurate object recognition in scenarios with complex object coverage in hyperspectral images.

[0156] Step 5 is as follows: First, extract the multi-scale feature map Linear map to key tensor Sum tensor At the same time, it directly As a query tensor , without performing a linear mapping operation; querying the tensor With key tensor Perform inner product operation and pass the scaling factor Scaling is performed to obtain the attention score of the channel feature map, where for Then, a softmax operation is applied to these channel feature maps to generate Channel attention map of Perform multiplication to obtain the channel attention map after multiplication; next, in order to calculate the normalized weight of the channel attention map , using the softmax function; the softmax function is used to generate the weights of the multi-scale feature map, which helps to enhance the expression of important features while suppressing unimportant features; then, the channel attention map is combined with Multiplication operations are performed to obtain a weighted channel attention map with important feature expressions; the weighted attention mechanism is used to fuse important features to improve classification accuracy.

[0157] Experimental results analysis

[0158] Experimental data set division and selection of comparison algorithms

[0159] In the experiments, the present invention selected four publicly available hyperspectral image datasets—PU, IP, Honghu, and Houston—to validate the classification performance of the proposed method. For the PU, Honghu, and Houston datasets, the training, validation, and test sets used for DNIGAT-CFF training consisted of 1%, 1%, and 98% of the total number of samples in each category, respectively, randomly sampled. For the IP dataset, the training, validation, and test sets used for DNIGAT-CFF training consisted of 5%, 5%, and 90% of the total number of samples in each category, respectively. To verify the effectiveness of DNIGAT-CFF, nine representative methods were selected for comparative classification results: SSRN, MDBNet, DBPFA, MSCA_MSDC, MVAHN, DGFNet, FTINet, MRCAGCFN, and DSNet. A detailed description of these nine methods is shown below.

[0160] SSRN: SSRN uses two cascaded residual blocks to extract important spectral and spatial features. The residual connection is introduced to mitigate accuracy degradation. The residual block consists of a 3D convolutional layer. Furthermore, the inclusion of BatchNorm in the 3D convolutional layer improves SSRN's classification accuracy even with imbalanced training sets.

[0161] MDBNet: MDBNet is designed to extract and fuse multi-scale features. Specifically, it first uses a multi-scale feature module to extract fine-grained spectral-spatial features while expanding the receptive field. Subsequently, a feature interaction module is introduced, which employs residual and dense connections to capture important spectral-spatial features. Furthermore, a shuffle attention mechanism is used to adaptively weight the spectral-spatial features, further improving hyperspectral image classification performance.

[0162] DBPFA: In order to fully fuse spectral and spatial features under small sample conditions, DBPFA first adopts a dual-branch CNN to extract spectral and spatial features, and then uses polarized attention to fuse the extracted features and learn spatial context information.

[0163] MSCA_MSDC: To learn multi-scale features in a lightweight network architecture, MSCA_MSDC uses 1D convolution and 2D depth-separable convolution for feature extraction. In addition, a channel attention mechanism is used to highlight important features between different channels.

[0164] MVAHN: To extract multiple types of features, MVAHN first uses a CNN to extract spectral-spatial features. The CNN is then embedded in a Transformer to learn local-global features. Finally, a GCN module is introduced in the parallel branch of the Transformer to mine pixel-level relationships in hyperspectral image data.

[0165] DGFNet: To learn correlations between two different dimensions, DGFNet designs spatial and spectral branches. In the spatial branch, GAT is used to learn relationships within features. Furthermore, graph pooling and local guidance modules are introduced to mitigate overfitting and learn important features. In the spectral branch, a weighted approach is used to acquire spectral features. Finally, a linear layer is used to fuse spectral and spatial features.

[0166] FTINet: To capture the relationship between hyperspectral image data, FTINet uses CNN and attention mechanism to learn local-global information representation. In addition, it combines spectral and graph topology to enhance information interaction and transmission.

[0167] MRCAGCFN: MRCAGCFN is a patch-based hybrid GCN-CNN model. First, it employs a multi-scale random shape CNN to learn patch-based convolutional features. Subsequently, a novel GCN module is used to extract patch-based GCN features. Finally, an adaptive local feature transformation module is designed to convert CNN and GCN features into pixel-level features for hyperspectral image classification.

[0168] DSNet: To distinguish mixed pixels, DSNet employs a dual-branch sub-pixel guided model. Specifically, it introduces an autoencoder-based unmixing architecture to automatically extract sub-pixel information and convolutional features. Furthermore, it learns the nonlinear physical properties of sub-pixels. Finally, a sub-pixel fusion method is designed to efficiently fuse sub-pixel and pixel information.

[0169] Experimental parameter settings

[0170] During the experiment, the Adam optimizer was used to update the parameters of DNIGAT-CFF. Cosine annealing was used to decay the learning rate. After 50 training epochs, DNIGAT-CFF stopped learning because the model loss stopped decreasing and fluctuated within a certain range. The cross-entropy loss function was used as the model loss function. The learning rates for PU, IP, Houston, and Honghu were set to {0.0005, 0.0009, 0.001, 0.001}, respectively. The patch size used for model training was uniformly set to 9×9 across all four datasets. The batch size was set to 64.

[0171] Comparative analysis of experimental results

[0172] In order to verify the classification performance of DNIGAT-CFF, an experimental comparison was conducted with nine different comparison methods on four different hyperspectral image datasets. The classification results of DNIGAT-CFF and the other nine comparison methods on the four datasets, including the classification results of each class, OA, AA and Kappa coefficient, are shown in Tables 1.1-1.4. The classification results are the average classification results with standard deviations calculated based on the results of ten iterations. In addition, the corresponding full-pixel classification maps of each method on the four datasets are shown in Figure 8-11 The following is a comprehensive discussion of the classification results on the four datasets.

[0173] Experimental Results on the PU Dataset: Table 1.1 shows the classification results of DNIGAT-CFF and nine comparison methods. As can be seen from the table, DNIGAT-CFF's OA, AA, and K coefficient values ​​all exceed those of the nine comparison methods, demonstrating its superior classification performance. The MVAHN method, a hybrid model combining CNN, Transformer, and GCN, performed well in distinguishing the C5 and C7 classes in the PU dataset, but its OA value was still approximately 0.62% lower than that of the present invention. SSRN employs stacked CNN layers and a residual strategy to alleviate the vanishing gradient problem to some extent. However, using only a CNN as the backbone network failed to achieve superior classification performance, resulting in the worst classification performance. Furthermore, DGFNet, FTINet, and MRCAGCFN, methods that combine GCN and CNN, showed better classification results than MDBNet, DBPFA, MSCA_MSDC, and DSNet, demonstrating the advantages of combining CNN and GCN for object recognition in the PU dataset.

[0174] Figure 8 The following table shows the classification maps of nine comparison methods and DNIGAT-CFF on the PU dataset. Compared to the comparison methods, the classification maps of DNIGAT-CFF show smoother class edges. In particular, the red-framed areas of each classification map show the smoothest class edges. Furthermore, the classification maps of our method exhibit less salt-and-pepper noise than those of the other comparison methods.

[0175] Experimental Results on the IP Dataset: The classification results of DNIGAT-CFF and nine other comparison methods on the IP dataset are shown in Table 1.2. DNIGAT-CFF achieved a classification accuracy of approximately 97.63% using the OA metric, which is higher than the classification accuracy of the other comparison methods and demonstrates superior classification performance. Furthermore, the inter-class differences in the IP dataset are relatively significant, which facilitates the identification of feature classes by different classification methods. Notably, DNIGAT-CFF, MSCA_MSDC, and MVAHN all achieved 100% classification accuracy for the C8 and C9 categories. SSRN's classification capabilities were limited, making it difficult to effectively identify the C1, C7, and C9 categories in the IP dataset. However, it was not the worst-performing model. Among all methods, the worst performance occurred with MDBNet, which failed to achieve satisfactory classification results on the IP dataset.

[0176] Figure 9 Full-pixel classification maps of nine comparison methods and DNIGAT-CFF are presented. MDBNet, FTINet, MRCAGCFN, and DSNet exhibit significant salt-and-pepper noise in their full-pixel classification maps, making it difficult to effectively distinguish between object classes. Compared to other comparison methods, DNIGAT-CFF exhibits smoother class boundaries in its classification maps, facilitating the recognition of different objects.

[0177] Experimental results on the Honghu dataset: As shown in Table 1.3, DGFNet achieved 100% classification accuracy on the C8 and C15 categories, reflecting its effective feature extraction capabilities achieved through its GCN-based backbone network. Furthermore, DNIGAT-CFF and MVAHN outperformed other classification methods across all three classification metrics, both of which primarily rely on CNN- and GCN-based architectures. Compared to other methods, MDBNet achieved the worst classification performance due to its low classification accuracy on the C7, C8, and C15 categories. In particular, MDBNet's classification accuracy on the C15 category was only approximately 39.99%, resulting in a decrease in the AA value. Although DNIGAT-CFF did not achieve optimal classification accuracy in some categories, its OA, AA, and K metrics were all higher than those of SSRN, DGFNet, and other comparison methods, highlighting its superior classification performance.

[0178] from Figure 10 As can be seen, the full-pixel classification maps of SSRN, MDBNet, FTINet, and MRGCFN exhibit significant salt-and-pepper noise. In addition to these classification methods, DNIGAT-CFF and other comparison methods exhibit smoother full-pixel classification maps. Comparing each map, the classification map of our invention exhibits the smoothest class edges.

[0179] Experimental Results on the Houston Dataset: Table 1.4 compares the classification results of DNIGAT-CFF with those of nine other comparison methods on the Houston dataset. As can be seen from the table, DNIGAT-CFF achieves superior classification performance compared to the other methods. For the C3 category, the OA classification accuracy of DBPFA, MVAHN, and DNIGAT-CFF, all of which are hybrid models, all reached 100%. Meanwhile, the OA values ​​of SSRN, MVAHN, and DSNet on the Houston dataset also achieved 100% classification accuracy for the C6 category. Furthermore, although MVAHN performed second only to the present invention on the Houston dataset, it exhibited superior classification results compared to the other comparison methods. SSRN achieved a classification accuracy of 0% for the C13 category. MSCA_MSDC achieved a classification accuracy of 0% for the C6 category, demonstrating the limitations of these methods. The Houston dataset suffers from an imbalanced class distribution, making pixel-level classification challenging for classification models. In addition, the results achieved by the proposed method on OA, AA and K are better than those of the other nine compared methods, demonstrating its excellent feature extraction and classification performance.

[0180] from Figure 11 It can be seen that the full pixel classification maps of SSRN, FTINet and DSNet show many categories of misclassification. In summary, compared with the other nine comparison methods, the proposed method produces classification results with smoother edges, which is beneficial for the recognition of ground objects.

[0181] Model Discussion

[0182] Analysis of visualization features of different methods

[0183] In order to verify the feature extraction capability of DNIGAT-CFF, we use the t-distributed stochastic neighbor embedding (T-SNE) technique to visualize the features, and use the IP dataset as an example to compare the feature extraction results of DNIGAT-CFF with those of nine comparison methods. Figure 12As shown, the feature visualizations generated by MDBNet, MRCAGCFN, and DSNet show a relatively chaotic distribution of features. Furthermore, the classification accuracy of these methods is significantly lower than that of the other methods. In contrast, while the feature visualizations generated by MVAHN show a tendency to separate features belonging to different categories, features within the same category also show a trend of separation. In the feature visualizations generated by SSRN, DBPFA, MSCA_MSDC, and DGFNet, while features of different categories are mixed in some areas, overall, features of different categories appear relatively separate, which is conducive to good classification results. The feature visualization generated by DNIGAT-CFF shows that features of different categories are separated from each other, while features of the same category are clustered together, demonstrating better classification performance.

[0184] Ablation experimental test

[0185] In order to verify the effectiveness of different modules in DNIGAT-CFF, this paper conducts ablation experiments on four datasets and uses OA as the evaluation indicator. The results of the ablation experiments are shown in Figure 2. Figure 13 As shown, the following is a detailed analysis of the experimental results after ablation of different modules.

[0186] In order to verify the effectiveness of the coupled convolution block, the present invention removes two layers of coupled convolution blocks from the proposed method and names it M_1. Figure 13 As shown in the results for OA, M_1 performs worse than M_7 on all four datasets, demonstrating the effectiveness of the coupled convolutional block. Specifically, on the PaviaU, IP, Honghu, and Houston datasets, the coupled convolutional block improves overall accuracy by approximately 1.83%, 0.9%, 0.55%, and 2.48%, respectively. This phenomenon demonstrates that the coupled convolutional block is capable of extracting differential features that are beneficial for land cover identification.

[0187] In order to verify the effectiveness of the attention network of the cross-hop node interaction graph, this paper replaces it with the standard GAT and names the model M_2. The experimental results are shown in Figure 13 As shown in the figure, M_2 has lower OA values ​​than M_7 on all four datasets, confirming the effectiveness of cross-hop node interactions in feature learning and their positive impact on classification accuracy. For the Houston dataset (where some categories have fewer training samples and are more challenging to classify), GAT with cross-hop node interactions significantly improves GAT's feature extraction capabilities by enhancing inter-node interactions, increasing the OA value by approximately 2.12%. Furthermore, GAT with cross-hop node interactions improves the OA values ​​by approximately 1.05%, 0.99%, and 1.14% on the PaviaU, IP, and Honghu datasets, respectively.

[0188] In order to verify the effectiveness of the cross-scale guided feature fusion module, this paper removes this module from the proposed method and conducts experiments. Figure 13 As shown in the figure, the results show that the OA value of M_4 is lower than that of M_7, indicating that this module significantly contributes to the classification results. Specifically, on the PaviaU, IP, Honghu, and Houston datasets, this module improves OA by approximately 0.78%, 2.62%, 0.42%, and 2.31%, respectively. This demonstrates that the cross-scale guided feature fusion module effectively utilizes the extracted features to enhance land cover recognition. Furthermore, the important features learned by the weighted attention mechanism play a key role in classification. This is confirmed by the experimental result that the OA value of M_5 is lower than that of M_7. Across the four datasets, when the weighted attention mechanism is removed from DNGAT-CFF, the corresponding OA results decrease by approximately 0.95%, 1.56%, 0.7%, and 1.9% on the PaviaU, IP, Honghu, and Houston datasets, respectively. These results further demonstrate that the weighted attention mechanism helps highlight the most important information in the fused multi-scale features.

[0189] In addition, the contributions of the GAT-based branch and the multi-scale feature fusion-based branch are also verified. Figure 13 As shown in the figure, on the PaviaU, IP, Honghu, and Houston datasets, the OA values ​​of M_3 are approximately 3.72%, 1.83%, 2.52%, and 5.58% lower than those of M_7, respectively, while the OA values ​​of M_6 are approximately 0.57%, 2.68%, 1.65%, and 2.13% lower, respectively. These results further highlight the importance of the two branches in improving the classification performance of DNIGAT-CFF on the four datasets.

[0190] Table 1.1 Classification results of different methods on PU dataset

[0191]

[0192] Table 1.2 Classification results of different methods on IP dataset

[0193]

[0194] Table 1.3 Classification accuracy results of different methods on the Honghu dataset

[0195]

[0196] Table 1.4 Classification accuracy results of different methods on the Houston dataset

[0197]

[0198] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method based on a cross-hop node interaction graph attention network, characterized by: The following steps are involved: Step 1: Construct the overall DNIGAT-CFF model architecture: Display the overall network architecture in the form of a network diagram; Step 2: Extract differential features based on coupled convolution blocks: Given the complex ground coverage of hyperspectral images, extracting key features is crucial for classification accuracy. To extract differential features from hyperspectral images, a coupled convolution block is introduced. It consists of two convolutional layers with different convolution kernel sizes, which are combined in parallel and the features extracted by different convolutional layers are fused through addition. Subsequently, a convolutional layer is applied to integrate the differential features extracted by the coupled convolution block. Step 3: Enhance spectral-spatial feature learning with the cross-hop node interaction graph attention network: Construct adjacency matrices of spectral and spatial node interactions from the spectral and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance spectral and spatial feature learning; Step 4: Multi-scale cross-guided feature fusion (CGFF): In the multi-scale convolutional feature fusion sub-network, 2D pyramid convolution is used as the backbone network to extract multi-scale features. Convolution kernels of sizes 1×1, 3×3, and 5×5 are used to obtain feature maps of different scales. 、 and ;use Guide separately and , information interaction is achieved through element-by-element multiplication and dimension-by-dimension summation operations, fully integrating multi-scale features; Step 5: Weighted attention mechanism highlights important fusion features: extract the multi-scale feature map Linear map to key tensor Sum tensor , directly As a query tensor , query tensor With key tensor Perform inner product operation and obtain the attention score of channel feature map by scaling by proportional factor, apply softmax operation to channel feature map to generate channel attention map, channel attention map and value tensor Perform multiplication operations and use the softmax function to generate the weights of the multi-scale feature maps to obtain the weighted channel attention map with important feature expressions.

2. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 1 is characterized by: Step 1: Take the Pavia University data cube and represent it as ,in Represents the size of the space, Indicates the size of the spectral channel; Random cropping is performed and fed into DNIGAT-CFF for pixel-by-pixel ground cover classification.

3. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 2 is characterized by: Specifically, step 1 includes: DNIGAT-CFF is composed of a GAT-based subnetwork and a multi-scale convolutional feature fusion subnetwork; in the GAT-based subnetwork, two layers of coupled convolution blocks are used to extract differentiated spectral-spatial features; Subsequently, in order to enhance the learning of spectral and spatial features, the attention network of the interaction graph of nodes with different hop numbers is operated in a one-shot manner; specifically, the adjacency matrices of spectral and spatial node interactions are constructed from the spectral channels and spatial channels of the extracted differentiated spectral-spatial features, respectively, to help the one-shot DNIGAT module enhance the learning of spectral and spatial features; in the multi-scale convolutional feature fusion sub-network, considering the correlation and complementarity between features of different scales, a multi-scale cross-guided feature fusion module is designed to fully fuse multi-scale features; then, a weighted attention mechanism is applied to highlight the important features in the fused multi-scale features; finally, the output features based on the multi-scale feature fusion sub-network and the GAT sub-network are fused through an adaptive feature fusion method to generate features for hyperspectral image classification. ; The specific operation is shown in formula (1.1): in, is a hyperparameter used to control the feature weights of the GAT sub-network, and Control the feature weights of the feature fusion sub-network, both of which can adaptively adjust their sizes during training; and Represent the features extracted by the GAT-based sub-network and the feature fusion sub-network, respectively.

4. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 1 is characterized by: The mathematical expressions in step 2 are shown in formula (1.2) and formula (1.3): in, Indicates the The feature map of the layer, and is the convolution kernel, and is the bias term, Indicates batch normalization operation; Mish represents the non-monotonic self-regularization activation function, which improves the classification accuracy of the model. The detailed mathematical expression of Mish is shown in formula (1.4): in, and Represent the output and input of the mish function respectively, Represents a multiplication operation.

5. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 1 is characterized by: In step 3, the given graph ,in represents the binary adjacency matrix of the graph, Represents the feature matrix of the node, each node consists of a size of vector description; in addition, if , then it means the node With node There is a connection between them; remember the target node The neighborhood set of , indicating that all nodes directly connected nodes; in addition, nodes Itself is also included in its neighborhood set, forming an extended neighborhood set, denoted as ; To visually represent the nodes degrees, using the symbol To indicate that ; Correspondingly, the expansion is expressed as ; Models based on graph neural networks have received widespread attention in processing complex ground coverage scenarios in hyperspectral images. They learn the features of target nodes by aggregating the features of neighboring nodes. Among these models, the GAT-based model shows superior performance in feature extraction, as shown in formula (1.5): in, Indicates the target node In the The feature representation of the layer, is a learnable weight matrix that Perform linear transformation; Can be obtained from the initial feature matrix, Represents the comparable weight coefficients obtained by applying the attention mechanism and softmax function to the input node features. represents the activation function; The above function is usually performed as a weighted sum, where As a neighbor node However, the limitation of this weighted summation is that it cannot model the information interaction between adjacent node representations; the information interaction between the target node and its neighboring nodes is conducive to the expression of important features of the target node; To effectively enhance the feature representation of target nodes in spectral or spatial channels, a graph attention network with different hop counts of nodes is proposed. This network leverages the interactions between nodes with different hop counts, enabling the graph attention network to extract more critical spectral and spatial features from hyperspectral images. Unlike traditional multi-hop graph methods, a graph strategy with different hop counts is introduced. This strategy breaks the limitation of fixed hop counts by innovatively fusing two multi-hop structures with different hop counts, thereby achieving multi-level effective information interaction between nodes. It is worth noting that the use of adjacency matrices with different hop counts helps avoid the problem of node self-interaction. In addition, different hop counts contribute differently to the feature representation of target nodes. The detailed construction process of DNIGAT is as follows: Two adjacency matrices with different hop counts are constructed according to the spatial or spectral channels of the hyperspectral image: Hop adjacency matrix and Hop adjacency matrix ,in represents the batch size, Represents the spatial dimension or spectral dimension of the feature graph; in order to promote the interaction of nodes with the same hop and different hops of the target node, the following operations are applied to achieve node interaction between different hops; first, and By adding element by element, we can generate , which records the Jump and The information of the jump node is shown in formula (1.6): in, Represents a union operation; Then, a window multi-head self-attention mechanism is adopted to promote the interaction between nodes on the same hop and different hops from a local perspective; subsequently, a sliding window multi-head self-attention block is introduced to further promote the interaction between nodes on the same hop and different hops from a global perspective.

6. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 5 is characterized by: The step 3 specifically includes the following steps: 3.

1. Using WMS for local range node interaction: matrix After the deformation operation, a non-overlapping window partitioning method is used for uniform segmentation; (The area size is ) is evenly divided into multiple non-overlapping windows ,in (j=1,2,…, D / ),and Indicates the spatial size of the local window; then, in each divided sub-window A multi-head self-attention mechanism is applied on (j=1,2,…, D / ) to promote node interaction within each sub-window; the specific operation is as follows: Will As input to MHSA; then, Input linear layer and construct query matrix respectively , bond matrix Sum Matrix ; The specific operation is shown in formula (1.7): in, , and is the linear projection matrix, , and represents the learnable parameter matrix; subsequently, the attention calculation is performed on these three constructed matrices; in addition, the position mask is added in each attention head To mark the location information; the specific operations are shown in formula (1.8) and formula (1.9): in, represents the softmax function, express The dimension size, represents the number of heads in MHSA, Represents the multi-head self-attention mechanism; The operation of using WMS to interact with local nodes is shown in formula (1.10): in, express The result of the interaction between the local scope nodes, Represents the window multi-head self-attention mechanism; 3.

2. Use SMS for global node interaction: Considering that node interactions occur only in each Therefore, there is a lack of global node interaction between different windows; for this reason, Apply SMS operations to facilitate node interactions within the same hop count and between different hop counts from a global perspective; Therefore, in A sliding window operation is applied on The interaction between windows is introduced; then, Apply MHSA to promote node interaction in different windows; specifically, Input MHSA to linearly construct the query matrix , bond matrix Sum Matrix ; Subsequently, the attention calculation is performed based on these three linearly constructed matrices, as shown in formula (1.11): in, express The local-global scope node interaction results; These operations not only enable interactions between nodes within the same hop count, but also facilitate interactions between nodes with different hop counts; 3.3 One-Shot DNIGAT Module Based on the adjacency matrix of nodes with different hop counts, a DNIGAT model is designed to enhance the extraction of spectral and spatial features by utilizing the interactions between nodes with different hop counts. The detailed operation process of the graph attention layer based on the interactions between nodes with different hop counts is shown in formulas (1.12) and (1.13). in, Representation node In the The feature representation of the layer, and Both represent learnable weight matrices, represents the u-th layer, represents the activation function, Represents GAT based on the interaction of nodes with different hop counts, represents the Leaky ReLU function, Represents the shared attention mechanism; In order to further alleviate the over-smoothing problem caused by multiple stacked graph attention layers, a graph attention network GAT with multiple nodes interacting with different hop counts is adopted and constructed in a one-shot connection mode. The corresponding specific operations are shown in formula (1.14): in, Indicates channel-wise concatenation operation; here, Set to 3; finally, the dropout layer and ELU activation function are used to help alleviate overfitting and improve the network training process.

7. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 1 is characterized by: The step 4 is specifically as follows: using 2D pyramid convolution as the backbone network to extract multi-scale features; CGFF uses a 1×1 convolution kernel to replace the 7×7 convolution kernel, while keeping the sizes of other convolution kernels unchanged, focusing on extracting finer features; specifically, the feature map It is input into 2D PConv for feature extraction, and then feature maps of different scales are obtained. , and , each feature map has a specific receptive field; at the same time, 2D batch normalization operation and mish activation function are introduced in each 2D convolution layer; In order to fully integrate and utilize the extracted multi-scale features, CGFF first uses Guide separately and ; Then, and Combined, so as to maximize the complementarity and correlation between the features of each scale; the detailed fusion operation is shown in formula (1.15), formula (1.16) and formula (1.17): in, , and They are respectively for , and The constructed trainable transformation matrix; Based on the above operations, use respectively and To guide, and realize information interaction through element-by-element multiplication and dimension-by-dimension summation operations; the detailed mathematical operations are shown in formula (1.18) and formula (1.19): in, represents the sum operation by dimension, represents the matrix transformation operation, Represents a slicing operation acting on the spatial dimension, represents an element-wise multiplication operation; subsequently, and The combination is performed through element-by-element summation and processed sequentially through the Leaky ReLU (LReLU) function and the softmax() function, as shown in formula (1.20): Based on the above operations, and Integration is performed to achieve full fusion of each scale feature; the specific fusion operation is shown in formula (1.21): in, represents the ReLU activation function, express , and The result after fusion; After being processed by the multi-scale feature fusion module, the complementarity and correlation between feature maps of different scales are fully exploited, and feature maps with different receptive fields are fully fused, thereby retaining the diversity of features; in addition, the full fusion of multi-scale features enables more accurate object recognition in scenarios with complex object coverage in hyperspectral images.

8. The hyperspectral image classification method based on a cross-hop node interaction graph attention network according to claim 1 is characterized by: The step 5 is specifically as follows: first, the extracted multi-scale feature map Linear map to key tensor Sum tensor At the same time, it directly As a query tensor , without performing a linear mapping operation; querying the tensor With key tensor Perform inner product operation and pass the scaling factor Scaling is performed to obtain the attention score of the channel feature map, where for Then, a softmax operation is applied to these channel feature maps to generate Channel attention map of Perform multiplication to obtain the channel attention map after multiplication; next, in order to calculate the normalized weight of the channel attention map , using the softmax function; the softmax function is used to generate the weights of the multi-scale feature map, which helps to enhance the expression of important features while suppressing unimportant features; then, the channel attention map is combined with Perform multiplication operations to obtain a weighted channel attention map with important feature expressions; The weighted attention mechanism is used to highlight important features in the fused multi-scale features while suppressing unimportant features, thereby improving classification accuracy.

Images