Hyperspectral image classification method based on multi-feature collaborative attention dynamic hypergraph convolution

Abstract

The invention relates to a hyperspectral image classification method based on multi-feature collaborative attention dynamic hypergraph convolution, and the method comprises the steps: inputting a hyperspectral image into a convolution branch, and carrying out the multi-scale feature extraction, thereby obtaining a multi-scale feature graph, acquiring a global feature map and a local feature map of the hyperspectral image by using a global-local fusion module in a convolution branch, and generating pixel-level features according to the multi-scale feature map, the global feature map and the local feature map; constructing a space incidence matrix and a spectrum incidence matrix of superpixel nodes of the hyperspectral image by using the hypergraph branches, fusing to obtain a multi-view hypergraph, extracting static hypergraph features and dynamic hypergraph features through static hypergraph convolution and dynamic hypergraph convolution based on the multi-view hypergraph, and generating superpixel-level features after fusion; and carrying out weighted fusion on the pixel-level features and the super-pixel-level features, and outputting a classification result through a SoftMax layer according to a fusion result. Therefore, the complex high-order relation in the hyperspectral image can be effectively captured, and accurate classification of the hyperspectral image is realized.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution. Background Technology

[0002] Hyperspectral imaging (HSI) is an imaging technique that combines spectral and spatial information. Unlike traditional RGB images, it captures the spectral features of a target object across hundreds of consecutive narrow bands in the visible to near-infrared wavelength range. Hyperspectral images not only contain rich spatial positioning information but also record continuous, high-resolution spectral data. Therefore, HSI is widely used in various fields such as geological exploration, disaster early warning, agricultural monitoring, and medical diagnosis.

[0003] Hyperspectral Image Classification (HSIC) is the process of assigning each pixel in a hyperspectral image to a specific land cover category based on its features. HSIC has been a hot research topic in remote sensing. Early HSIC methods primarily relied on spectral feature matching (such as spectral angle mapping) and statistical models (such as maximum likelihood classification). These methods are simple and fast, but are significantly affected by human factors. With technological advancements, machine learning algorithms have also been applied to HSIC, achieving good results. Machine learning is an artificial intelligence technique that enhances statistical learning algorithms by incorporating domain expertise. The classification process involves extracting discriminative features from hyperspectral data using methods such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), band selection, and Gray-Level Co-occurrence Matrix (GLCM). These techniques effectively address the inherent redundancy and complexity of high-dimensional datasets. Subsequently, machine learning models, including Support Vector Machines (SVM), Random Forests (RF), and ensemble learning algorithms, were trained and used as classifiers to achieve optimal pattern recognition performance. However, as the number of spectral bands in hyperspectral data gradually increases, the nonlinear modeling ability of machine learning models is limited because they can only utilize shallow features and cannot accurately identify the mapping relationship between features and classes.

[0004] In recent years, deep learning algorithms have become a cutting-edge application and research hotspot in the field of HSIC (High-Speed ​​Interference Centering), greatly addressing the limitations of traditional machine learning models. Compared to shallow feature extraction, deep learning can progressively extract diverse and abstract structured features. After extracting discriminative features, deep learning utilizes the SoftMax function to achieve end-to-end HSIC. Convolutional Neural Networks (CNNs) extract local features through convolution operations, reducing the number of parameters and improving computational efficiency through weight sharing, making them the most widely used feature extraction method. Subsequently, various CNN architectures have been proposed to enhance spectral spatial representation capabilities, such as 2D-CNN, 3D-CNN, and DCNN. However, due to the limitations of fixed convolution operations, CNNs face certain challenges in HSIC. On the one hand, a fixed receptive field may limit the model's expressive power; on the other hand, CNNs cannot capture global information and cannot handle long-distance pixel dependencies.

[0005] Graph Neural Networks (GNNs) learn feature representations by utilizing information from neighboring nodes in a graph structure, enabling rapid information propagation between nodes and capturing long-range features. The initial graph-based HSIC method treated each pixel of the HSI as a graph node, resulting in very high computational costs. To address this, one solution is to use a mini-batch strategy, loading only a subset of nodes and edges for training in each iteration, thus avoiding full graph computation. However, this approach may ignore global context and lose complete graph information. Another approach is to optimize the graph structure, such as constructing sparse graphs or using superpixel nodes instead of pixel nodes. However, sparsification may lead to information loss, making it difficult to model long-range dependencies. Superpixel segmentation methods, which preserve basic structural information while overcoming the limitations of the original Euclidean data structure, are currently the dominant solution.

[0006] Both CNNs and GNNs are deep learning methods capable of capturing deep features. CNNs excel at handling local information and fine-grained features in images, but struggle with modeling global information. GCNs, on the other hand, can consider the global graph structure and capture complex relationships between nodes, but their computational cost is relatively high. Although superpixel-based graph methods significantly reduce computational complexity, the superpixel segmentation process inevitably leads to the loss of internal details, which may hinder the classification of fine-grained objects. By combining these two methods, hybrid networks can utilize both local features and graph structural information of images, thereby improving the ability to recognize complex patterns.

[0007] Although the above studies have demonstrated the effectiveness of hybrid networks, several problems still need to be addressed. For example, (1) land cover distributions in real-world scenarios often have high-order interaction relationships. However, previous graph-based studies have typically relied on simple graphs, i.e., using pairwise interaction relationships between low-order connections between nodes. This approach is insufficient to fully characterize the complex interactions between various land cover distributions, which seriously affects the rationality of feature representation; (2) the relationships between HSIs are often complex and dynamic, and traditional static graph structures cannot adequately capture these constantly changing relationships. Summary of the Invention

[0008] This application provides a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution to solve the problem that traditional hyperspectral image classification methods cannot effectively capture the complex high-order relationships in HSI, resulting in low classification accuracy.

[0009] This application provides a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution. The multi-feature collaborative attention dynamic hypergraph convolution includes a convolution branch and a hypergraph branch. The method includes the following steps: acquiring a hyperspectral image; inputting the hyperspectral image into the convolution branch for multi-scale feature extraction to obtain a multi-scale feature map, and using a global-local fusion module in the convolution branch to obtain a global feature map and a local feature map of the hyperspectral image, and generating pixel-level features based on the multi-scale feature map, the global feature map, and the local feature map; constructing a spatial correlation matrix and a spectral correlation matrix of the superpixel nodes of the hyperspectral image using the hypergraph branch, and fusing the spatial correlation matrix and the spectral correlation matrix to obtain a multi-view hypergraph; extracting static hypergraph features and dynamic hypergraph features based on the multi-view hypergraph through static hypergraph convolution and dynamic hypergraph convolution respectively, and fusing the static hypergraph features and the dynamic hypergraph features to generate superpixel-level features; weighted fusion based on the pixel-level features and the superpixel-level features, and outputting a classification result through a SoftMax layer based on the fusion result.

[0010] Optionally, the step of inputting the hyperspectral image into the convolutional branch for multi-scale feature extraction to obtain a multi-scale feature map includes: performing multi-scale feature extraction using the multi-scale feature extraction module in the convolutional branch, wherein the calculation formula of the multi-scale feature extraction module is:

[0011]

[0012] in, For output features, Given the input feature map, W1 and W2 are linear layers, and C... 3×3 It is a 3×3 depthwise convolutional layer. It is a 3×3 convolutional layer with a dilation rate of 2 and a depth of dilation. It is a 3×3 convolutional layer with a dilation rate of 3 and a depth of dilation.

[0013] Optionally, before extracting static and dynamic supergraph features based on the multi-view supergraph through static and dynamic supergraph convolutions respectively, the method includes: calculating an attention score map between superpixels using the superpixel attention module of the supergraph branch, and calculating an attention score map between feature maps using the feature attention module of the supergraph branch, so as to highlight superpixel features and feature maps based on the attention score map between superpixels and the attention score map between feature maps, and performing feature extraction on the multi-view supergraph based on the highlighted superpixel features and feature maps.

[0014] Optionally, the calculation formula for the superpixel attention module is:

[0015]

[0016] in, The output is the feature map, where α is a learnable scale parameter. To perform self-attention operations on superpixels, Let Q be the input superpixel, Q be the query matrix, and K be the input superpixel. T Let V be the transpose of the key matrix, and let V be the value matrix. For attention score graphs,

[0017] The calculation formula for the feature attention module is as follows:

[0018]

[0019] in, The output is the feature map, where μ is a learnable parameter. To perform self-attention operations on the feature map, E b For attention score graphs, This is the weighted feature map.

[0020] Optionally, the global-local fusion module is:

[0021]

[0022] in, For the input features of the i-th layer, For the output features of the i-th layer, C 1×1 For 1x1 convolution, BN(·) is a batch normalized layer, and φ(·) is the GELU nonlinear activation function.

[0023] Optionally, when extracting static and dynamic hypergraph features based on the multi-view hypergraph through static hypergraph convolution and dynamic hypergraph convolution, the process includes: transforming the multi-view hypergraph using a node convolution transformation mechanism and a hyperedge convolution transformation mechanism.

[0024] Optionally, the node convolution transformation mechanism is as follows:

[0025]

[0026]

[0027] in, For the node features of layer l, k l b is a trainable weight vector. l Let T be the bias, and T be the learnable transformation matrix. σ represents the super-edge features of the aggregation, and σ is a non-linear activation function.

[0028] Optionally, the super-edge convolution transformation mechanism is as follows:

[0029]

[0030]

[0031] in, For node v i All connected hyperedges, To learn the weights of hyperedges from hyperedges using a multilayer perceptron, F is related to node v i The weights of all connected hyperedges, For node v i The number of all connected superedges. For updated node features,

[0032] Optionally, before constructing the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image using the hypergraph branch, the method includes: clustering pixels with similar spatial and spectral features in the hyperspectral image into superpixels using a linear iterative clustering method.

[0033] Optionally, when generating pixel-level features based on the multi-scale feature map, the global feature map, and the local feature map, the process includes: adjusting the feature channels of the multi-scale feature map, the global feature map, and the local feature map using a preset convolutional layer; shuffling the feature channels and dividing the shuffled feature channels into multiple groups of feature channels to extract the spatial features of each group of feature channels; and concatenating the spatial features of each group of feature channels to obtain the pixel-level features based on the concatenation result.

[0034] A second aspect of this application provides a hyperspectral image classification system using multi-feature collaborative attention dynamic hypergraph convolution. The multi-feature collaborative attention dynamic hypergraph convolution includes a convolution branch and a hypergraph branch. The system includes: an acquisition module for acquiring a hyperspectral image; and a first generation module for inputting the hyperspectral image into the convolution branch for multi-scale feature extraction to obtain a multi-scale feature map, and using a global-local fusion module in the convolution branch to obtain a global feature map and local feature maps of the hyperspectral image, and generating an image based on the multi-scale feature map, the global feature map, and the local feature map. The system comprises: a pixel-level feature module; a second generation module, used to construct the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image using the hypergraph branch, and to fuse the spatial correlation matrix and the spectral correlation matrix to obtain a multi-view hypergraph; based on the multi-view hypergraph, extracting static hypergraph features and dynamic hypergraph features through static hypergraph convolution and dynamic hypergraph convolution respectively, and fusing the static hypergraph features and the dynamic hypergraph features to generate superpixel-level features; and a classification module, used to perform weighted fusion based on the pixel-level features and the superpixel-level features, and output the classification result through a SoftMax layer according to the fusion result.

[0035] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hyperspectral image classification method of multi-feature collaborative attention dynamic hypergraph convolution as described in the above embodiments.

[0036] A fourth aspect of this application provides a computer program product having a computer program stored thereon, which is executed by a processor to implement the hyperspectral image classification method of multi-feature collaborative attention dynamic hypergraph convolution as described in the above embodiments.

[0037] In the above implementation, the hyperspectral image is input into the convolutional branch for multi-scale feature extraction to obtain a multi-scale feature map. The global-local fusion module in the convolutional branch is used to obtain the global and local feature maps of the hyperspectral image. Pixel-level features are generated based on the multi-scale feature map, global feature map, and local feature map. The hypergraph branch is used to construct the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image, and these are fused to obtain a multi-view hypergraph. Based on the multi-view hypergraph, static and dynamic hypergraph features are extracted through static and dynamic hypergraph convolution, and then fused to generate superpixel-level features. The pixel-level features and superpixel-level features are weighted and fused, and the classification result is output through a SoftMax layer based on the fusion result. This solves the problem that traditional hyperspectral image classification methods cannot effectively capture the complex higher-order relationships in HSI, resulting in low classification accuracy. It can more effectively capture the complex higher-order relationships in HSI in different tasks, thereby enhancing the model's stability and adaptability, and achieving higher classification accuracy.

[0038] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0039] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0040] Figure 1 This is a flowchart of a hyperspectral image classification method based on multi-feature collaborative attention dynamic hypergraph convolution according to an embodiment of this application;

[0041] Figure 2 This is a flowchart of a hyperspectral image classification method based on a multi-feature collaborative attention dynamic hypergraph convolution according to an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of a hyperspectral image classification system based on multi-feature collaborative attention dynamic hypergraph convolution according to an embodiment of this application;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0044] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0045] The following describes a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution according to embodiments of this application, with reference to the accompanying drawings. Addressing the problem mentioned in the background art that traditional hyperspectral image classification methods cannot effectively capture complex high-order relationships in HSI, resulting in low classification accuracy, this application provides a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution. In this method, the hyperspectral image is input into a convolution branch for multi-scale feature extraction to obtain a multi-scale feature map. The global-local fusion module in the convolution branch is used to obtain the global and local feature maps of the hyperspectral image. Pixel-level features are generated based on the multi-scale feature map, global feature map, and local feature map. The spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image are constructed using a hypergraph branch and fused to obtain a multi-view hypergraph. Static and dynamic hypergraph features are extracted based on the multi-view hypergraph through static and dynamic hypergraph convolution, and fused to generate superpixel-level features. The pixel-level features and superpixel-level features are weighted and fused, and the classification result is output through a SoftMax layer based on the fusion result. This solves the problem that traditional hyperspectral image classification methods cannot effectively capture complex higher-order relationships in HSI, resulting in low classification accuracy. It can capture complex higher-order relationships in HSI more effectively in different tasks, thereby enhancing the stability and adaptability of the model and achieving higher classification accuracy.

[0046] Unlike simple graphs, which can only model pairwise relationships between nodes, hypergraphs use hyperedges to connect multiple nodes, allowing for the modeling of complex interactions. Therefore, hypergraphs represent an extension of simple graphs. Existing research has shown that hypergraphs are effective in capturing high-order relationships between nodes, which is crucial for complex data structures in many practical applications. Currently, research on hypergraph-based HSI classification is relatively limited. Therefore, this application proposes a novel Multi-Feature Collaborative Attention Dynamic Hypergraph Convolutional Network (MADHG) for accurate classification of hyperspectral images.

[0047] Specifically, inspired by the widespread application of pyramid convolution in object detection, this application introduces a multi-scale feature extraction module. This module utilizes more discriminative high- and low-level features to generate complementary feature representations for fine-grained HSIC. An attention mechanism captures long-range dependencies within HSI by computing relationships between input data elements, thereby enhancing the model's flexibility and expressiveness. Existing CNN-based HSIC methods have achieved great success by learning deep local spatial and spectral features. However, the analytical performance of HSI can be further improved by effectively considering and utilizing global features. Therefore, this application introduces a global-local fusion module combining CNN and a self-attention mechanism, considering both local and global information to improve classification performance.

[0048] To further explore higher-order interactions in HSI, a dynamic hypergraph convolutional neural network is constructed to achieve HSIC. Specifically, a multi-view hypergraph is built from spectral and spatial directions to more comprehensively represent the complex interactions in HSI.

[0049] Inspired by spatial and spectral attention mechanisms, a superpixel attention module and a feature attention module are proposed to compute the correlation between superpixels and their features, helping the supergraph focus on key regions and supplement missing detailed features.

[0050] Since static hypergraph convolutional neural networks (HCNNs) can stably model graph structures, while dynamic hypergraph HCNNs can flexibly adjust graph structures according to task requirements and integrate multiple information sources, a static-dynamic fusion hypergraph HCNN was designed to ensure more effective capture of complex high-order relationships in different tasks, thereby enhancing the model's stability and adaptability. Finally, a weighted method was used to fuse convolution and hypergraph feature extraction.

[0051] It should be noted that the specific implementation of this application is written in the PYTHON language and implemented using the classic deep learning framework PYTORCH.

[0052] Specifically, Figure 1 This is a flowchart illustrating a hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution provided in an embodiment of this application.

[0053] like Figure 1 As shown, this hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution includes the following steps:

[0054] Among them, the multi-feature collaborative attention dynamic hypergraph convolution includes a convolutional branch and a hypergraph branch.

[0055] In step S101, a hyperspectral image is acquired.

[0056] In step S102, the hyperspectral image is input into the convolution branch for multi-scale feature extraction to obtain a multi-scale feature map. The global-local fusion module in the convolution branch is used to obtain the global and local feature maps of the hyperspectral image. Pixel-level features are generated based on the multi-scale feature map, global feature map, and local feature map.

[0057] Optionally, in some embodiments, the hyperspectral image is input into a convolutional branch for multi-scale feature extraction to obtain a multi-scale feature map, including: using a multi-scale feature extraction module in the convolutional branch to perform multi-scale feature extraction, wherein the calculation formula of the multi-scale feature extraction module is:

[0058]

[0059] in, For output features, Given the input feature map, W1 and W2 are linear layers, and C... 3×3 It is a 3×3 depthwise convolutional layer. It is a 3×3 convolutional layer with a dilation rate of 2 and a depth of dilation. It is a 3×3 convolutional layer with a dilation rate of 3 and a depth of dilation.

[0060] Optionally, in some embodiments, the global-local fusion module is:

[0061]

[0062] in, For the input features of the i-th layer, For the output features of the i-th layer, C 1×1 For 1x1 convolution, BN(·) is a batch normalized layer, and φ(·) is the GELU nonlinear activation function.

[0063] Optionally, in some embodiments, when generating pixel-level features based on multi-scale feature maps, global feature maps, and local feature maps, the process includes: adjusting the feature channels of multi-scale feature maps, global feature maps, and local feature maps using a preset convolutional layer; shuffling the feature channels and dividing the shuffled feature channels into multiple groups of feature channels to extract the spatial features of each group of feature channels; and splicing the spatial features of each group of feature channels to obtain pixel-level features based on the splicing result.

[0064] Specifically, such as Figure 2 As shown, in the convolution branch, the input hyperspectral image (HSI) first undergoes a multi-scale feature extraction module to fully utilize the rich spectral features in the HSI and suppress noise. Then, the fused multi-scale features are restored to their original size and residually connected to the input HSI. Next, the features pass through a global-local fusion module to capture the long-range dependencies and neighborhood spectral correlations of the HSI, resulting in global and local feature maps of the hyperspectral image. After this, a 1×1 convolution kernel is used to extract spectral features, and the channel dimensions of the feature maps are adjusted. Then, a channel shuffling operation is performed, dividing the input features into multiple non-overlapping groups along the channel dimensions to enhance cross-channel interaction. Then, a 3×3 depthwise convolution is used within each group to extract spatial features. Finally, the results from each group are concatenated. Repeating this process twice yields the final pixel-level feature F extracted by the convolution branch. Convolution .

[0065] The multi-scale feature extraction module addresses the issue that HSI contains numerous spectral bands, and a single spectral feature may be insufficient to fully represent the information in an image. To capture features at different scales and enhance the network's feature representation capability and robustness, a multi-scale feature extraction module is designed, utilizing contextual information to enrich features. This module first expands the feature channels of the input HSI using linear layers. Then, features are processed on three parallel paths, and cascaded operations are introduced to enhance feature representation. This module uses multi-scale expanded convolutions to extract features at different scales. Specifically, three 3×3 expanded convolutions are used, with expansion rates of 1, 2, and 3, respectively. Finally, the cascaded features are passed through another linear layer to restore the original feature size for residual connection with the input. In detail, given a feature map... The calculation process of the multi-scale feature extraction module is shown in Equation (1).

[0066] Among them, the global-local fusion module: Convolutional operations are limited by the receptive field and local attention, which often leads to excessive attention to local features, thereby distorting the sequence information. In contrast, the self-attention mechanism can effectively model the sequence information by establishing the relationship between different pixels, enabling the network to fully learn global information. Therefore, a global-local fusion module composed of parallel convolution and self-attention mechanism is proposed to promote the simultaneous extraction of global and local features. In the global branch, a feature attention module is used to capture more global HSI features. In the local branch, two 1x1 convolutional layers are used for feature extraction and to transform and restore the feature channels respectively. The batch normalization layer accelerates training by adjusting the data distribution of features and reducing the parameter initialization requirements. The GELU activation function enhances the nonlinear transformation of features, as shown in Equation (2). Repeating this process twice yields the output of the local branch.

[0067] In step S103, the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image are constructed using the hypergraph branch, and the spatial correlation matrix and spectral correlation matrix are fused to obtain a multi-view hypergraph. Based on the multi-view hypergraph, static hypergraph features and dynamic hypergraph features are extracted by static hypergraph convolution and dynamic hypergraph convolution, respectively, and the static hypergraph features and dynamic hypergraph features are fused to generate superpixel-level features.

[0068] Optionally, in some embodiments, before constructing the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image using hypergraph branches, the method includes: clustering pixels with similar spatial and spectral features in the hyperspectral image into superpixels using a linear iterative clustering method.

[0069] Optionally, in some embodiments, before extracting static and dynamic supergraph features based on the multi-view supergraph through static and dynamic supergraph convolutions respectively, the method includes: using the superpixel attention module of the supergraph branch to calculate the attention score map between superpixels, and the feature attention module of the supergraph branch to calculate the attention score map between feature maps, so as to highlight the superpixel features and feature maps based on the attention score map between superpixels and the attention score map between feature maps, and to extract features from the multi-view supergraph based on the highlighted superpixel features and feature maps.

[0070] Optionally, in some embodiments, the calculation formula for the superpixel attention module is:

[0071]

[0072] in, The output feature map, where α is a learnable scaling parameter (used to control Q·K before applying the softmax function). T (size of the matrix) To perform self-attention operations on superpixels, Let Q be the input superpixel, Q be the query matrix, and K be the input superpixel. T Let V be the transpose of the key matrix, and let V be the value matrix. For attention score graphs,

[0073] The calculation formula for the feature attention module is:

[0074]

[0075] in, The output is the feature map, where μ is a learnable parameter. To perform self-attention operations on the feature map, E b For attention score graphs, This is the weighted feature map.

[0076] Optionally, in some embodiments, when extracting static and dynamic hypergraph features based on the multi-view hypergraph through static hypergraph convolution and dynamic hypergraph convolution respectively, the method includes: transforming the multi-view hypergraph using a node convolution transformation mechanism and a hyperedge convolution transformation mechanism.

[0077] Optionally, in some embodiments, the node convolution transformation mechanism is as follows:

[0078]

[0079]

[0080] in, For the node features of layer l, k l b is a trainable weight vector.l Let T be the bias, and T be the learnable transformation matrix. σ represents the super-edge features of the aggregation, and σ is a non-linear activation function.

[0081] Optionally, in some embodiments, the superedge convolution transformation mechanism is as follows:

[0082]

[0083]

[0084] in, For node v i All connected hyperedges, To learn the weights of hyperedges from hyperedges using a multilayer perceptron, F is related to node v i The weights of all connected hyperedges, For node v i The number of all connected superedges. For updated node features,

[0085] Specifically, such as Figure 2 As shown, in the hypergraph branch, the Simple Linear Iterative Clustering (SLIC) method is first executed to divide pixels into superpixel blocks S∈{1,2,...,Z}, where Z is the number of superpixels. Then, the spatial correlation matrix H between superpixel nodes is constructed from both the spatial and spectral directions. spa and spectral correlation matrix H spe .

[0086] H spa and H spe The multi-view hypergraph G is obtained by addition and fusion. multiview Multi-view hypergraphs, by simultaneously considering spectral and spatial distances, can better represent the nonlinear higher-order relationships in HSI. Then, superpixel nodes are sequentially passed through a superpixel attention module and a feature attention module to better utilize the correlation between superpixels and features.

[0087] To avoid oversmoothing issues, most current GCN models are not very deep. Therefore, this application uses only two hypergraph convolution layers, combining static and dynamic hypergraphs in parallel. Specifically, the network extracts static and dynamic hypergraph features through one static hypergraph convolution layer and one dynamic hypergraph convolution layer, and then fuses the extracted static and dynamic hypergraph features to obtain the final extracted superpixel features F. Hypergraph .

[0088] The self-attention mechanism, by calculating the correlation between elements in the input features, allows the model to consider global information when processing each element, while highlighting more important information.

[0089] This application introduces a self-attention mechanism to construct two attention modules: a superpixel attention module and a feature attention module. Some superpixel regions may contain more important information, while others may be background or noise. The superpixel attention module can weight different superpixels, helping the model to better focus on key regions in the HSI and suppress interference from irrelevant information. The feature attention module emphasizes the interdependencies between feature maps, allowing the model to weight different features according to their importance, highlighting important features and suppressing unimportant features.

[0090] The superpixel attention module uses an attention mechanism to flexibly learn important information between superpixels, assigning weights to each superpixel instead of treating all superpixels equally as in traditional graph algorithms. It directly computes the attention score map between each superpixel from the input superpixel features. Specifically, given a superpixel's feature map... This module first performs tensor operations to reshape the feature dimensions for subsequent computation. Then, it initializes three transformation matrices W. q and W k With W v The input features are multiplied to obtain Q, K, and V, which are then used for attention calculation, where K is transposed as... Finally, the attention features obtained are added to the input features to obtain the final output result. The output of the superpixel attention module is shown in Equation (3).

[0091] The feature attention module employs a self-attention mechanism to capture the dependencies between different feature maps, assigning appropriate weights to each feature map. This module directly computes the attention score map between each feature map from the input features. Then, the obtained attention features are added to the input features to obtain the final features. Specifically, given a feature map... Directly on input features Attention calculation is performed, and the feature attention map is subjected to max pooling with a kernel size of 1. According to equation (9), the calculation formula (4) for the feature attention module is obtained.

[0092]

[0093] in, Feature attention map measures the influence of feature map i on feature map j. MaxPool(·) represents max pooling along the feature channel dimension.

[0094] Pixel-to-superpixel transformation: Directly treating the pixels of HSI as nodes in the graph would result in an excessively large graph size and overly complex calculations in subsequent steps. Therefore, the Simple Linear Iterative Clustering (SLIC) method is used to segment the HSI into superpixels S∈{1,2,...,Z}, where Z=(H×W) / λ, and λ is the superpixel segmentation scale.

[0095] Specifically, SLIC uses the K-means clustering algorithm to cluster pixels with similar spatial and spectral information into superpixels. These superpixels are then used as nodes in the graph to reduce computational complexity, where the feature of each node is the mean of the features of all pixels within the corresponding superpixel. Since the features output by the convolutional branches and the final generated predicted labels are both pixel-level data, a correlation matrix between pixels and superpixels is used. This allows for the conversion between pixels and superpixels.

[0096]

[0097] in, This represents the flattened HSI data. express The i-th pixel in S j Let X represent the j-th superpixel, and Flatten(X) is the flattening operation performed on the input HSI data.

[0098] Then, the conversion between pixels and superpixels can be achieved through an encoder and a decoder, as shown below:

[0099]

[0100] X new =Decoder(S;A)=Reshape(AS); (12)

[0101] in, Let A be the column normalization, S be the feature matrix of all superpixel nodes, and Reshape(·) represent restoring the spatial dimension of the flat data. new This represents the pixel features obtained from superpixel features. Encoder(X;A) is the encoder that converts pixels into superpixels, and Decoder(S;A) is the decoder that converts superpixels into pixels.

[0102] Hypergraph Construction: In this application, an encoder can be used to easily encode HSI into graph nodes, with each node corresponding to a superpixel in the HSI. A hypergraph structure can be constructed by considering the similarity or relationship between input nodes. Traditional HSI hypergraph construction uses the k-nearest neighbor (KNN) algorithm to generate spectral feature hyperedges, but often ignores the inherent spatial features of the segmented region. Therefore, for a given node, based on the spatial Euclidean distance and spectral Euclidean distance between nodes, the k nearest nodes are selected to form a hyperedge e that includes the node itself. spa and e spe This process continues until all nodes have been traversed. This generates two types of hyperedges: E spa and E spe Then, based on these hyperedges, the spatial correlation matrix H was constructed. spa and spectral correlation matrix H spe The spatial hypergraph G was obtained. spa and spectral hypergraph G spe To better represent the nonlinear higher-order relations in HSI and reduce the computational complexity of the method, a weighted fusion is used to combine the two hypergraphs to obtain the multiview hypergraph G. multiview .

[0103] Hypergraph Convolution: After constructing the hypergraph, features are propagated through hypergraph convolution. Specifically, hypergraph convolution includes two steps: aggregating node features onto hyperedges to construct hyperedge features; and propagating the aggregated hyperedge features back to the nodes to complete the node feature update. Given node features... The process of clustering node features onto a hyperedge is described as follows: The process of propagating hyperedge features back to nodes is described as follows: Therefore, the propagation of information on the constructed hypergraph can be represented as:

[0104]

[0105] in, Represents the node features of the l-th layer. This represents the updated node characteristics. Let Θ represent the aggregated hyperedge features of the l-th layer, and W be the weight matrix of the hyperedge. For simplicity, this application sets W as the identity matrix. (l) σ represents the learnable parameters of the hypergraph convolutional layer, and σ represents the nonlinear activation function.

[0106] Dynamic Hypergraph Convolutional Neural Network: Most graph / hypergraph-based neural networks only use the initial graph / hypergraph structure, ignoring the transformations that occur after features pass through hidden layers, and the structures corresponding to these transformed features may also change. To address this issue, this application employs a Dynamic Hypergraph Convolutional Neural Network (DHGCN) to dynamically adjust the hypergraph structure.

[0107] DHGCN also consists of two parts: hypergraph construction and hypergraph convolution. This method also utilizes the KNN algorithm to construct a dynamic hypergraph. Relationships between nodes are mined by calculating the Euclidean distance between each pair of nodes and the cosine similarity between their features. Therefore, two correlation matrices H are obtained. loc and H fea In HGCN, the node-hyperedge-node interactions in hypergraph convolution are essentially a simple feature summation, assuming all nodes have equal importance. This can lead to an overestimation of the importance of irrelevant nodes. Dynamic hypergraph convolution employs node-level and hyperedge-level convolution transformation mechanisms to transform input features, dynamically extracting more complex higher-order features.

[0108] Node-based convolution transformation mechanism: The neighborhood of a node is similar to the pixel neighborhood in an image, often considered a form of local structure. Depthwise convolution can effectively capture this local spatial relationship. Therefore, node-based convolution performs a depthwise convolution operation on the input node features to obtain a learnable transformation matrix. The transformation matrix is ​​then normalized using the softmax activation function. Given the input node features... As shown in Equation (5), the transformation matrix is ​​then multiplied by the input node features to update the features of each node. Subsequently, the convolutional layer aggregates the node features after weighting the neighbors onto each hyperedge to obtain the hidden layer features of each hyperedge, as shown in Equation (6).

[0109] Hyperedge convolution transformation mechanism: Hyperedges vary greatly in structure and size, and the correlation between hyperedges may depend on information from the entire graph. MLPs can directly model global dependencies through fully connected structures, while convolution focuses more on local relationships. Hyperedge convolution uses MLPs to model global dependencies connected to node v. i Each hyperedge is assigned a weight as shown in Equation (7). Then, based on the calculated weights, the features of each hyperedge are aggregated onto the nodes to obtain the updated node features. As shown in equation (8).

[0110] In step S104, the pixel-level features and superpixel-level features are weighted and fused, and the classification result is output through the SoftMax layer according to the fusion result.

[0111] Specifically, pixel-level features F Convolution and superpixel-level features F Hypergraph Weighted fusion uses a multilayer perceptron to adjust the channel length of the feature vector to match the number of classes, and then passes it through a SoftMax layer to obtain the classification result of the hyperspectral image.

[0112] The following comparative experiments will verify the beneficial effects of this application:

[0113] The data used in this experiment are the commonly used University of Pavia dataset, Indian Pine dataset, and Salinas dataset.

[0114] The University of Pavia dataset was collected by a reflectance optical spectral imaging system sensor. This spectrometer captured continuous images in 115 bands within the wavelength range of 430–860 nm, with a spatial resolution of 1.3 meters. After removing 12 noise and water vapor absorption bands, the final University of Pavia dataset consists of 103 spectral bands. The dataset size is 610 × 340, containing a total of 207,400 pixels. Of these, 42,776 pixels have basic ground truth labels, divided into 9 different categories. Details of each category and the dataset partitioning are shown in Table 1.

[0115] The Indian Pine dataset was acquired using the Airborne Visible Infrared Imaging Spectrometer (AVIRIS). This spectrometer captured continuous images across 220 bands in the 400–2500 nm wavelength range with a spatial resolution of 20 meters. After removing 20 noise bands and water vapor absorption bands, the final Indian Pine dataset consists of 200 spectral bands. The dataset is 145 × 145 pixels in size and contains a total of 21,025 pixels. Of these, 10,249 pixels are labeled into 16 different categories; Table 2 provides details for each category and dataset partitions.

[0116] The Salinas dataset was collected by the AVIRIS sensor. This spectrometer captured continuous images in 224 bands within the wavelength range of 400–2500 nm, with a spatial resolution of 3.7 meters. After removing 20 noise bands and water vapor absorption bands, the final dataset consists of 204 spectral bands. The dataset is 512 × 217 pixels in size, comprising 54,129 pixels, and is divided into 16 categories for research and analysis. Table 3 provides detailed information on each category and dataset partitions.

[0117] 2D-CNN, GCN, CEGCN (CNN-enhanced Graph Convolutional Network), WFCG (Weighted Feature Fusion of CNN and Graph Attention Network), AMGCFN (Attention Multihop Graph and Multiscale Convolutional Fusion Network), DF2Net (Differential Feature Fusion Network), HGCN-MHF (Hypergraph Convolutional Network with Multiple Hyperedges Fusion), and this application were used for hyperspectral image classification.

[0118] Table 1

[0119]

[0120] Table 2

[0121]

[0122]

[0123] Table 3

[0124]

[0125]

[0126] Classification evaluation indicators: To quantitatively test the performance of the model, four indicators were selected: overall accuracy (OA), partial accuracy (PA), average accuracy (AA), and Kappa coefficient. All experimental results were independently repeated 10 times and then the average value was taken to reduce the influence of random factors.

[0127] (1) Overall accuracy

[0128] Overall precision refers to the number of samples that are correctly classified. The percentage of the total number of samples N in the test set represents the overall accuracy. The higher the overall accuracy, the better the classification performance of the model. The calculation formula is as follows:

[0129]

[0130] (2) Category precision

[0131] Category precision refers to the precision of each category in the classification, which is calculated by multiplying the number of correctly classified samples by the number of samples in each category by X. ii This represents the total number of samples of this type in the test set. The percentage represents the accuracy of the classification of a class. The higher the class accuracy, the higher the classification accuracy of the model in that class. The calculation formula is as follows:

[0132]

[0133] (3) Average accuracy

[0134] Average precision refers to the sum of category precisions. Dividing by the total number of classes n, mean precision reflects the stability of the model. In imbalanced classification tasks, mean precision often better reflects the true performance of the model, and its calculation formula is as follows:

[0135]

[0136] (4) Kappa coefficient

[0137] The Kappa coefficient is a consistency test index used to calculate the agreement rate between predicted results and actual ground features. The calculated Kappa value is between -1 and 1, but typically falls between 0 and 1. A larger Kappa value indicates stronger consistency. The specific calculation formula is as follows:

[0138]

[0139] in, and Consistent, both indicating that category i is the predicted label.

[0140] Table 4 shows the comparison results of various methods on the University of Pavia dataset, including class accuracy, OA, AA, and Kappa. Table 5 shows the comparison results of various methods on the Indian Pine dataset, including class accuracy, OA, AA, and Kappa. Table 6 shows the comparison results of various methods on the Salinas dataset, including class accuracy, OA, AA, and Kappa.

[0141] Table 4

[0142]

[0143]

[0144] Table 5

[0145]

[0146]

[0147] Table 6

[0148]

[0149]

[0150] As can be seen from Table 4-6, this application can obtain higher OA, AA and Kappa scores, indicating that this application has stronger classification ability.

[0151] Therefore, it can be concluded that, compared with traditional hyperspectral image classification methods, this application can flexibly mine higher-order interrelationships of HSI and has higher classification accuracy. Multi-scale features are integrated through a multi-scale feature extraction module and a global-local fusion module, enabling simultaneous extraction of global and local features. Furthermore, a static-dynamic multi-view hypergraph neural network is constructed to dynamically explore higher-order topological information of HSI, improving the network's feature extraction capability. Finally, by fusing features from different levels, the network's classification performance and robustness are enhanced.

[0152] The hyperspectral image classification method based on multi-feature collaborative attention dynamic hypergraph convolution proposed in this application involves inputting the hyperspectral image into a convolution branch for multi-scale feature extraction to obtain a multi-scale feature map. The global-local fusion module within the convolution branch is used to acquire the global and local feature maps of the hyperspectral image. Pixel-level features are generated based on the multi-scale, global, and local feature maps. The hypergraph branch is used to construct the spatial and spectral correlation matrices of the hyperspectral image's superpixel nodes, which are then fused to obtain a multi-view hypergraph. Static and dynamic hypergraph features are extracted based on the multi-view hypergraph through static and dynamic hypergraph convolution, and then fused to generate superpixel-level features. The pixel-level and superpixel-level features are weighted and fused, and the classification result is output through a SoftMax layer based on the fusion result. This solves the problem that traditional hyperspectral image classification methods cannot effectively capture complex higher-order relationships in HSI, leading to low classification accuracy. This method can more effectively capture complex higher-order relationships in HSI across different tasks, thereby enhancing the model's stability and adaptability, and achieving higher classification accuracy.

[0153] Next, referring to the accompanying drawings, a hyperspectral image classification system based on multi-feature collaborative attention dynamic hypergraph convolution proposed in this application is described.

[0154] Figure 3This is a block diagram of a hyperspectral image classification system with multi-feature collaborative attention dynamic hypergraph convolution according to an embodiment of this application.

[0155] like Figure 3 As shown, the hyperspectral image classification system 10 with multi-feature collaborative attention dynamic hypergraph convolution includes: an acquisition module 100, a first generation module 200, a second generation module 300, and a classification module 400.

[0156] The system comprises the following modules: an acquisition module 100 for acquiring a hyperspectral image; a first generation module 200 for inputting the hyperspectral image into a convolutional branch for multi-scale feature extraction to obtain a multi-scale feature map, and using a global-local fusion module in the convolutional branch to obtain global and local feature maps of the hyperspectral image, and generating pixel-level features based on the multi-scale feature map, global feature map, and local feature map; a second generation module 300 for constructing a spatial correlation matrix and a spectral correlation matrix of the superpixel nodes of the hyperspectral image using a hypergraph branch, fusing the spatial and spectral correlation matrices to obtain a multi-view hypergraph, extracting static and dynamic hypergraph features based on the multi-view hypergraph through static and dynamic hypergraph convolution, and fusing the static and dynamic hypergraph features to generate superpixel-level features; and a classification module 400 for weighted fusion of pixel-level features and superpixel-level features, and outputting classification results through a SoftMax layer based on the fusion result.

[0157] Optionally, in some embodiments, the first generation module 200 is further configured to: perform multi-scale feature extraction using a multi-scale feature extraction module in the convolution branch, wherein the calculation formula for the multi-scale feature extraction module is:

[0158]

[0159] in, For output features, Given the input feature map, W1 and W2 are linear layers, and C... 3×3 It is a 3×3 depthwise convolutional layer. It is a 3×3 convolutional layer with a dilation rate of 2 and a depth of dilation. It is a 3×3 convolutional layer with a dilation rate of 3 and a depth of dilation.

[0160] Optionally, in some embodiments, before extracting static and dynamic supergraph features based on the multi-view supergraph through static and dynamic supergraph convolutions respectively, the second generation module 300 is further configured to: calculate an attention score map between superpixels using the superpixel attention module of the supergraph branch, and calculate an attention score map between feature maps using the feature attention module of the supergraph branch, so as to highlight superpixel features and feature maps based on the attention score map between superpixels and the attention score map between feature maps, and to perform feature extraction on the multi-view supergraph based on the highlighted superpixel features and feature maps.

[0161] Optionally, in some embodiments, the calculation formula for the superpixel attention module is:

[0162]

[0163] in, The output is the feature map, where α is a learnable scale parameter. To perform self-attention operations on superpixels, Let Q be the input superpixel, Q be the query matrix, and K be the input superpixel. T Let V be the transpose of the key matrix, and let V be the value matrix. For attention score graphs,

[0164] The calculation formula for the feature attention module is:

[0165]

[0166] in, The output is the feature map, where μ is a learnable parameter. To perform self-attention operations on the feature map, E b For attention score graphs, This is the weighted feature map.

[0167] Optionally, in some embodiments, the global-local fusion module is:

[0168]

[0169] in, For the input features of the i-th layer, For the output features of the i-th layer, C 1×1 For 1x1 convolution, BN(·) is a batch normalized layer, and φ(·) is the GELU nonlinear activation function.

[0170] Optionally, in some embodiments, when extracting static and dynamic hypergraph features based on the multi-view hypergraph through static hypergraph convolution and dynamic hypergraph convolution respectively, the second generation module 300 is further configured to: transform the multi-view hypergraph using a node convolution transformation mechanism and a hyperedge convolution transformation mechanism.

[0171] Optionally, in some embodiments, the node convolution transformation mechanism is as follows:

[0172]

[0173]

[0174] in, For the node features of layer l, k l b is a trainable weight vector. l Let T be the bias, and T be the learnable transformation matrix. σ represents the super-edge features of the aggregation, and σ is a non-linear activation function.

[0175] Optionally, in some embodiments, the superedge convolution transformation mechanism is as follows:

[0176]

[0177]

[0178] in, For node v i All connected hyperedges, To learn the weights of hyperedges from hyperedges using a multilayer perceptron, F is related to node v i The weights of all connected hyperedges, For node v i The number of all connected superedges. For updated node features,

[0179] Optionally, in some embodiments, before constructing the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image using the hypergraph branch, the second generation module 300 is further configured to: cluster pixels with similar spatial and spectral features in the hyperspectral image into superpixels using a linear iterative clustering method.

[0180] Optionally, in some embodiments, when generating pixel-level features based on multi-scale feature maps, global feature maps, and local feature maps, the first generation module 200 is further configured to: adjust the feature channels of multi-scale feature maps, global feature maps, and local feature maps using preset convolutional layers; shuffle the feature channels and divide the shuffled feature channels into multiple groups of feature channels to extract the spatial features of each group of feature channels; and splice the spatial features of each group of feature channels to obtain pixel-level features based on the splicing result.

[0181] It should be noted that the explanation of the above-described embodiment of the hyperspectral image classification method with multi-feature collaborative attention dynamic hypergraph convolution also applies to the hyperspectral image classification system with multi-feature collaborative attention dynamic hypergraph convolution in this embodiment, and will not be repeated here.

[0182] The hyperspectral image classification system based on the multi-feature collaborative attention dynamic hypergraph convolution proposed in this application involves inputting the hyperspectral image into a convolution branch for multi-scale feature extraction to obtain a multi-scale feature map. The global-local fusion module within the convolution branch acquires the global and local feature maps of the hyperspectral image. Pixel-level features are generated based on the multi-scale, global, and local feature maps. The hypergraph branch constructs the spatial and spectral correlation matrices of the hyperspectral image's superpixel nodes and fuses them to obtain a multi-view hypergraph. Static and dynamic hypergraph features are extracted based on the multi-view hypergraph through static and dynamic hypergraph convolution, and then fused to generate superpixel-level features. The pixel-level and superpixel-level features are weighted and fused, and the classification result is output through a SoftMax layer based on the fusion result. This solves the problem of traditional hyperspectral image classification methods failing to effectively capture complex higher-order relationships in HSI, leading to low classification accuracy. The system can more effectively capture complex higher-order relationships in HSI across different tasks, thereby enhancing the model's stability and adaptability, and achieving higher classification accuracy.

[0183] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0184] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0185] When the processor 402 executes the program, it implements the hyperspectral image classification method with multi-feature collaborative attention dynamic hypergraph convolution provided in the above embodiments.

[0186] Furthermore, electronic devices also include:

[0187] Communication interface 403 is used for communication between memory 401 and processor 402.

[0188] The memory 401 is used to store computer programs that can run on the processor 402.

[0189] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0190] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0191] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0192] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0193] This application also provides a computer program product on which a computer program is stored, which, when executed by a processor, implements the hyperspectral image classification method of multi-feature collaborative attention dynamic hypergraph convolution as described above.

[0194] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0195] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0196] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0197] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be specifically implemented in any computer program product for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer program product" can be any means that can contain, store, communicate, propagate, or transmit a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer program products include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, the computer program product can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0198] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0199] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer program product, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0200] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer program product.

[0201] The computer program product mentioned above may be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A hyperspectral image classification method using multi-feature collaborative attention dynamic hypergraph convolution, characterized in that, The multi-feature collaborative attention dynamic hypergraph convolution includes a convolutional branch and a hypergraph branch, and includes the following steps: Acquire hyperspectral images; The hyperspectral image is input into the convolution branch for multi-scale feature extraction to obtain a multi-scale feature map. The global and local feature maps of the hyperspectral image are obtained by using the global-local fusion module in the convolution branch. Pixel-level features are generated based on the multi-scale feature map, the global feature map, and the local feature map. The spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image are constructed using the hypergraph branches, and the spatial correlation matrix and the spectral correlation matrix are fused to obtain a multi-view hypergraph. Based on the multi-view hypergraph, static hypergraph features and dynamic hypergraph features are extracted by static hypergraph convolution and dynamic hypergraph convolution, respectively, and the static hypergraph features and the dynamic hypergraph features are fused to generate superpixel-level features. The pixel-level features and superpixel-level features are weighted and fused, and the classification result is output through the SoftMax layer based on the fusion result.

2. The method according to claim 1, characterized in that, The step of inputting the hyperspectral image into the convolutional branch for multi-scale feature extraction to obtain a multi-scale feature map includes: Multi-scale feature extraction is performed using the multi-scale feature extraction module in the convolution branch. The calculation formula for the multi-scale feature extraction module is as follows: in, For output features, Given the input feature map, W1 and W2 are linear layers, and C... 3×3 It is a 3×3 depthwise convolutional layer. It is a 3×3 convolutional layer with a dilation rate of 2 and a depth of dilation. It is a 3×3 convolutional layer with a dilation rate of 3 and a depth of dilation.

3. The method according to claim 1, characterized in that, Before extracting static and dynamic hypergraph features based on the multi-view hypergraph using static and dynamic hypergraph convolution respectively, the process includes: The superpixel attention module of the supergraph branch is used to calculate the attention score map between superpixels, and the feature attention module of the supergraph branch is used to calculate the attention score map between feature maps. The superpixel features and feature maps are highlighted based on the attention score map between superpixels and the attention score map between feature maps. Feature extraction is performed on the multiview supergraph based on the highlighted superpixel features and feature maps.

4. The method according to claim 3, characterized in that, The calculation formula for the superpixel attention module is as follows: in, The output is the feature map, where α is a learnable scale parameter. To perform self-attention mechanism operations on superpixels, Let K be the input superpixel, Q be the query matrix, and K be the input superpixel. T Let V be the transpose of the key matrix, and let V be the value matrix. For attention score graphs, The calculation formula for the feature attention module is as follows: in, The output is the feature map, where μ is a learnable parameter. To perform self-attention operations on features, E b For attention graph scoring graph, This is the weighted feature map.

5. The method according to claim 1, characterized in that, The global-local fusion module is: in, For the input features of the i-th layer, For the output features of the i-th layer, C 1×1 For 1x1 convolution, BN(·) is a batch normalized layer, and φ(·) is the GELU nonlinear activation function.

6. The method according to claim 1, characterized in that, When extracting static hypergraph features and dynamic hypergraph features based on the multi-view hypergraph using static hypergraph convolution and dynamic hypergraph convolution respectively, the following steps are included: The multi-view hypergraph is transformed using node convolution transformation mechanism and hyperedge convolution transformation mechanism.

7. The method according to claim 6, characterized in that, The node convolution transformation mechanism is as follows: in, For the node features of layer l, k l b is a trainable weight vector. l Let T be the bias, and T be the learnable transformation matrix. σ represents the super-edge features of the aggregation, and σ is a non-linear activation function.

8. The method according to claim 6, characterized in that, The hyperedge convolution transformation mechanism is as follows: in, For node v i All connected hyperedges, To learn the weights of hyperedges from hyperedges using a multilayer perceptron, F is related to node v i The weights of all connected hyperedges, For node v i The number of all connected superedges. For updated node features, 9. The method according to claim 6, characterized in that, Before constructing the spatial correlation matrix and spectral correlation matrix of the superpixel nodes of the hyperspectral image using the hypergraph branches, the process includes: Pixels with similar spatial and spectral features in the hyperspectral image are clustered into superpixels using a linear iterative clustering method.

10. The method according to claim 1, characterized in that, When generating pixel-level features based on the multi-scale feature map, the global feature map, and the local feature map, the process includes: The feature channels of the multi-scale feature map, the global feature map, and the local feature map are adjusted using preset convolutional layers; The feature channels are shuffled and then divided into multiple groups of feature channels to extract the spatial features of each group of feature channels. The spatial features of each set of feature channels are spliced ​​together to obtain the pixel-level features based on the splicing results.

Images