Hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative network

Abstract

This invention discloses a hyperspectral remote sensing image classification method based on a multipath spectral spatial collaborative network, belonging to the field of image classification technology. The method includes: acquiring the hyperspectral remote sensing image to be processed and constructing a feature tensor; inputting the feature tensor into a multipath spectral spatial collaborative network, performing spectral-spatial collaborative enhancement on the feature tensor using a two-dimensional attention mechanism to obtain spectral-spatial collaborative enhancement features; extracting multi-scale local features from the feature tensor to obtain multi-scale context extraction features; performing superpixel segmentation and graph convolution aggregation on the feature tensor to obtain superpixel feature aggregation features; and adaptively fusing the spectral-spatial collaborative enhancement features, multi-scale context extraction features, and superpixel feature aggregation features to obtain the classification result of the hyperspectral remote sensing image to be processed. This invention can flexibly adjust the contribution ratio of features from each path, significantly improving the model's classification robustness and generalization ability in complex scenes.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network. Background Technology

[0002] The "image-spectral integration" characteristic of hyperspectral remote sensing images provides a unique advantage for the detailed identification of ground features, and has important application value in fields such as precision agriculture, environmental monitoring, and geological exploration. Hyperspectral remote sensing image classification is a fundamental task in these applications, and its core challenge lies in how to effectively extract spectral-spatial feature information and adaptively fuse it.

[0003] In existing technologies, some researchers have designed a CNN-GCN ensemble framework, which constructs two parallel paths: a CNN path and a GCN path. The CNN path uses 2D or 3D convolutional layers to extract local spectral-spatial features from hyperspectral images, obtaining local feature maps. The GCN path first performs superpixel segmentation on the image, constructing a superpixel node map; then, it uses a graph convolutional network to propagate and aggregate features on the superpixel map to model long-range spatial dependencies between land features, obtaining global topological features. The local feature map and global topological features are statically fused into a single feature vector, and the fused feature is input into a fully connected layer or classifier to obtain the land feature category probability.

[0004] However, when faced with different land cover distributions, spectral feature differences, or sample scarcity, this existing technology cannot dynamically optimize the contribution ratio of each path feature, making it difficult to fully explore the complementary advantages of multipath heterogeneous features, thus affecting the robustness and generalization ability of classification performance. Summary of the Invention

[0005] Therefore, it is necessary to provide a hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks to address the aforementioned technical problems.

[0006] This invention provides a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network, comprising: Acquire the hyperspectral remote sensing image to be processed, and construct a feature tensor based on the spatial height, spatial width, and number of spectral bands of the hyperspectral remote sensing image to be processed; The feature tensor is input into the multipath spectral spatial collaborative network, which includes a feature extraction module and a meta-weighted feature fusion module connected in sequence. The feature extraction module includes a parallel spectral spatial collaborative enhancement module, a multi-scale context extraction module, and a superpixel feature aggregation module. The spectral-spatial co-enhancement module performs a two-dimensional attention mechanism on the feature tensor to extract discriminative features, resulting in spectral-spatial co-enhancement features. Multi-scale local feature extraction is performed on the feature tensor by a multi-scale context extraction module to capture spatial context information under different receptive fields and obtain multi-scale context extraction features. The superpixel feature aggregation module performs superpixel segmentation and graph convolution aggregation on the feature tensor to model the long-range spatial dependency between superpixels and aggregate region-level features to obtain superpixel feature aggregation features. The meta-weighted feature fusion module adaptively fuses spectral-spatial co-enhanced features, multi-scale contextual extraction features, and superpixel feature aggregation features to dynamically integrate heterogeneous features through a meta-learning strategy, thereby obtaining the classification results of the hyperspectral remote sensing image to be processed.

[0007] Optionally, the feature tensor is enhanced using a two-dimensional attention mechanism via a spectral-spatial co-enhancement module to extract discriminative features, resulting in spectral-spatial co-enhancement features, specifically including: The spectral spatial collaborative enhancement module includes: a feature enhancement module, a spatial-spectral attention module, and a first multi-scale convolution module connected in sequence. The spatial-spectral attention module includes: a parallel spectral attention branch and a spatial attention branch. Based on the following formula, the feature enhancement module uses global average pooling and global max pooling operations to extract features from the feature tensor, thus obtaining the spectral attention weights: ; The enhanced hyperspectral features are obtained by optimizing the feature tensor using spectral attention weights based on the following formula: ; The enhanced hyperspectral features are grouped based on spectral similarity to obtain multiple feature groups; For each feature group, channel dependencies are extracted using the spectral attention branch based on the following formula to obtain the spectral attention features: ; Based on the following formula, spatial attention features are obtained by mining local spatial correlations between pixels through spatial attention branches: ; Based on the following formula, the fused features obtained from spectral attention features and spatial attention features are processed in parallel by using multiple convolutional layers with different receptive fields through the first multi-scale convolutional block, resulting in spectral-spatial co-enhanced features: ; in, This indicates a global average pooling operation. This represents the global max pooling operation. Representative feature transformation function, It is the Sigmoid activation function. Indicates the first Grouping by features This is a feature encoding operation consisting of 1×1 convolution and batch normalization (BN). This is the attention weight vector corresponding to this group. This represents element-wise multiplication. and For learnable fusion coefficients, This represents a 3D convolution operation with a kernel size of 1×3×3. For upsampling operation, For activation function, Indicates the core size is Convolution operation, The fusion process representing multi-scale features.

[0008] Optionally, a multi-scale local feature extraction module is used to extract features from the feature tensor at multiple scales to capture spatial context information under different receptive fields, resulting in multi-scale context extraction features, specifically including: The multi-scale context extraction module includes: a second multi-scale convolution module and an MHBC module connected in sequence; Based on the following formula, the second multi-scale convolutional block uses multiple convolutional layers with different receptive fields to extract local features of the feature tensor at multiple scales, and performs feature fusion on the initial multi-scale features through cross-connection to obtain the initial multi-scale features; ; ; The initial multi-scale features are concatenated along the channel dimension based on the following formula to obtain the multi-scale fused features: ; The channel dimension is uniformly divided into multiple subspaces using a multi-head attention mechanism, and the spatial attention weights are determined within each subspace based on the following formula: ; The outputs of each attention head are concatenated along the channel dimension based on the following formula to obtain multi-scale context extraction features: ; in, Indicates the first Layer input feature tensor, Perform channel dimension transformation to reduce computational complexity. Representing the In-layer dimensions are Depth-separable convolution operations are used, and BN is a batch normalization process. Represents the ReLU activation function. For multi-scale fusion features, Indicates the first The feature sub-tensor corresponding to each attention head This is a global average pooling operation. represent Convolutional layer It is the Sigmoid activation function. This represents element-wise multiplication. This indicates a channel dimension splicing operation.

[0009] Optionally, the feature tensor is segmented into superpixels and aggregated using a superpixel feature aggregation module to model the long-range spatial dependencies between superpixels and aggregate region-level features, resulting in superpixel feature aggregation features, specifically including: The superpixel feature aggregation module includes: a linear clustering module and multiple upsampled map convolutional networks connected in sequence; Based on the total number of pixels in the hyperspectral remote sensing image to be processed, the input feature tensor is divided into multiple homogeneous regions using a linear clustering module to obtain the superpixel matrix: ; Based on determining the similarity between superpixel nodes using Mahalanobis distance and constructing an adjacency matrix, a superpixel-level undirected graph structure is obtained: ; ; ; ; The initial feature representation of each superpixel node is obtained by mean pooling of pixel features within the region based on the following formula: ; Based on the following formula, the superpixel node features are propagated sequentially through each upsampled image convolutional network, and the superpixel nodes and pixel-level features are updated through region-level adjacency relationships to obtain multiple superpixel-level features: ; ; ; Based on the following formula, spatial features of different granularities are extracted through graph convolution and pooling operations, and all superpixel-level features are mapped back to the pixel level through upsampling to obtain superpixel feature aggregation features: ; in, Indicates the first One superpixel region For pixels eigenvectors, This represents the corresponding node features. and Representing the node set and edge set respectively. It is an adjacency matrix. It is a symmetric positive semi-definite matrix. To add a self-loop adjacency matrix, This is the corresponding degree matrix. For trainable weight matrix, It is a non-linear activation function. Represents pixel-level features. Represents superpixel-level features. This is a feature aggregation for superpixel features.

[0010] Optionally, the spectral-spatial co-enhanced features, multi-scale context extraction features, and superpixel feature aggregation features are adaptively fused through a meta-weighted feature fusion module to dynamically integrate heterogeneous features through a meta-learning strategy, thereby obtaining the classification results of the hyperspectral remote sensing image to be processed. Specifically, this includes: The output features from the three branches—spectral spatial co-enhancement results, multi-scale contextual extraction features, and superpixel feature aggregation results—are concatenated along the channel dimension based on the following formula to obtain the fused features: ; Based on the following formula, the dimensionality transformation of the fused features and the classification features are performed through a fully connected layer to obtain the direct fused classification probability: ; Based on the following formula, the output features of the three branches are interacted through a cross-branch attention mechanism to mine complementary information between different feature representations and obtain enhanced features: ; The enhancement classification probability is obtained by probabilistic prediction of the enhancement features using a fully connected layer and a softmax function, based on the following formula: ; The final classification result is obtained by weighting and summing the direct fusion classification probability and the reinforcement classification probability using the following formula through a balancing parameter: ; in, This represents the output features of the multi-scale context extraction module. This represents the output features of the superpixel feature aggregation module. This indicates the output characteristics of the spectral-spatial synergistic enhancement module. and These are the weight matrix and bias term of the classification layer, respectively. This indicates a cross-branch attention operation. Hyperparameters contributing to attention control These are the balancing parameters.

[0011] Optionally, training a multipath spectral spatial cooperative network includes: Acquire historical hyperspectral images and their corresponding historical spectral-spatial co-enhanced features, historical multi-scale contextual extraction features, historical superpixel feature aggregation features, and historical image classification results; Historical hyperspectral images are input into a multipath spectral spatial collaborative network. The feature extraction module processes the historical hyperspectral images to obtain predicted spectral-spatial collaborative enhancement features, predicted multi-scale context extraction features, and predicted superpixel feature aggregation features. The meta-weighted feature fusion module performs adaptive fusion to obtain the predicted image classification results. The spatial-spectral co-enhancement classification loss for supervised spectral-spatial co-enhancement is determined based on the following formula, using predicted spectral-spatial co-enhancement features and historical spectral-spatial co-enhancement features: ; The extraction classification loss for supervised multi-scale context extraction is determined based on the predicted multi-scale context extraction features and the historical multi-scale context extraction features, according to the following formula: ; The aggregation classification loss used to supervise the structural features of the superpixel map is determined based on the following formula, which combines the aggregated features of predicted superpixel features with the aggregated features of historical superpixel features: ; The main fusion classification loss used to supervise the adaptive fusion of multi-branch heterogeneous features is determined based on the following formula, using the predicted image classification results and the historical image classification results: ; The total loss function is obtained by weighted summing of the spatial spectrum collaborative classification loss, multi-scale context extraction loss, aggregate classification loss, and main fusion classification loss based on the following formula: ; in, This represents the spatial spectrum collaborative classification loss. Indicates the number of valid label pixels. Represents the numerically stable term. Indicates the height of the hyperspectral image. Indicates the width of the hyperspectral image. This represents the One-Hot true label for the p-th pixel. This indicates the predicted superpixel feature aggregation feature. Indicates a binary mask. This represents the classification loss for extracting multi-scale contextual features. Represents the aggregate classification loss. Indicates the main fusion classification loss, Represents the total loss function. Indicates the weighting parameter; Based on the total loss function, the parameters of all modules in the multipath spectral spatial cooperative network are updated using the backpropagation algorithm to obtain the trained multipath spectral spatial cooperative network.

[0012] The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks provided in this invention has the following advantages compared with the prior art: This invention employs a meta-learning strategy to perceive and evaluate the content quality, discriminative power, and complexity of the current scene's input features across three paths in real time. It then dynamically learns a set of optimal, variable fusion weight coefficients for the feature map of each path. This adaptive weighting mechanism, based on the feature content itself, enables the multipath spectral spatial collaborative network to flexibly adjust the contribution ratio of each path's features when faced with different land cover distributions, spectral feature differences, or sample scarcity. It fully leverages and integrates the complementary advantages of heterogeneous multipath features, significantly improving the model's classification robustness and generalization ability in complex scenarios. Attached Figure Description

[0013] Figure 1 This is a model framework diagram of a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network provided in one embodiment; Figure 2 This is a spectral spatial attention structure diagram of a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network provided in one embodiment. Figure 3 This is a diagram of the internal structure of a CNN block in a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network, as provided in one embodiment. Figure 4 This is a diagram illustrating the superpixel and pixel feature conversion process of a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network provided in one embodiment. Figure 5 This is an example of experimental results from the PU dataset for a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network, as provided in one embodiment. Figure 5 (a) in the image is a pseudo-color image. Figure 5 (b) in the image is the actual label image. Figure 5 (c) in the figure shows the experimental results of DBDA. Figure 5(d) in the figure represents the experimental results of A2S2K. Figure 5 (e) in the figure represents the experimental results for FECNe. Figure 5 Figure (f) in the figure shows the experimental results from CEGCN. Figure 5 (g) in the figure represents the experimental results for H2. Figure 5 (h) in the figure represents the experimental results of HCGN. Figure 5 In the figure (i), the experimental results of GTFN are shown. Figure 5 (j) in the figure represents the experimental results of DBCTNet. Figure 5 (k) in the figure represents the experimental results of GSCViT. Figure 5 (l) in the figure represents the experimental results of Tri_CNN. Figure 5 (m) in the figure represents the experimental results of FDGC. Figure 5 In the figure, (n) represents the experimental results of SSGRN. Figure 5 (o) in the figure represents the experimental results of MIM. Figure 5 (p) in the figure represents the experimental results of MPS³N; Figure 6 The image shows the results of different methods of hyperspectral remote sensing image classification based on multipath spectral spatial cooperative networks provided in one embodiment on training samples of the Indian Pines dataset, Pavia University dataset, Kennedy Space Center dataset, and WHU-Hi-HongHu dataset. Figure 6 (a) in the figure shows the results of different methods on the training samples of the IndianPines dataset. Figure 6 (b) in the figure shows the results of different methods on the training samples of the Pavia University dataset. Figure 6 (c) in the figure shows the results of different methods on the training samples of the Kennedy Space Center dataset. Figure 6 (d) in the figure shows the results of different methods on the training samples of the WHU-Hi-HongHu dataset; Figure 7 The figure shows the results of different fusion methods of the hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative network provided in one embodiment on the Indian Pines dataset, Pavia University dataset, Kennedy SpaceCenter dataset, and WHU-Hi-HongHu dataset. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0015] Hyperspectral data consists of dozens to hundreds of continuous spectral bands. On the one hand, this provides rich spectral information for ground object identification, but on the other hand, it also brings significant spectral redundancy problems. The high correlation between adjacent bands and the interference of noisy bands cause the truly discriminative spectral features to be submerged in a large amount of redundant information.

[0016] Early research primarily employed linear dimensionality reduction methods such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), as well as nonlinear manifold learning techniques like Local Linear Embedding (LLE) and Laplacian Eigenmaps (LE), to remove redundant information by reducing spectral dimensionality during preprocessing. However, these methods are independent of subsequent classification tasks, struggle to capture the complex nonlinear structures between spectral features, exhibit poor stability in small-sample scenarios, and fail to achieve an ideal balance between redundancy suppression and discriminative feature preservation. With the development of deep learning, attention mechanisms have been introduced into spectral-spatial feature extraction. Examples include lightweight channel attention modules, efficient spatial attention mechanisms, grouped multi-attention mechanisms, and dual-attention models. While these have significantly improved feature discriminative power, they generally lack attention to local fine structures, and still fall short in subtle feature extraction and spatial structure modeling.

[0017] As spectral feature extraction gradually evolves towards "spectral-spatial synergy," effectively extracting spatial context information and capturing the spatial distribution structure of ground features has become crucial for improving classification performance. In recent years, deep learning networks have become the mainstream technology for hyperspectral classification due to their powerful feature learning capabilities. Among them, convolutional neural networks (CNNs) have been widely used due to their excellent local feature extraction capabilities—from the initial two-dimensional convolution to three-dimensional convolution that better fits the characteristics of hyperspectral data, and then to depthwise separable convolution. Although progress has been made in optimizing feature extraction efficiency and computational complexity, the inherent characteristics of "local connectivity" and "weight sharing" make it difficult to capture long-range spatial dependencies between pixels and insufficient for modeling the global topological structure of complex ground features. The emergence of Graph Convolutional Networks (GCNs) has provided a new approach to balancing local features and long-range dependencies. Some scholars have proposed superpixel graph convolutional networks, some have proposed mini-batch graph construction strategies, some have proposed multi-hop graph convolutional frameworks, and some have proposed dynamic graph convolutional networks. These methods achieve local information preservation and long-range dependency capture by constructing superpixel topological relationships. However, these methods still face a core contradiction: if the number of graph convolutional layers is too shallow, the global context perception is insufficient; if the number of layers is too deep, the problem of excessive smoothing of node features homogenization will easily occur. At the same time, they often ignore the complementarity between local features and global dependencies, and fail to completely solve the problem of accurate modeling of complex spatial structures.

[0018] With the continuous development of spectral-spatial feature extraction technology, a single feature extraction path struggles to simultaneously address the dual challenges of spectral redundancy suppression and complex spatial structure modeling. Therefore, the effective fusion of spectral-spatial features acquired through multiple paths remains a pressing issue. For example, some scholars have designed the CNN-GCN ensemble framework to attempt to obtain more comprehensive feature representations, but its simple static feature concatenation fusion strategy fails to achieve adaptive integration of heterogeneous features.

[0019] Furthermore, although existing studies have attempted to introduce more refined multi-scale strategies or utilize Transformers to enhance global fusion capabilities, these methods mostly focus on the fusion of multi-source image data. How to dynamically and adaptively fuse multi-path heterogeneous features derived within the same hyperspectral image based on the feature content itself remains a key challenge.

[0020] This invention provides a hyperspectral remote sensing image classification method based on a multipath spectral spatial cooperative network, comprising: Acquire the hyperspectral remote sensing image to be processed, and construct a feature tensor based on the spatial height, spatial width, and number of spectral bands of the hyperspectral remote sensing image to be processed.

[0021] The feature tensor is input into the multipath spectral spatial collaborative network, which includes a feature extraction module and a meta-weighted feature fusion module connected in sequence. The feature extraction module includes a parallel spectral spatial collaborative enhancement module, a multi-scale context extraction module, and a superpixel feature aggregation module.

[0022] The spectral-spatial co-enhancement module performs spectral-spatial co-enhancement on the feature tensor using a two-dimensional attention mechanism to extract discriminative features, thus obtaining spectral-spatial co-enhancement features.

[0023] The feature tensor is subjected to multi-scale local feature extraction by a multi-scale context extraction module to capture spatial context information under different receptive fields and obtain multi-scale context extraction features.

[0024] The superpixel feature aggregation module performs superpixel segmentation and graph convolution aggregation on the feature tensor to model the long-range spatial dependency between superpixels and aggregate region-level features to obtain superpixel feature aggregation features.

[0025] The meta-weighted feature fusion module adaptively fuses spectral-spatial co-enhanced features, multi-scale contextual extraction features, and superpixel feature aggregation features to dynamically integrate heterogeneous features through a meta-learning strategy, thereby obtaining the classification results of the hyperspectral remote sensing image to be processed.

[0026] A specific embodiment of the present invention is provided: The multipath spectral spatial cooperative network (MPS³N) structure proposed in this invention is as follows: Figure 1 As shown, the network consists of a feature extraction module and a fusion module: 1) The Spectral-Spatial Co-Enhancement (SSCE) module uses a two-dimensional attention mechanism to achieve interactive enhancement and redundancy suppression of spectral and spatial features, driving a multi-scale convolutional network to deeply mine the enhanced features, thereby extracting more discriminative feature representations.

[0027] 2) Multi-Scale Context Extraction (MSCE) module, used to extract discriminative multi-scale spectral-spatial features.

[0028] 3) Superpixel Feature Aggregation (SFA) module, which uses the superpixel map structure to establish long-range semantic associations between ground features.

[0029] 4) Meta-Weighted Feature Fusion (MW-FF) module: It adaptively fuses heterogeneous features from different branches through meta-learning strategy, thereby improving the model’s discrimination ability and robustness in multiple scenarios.

[0030] Representing hyperspectral remote sensing images as feature tensors ,in , and These represent the spatial height, spatial width, and number of spectral bands of a hyperspectral remote sensing image, respectively.

[0031] 1. Spectral Spatial Co-enhancement Module (SSCE).

[0032] 1.1 Feature Enhancement Module (FE).

[0033] To enhance the effective discriminative information in the spectral dimension and suppress the interference of redundant information, a spectral attention weight is constructed based on a global context-aware mechanism in the spectral-spatial collaborative enhancement feature extraction path.

[0034] First, the feature tensor is input, and then spectral attention weights are generated through dual-path feature extraction using feature enhancement methods, combined with nonlinear transformations. (1) in, and These represent the global average pooling operation and the global max pooling operation, respectively. Representative feature transformation function, The sigmoid activation function is used. Feature optimization based on spectral attention weights can be expressed as:

[0035] (2) This optimization process maintains feature stability through residual connections while enhancing the representational strength of discriminative spectral bands. Subsequently, the enhanced features are grouped based on spectral similarity, and spectral-spatial attention filtering is performed on each group.

[0036] 1.2 Spectral-Spatial Attention Module.

[0037] In the spectral-spatial attention screening stage, a multi-group, dual-dimensional attention collaboration mechanism is employed: spectral attention captures channel dependence, spatial attention mines spatial correlations, and multi-scale fusion enhances feature representation. Its spectral-spatial attention structure is as follows: Figure 2 As shown.

[0038] Hyperspectral features after feature enhancement Grouping by spectral similarity, grouping for each feature. Two paths, spectral attention and spatial attention, are introduced for collaborative modeling: the spectral attention branch focuses on the semantic dependencies between channels, and feature optimization is achieved through encoding layers and attention weighting. Its core expression is: (3) in, Indicates the first Grouping by features This is a feature encoding operation consisting of 1×1 convolution and batch normalization (BN). This is the attention weight vector corresponding to this group. Used to achieve weight normalization. This represents element-wise multiplication. and These are learnable fusion coefficients used to adaptively adjust the fusion ratio between the original features and the attention-weighted features.

[0039] 1.3 First multi-scale convolutional module (CNN Block 1).

[0040] The spatial attention branch focuses on uncovering local spatial relationships between pixels, modeling spatial information through dimensionality expansion, convolutional dimensionality reduction, and upsampling recovery, expressed as: (4) In the formula, This represents a 3D convolution operation with a kernel size of 1×3×3, used for spatial information extraction and dimensionality reduction of features after dimensional expansion. This is an upsampling operation (using nearest neighbor interpolation) used to restore the dimensionality-reduced features to their original size. The activation function is used to weight and enhance the original grouped features through element-wise multiplication, and finally it is connected with the original feature residuals to preserve the basic information.

[0041] To further capture multi-scale spatial context, the module employs convolutional layers with three different receptive fields: 7×7, 5×5, and 3×3. and Features after channel splicing and fusion Parallel processing is performed to output spectral-spatial co-enhanced features: (5) in, Indicates the core size is Convolution operation, The fusion process representing multi-scale features.

[0042] SSCE for input data The feature extraction process is summarized as follows: 1) Input data is used for feature enhancement, and the enhanced features are obtained through equations (1) and (2); 2) Group the data based on spectral similarity and obtain the features of the grouped data. ; 3) Using equations (3) and (4), the grouped features are fed into the two-dimensional attention method to obtain the fused features. ; 4) The data is fed into a CNN block for local feature extraction and then concatenated along the dimensional lines to obtain spectral-spatial co-enhanced features. .

[0043] 2. Multi-scale Context Extraction Module (MSCE).

[0044] To enhance the ability to depict local details, a multi-scale context extraction module is proposed (consisting of two modules: a second multi-scale convolution module (CNN Block2) and a multi-head convolution module MHBC).

[0045] The internal structures of CNN Block 1 and CNN Block 2 are as follows: Figure 3 As shown, three convolutional groups with different receptive fields were constructed to capture multi-scale local detail features. The computation process can be described as follows: (6) in, Indicates the first Layer input feature tensor, Perform channel dimension transformation to reduce computational complexity. Representing the In-layer dimensions are Depth-separable convolution operations are used, and BN is a batch normalization process. This represents the ReLU activation function.

[0046] To promote information interaction and complementarity among features at different scales, cross-connections are introduced on three convolutional groups with different receptive fields. This allows the convolutional modules that capture large-scale and small-scale information to be fully fused before entering the next layer. The fusion process is as follows: (7) Through a three-layer CNN block, discriminative local features were obtained from three different receptive fields. To better complement information, these features were concatenated along the channel dimension. (8) To further enhance feature discriminativeness, a multi-head convolutional module is used to strengthen the discriminative representation capability of the features. First, the features output by the CNN Block are processed... The channel dimension is uniformly divided into N independent subspaces, and spatial attention weights are calculated in each subspace:

[0047] (9) in, Indicates the first The feature sub-tensor corresponding to each attention head This is a global average pooling operation. represent Convolutional layer The sigmoid activation function is used. The enhanced outputs of each attention head are concatenated along the channel dimension to reorganize features, resulting in multi-scale context-extracted features.

[0048] (10) in, This represents element-wise multiplication. This indicates a channel dimension splicing operation.

[0049] MSCE for input data The feature extraction process is summarized as follows: 1) Initialize the module and pre-process the channel to obtain the adaptation features. ; 2) The initial features are extracted from 3 sets of CNNBlocks obtained by equation (6). , , The characteristic is obtained by summing according to equation (2). ; 3) with As input, the local features extracted by the second-layer CNN Block are obtained according to equations (6) and (7) and then summed to obtain... ; 4) with As input, each branch obtains the features extracted by the third-layer CNN Block through equations (6) and (7), and obtains multi-scale features by channel concatenation according to equation (8). ; 5) Through MHBC Enhancement is performed, and multi-scale context extraction features are obtained according to equations (9) and (10). .

[0050] 3. Superpixel feature aggregation module (SFA).

[0051] 3.1 Linear Clustering Module (SLLC).

[0052] To accurately capture long-range spatial dependencies between ground features and aggregate region-level features, we first establish the association between pixels and regions through superpixel segmentation. Let the total number of pixels in the input image be... ( (Image height and width), number of superpixels: ( (where is the segmentation scale of the superpixels), and the set of superpixel regions is denoted as . ( Indicates the first (multi-pixel regions), the pixel feature set is ( For the first (Feature vectors of 1 pixel). Define the matrix to which the superpixel belongs. Its elements satisfy:

[0053] (11) When constructing a region-level graph structure for superpixels, define the superpixel adjacency matrix. Its elements satisfy: (12) The superpixel and pixel feature conversion process is as follows Figure 4 As shown, the input image is first divided into multiple homogeneous regions based on a superpixel segmentation algorithm, constructing a region-level graph structure. Each superpixel region is considered a node in the graph, and the node features are obtained by mean pooling of the pixel features within the region.

[0054] (13) in, Indicates the first One superpixel region For pixels eigenvectors, This represents the corresponding node features.

[0055] When modeling graph structures, undirected graphs It is used to represent the relationship between superpixels, where and Representing the node set and edge set respectively. Given an adjacency matrix, the similarity between nodes is calculated using a learnable Mahalanobis distance metric.

[0056] (14) in, Given a symmetric positive semi-definite matrix, the optimal feature similarity metric is automatically learned through backpropagation. Based on this distance metric, the elements of the adjacency matrix are defined as follows:

[0057] (15) 3.2 Upsampled Graph Convolutional Network (UGCN).

[0058] In the feature propagation stage, lightweight graph convolution is used to achieve information exchange between nodes. Based on the symmetric normalized graph Laplacian matrix, the graph convolution operation is expressed as:

[0059] (16) in, To add a self-loop adjacency matrix, This is the corresponding degree matrix. For trainable weight matrix, It is a non-linear activation function.

[0060] During feature updates, both superpixel nodes and pixel-level features are updated simultaneously using region-level adjacency relationships. For nodes belonging to the... The feature update process for pixels in a region can be described as follows:

[0061] (17) in, and These represent the pixel features before and after the update, respectively. This strategy enables each pixel to aggregate the global neighborhood information of its superpixel.

[0062] To further mine multi-scale information, a four-level feature pyramid architecture was constructed. Each level extracts spatial features of different granularities through graph convolution and pooling operations. Finally, multi-scale feature fusion is achieved through upsampling and channel concatenation to obtain superpixel feature aggregation features.

[0063] (18) This architecture enables the model to simultaneously capture local detailed features and global semantic context, achieving collaborative optimization of multi-scale features.

[0064] SFA for input data The feature extraction process is summarized as follows: 1) Perform superpixel segmentation on the input data and obtain the adjacency matrix using equations (11)-(15). and the matrix to which the superpixel belongs ; 2) According to equation (16), the features are aggregated by graph convolution; 3) Through four-layer upsampling roll operation, update pixel-level and superpixel-level features simultaneously according to equation (17); 4) Concatenate the features extracted from each upsampled image convolution according to the channel dimension to obtain superpixel feature aggregation features. .

[0065] 4. Meta-weighted feature fusion module (MW-FF).

[0066] To adaptively fuse features extracted from multiple paths, a dynamic weighted summation method is used to fuse output features from the spectral-spatial co-enhancement path, the multi-scale context extraction path, and the superpixel feature aggregation path. First, a feature concatenation strategy is employed to fuse features from different branches:

[0067] (19) in, , and These represent the output features of the multi-scale context extraction path, the superpixel feature aggregation path, and the spectral-spatial co-enhancement path, respectively.

[0068] The fused features undergo dimensionality transformation and classification through a fully connected layer: (20) in, and These are the weight matrix and bias term of the classification layer, respectively. The softmax function converts the original scores into a class probability distribution.

[0069] To further enhance feature interaction, a cross-branch attention mechanism is used to mine complementary information between different feature representations: (twenty one) (twenty two) in, This indicates a cross-branch attention operation. Hyperparameters contributing to attention control. This mechanism effectively captures the correlation between different feature modalities, improving the discriminative power of fused features.

[0070] The final classification strategy combines the direct fusion output with the results of independent predictions from each path: (twenty three) in, This is a balancing parameter used to adjust the relative importance of feature-level fusion and prediction-level fusion.

[0071] MW-FF for output characteristics from three paths The feature fusion process is summarized as follows: 1) The features on the three paths are spliced ​​according to the channel dimension, and the classification probability Y of the fused features is obtained by equation (19) and equation (20); 2) Information mining is performed on the features of the three branches through cross-branch attention, and the results are added to the fused features to obtain enhanced features. ; 3) According to equation (23), the fusion classification probability Y and the reinforcement classification probability are combined. By balancing parameters Determine the classification results of the multipath spectral spatial cooperative network .

[0072] 5. Model training.

[0073] Acquire historical hyperspectral images and their corresponding historical spectral-spatial co-enhanced features, historical multi-scale contextual extraction features, historical superpixel feature aggregation features, and historical image classification results.

[0074] Historical hyperspectral images are input into a multipath spectral spatial collaborative network. The feature extraction module processes these images to obtain predicted spectral-spatial collaborative enhancement features, predicted multi-scale context extraction features, and predicted superpixel feature aggregation features. An adaptive fusion module then performs meta-weighted feature fusion to obtain the predicted image classification result.

[0075] The spatial-spectral co-enhancement classification loss for supervised spectral-spatial co-enhancement is determined based on the following formula, using predicted spectral-spatial co-enhancement features and historical spectral-spatial co-enhancement features: (twenty four) The extraction classification loss for supervised multi-scale context extraction is determined based on the predicted multi-scale context extraction features and the historical multi-scale context extraction features, according to the following formula: (25) The aggregation classification loss used to supervise the structural features of the superpixel map is determined based on the following formula, which combines the aggregated features of predicted superpixel features with the aggregated features of historical superpixel features: (26) The main fusion classification loss used to supervise the adaptive fusion of multi-branch heterogeneous features is determined based on the following formula, using the predicted image classification results and the historical image classification results: (27) The total loss function is obtained by weighted summation of the spatial spectrum collaborative classification loss, multi-scale context extraction loss, aggregate classification loss, and main fusion classification loss based on the following formula: (28) in, This represents the spatial spectrum collaborative classification loss. This represents the multi-scale context extraction loss. Indicates the number of valid label pixels. This represents a numerically stable term (in this invention, it is uniformly set to avoid log(0) overflow). Indicates the height of the hyperspectral image. Indicates the width of the hyperspectral image. This represents the One-Hot true label for the p-th pixel. This indicates the predicted superpixel feature aggregation feature. Indicates a binary mask. Represents the aggregate classification loss. Indicates the main fusion classification loss, Represents the total loss function. This represents the weighting parameter.

[0076] Based on the total loss function, the parameters of all modules in the multipath spectral spatial cooperative network are updated using the backpropagation algorithm to obtain the trained multipath spectral spatial cooperative network.

[0077] 6. Experiment.

[0078] 6.1 Dataset.

[0079] To verify the effectiveness and robustness of the proposed MPS³N network, a comparative experiment was conducted using the Pavia University dataset (PU). PU is derived from ROSIS sensor observation data. Five samples from each labeled class were randomly selected as the training set, five as the validation set, and the remaining samples as the test set.

[0080] 6.2 Experimental setup.

[0081] Implemented using Python 3.9 and PyTorch 1.5.1 frameworks, employing an NVIDIA A100 GPU equipped with CUDA 11.3. Model optimization utilizes the Adam optimizer, with a uniform training iteration of 500 times. To ensure fairness in evaluation, random sampling was performed 20 times, and the average accuracy was recorded. Overall accuracy (OA), class average accuracy (AA), and Kappa coefficient were used to measure model performance metrics.

[0082] To systematically evaluate the performance of the proposed MPS³N network in hyperspectral image classification, experiments were conducted using 14 state-of-the-art methods covering five mainstream architectures as benchmarks. To ensure the fairness and validity of the comparison results, all comparison methods were tested using the optimal parameter configurations published in their original papers. These methods include: DBDA, A2S2K, and FECNet based on convolutional neural networks (CNNs); CEGCN, H2, and HCGN based on graph convolutional networks (GCNs); GTFN, DBCTNet, and GSCVIT based on Transformers; TriCNN, FDGC, and SSGRN based on a three-branch design; and the MIM model based on the Mamba architecture. Four public datasets with significant differences in spectral features, spatial structure, and sample distribution were selected to verify the classification accuracy, generalization ability, and overall effectiveness of the MPS³N model in different scenarios.

[0083] 6.3 Comparative Experiment.

[0084] The experimental results on the PU dataset are shown in Table 1. MS³N achieved competitive overall performance in the urban scene hyperspectral classification task, with overall accuracy (OA), average accuracy (AA) and Kappa coefficient reaching 87.88%, 91.21% and 84.38% respectively, all of which are better than the various mainstream models compared.

[0085] This dataset exhibits significant spectral overlap and spatial structural complexity. MS³N, through its multi-branch spectral feature extraction and attention fusion mechanism, demonstrates outstanding performance in typical difficult categories: it achieves a classification accuracy of 81.91% in the spectrally similar gravel category, which is 1.8 times that of A2S2K; and it achieves an accuracy of 99.39% in small sample shaded regions, significantly outperforming models such as MIM, demonstrating excellent detail preservation and sample robustness.

[0086] It is worth noting that some GCN-based methods, such as CEGCN (OA of 85.13%) and H2 (OA of 84.43%), show little difference in overall accuracy compared to the method presented in this paper, especially in categories with clear structures (such as metal sheets and asphalt). This is attributed to their ability to model spatial context using graph structures. However, MS³N, through multi-branch spectral feature extraction and dynamic fusion mechanisms, performs better in categories with severe spectral overlap (such as gravel) and small sample categories (such as shadows), demonstrating more comprehensive scene adaptability and feature discrimination capabilities.

[0087] Table 1 Experimental results of the PU dataset like Figure 5As shown, from the perspective of classification visual effects, MS³N outperforms the comparison models in terms of boundary preservation and region consistency. Although methods such as CEGCN and H2 are comparable in some homogeneous areas, they still suffer from overly blurred transitions or local misclassifications at complex boundaries. MS³N, through the synergistic optimization of spectral and spatial features, effectively improves the discrimination accuracy of detailed features while maintaining the overall structural smoothness, especially demonstrating superior semantic coherence and classification robustness at irregular building edges and scattered shadow areas.

[0088] 6.4 Comparative analysis under different training sample sizes.

[0089] like Figure 6 As shown, the overall classification accuracy (OA) of MPS³N is compared with that of several mainstream hyperspectral image classification methods on the Indian Pines dataset (IP), Pavia University dataset (PU), Kennedy Space Center dataset (KSC), and WHU-Hi-HongHu dataset (HU) as the number of training samples per class varies. To standardize the evaluation process, this section sets the number of training and validation samples for each class to 5, 10, 15, 20, and 25, respectively, with the remaining samples used as the test set. For classes with insufficient total samples, they are evenly divided into training, validation, and test sets to ensure the rationality of the experimental setup.

[0090] The line graphs for each dataset illustrate the trends in OA (Adjustment Ability) for different methods under varying proportions of training samples in each class: Overall, the classification performance of most methods improves with the increase in the number of training samples in each class. This phenomenon can be attributed to the fact that the increase in training samples provides the model with richer spectral-spatial feature information, helping the model to learn the discriminative representations of each class more fully, thereby alleviating the underfitting problem caused by insufficient samples.

[0091] While most methods show a steady improvement in classification performance as the number of training samples gradually increases, some methods still experience performance fluctuations or sluggish growth on specific datasets, with varying specific manifestations and causes. In the PU dataset, DBDA's overall classification accuracy slightly decreased from 78.52% to 78.45% when the sample size increased from 5 to 10, and also dropped from 92.59% to 91.93% when increasing from 15 to 20. Similarly, in the KSC and HU datasets, the method showed an accuracy decline at 20 and 25 samples, respectively. These anomalies mainly stem from feature interference caused by edge-mixed pixels and inter-class spectral confusion introduced during sample expansion. DBDA, lacking an effective noise suppression mechanism, struggles to accurately distinguish discriminative features from interference information during training. On the IP dataset, Tri_CNN's performance also exhibited phased fluctuations: it only slowly increased from 5 to 15 samples, then decreased to 61.15% at 20 samples, and finally only recovered to 70.42% at 25 samples. This phenomenon is closely related to the class imbalance of the dataset. Furthermore, the model's reliance on local convolutional structures and the lack of cross-scale fusion and noise suppression mechanisms limit its generalization ability when dealing with edge-mixed pixels. In addition, GSCViT's performance on the KSC dataset also exhibits significant fluctuations; its accuracy drops sharply from 86.24% to 68.29% when the number of samples increases from 5 to 10, only gradually recovering with further increases in the number of samples. This behavior reflects the high sensitivity of its Transformer-based attention mechanism to the quality of training samples. When the sample size is limited and inter-class confusion exists, the lack of effective attention constraints causes the model to capture too many interfering features, leading to performance degradation.

[0092] In contrast, MPS³N and HCGN demonstrated consistently improving OA performance across all datasets with increasing training sample size. MPS³N's robustness stems from the synergistic effect of its multi-path spectral-spatial features, while HCGN, although exhibiting some robustness to noise through graph structure enhancement, still falls short of MPS³N's overall performance. Notably, MPS³N achieved superior classification accuracy across all training sample settings, fully demonstrating its excellent feature extraction and noise suppression capabilities across multiple scenarios and sample sizes.

[0093] 6.5 Analysis of MW-FF method and comparative fusion method.

[0094] To objectively evaluate the effectiveness of the proposed Meta-Weighted Feature Fusion (MW-FF) method, this section compares its performance with four mainstream fusion strategies: feature concatenation (Concat), element-wise summation (Sum), maximum value (Max), and average value (Average). Figure 7As shown, the trajectory of the MW-FF curve consistently outperforms other comparative methods across all test datasets, demonstrating a comprehensive and consistent performance advantage. Specifically, this advantage manifests in two aspects: First, compared to static fusion methods with fixed parameters such as Sum, Max, and Average, MW-FF achieves a significant lead in classification accuracy, especially in complex scenarios like HU, where its performance improvement exceeds twenty percentage points. This fully reveals the necessity of dynamic adaptive weighting mechanisms when handling complex spectral-spatial features. Second, even compared to feature concatenation methods with the second-best performance, MW-FF maintains a stable and significant accuracy gain across all datasets. This phenomenon strongly suggests that MW-FF's success is not simply due to the preservation of feature information, but more importantly, its intelligent filtering and enhancement mechanism implemented through a meta-weighted network. This mechanism dynamically optimizes the contribution ratio of different branch features based on the characteristics of the input samples, thereby achieving a more accurate and robust feature fusion effect.

[0095] 6.6 Ablation experiment.

[0096] To systematically evaluate the contribution of each module in the proposed MPS³N model, comprehensive ablation experiments were conducted, and the results are shown in Table 2. The experiments compared the overall accuracy (OA) of the complete MPS³N with versions that omitted specific modules. Specifically, the following abbreviations are used to denote the modification methods: MPS³N a This indicates the removal of the multi-scale context extraction branch, MPS³N b This indicates the removal of the superpixel feature aggregation branch, MPS³N c represents the removal of the spectral-spatial co-enhancement branch, FE represents the removal of the feature enhancement module, MHBC represents the removal of the multi-head convolution module, and CAM represents the removal of the cross attention module.

[0097] Table 2. Effectiveness Indicators of Different Modules in MPS³N 1) Contributions of Different Branches: The three proposed branches collectively constitute the basic architecture of MPS³N. To evaluate the independent contribution of each branch, this paper compares the overall accuracy (OA) generated by the complete MPS³N with the results generated by the MPS³N version omitting one of the networks. The results are shown in Table IX. The overall accuracy of MPS³N is higher than all improved methods, especially in scenarios with fewer training samples (such as the IP and PU datasets), where the performance advantage of MPS³N is more obvious. These results validate the necessity of the three branches, which improve classification performance from three dimensions: spatial local features, spatial global relationships, and spectral-spatial collaboration, respectively.

[0098] 2) Contribution of the Feature Enhancement Module (FE): The Feature Enhancement Module (FE) recalibrates the importance of spectral dimensions through global attention. Removing this module results in an average performance decrease of 2.1%, indicating that feature enhancement has a positive effect on spectral redundancy processing in hyperspectral data, especially in providing supplementary information in band selection.

[0099] 3) Contribution of the Multi-Head Convolutional Module (MHBC): Removing this module resulted in an average performance decrease of 3.12% for MPS³N, from 87.88% to 83.96% on the PU dataset and from 85.08% to 80.50% on the HU dataset. This indicates that the module effectively suppresses feature redundancy and improves model robustness. In complex scenarios (such as HU and PU), the performance degradation is more significant, suggesting that the MHBC mechanism enhances the model's ability to discriminate complex terrain features through multi-view feature learning.

[0100] 4) Contribution of the Cross-Attention Module (CAM): CAM is the core of multi-source feature fusion. As shown in Table IX of the ablation results, removing this module results in the most severe performance degradation, decreasing from 83.86% to 68.22% (a decrease of 15.64%) on the IP dataset and from 85.08% to 70.28% (a decrease of 14.80%) on the HU dataset. This indicates that the lack of cross-modal interaction prevents the effective fusion of features from the MSCE and SEA branches, significantly weakening the overall model performance. This is especially true in small-sample training scenarios (such as the IP dataset), where the complementarity of different modal features can alleviate the data scarcity problem.

[0101] 7. Effect analysis.

[0102] 7.1 Global deep spectral-spatial feature extraction and fusion is achieved through three complementary feature extraction paths: spectral-spatial synergistic enhancement, superpixel feature aggregation, and multi-scale context extraction, as well as an adaptive meta-weighted feature fusion method.

[0103] 7.2 To address the feature redundancy problem caused by high correlation in spectral bands, a multi-scale feature extraction method guided by spectral-spatial collaboration is constructed. By using a dual-dimensional attention mechanism to locate key feature regions and accurately suppress spectral redundancy, the multi-scale feature extraction module is driven to deeply mine the enhanced key features, thereby achieving synergistic enhancement of spectral and spatial features and effectively improving the discriminative ability of features.

[0104] 7.3 To address the problem that traditional static fusion strategies are difficult to adapt to the feature distribution of different scenarios, an adaptive meta-weighted feature fusion method is proposed. This method adaptively fuses features from different branches through a meta-learning strategy, thereby improving the model's discriminative ability and robustness in different scenarios.

[0105] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks, characterized in that, include: Acquire the hyperspectral remote sensing image to be processed, and construct a feature tensor based on the spatial height, spatial width, and number of spectral bands of the hyperspectral remote sensing image to be processed; The feature tensor is input into a multipath spectral spatial collaborative network, which includes a feature extraction module and a meta-weighted feature fusion module connected in sequence. The feature extraction module includes a parallel spectral spatial collaborative enhancement module, a multi-scale context extraction module, and a superpixel feature aggregation module. The spectral-spatial co-enhancement module performs a two-dimensional attention mechanism on the feature tensor to extract discriminative features, resulting in spectral-spatial co-enhancement features. Multi-scale local feature extraction is performed on the feature tensor by a multi-scale context extraction module to capture spatial context information under different receptive fields and obtain multi-scale context extraction features. The superpixel feature aggregation module performs superpixel segmentation and graph convolution aggregation on the feature tensor to model the long-range spatial dependency between superpixels and aggregate region-level features to obtain superpixel feature aggregation features. The meta-weighted feature fusion module adaptively fuses spectral-spatial co-enhanced features, multi-scale contextual extraction features, and superpixel feature aggregation features to dynamically integrate heterogeneous features through a meta-learning strategy, thereby obtaining the classification results of the hyperspectral remote sensing image to be processed.

2. The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks as described in claim 1, characterized in that, The spectral-spatial co-enhancement of the feature tensor through a two-dimensional attention mechanism via the spectral-spatial co-enhancement module to extract discriminative features, resulting in spectral-spatial co-enhancement features, specifically includes: The spectral spatial collaborative enhancement module includes: a feature enhancement module, a spatial spectral attention module, and a first multi-scale convolution module connected in sequence. The spatial spectral attention module includes: a parallel spectral attention branch and a spatial attention branch. Based on the following formula, the feature enhancement module uses global average pooling and global max pooling operations to extract features from the feature tensor, thus obtaining the spectral attention weights: ； The enhanced hyperspectral features are obtained by optimizing the feature tensor using spectral attention weights based on the following formula: ； The enhanced hyperspectral features are grouped based on spectral similarity to obtain multiple feature groups; For each feature group, channel dependencies are extracted using the spectral attention branch based on the following formula to obtain the spectral attention features: ； Based on the following formula, spatial attention features are obtained by mining local spatial correlations between pixels through spatial attention branches: ； Based on the following formula, the fused features obtained from spectral attention features and spatial attention features are processed in parallel by using multiple convolutional layers with different receptive fields through the first multi-scale convolutional block, resulting in spectral-spatial co-enhanced features: ； in, This indicates a global average pooling operation. This represents the global max pooling operation. Representative feature transformation function, It is the Sigmoid activation function. Indicates the first Grouping by features This is a feature encoding operation consisting of 1×1 convolution and batch normalization (BN). This is the attention weight vector corresponding to this group. This represents element-wise multiplication. and For learnable fusion coefficients, This represents a 3D convolution operation with a kernel size of 1×3×3. For upsampling operation, For activation function, Indicates the core size is Convolution operation, The fusion process representing multi-scale features.

3. The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks as described in claim 1, characterized in that, The step involves using a multi-scale context extraction module to extract local features from the feature tensor at multiple scales to capture spatial context information under different receptive fields, thereby obtaining multi-scale context extraction features. Specifically, this includes: The multi-scale context extraction module includes: a second multi-scale convolution module and an MHBC module connected in sequence; Based on the following formula, the second multi-scale convolutional block uses multiple convolutional layers with different receptive fields to extract local features of the feature tensor at multiple scales, and performs feature fusion on the initial multi-scale features through cross-connection to obtain the initial multi-scale features; ； ； The initial multi-scale features are concatenated along the channel dimension based on the following formula to obtain the multi-scale fused features: ； The channel dimension is uniformly divided into multiple subspaces using a multi-head attention mechanism, and the spatial attention weights are determined within each subspace based on the following formula: ； The outputs of each attention head are concatenated along the channel dimension based on the following formula to obtain multi-scale context extraction features: ； in, Indicates the first Layer input feature tensor, Perform channel dimension transformation to reduce computational complexity. Representing the In-layer dimensions are Depth-separable convolution operations are used, and BN is a batch normalization process. Represents the ReLU activation function. For multi-scale fusion features, Indicates the first The feature sub-tensor corresponding to each attention head This is a global average pooling operation. represent Convolutional layer It is the Sigmoid activation function. This represents element-wise multiplication. This indicates a channel dimension splicing operation.

4. The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative network as described in claim 1, characterized in that, The superpixel feature aggregation module performs superpixel segmentation and graph convolution aggregation on the feature tensor to model the long-range spatial dependencies between superpixels and aggregate region-level features to obtain superpixel feature aggregation features, specifically including: The superpixel feature aggregation module includes: a linear clustering module and multiple upsampled map convolutional networks connected in sequence; Based on the total number of pixels in the hyperspectral remote sensing image to be processed, the input feature tensor is divided into multiple homogeneous regions using a linear clustering module to obtain the superpixel matrix: ； Based on determining the similarity between superpixel nodes using Mahalanobis distance and constructing an adjacency matrix, a superpixel-level undirected graph structure is obtained: ； ； ； ； The initial feature representation of each superpixel node is obtained by mean pooling of pixel features within the region based on the following formula: ； Based on the following formula, the superpixel node features are propagated sequentially through each upsampled image convolutional network, and the superpixel nodes and pixel-level features are updated through region-level adjacency relationships to obtain multiple superpixel-level features: ； ； ； Based on the following formula, spatial features of different granularities are extracted through graph convolution and pooling operations, and all superpixel-level features are mapped back to the pixel level through upsampling to obtain superpixel feature aggregation features: ； in, Indicates the first One superpixel region For pixels eigenvectors, This represents the corresponding node features. and Representing the node set and edge set respectively. It is an adjacency matrix. It is a symmetric positive semi-definite matrix. To add a self-loop adjacency matrix, This is the corresponding degree matrix. For trainable weight matrix, It is a non-linear activation function. Represents pixel-level features. Represents superpixel-level features. This is a feature aggregation for superpixel features.

5. The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks as described in claim 1, characterized in that, The meta-weighted feature fusion module adaptively fuses spectral-spatial co-enhanced features, multi-scale context extraction features, and superpixel feature aggregation features to dynamically integrate heterogeneous features through a meta-learning strategy, thereby obtaining the classification result of the hyperspectral remote sensing image to be processed. Specifically, this includes: The output features from the three branches—spectral spatial co-enhancement results, multi-scale contextual extraction features, and superpixel feature aggregation results—are concatenated along the channel dimension based on the following formula to obtain the fused features: ； Based on the following formula, the dimensionality transformation of the fused features and the classification features are performed through a fully connected layer to obtain the direct fused classification probability: ； Based on the following formula, the output features of the three branches are interacted through a cross-branch attention mechanism to mine complementary information between different feature representations and obtain enhanced features: ； The enhancement classification probability is obtained by probabilistic prediction of the enhancement features using a fully connected layer and a softmax function, based on the following formula: ； The final classification result is obtained by weighting and summing the direct fusion classification probability and the reinforcement classification probability using the following formula through a balancing parameter: ； in, This represents the output features of the multi-scale context extraction module. This represents the output features of the superpixel feature aggregation module. This indicates the output characteristics of the spectral-spatial synergistic enhancement module. and These are the weight matrix and bias term of the classification layer, respectively. This indicates a cross-branch attention operation. Hyperparameters contributing to attention control These are the balancing parameters.

6. The hyperspectral remote sensing image classification method based on multipath spectral spatial cooperative networks as described in claim 1, characterized in that, Training the multipath spectral spatial cooperative network specifically includes: Acquire historical hyperspectral images and their corresponding historical spectral-spatial co-enhanced features, historical multi-scale contextual extraction features, historical superpixel feature aggregation features, and historical image classification results; Historical hyperspectral images are input into the multipath spectral spatial collaborative network. The feature extraction module processes the historical hyperspectral images to obtain predicted spectral-spatial collaborative enhancement features, predicted multi-scale context extraction features, and predicted superpixel feature aggregation features. The meta-weighted feature fusion module performs adaptive fusion to obtain the predicted image classification result. The spatial-spectral co-enhancement classification loss for supervised spectral-spatial co-enhancement is determined based on the following formula, using predicted spectral-spatial co-enhancement features and historical spectral-spatial co-enhancement features: ； The extraction classification loss for supervised multi-scale context extraction is determined based on the predicted multi-scale context extraction features and the historical multi-scale context extraction features, according to the following formula: ； The aggregation classification loss used to supervise the structural features of the superpixel map is determined based on the following formula, which combines the aggregated features of predicted superpixel features with the aggregated features of historical superpixel features: ； The main fusion classification loss used to supervise the adaptive fusion of multi-branch heterogeneous features is determined based on the following formula, using the predicted image classification results and the historical image classification results: ； The total loss function is obtained by weighted summing of the spatial spectrum collaborative classification loss, multi-scale context extraction loss, aggregate classification loss, and main fusion classification loss based on the following formula: ； in, This represents the spatial spectrum collaborative classification loss. Indicates the number of valid label pixels. Represents the numerically stable term. Indicates the height of the hyperspectral image. Indicates the width of the hyperspectral image. This represents the One-Hot true label for the p-th pixel. This indicates the predicted superpixel feature aggregation feature. Indicates a binary mask. This represents the classification loss for multi-scale context extraction. Represents the aggregate classification loss. Indicates the main fusion classification loss, Represents the total loss function. Indicates the weighting parameter; Based on the total loss function, the parameters of all modules in the multipath spectral spatial cooperative network are updated using the backpropagation algorithm to obtain the trained multipath spectral spatial cooperative network.

Images