A hyperspectral image ground object classification method based on spectral graph convolution and visual state space model cooperation

Abstract

The application relates to a hyperspectral image ground object classification method based on cooperation of spatial spectrum graph convolution and visual state space model. The method comprises the following steps: firstly, pre-processing, principal component analysis dimension reduction and spatial spectrum graph structure data set construction are carried out on the hyperspectral image; secondly, an SSGM-Net network is constructed, local spatial spectrum information is aggregated through spatial spectrum graph convolution coding branch, global long-range dependence is modeled in linear complexity by combining a bidirectional spatial spectrum selective scanning branch, and an adaptive gate fusion submodule is used to output ground object category probability; then, a joint loss function composed of a weighted cross-entropy loss, a graph Laplacian smoothing constraint and a prototype contrast loss is used for training; finally, a ground object classification map is generated through sliding window prediction, superpixel majority voting and full connection conditional random field post-processing. The application can significantly improve the classification accuracy and boundary consistency under the condition of a small amount of labeled samples, and can effectively serve fine agriculture and ecological monitoring.

Description

Technical Field

[0001] This invention relates to the field of intelligent interpretation technology of hyperspectral remote sensing images, specifically to a hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model. Background Technology

[0002] Hyperspectral remote sensing imagery, while acquiring continuous spectral curves of ground features, also preserves two-dimensional spatial distribution information, forming a three-dimensional data cube with both image and spectral characteristics. This provides rich discriminative criteria for refined ground feature identification and has been widely applied in fields such as precision agriculture crop type identification, ecological environment monitoring, mineral resource exploration, and land resource surveys. The core task of hyperspectral imagery ground feature classification is to assign a unique ground feature category label to each pixel in the image; the classification accuracy directly determines the reliability of subsequent quantitative inversion and application analysis.

[0003] With the rapid development of deep learning, hyperspectral image classification methods based on convolutional neural networks, recurrent neural networks, and Transformers have been proposed, achieving classification performance superior to traditional machine learning methods when sufficient labeled samples are available. However, existing technologies still have the following technical problems:

[0004] (1) Convolutional neural networks extract features with regular square convolution kernels. Limited by the local receptive field, they are difficult to characterize the irregular spatial morphology of ground objects and global long-range spectral dependence. In addition, the number of stacked layers will lead to a sharp increase in the number of parameters and computation. Although the Transformer method based on self-attention mechanism has global modeling capabilities, its computational complexity is quadratic with the sequence length, making it difficult to handle the long sequences brought about by the massive number of pixels and hundreds of bands in hyperspectral images.

[0005] (2) Existing methods usually construct samples with regular square neighborhoods or fixed-size windows, and classify each pixel as an independent and identically distributed sample, which severs the spatial context association between pixels. In heterogeneous areas such as the boundaries of crop plots and the forest-grass transition zone, salt-and-pepper noise is easily generated, resulting in fragmented classification patches and poor boundary fit.

[0006] (3) Hyperspectral images suffer from a severe Hughes phenomenon, meaning that when the number of training samples is limited, the classifier performance decreases as the feature dimension increases. Simultaneously, the distribution of land cover categories is extremely uneven, with smaller categories easily being overwhelmed by larger ones. Existing cross-entropy loss functions lack explicit constraints on intra-class feature compactness and spatial consistency. Therefore, there is an urgent need for a hyperspectral image land cover classification method that can simultaneously model local spatial-spectral structure and global long-range dependencies with a limited number of labeled samples, while maintaining controllable computational complexity. Summary of the Invention

[0007] To address the technical problems of existing technologies, such as difficulty in depicting irregular spatial morphology of ground features, fragmented spatial context relationships between pixels leading to broken classification patches, and the difficulty in handling long hyperspectral sequences due to the increasing computational complexity of self-attention methods with the square of sequence length, this invention aims to provide a hyperspectral image ground feature classification method based on the synergy of spatial spectral graph convolution and a visual state space model. The specific technical solution adopted is as follows: The method includes: acquiring the hyperspectral remote sensing image to be classified and performing radiometric calibration, atmospheric correction, bad line repair, and minimum noise separation preprocessing to obtain a preprocessed image; performing principal component analysis on the preprocessed image to obtain a dimensionality-reduced image; cropping spatial spectral cube samples pixel by pixel on the dimensionality-reduced image according to a preset spatial window size to obtain a spatial spectral cube sample set; performing superpixel segmentation on the preprocessed image to obtain a superpixel segmentation result; using the superpixels in the superpixel segmentation result as graph nodes, constructing node adjacency relationships according to spatial spectral composite kernel similarity to obtain a spatial spectral graph structure sample set; designing a spatial spectral graph convolutional coding branch based on a multi-hop attention graph convolutional network to integrate the spatial spectral graph structure. The sample set is converted into spatial spectral image structural features. A bidirectional spatial-spectral selective scanning branch is designed based on a selective state-space model to flatten and scan the spatial spectral image structural features in both forward and reverse spatial raster sequences, obtaining global long-range dependency features. An adaptive gated fusion submodule is designed to perform channel-by-channel gated fusion of the spatial spectral image structural features and the global long-range dependency features, obtaining fused classification features. A classification head is then concatenated after the fused classification features to form the SSGM-Net network. A joint loss function is designed based on the characteristics of uneven distribution of land cover categories in hyperspectral images, sparse labeled samples, and large spectral dispersion of similar land cover types, to obtain training and supervision signals. The SSGM-Net network is trained using these training and supervision signals to obtain a trained network model. The trained network model is then used to classify the hyperspectral remote sensing images to be classified pixel by pixel, obtaining initial classification results. The initial classification results are post-processed using superpixel majority voting and fully connected conditional random fields to obtain a land cover classification map. The overall accuracy, average accuracy, and Kappa coefficient of the land cover classification map are evaluated.

[0008] Furthermore, the hyperspectral remote sensing image acquisition and preprocessing method includes: selecting hyperspectral remote sensing images with a spatial resolution of not less than 30 meters, a spectral resolution better than 10 nanometers, and not less than 100 bands as data sources, including Gaofen-5 AHSI, Zhuhai-1 OHS or AVIRIS, and ROSIS airborne hyperspectral sensors; performing radiometric calibration and atmospheric correction on the original images sequentially to eliminate sensor response differences and atmospheric scattering and absorption effects; using a neighborhood pixel linear interpolation method to repair bad lines, dead pixels, and stripe noise; and using minimum noise separation transform to perform noise whitening and feature extraction on the repaired images, removing noise-dominant components with a signal-to-noise ratio lower than a set threshold to obtain images with a spatial dimension of [missing information]. The number of bands is The preprocessed image, where bad line restoration uses neighborhood pixel linear interpolation, is calculated using the following formula:

[0009]

[0010] in, Indicates the first In the band, located at the _ _ band Line 1 The grayscale values ​​of the cells in the column; This indicates the grayscale value of the restored pixel; , This indicates the total number of bands.

[0011] Furthermore, the method for obtaining the principal component analysis and spatial spectral cube sample set includes: Since hyperspectral images have a large number of bands and strong correlations between adjacent bands, directly using all bands in network computation would lead to severe redundancy and the curse of dimensionality. Therefore, principal component analysis is used to project the original bands onto an uncorrelated low-dimensional principal component space; the band covariance matrix of the preprocessed image is calculated and eigenvalue decomposition is performed. The covariance matrix and characteristic equation of the principal component analysis are shown below:

[0012]

[0013]

[0014] in, Indicates by A data matrix consisting of spectral vectors of pixels, each spectral vector having a dimension of . ; Represents the sample spectral mean vector; Represents the band covariance matrix; This represents a diagonal matrix composed of eigenvalues; This represents the projection matrix formed by eigenvectors arranged in columns; Represents the low-dimensional data after projection; take the first digit. The eigenvectors corresponding to the largest eigenvalues ​​form a projection matrix, which projects the preprocessed image into a low-dimensional space, resulting in a band number of [value missing]. Dimensional reduction image, The value ranges from 15 to 40; centered on each pixel in the dimension-reduced image, according to the preset spatial window size. Trim the empty spectral cube samples, and the dimension of the empty spectral features of each sample is... The size of the spatial window is an odd number and its value ranges from 7 to 11.

[0015] Furthermore, the method for obtaining the spatial spectral map structure sample set includes: since pixel-level regular square neighborhoods are difficult to fit the true boundaries of ground objects, in order to ensure the homogeneity and semantic integrity of graph nodes, the preprocessed image is first subjected to superpixel segmentation; a simple linear iterative clustering algorithm is used to perform superpixel segmentation on the preprocessed image, and spectral distance and spatial distance are measured simultaneously during clustering. The distance measurement calculation formula is as follows:

[0016]

[0017] in, This represents the spectral distance between a pixel and its cluster center in the reduced-dimensional image. This represents the spatial Euclidean distance between the two. , These are the normalization factors for spectral distance and spatial distance, respectively. The cluster centers are iteratively updated until convergence, resulting in a superpixel segmentation result consisting of several superpixels with consistent internal spectra and boundaries conforming to the contours of ground features. Each superpixel is considered a graph node, and the mean of the spatial-spectral features of all pixels within the superpixel is used as the node feature. The similarity of the spatial-spectral composite kernel is used to determine whether two nodes are adjacent. The formula for calculating the spatial-spectral composite kernel similarity is shown below:

[0018]

[0019] in, , They represent the first The and the first The mean vector of the spatial spectral features of each node; This represents the spatial Euclidean distance between the centroids of two nodes; The spectral kernel bandwidth ranges from 0.5 to 2.0. The spatial kernel bandwidth ranges from 3 to 15 pixels. This represents the similarity of the spatial spectral composite kernel between two nodes. An edge is retained if the similarity is greater than a similarity threshold; otherwise, it is set to zero. Only the edge with the highest similarity is retained for each node. Use edge strips to obtain sparse adjacency relationships. The value range is 4 to 12; the adjacency relationship is symmetrically normalized to obtain the normalized adjacency matrix. The symmetric normalization calculation formula is as follows:

[0020]

[0021] in, This represents the adjacency matrix composed of spatial spectrum composite kernel similarity; Represents the identity matrix; Representing the adjacency matrix The degree matrix; Represents the normalized adjacency matrix; This represents the Laplace matrix of the symmetric normalized graph.

[0022] Furthermore, the SSGM-Net network is composed of a spatial spectrogram convolutional coding branch, a bidirectional spatial spectrogram selective scanning branch, an adaptive gated fusion submodule, and a classification head connected in series. The spatial spectrogram convolutional coding branch is used to aggregate heterogeneous neighborhood information on a non-Euclidean spatial spectrogram. The bidirectional spatial spectrogram selective scanning branch is used to capture global long-range spectral dependence with linear complexity. The adaptive gated fusion submodule is used to adaptively weighted and complementaryly fuse the two features channel by channel.

[0023] Furthermore, the spatial spectral map convolutional coding branch is composed of... The multi-hop convolutional layers are constructed by connecting them in a series. The value range is 2 to 4; each layer first performs neighborhood aggregation using a normalized adjacency matrix to obtain the neighborhood aggregation result, and then calculates the attention coefficients of the neighborhood nodes using the learnable attention vector to weight the aggregation result, the th... The formulas for calculating the neighborhood aggregation and attention coefficient of a layer are shown below:

[0024]

[0025]

[0026]

[0027] in, Indicates the first The node feature matrix of the layer; Indicates the first The learnable weight matrix of the layer; Represents a node The set of adjacent nodes; Represents a node For nodes Attention coefficient; Represents a learnable attention vector; The activation function is represented by ReLU or LeakyReLU. To account for neighborhood information of different orders, the outputs of each layer are concatenated along the channel dimension and dimensionality is reduced by linear mapping to obtain the empty spectral structure features. The multi-hop feature concatenation calculation formula is shown below:

[0028]

[0029] in, Indicates the first The output of the layer graph convolutional layer; Represents the learnable weight matrix for multi-hop feature fusion; This indicates the structural features of the empty spectrum.

[0030] Furthermore, the method for obtaining the bidirectional spatial-spectral selective scanning branch includes: since the computational complexity of the self-attention mechanism is quadratic with the sequence length, directly modeling all graph nodes of the entire image is too costly, so a selective state-space model with linear complexity is introduced; the nodes in the spatial-spectral structural features output by the spatial-spectral convolutional coding branch are sorted by superpixel centroid row and column coordinates in a forward raster order to obtain a forward sequence, and in reverse order to obtain a reverse sequence; the two sequences are modeled separately using a continuous-time state-space model, and the zero-order preservation method is used to discretize them into a recursive form. The continuous state-space model and the discretization calculation formula are shown below:

[0031]

[0032]

[0033]

[0034] in, Indicates the input sequence in The value at time; Represents the hidden state vector; Indicates the output; Represents the state transition matrix; , , These represent the input projection matrix, the output projection matrix, and the direct connection matrix, respectively. Indicates the discretization step size; , These represent the discretized state transition matrix and the input projection matrix, respectively. , , They represent the discretized first and second digits respectively. The hidden states, inputs, and outputs at each position are defined. To enable selective memory by allowing the parameters of the state-space model to adapt to the input, the discretization step size, input projection matrix, and output projection matrix are set as functions of the input. The formulas for calculating the selective parameters are as follows:

[0035]

[0036]

[0037] in, Indicates the node sequence at the 1st Input features at each location; , , These are the learnable weight matrices used to generate the discretization step size, the input projection matrix, and the output projection matrix, respectively. For learnable bias terms; , , These represent the discretization step size, input projection matrix, and output projection matrix, respectively, which adapt to the input. Represents the node feature tensor; , These represent the forward scan and reverse scan operators, respectively; The learnable weight matrix represents the output projection; This represents a global long-range dependency feature.

[0038] Furthermore, the fusion process of the adaptive gating fusion submodule is as follows: Since the spatial spectral structure features are good at characterizing local heterogeneous neighborhood structures, while the global long-range dependency features are good at characterizing the spectral context of the entire image, the importance of the two features changes dynamically with position and category. Therefore, it is not appropriate to use direct addition or simple concatenation with fixed weights. The spatial spectral structure features and the global long-range dependency features are concatenated along the channel dimension, and channel-wise gating weights are generated by one-dimensional convolution and the Sigmoid activation function. The two branch features are then fused by complementary weighting according to the gating weights. The formula for calculating the fused classification features is as follows:

[0039]

[0040]

[0041] in, Indicates the structural features of the empty spectrum; Indicates global long-range dependency features; This represents the learnable weight matrix that generates the gated weights; This represents the channel-by-channel gating weight, with a value ranging from 0 to 1; This indicates element-wise multiplication; Indicates fusion classification features; Indicates the first The fusion classification features of each node; , These represent the weight matrix and bias term of the classification head, respectively; Indicates the first The probability distribution of each node belonging to a category.

[0042] Furthermore, the method for obtaining the joint loss function includes: addressing the problems of uneven distribution of land cover categories, scarce labeled samples, and large spectral dispersion of similar land cover categories in hyperspectral images, introducing a graph Laplace smoothing constraint term and a prototype contrast loss on top of the weighted cross-entropy loss. The weighted cross-entropy loss sets category weights according to the reciprocal of the number of samples in each category to alleviate category imbalance; the graph Laplace smoothing constraint term makes the feature representations of adjacent nodes in the graph more consistent to enhance spatial smoothness; and the prototype contrast loss uses the mean of the current batch features of each category as the prototype and adjusts the distribution sharpness with a temperature coefficient. The calculation formulas for the weighted cross-entropy loss, the graph Laplace smoothing constraint term, and the prototype contrast loss are shown below:

[0043]

[0044]

[0045]

[0046]

[0047] in, Indicates the total number of training samples; Indicates the total number of land feature categories; Indicates the first Number of samples in the class; Indicates the first Class weights; Indicates the first The sample at the th Real labels on the class; This represents the predicted probability output by the network. This represents the weighted cross-entropy loss; Represents the fusion classification feature matrix; Represents the Laplace matrix of a symmetric normalized graph; Represents the trace of a matrix; This represents the graph Laplace smoothing constraint term; Indicates the first The prototype feature vector of the class, derived from the current batch of the first... The mean of the features of the class samples is calculated; This represents the temperature coefficient, with a value ranging from 0.1 to 1.0. Indicates prototype contrast loss; , This is the multi-task balancing coefficient.

[0048] Furthermore, the training method of the SSGM-Net network includes: dividing the samples by randomly selecting a fixed proportion of pixels from each class as the training set and the remainder as the test set; using the AdamW optimizer, with an initial learning rate set to... The weight decay coefficient is set to The learning rate is dynamically adjusted according to the cosine annealing strategy; the batch size is set to 64, and the training iterations are conducted for no less than 300 epochs; the multi-task balancing coefficient is set to... , The temperature coefficient is set to Random dropout is used after both the graph convolutional layer and the selective scan layer, with the dropout probability set to 0.2 to 0.5. Training is stopped when the overall loss of the validation set no longer decreases for 30 consecutive epochs, and the optimal model weights are saved.

[0049] Furthermore, the superpixel majority voting and fully connected conditional random field post-processing method includes: since the spectral consistency within a superpixel ensures a high degree of uniformity in its land cover category, voting is first performed on a superpixel-by-unit basis to eliminate sporadic misclassifications within the superpixel; the number of pixels of each category within each superpixel is counted, and the majority category is used as the category label for that superpixel. The formula for calculating the decision with the most votes is as follows:

[0050]

[0051] in, Indicates the first The set of pixels within a superpixel; Indicates the first The initial predicted category for each pixel; This indicates an indicator function that takes the value 1 if the condition is true and 0 otherwise. Indicates the first Voting category labels for each superpixel; fully connected conditional random field inference is performed using an energy function composed of a univariate potential function and a binary potential function. The univariate potential function takes the negative log probability of the class output by the network, and the binary potential function is composed of a weighted average of spatial spectral feature similarity and spatial proximity. The calculation formulas for the energy function and the binary potential function are shown below:

[0052]

[0053]

[0054] in, Represents the energy function; Represents a univariate potential function; Represents a binary potential function; This represents the label compatibility function, which takes the value 1 when two pixel labels are different and 0 otherwise. Represents the Euclidean distance between the spatial spectral features of two pixels; This represents the spatial Euclidean distance between two pixels; Indicates the spectral kernel bandwidth; The spatial kernel bandwidth is represented by the label configuration that minimizes the energy function by means of the mean field approximation iterative solution, with 10 iterations, to obtain the optimized land cover classification map.

[0055] Furthermore, the accuracy evaluation method includes: using overall accuracy, average accuracy, and Kappa coefficient as quantitative evaluation indicators, wherein overall accuracy is the proportion of correctly classified pixels to the total number of pixels, average accuracy is the arithmetic mean of the accuracy of producers in each category, and the Kappa coefficient is the overall accuracy after deducting the influence of random consistency. The calculation formulas for overall accuracy, average accuracy, and Kappa coefficient are as follows:

[0056]

[0057] in, Indicates overall accuracy; Indicates average precision; Represents the Kappa coefficient; This represents the total number of pixels that were correctly classified. This represents the total number of pixels involved in the evaluation. Indicates the total number of land feature categories; Indicates the first The number of pixels in the class that were correctly classified; Indicates the first The actual number of pixels in the class; The expected proportion of random consistency is defined by the following formula: ; Indicates that it is predicted to be the first The number of pixels in the class.

[0058] This invention offers the following superior advantages: Firstly, by performing radiometric calibration, atmospheric correction, bad line repair, and minimum noise separation preprocessing on hyperspectral remote sensing images, it eliminates sensor response differences, atmospheric influences, and fringe noise, suppressing the interference of noise-dominant components on subsequent feature extraction. Secondly, due to the large number of bands and strong correlation between adjacent bands in hyperspectral images, principal component analysis is used to project the original bands into an uncorrelated low-dimensional principal component space, significantly reducing the feature dimension while retaining most of the information and alleviating the Hughes phenomenon. Thirdly, since pixel-level regular square neighborhoods are difficult to fit the true boundaries of ground features, a simple linear iterative clustering algorithm is used for superpixel segmentation, obtaining superpixels with consistent internal spectra and boundaries that fit the outlines of ground features as graph nodes. Sparse adjacency relationships are constructed based on spatial-spectral composite kernel similarity, allowing the graph structure to simultaneously carry prior spectral similarity and spatial proximity. Fourthly, since the adjacency relationships on the spatial-spectral graph are non-Euclidean, a multi-hop attention graph convolutional layer is used to adaptively weight and aggregate heterogeneous neighborhood information. Multi-hop stitching also takes into account neighborhood information of different orders, enhancing the handling of irregular ground features. The system effectively depicts spatial morphology. Since the complexity of the self-attention mechanism is quadratically related to the sequence length, it struggles to handle long sequences composed of massive pixels and hundreds of bands. Therefore, a selective state-space model is introduced, using parameters as input functions to achieve selective memorization. This achieves global long-range spectral dependence modeling with linear complexity, and scanning is performed in both forward and reverse raster sequences to eliminate directional bias in unidirectional scanning. Because the importance of local structural features and global contextual features dynamically changes with location and category, an adaptive gated fusion submodule is designed for channel-by-channel complementary weighted fusion, avoiding the expression bottlenecks of direct addition and simple splicing. Due to the uneven distribution of land cover categories and the scarcity of labeled samples, a graph Laplacian smoothing constraint and prototype contrast loss are introduced on top of the weighted cross-entropy loss to enhance spatial smoothness and intra-class feature compactness, respectively, suppressing inter-class confusion. Since single-point pixel prediction is prone to salt-and-pepper noise, superpixel majority voting and fully connected conditional random fields are used for post-processing, preserving land cover boundary details while eliminating sporadic misclassifications, resulting in complete classification patches with well-fitting boundaries. This invention fully considers the characteristics of hyperspectral imagery, such as map-spectrum integration, high dimensionality, sparse labeled samples, and class imbalance. It designs a unique framework for superpixel spatial spectrum construction, multi-hop attention map convolution, bidirectional spatial spectrum selective scanning, adaptive gated fusion, and joint loss supervision. Under the condition of a small number of labeled samples, it significantly improves the accuracy of land cover classification and the consistency of land cover boundaries, providing a reliable technical means for the fine interpretation of hyperspectral remote sensing images. Attached Figure Description

[0059] To more clearly illustrate the solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of a hyperspectral image land cover classification method based on the collaboration of spatial spectral map convolution and visual state space model, provided by an embodiment of the present invention.

[0061] Figure 2 This is a diagram of the SSGM-Net network structure provided in one embodiment of the present invention. Detailed Implementation

[0062] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a hyperspectral image land cover classification method based on the synergy of spatial spectral image convolution and visual state space model proposed according to the present invention. In the following description, different embodiments or one embodiment do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The specific scheme of the hyperspectral image land cover classification method based on the synergy of spatial spectral image convolution and visual state space model provided by the present invention is described in detail below with reference to the accompanying drawings.

[0063] Please see Figure 1 This paper illustrates a hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model, which includes the following steps.

[0064] Step S1: Acquire the hyperspectral remote sensing image to be classified and perform radiometric calibration, atmospheric correction, bad line repair and minimum noise separation preprocessing to obtain the preprocessed image.

[0065] The embodiments of this invention are mainly applied to scenarios such as crop type identification, forest and grassland cover classification, and urban feature mapping. In order to obtain high-quality hyperspectral data, hyperspectral remote sensing images covering the study area are first selected as the data source. This data source has complete spatial coverage and a cloud and fog coverage rate of less than 5%. Due to sensor response differences, atmospheric scattering absorption, and fringe noise during remote sensing imaging, the original images need to be strictly preprocessed to ensure the reliability of subsequent classification.

[0066] Preferably, in one embodiment of the present invention, a Gaofen-5 AHSI hyperspectral image is selected as the data source. This image has a spatial resolution of 30 meters, a spectral range of 400 nm to 2500 nm, a spectral resolution of 5 nm to 10 nm, and 330 bands. After removing water absorption bands and bands with low signal-to-noise ratios, 176 effective bands are retained. The image's DN value is converted to radiance using laboratory calibration coefficients. Atmospheric correction is performed using the FLAASH model, and radiance is converted to surface reflectance. Neighborhood pixel linear interpolation is used to repair bad lines, dead pixels, and stripe noise. The repaired image is then subjected to noise whitening and feature extraction using minimum noise separation transform. The 40 dominant noise components are removed based on the inflection point of the eigenvalue curve, resulting in a spatial dimension of [missing value]. The number of bands is Preprocessed images.

[0067] Step S2: Perform principal component analysis on the preprocessed image to obtain a dimension-reduced image; crop the spatial spectrum cube samples pixel by pixel on the dimension-reduced image according to the preset spatial window size to obtain a spatial spectrum cube sample set; perform superpixel segmentation on the preprocessed image to obtain superpixel segmentation results; use the superpixels in the superpixel segmentation results as graph nodes, construct the node adjacency relationship according to the spatial spectrum composite kernel similarity, and obtain a spatial spectrum graph structure sample set.

[0068] Because hyperspectral images have a large number of bands and strong correlations between adjacent bands, directly using all bands in network calculations would lead to serious redundancy and the curse of dimensionality. Therefore, it is necessary to reduce the spectral dimension. Since pixel-level regular square neighborhoods are difficult to fit the true boundaries of ground objects, it is necessary to construct graph nodes using superpixels as units.

[0069] Preferably, in one embodiment of the present invention, the band covariance matrix of the preprocessed image is calculated and eigenvalue decomposition is performed, and the first band covariance matrix is ​​taken. The eigenvectors corresponding to the largest eigenvalues ​​form a projection matrix, obtaining a dimensionality-reduced image with a cumulative contribution rate of no less than 99%; centering on each pixel in the dimensionality-reduced image, according to the spatial window size... The empty spectral cube samples are cropped, and the edge pixels are filled with mirror images. The dimension of the empty spectral features of each sample is [value missing]. A simple linear iterative clustering algorithm was used to perform superpixel segmentation on the preprocessed image. The number of superpixels was set to 1500, the compactness coefficient to 20, and the number of iterations to 20, resulting in approximately 1500 superpixel segmentation results. Each superpixel was treated as a graph node, and the mean of the spatial spectral features of all pixels within a superpixel was used as the node feature. The spectral kernel bandwidth was set... Space core bandwidth For each pixel, the node similarity is calculated using the spatial-spectral composite kernel similarity. Edges with similarities greater than 0.3 are retained, and only the edge with the highest similarity is kept for each node. Edges are used to obtain sparse adjacency relationships, and symmetric normalization is performed to obtain a normalized adjacency matrix. This allows for the construction of a sample set of empty spectral graph structures.

[0070] Step S3: Design a spatial spectrogram convolutional coding branch based on a multi-hop attention map convolutional network to transform the spatial spectrogram structure sample set into spatial spectrogram structure features; design a bidirectional spatial spectrogram selective scanning branch based on a selective state space model to flatten and scan the spatial spectrogram structure features in both forward and reverse spatial grating sequences to obtain global long-range dependency features; design an adaptive gated fusion submodule to perform channel-by-channel gated fusion of the spatial spectrogram structure features and global long-range dependency features to obtain fused classification features; and concatenate the classification head after the fused classification features to form the SSGM-Net network.

[0071] Since the adjacency relationships on the empty spectral graph are non-Euclidean, standard convolution cannot be directly applied to them. Therefore, graph convolution is used to aggregate neighborhood information. Because different neighboring nodes contribute differently to the central node, a learnable attention vector is introduced to calculate attention coefficients for weighted summation. Since the complexity of the self-attention mechanism is quadratically related to the sequence length, a selective state-space model is introduced to complete global modeling with linear complexity. Please refer to [link to relevant documentation]. Figure 2 This illustrates the SSGM-Net network structure provided in one embodiment of the present invention.

[0072] Preferably, in one embodiment of the present invention, the spatial spectrogram convolutional coding branch is set. The multi-hop attention graph convolutional layer has 128, 128, and 128 channels in each layer. Each layer first uses a normalized adjacency matrix. Neighborhood aggregation is performed, and then the attention coefficient is calculated using a learnable attention vector of dimension 256, with a negative slope of 0.2 and LeakyReLU as the activation function. The outputs of the three layers are concatenated along the channel dimension and then linearly mapped to reduce the dimension to 128 to obtain the empty spectral graph structure features. The state space dimension of the bidirectional spatial spectral selective scanning branch is set to 64. The graph nodes are sorted by centroid row and column coordinates in the forward raster to obtain the forward sequence, and the reverse sequence is obtained in reverse order. These are then fed into two parameter-independent selective scanning paths. Each path consists of depthwise separable convolution, SiLU activation, selective scanning, and layer normalization. The outputs from both paths are summed element-wise and then projected through the output matrix. Mapping to 128 dimensions yields global long-range dependency features. The adaptive gating fusion submodule will and The concatenation along the channel dimension results in 256 dimensions, which are then reduced to 128 dimensions by a one-dimensional convolution with a kernel size of 1, and channel-wise gated weights are generated by the Sigmoid function. ,according to Complementary weighted fusion is performed to obtain fused classification features. The classification head consists of a fully connected layer and a softmax function, and its output dimension is equal to the total number of land cover categories. .

[0073] Step S4: Based on the characteristics of uneven distribution of land cover categories, sparse labeled samples, and large spectral dispersion of similar land cover categories in hyperspectral images, a joint loss function is designed to obtain training supervision signals; the SSGM-Net network is trained based on the training supervision signals to obtain a trained network model.

[0074] Because the distribution of land cover categories in hyperspectral images is extremely uneven, equal weighting of all categories would cause small sample categories to be overwhelmed by large sample categories. Therefore, category weights are set according to the reciprocal of the number of samples in each category. Because the spectral dispersion within the same land cover is large and pixels at different locations are easily misclassified, a graph Laplacian smoothing constraint term is introduced to make the features of adjacent nodes more consistent. Because the scarcity of labeled samples leads to loose intra-class features, a prototype contrast loss is introduced to enhance intra-class compactness and inter-class separability.

[0075] Preferably, in one embodiment of the present invention, samples are divided by randomly selecting 5% of pixels from each class as the training set and the remaining 95% as the test set; class weights are set. The multi-task balancing factor is set to... , The temperature coefficient is set to The AdamW optimizer was used, with an initial learning rate set to... The weight decay coefficient is set to The learning rate is dynamically adjusted using a cosine annealing strategy; the batch size is set to 64, and training is performed iteratively for 300 epochs; random dropout is used after the graph convolutional layer and the selective scan layer, with a dropout probability of 0.3; training is stopped when the overall loss of the validation set no longer decreases for 30 consecutive epochs, the optimal model weights are saved, and the trained network model is obtained.

[0076] Step S5: Use the trained network model to classify the hyperspectral remote sensing image to be classified pixel by pixel to obtain the initial classification result; perform superpixel majority voting and fully connected conditional random field post-processing on the initial classification result to obtain the land cover classification map; evaluate the overall accuracy, average accuracy and Kappa coefficient of the land cover classification map.

[0077] Since the spectra within superpixels are highly consistent and their land cover categories are uniform, a majority vote is first conducted using superpixels as units. Since single-pixel predictions still have sporadic misclassifications and boundary spikes, a fully connected conditional random field is used for further smoothing, and the spectral differences at the land cover boundaries are explicitly preserved in the random field inference.

[0078] Preferably, in one embodiment of the present invention, the image to be classified is constructed into an empty spectral map structure sample set in the same manner as steps S1 to S2, and the trained network model is input to obtain the class probability of each pixel. The class corresponding to the maximum probability is taken as the initial classification result. For each superpixel, the number of pixels of each class is counted, and all pixels in the superpixel are uniformly assigned the class with the most votes to obtain the classification result after voting. The negative logarithmic probability of the class output by the network is used as a univariate potential function, and the spectral kernel bandwidth is set. Space core bandwidth For each pixel, the label configuration that minimizes the energy function is obtained by iterating 10 times using the mean field approximation to obtain the final land cover classification map. Using a high-precision real land surface map interpreted by humans as a reference, the confusion matrix is ​​statistically analyzed and the overall accuracy, average accuracy and Kappa coefficient are calculated. At the same time, the producer accuracy and user accuracy of each category are output to assist in the result analysis.

Claims

1. A hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model, characterized in that, The method includes: acquiring a hyperspectral remote sensing image to be classified and performing radiometric calibration, atmospheric correction, bad line repair, and minimum noise separation preprocessing to obtain a preprocessed image; performing principal component analysis on the preprocessed image to obtain a dimensionality-reduced image; cropping spatial spectrum cube samples pixel-by-pixel on the dimensionality-reduced image according to a preset spatial window size to obtain a spatial spectrum cube sample set; performing superpixel segmentation on the preprocessed image to obtain superpixel segmentation results; using superpixels in the superpixel segmentation results as graph nodes, constructing node adjacency relationships according to spatial spectrum composite kernel similarity to obtain a spatial spectrum graph structure sample set; designing a spatial spectrum graph convolutional coding branch based on a multi-hop attention graph convolutional network to convert the spatial spectrum graph structure sample set into spatial spectrum graph structure features; and designing a bidirectional spatial spectrum selective scanning branch based on a selective state space model to flatten and scan the spatial spectrum graph structure features according to forward and reverse spatial grating sequences to obtain... Global long-range dependency features are obtained; an adaptive gated fusion submodule is designed to perform channel-by-channel gated fusion of spatial spectral image structure features and global long-range dependency features to obtain fused classification features; a classification head is concatenated after the fused classification features to form an SSGM-Net network; a joint loss function is designed based on the characteristics of uneven distribution of land cover categories, sparse labeled samples, and large spectral dispersion of similar land cover categories in hyperspectral images to obtain training supervision signals; the SSGM-Net network is trained based on the training supervision signals to obtain a trained network model; the trained network model is used to classify the hyperspectral remote sensing images to be classified pixel by pixel to obtain initial classification results; the initial classification results are post-processed with superpixel majority voting and fully connected conditional random fields to obtain a land cover classification map; the overall accuracy, average accuracy, and Kappa coefficient of the land cover classification map are evaluated.

2. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The hyperspectral remote sensing image acquisition and preprocessing method includes: selecting hyperspectral remote sensing images with a spatial resolution of not less than 30 meters, a spectral resolution better than 10 nanometers, and not less than 100 bands as data sources, including Gaofen-5 AHSI, Zhuhai-1 OHS or AVIRIS, and ROSIS airborne hyperspectral sensors; performing radiometric calibration and atmospheric correction on the original images sequentially to eliminate sensor response differences and atmospheric scattering and absorption effects; using a neighborhood pixel linear interpolation method to repair bad lines, dead pixels, and fringe noise; and using minimum noise separation transform to perform noise whitening and feature extraction on the repaired images, removing noise-dominant components with a signal-to-noise ratio lower than a set threshold to obtain images with a spatial dimension of [missing information]. The number of bands is Preprocessed images.

3. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The method for obtaining the principal component analysis and spatial spectrum cube sample set includes: calculating the band covariance matrix of the preprocessed image and performing eigenvalue decomposition. The covariance matrix and characteristic equation of the principal component analysis are shown below: , , In the formula, Indicates by A data matrix consisting of spectral vectors of 1000 pixels; Represents the sample spectral mean vector; Represents the band covariance matrix; This represents a diagonal matrix composed of eigenvalues; This represents the projection matrix formed by eigenvectors arranged in columns; Represents the low-dimensional data after projection; take the first digit. The eigenvectors corresponding to the largest eigenvalues ​​form a projection matrix, which projects the preprocessed image into a low-dimensional space, resulting in a band number of [value missing]. Dimensional reduction image, The value ranges from 15 to 40; centered on each pixel in the dimension-reduced image, according to the preset spatial window size. Trim the empty spectral cube samples, and the dimension of the empty spectral features of each sample is... The size of the spatial window is an odd number and its value ranges from 7 to 11.

4. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The method for obtaining the spatial spectral image structure sample set includes: performing superpixel segmentation on the preprocessed image using a simple linear iterative clustering algorithm, simultaneously measuring spectral distance and spatial distance during clustering, with the distance measurement formula as follows: In the formula, This represents the spectral distance between a pixel and the cluster center; This represents the spatial Euclidean distance between the two. , These are the normalization factors for spectral distance and spatial distance, respectively; a superpixel segmentation result is obtained, consisting of several superpixels with consistent internal spectra and boundaries conforming to the contours of ground features; each superpixel is considered a graph node, and the average spatial-spectral feature of all pixels within the superpixel is used as the node feature; the adjacency of two nodes is determined by the spatial-spectral composite kernel similarity, and the formula for calculating the spatial-spectral composite kernel similarity is as follows: In the formula, , They represent the first The and the first The mean vector of the spatial spectral features of each node; This represents the spatial Euclidean distance between the centroids of two nodes; The spectral kernel bandwidth; For space kernel bandwidth; This represents the similarity of the spatial-spectral composite kernels of two nodes.

5. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 4, characterized in that, If the similarity of the spatial spectrum composite kernel between two nodes is greater than the similarity threshold, the edge is retained; otherwise, it is set to zero. Only the edge with the highest similarity is retained for each node. Use edges to obtain a sparse adjacency matrix. The value range is 4 to 12; The adjacency relationships are symmetrically normalized to obtain the normalized adjacency matrix. The formula for symmetric normalization is as follows: , In the formula, This represents the adjacency matrix composed of spatial spectrum composite kernel similarity; Represents the identity matrix; Representing the adjacency matrix The degree matrix; Represents the normalized adjacency matrix; This represents the Laplace matrix of the symmetric normalized graph.

6. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The spatial spectrum convolutional coding branch is composed of The multi-hop convolutional layers are constructed by connecting them in a series. The value range is 2 to 4; each layer first performs neighborhood aggregation using a normalized adjacency matrix to obtain the neighborhood aggregation result: Then, the attention coefficients of the neighboring nodes are calculated using the learnable attention vectors, and the aggregation result is weighted accordingly. , In the formula, Indicates the first The node feature matrix of the layer; Indicates the first The learnable weight matrix of the layer; Represents a node The set of adjacent nodes; Represents a node For nodes Attention coefficient; Represents a learnable attention vector; The activation function is represented; the outputs of each layer are concatenated along the channel dimension and dimensionality reduced by linear mapping to obtain the empty spectral structure features: In the formula, Represents the learnable weight matrix for multi-hop feature fusion; This indicates the structural features of the empty spectrum.

7. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The method for obtaining the bidirectional spatial-spectral selective scanning branch includes: sorting the nodes in the spatial-spectral image structural features output by the spatial-spectral image convolutional coding branch in a forward raster order according to the superpixel centroid row and column coordinates to obtain a forward sequence, and sorting them in reverse order to obtain a reverse sequence; and modeling the two sequences separately using a continuous-time state-space model. , The zero-order preservation method is used to discretize it into a recursive form. The discretization calculation formula is as follows: , , , In the formula, Indicates the input sequence in The value at time; Represents the hidden state vector; Indicates the output; Represents the state transition matrix; , , These represent the input projection matrix, the output projection matrix, and the direct connection matrix, respectively. Indicates the discretization step size; , Let represent the discretized state transition matrix and the input projection matrix, respectively. To enable selective memory by allowing the parameters of the state-space model to adapt to the input, the discretization step size, the input projection matrix, and the output projection matrix are set as functions of the input. The formula for calculating the selective parameters is as follows: , , In the formula, , , These are the learnable weight matrices used to generate the discretization step size, the input projection matrix, and the output projection matrix, respectively. The learnable bias term is used; the global long-range dependency feature is obtained by summing the forward and reverse scan results, followed by layer normalization and output projection. In the formula, Represents the node feature tensor; , These represent the forward scan and reverse scan operators, respectively; The learnable weight matrix represents the output projection; This represents a global long-range dependency feature.

8. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The fusion process of the adaptive gated fusion submodule is as follows: The spatial spectrogram structural features and global long-range dependency features are concatenated along the channel dimension, and channel-wise gate weights are generated through one-dimensional convolution and the Sigmoid activation function. The features of the two branches are then fused using complementary weighting according to these gate weights. The formula for calculating the fused classification features is as follows: , , In the formula, Indicates the structural features of the empty spectrum; Indicates global long-range dependency features; This represents the learnable weight matrix that generates the gated weights; Indicates the channel-by-channel gating weight; This indicates element-wise multiplication; Indicates fusion classification features; Indicates the first The fusion classification features of each node; , These represent the weight matrix and bias term of the classification head, respectively; Indicates the first The probability distribution of each node belonging to a category.

9. The hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The joint loss function consists of three terms: weighted cross-entropy loss, graphical Laplacian smoothing constraint term, and prototype contrast loss. The weighted cross-entropy loss sets the class weights according to the reciprocal of the number of samples in each class to alleviate class imbalance. , The graph Laplace smoothing constraint term makes the characteristic representations of adjacent nodes in the graph tend to be consistent to enhance spatial smoothness: The prototype contrastive loss uses the mean of the current batch features for each category as the prototype and adjusts the distribution sharpness using a temperature coefficient. The combined loss is In the formula, Indicates the total number of training samples; Indicates the total number of land feature categories; Indicates the first Number of samples in the class; Indicates the first The sample at the th Real labels on the class; This represents the predicted probability output by the network. Represents the trace of a matrix; Indicates the first The prototype feature vector of the class; Indicates the temperature coefficient; , This is the multi-task balancing coefficient.

10. A hyperspectral image land cover classification method based on the synergy of spatial spectral map convolution and visual state space model as described in claim 1, characterized in that, The superpixel majority voting and fully connected conditional random field post-processing method includes: counting the number of pixels of each category within each superpixel and using the majority category as the category label for that superpixel; the formula for calculating the decision with the most votes is as follows: In the formula, Indicates the first The set of pixels within a superpixel; Indicates the first The initial predicted category for each pixel; This indicates an indicator function that takes the value 1 if the condition is true and 0 otherwise. Indicates the first Voting category labels for each superpixel; then, using univariate and binary potential functions to construct an energy function for fully connected conditional random field inference: The univariate potential function takes the negative log-probability of the class output by the network, while the binary potential function is composed of a weighted average of spatial spectral feature similarity and spatial proximity. In the formula, This represents the tag compatibility function; Represents the Euclidean distance between the spatial spectral features of two pixels; This represents the spatial Euclidean distance between two pixels; Indicates the spectral kernel bandwidth; The spatial kernel bandwidth is represented; the label configuration that minimizes the energy function is obtained by iteratively solving the mean-field approximation to obtain the land cover classification map; the land cover classification map is evaluated by overall accuracy, average accuracy, and Kappa coefficient. , , In the formula, This represents the total number of pixels that were correctly classified. Indicates the first The number of pixels in the class that were correctly classified; This represents the expected proportion of random consistency.

Images