Hyperspectral image classification method fusing Mangbar and Chebyshev image convolution

Abstract

The invention provides a hyperspectral image classification method fusing Mangbar and Chebyshev image convolution, and relates to the technical field of hyperspectral image detection. The method comprises the steps that a hyperspectral image data set is collected and preprocessed, and each image is divided into a preset number of image blocks with each pixel point as the center; inputting the image block into a heterogeneous spatial convolution block for processing to obtain output data with different receptive fields; the output data is divided into two branches, one branch is processed through a band selection enhanced bidirectional Mangban branch, the long-range dependency relationship between the spatial features and the spectral features is obtained, and first output is obtained; the other branch is input into the re-parameterized Chebyshev graph volume integral branch for processing, the similarity relation between different pixels is obtained, and second output is obtained; and fusing the first output and the second output to obtain a double-branch fusion feature, and inputting the double-branch fusion feature into a classifier to obtain a final classification result. The method can improve the classification precision and reduce the calculation cost.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image detection, and in particular to a hyperspectral image classification method integrating Mamba and Chebyshev graph convolution. Background Art

[0002] Hyperspectral image detection technology has a wide range of applications in many fields, such as environmental monitoring, agricultural remote sensing, etc. Hyperspectral imaging technology can provide rich spectral information from ultraviolet to near-infrared, and can more accurately analyze objects and scenes in images. However, traditional hyperspectral image classification methods usually face many challenges, such as the effective fusion of spectral features and spatial information, and high computational complexity when processing large-scale data.

[0003] At present, hyperspectral image classification methods based on basic spectral information include support vector machines, nearest neighbors, decision trees, extreme learning machines, etc. These methods mainly rely on spectral features for pixel-level classification, but often ignore the spatial information of the image. Therefore, for complex image scenes, the classification accuracy is usually not high. In order to improve the classification accuracy, researchers have tried to combine spatial information with spectral information in recent years, and proposed hyperspectral image classification models based on convolutional neural networks or graph convolutional networks, which usually process spatial information through multi-layer convolution and use spectral information for feature extraction. However, the above methods still have the following problems:

[0004] (1) Most of the current hyperspectral image classification methods use independent network architectures to extract spatial features and spectral features respectively, and usually ignore the deep-level relationship between the two when processing. Although these methods can guarantee a certain classification accuracy in some cases, when fusing spatial scale and spectral scale information, they often use simple linear fusion methods or independent processing, which makes it difficult to effectively capture the complex nonlinear relationship between the two and the feature differences at different scales, resulting in the model being unable to fully explore the complementarity of spatial information and spectral features.

[0005] (2) Although traditional convolutional neural networks, transformer models and other networks have significantly improved classification accuracy compared to machine learning methods, their computational complexity and model complexity are still problems, especially in the processing of large-scale hyperspectral datasets. Since the resolution of hyperspectral images is generally high, it is difficult to improve the computational efficiency of traditional models when the size of the dataset increases. Therefore, when such models process large-scale hyperspectral images, the resources and time costs required are greatly increased. However, due to the limitations of the network sampling structure, general convolutional neural network models cannot process information in non-Euclidean space and it is difficult to effectively establish similarity relationships between pixels in the image.

[0006] (3) Existing hyperspectral classification models often use a single network architecture, which makes it difficult to take into account the processing of spectral features and spatial features while capturing the similarity relationship between pixels in the image. A single network architecture is difficult to achieve a balance between spectral features, spatial features and pixel relationship modeling. Convolutional neural networks are designed to focus on the extraction of spatial features, so the processing of spectral features is usually relatively simple, which may weaken the importance of spectral features. In addition, in complex hyperspectral scenes, there may be cross-regional spectral similarities between pixels, and these pixels may be ignored by convolutional neural networks; and graph convolutional neural networks will find it difficult to distinguish images with similar spectra but different spatial distributions if they do not combine spatial features.

[0007] Therefore, there is a problem in the prior art that it is difficult to fuse spectral features with spatial features. Summary of the invention

[0008] In view of the above problems, the purpose of the present invention is to provide a hyperspectral image classification method that integrates Mamba and Chebyshev graph convolution, which captures the long-range dependencies between spatial and spectral dimensions by enhancing the bidirectional Mamba model through band selection, and at the same time utilizes the feature propagation capability of reparameterized Chebyshev graph convolution in non-Euclidean space to deeply explore the complementarity of spatial and spectral features, significantly improve classification accuracy, reduce computational costs, and improve efficiency and stability when processing large-scale hyperspectral data.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] On the one hand, a hyperspectral image classification method integrating Mamba and Chebyshev graph convolution is provided, the method comprising the following steps:

[0011] S1. Collect a hyperspectral image dataset and divide it into a training set and a validation set in proportion.

[0012] S2, preprocessing the image data, dividing each image into a predetermined number of image blocks X with each pixel as the center;

[0013] S3: Input the image block X into the heterogeneous spatial convolution block for processing to obtain output data X with different receptive fields. out ;

[0014] S4, X out It is divided into two branches, one of which is processed by band-selectively enhancing the bidirectional Mamba branch to obtain the long-range dependency between the spatial and spectral features in the image;

[0015] S5. Use the band-selective Mamba model to generate channel importance scores S for different spectral channels, and perform weighted processing with the output of the band-selective enhanced bidirectional Mamba branch to obtain the first output YM ;

[0016] S6. X out The other branch inputs the re-parameterized Chebyshev graph convolution branch for processing to obtain the similarity relationship between different pixels in the image and obtain the second output Y;

[0017] S7, using the dual-branch integrated module to the first output Y M It is fused with the second output Y to obtain a dual-branch fusion feature, which is input into the classifier to obtain the final classification result.

[0018] Optionally, the step S1 specifically includes:

[0019] Collect multiple different public hyperspectral image datasets, with different spectral band numbers, coverage, image sizes, and ground sampling distances between different datasets;

[0020] The collected hyperspectral image dataset is divided into a training set and a validation set in a ratio of 11:1; the training set is used for model training, and the validation set is used to verify the model effect; the model with the best performance on the validation set is saved as the final model.

[0021] Optionally, the step S2 specifically includes:

[0022] Each image is divided into a predetermined number of image blocks X with each pixel as the center, which are used as the input of the model; input image block X∈R B×Cin×H×W , where B represents the batch size, C in is the number of input spectral channels, H and W are the height and width of the image block.

[0023] Optionally, in step S3, the processing process of the heterogeneous spatial convolution block includes:

[0024] Use a 1×1 convolution to map the input image block X to a high-dimensional embedding space to generate a high-dimensional feature X emb :

[0025]

[0026] Where X emb ∈R B×Cemb×H×W , C emb Set to 256, indicating the number of spectral channels after embedding;

[0027] For high-dimensional features X emb Perform 1×1 convolution and 3×3 convolution operations, add and fuse the two convolution results element by element to obtain a multi-scale feature combination, and then use a 1×1 convolution to restore the number of spectral channels to the number of spectral channels C of the original input. in , the operation is as follows:

[0028]

[0029] Where X out ∈R B×Cin×H×W , 1×1 convolution operation is used to extract local fine-grained features of each pixel, and 3×3 convolution operation is used to extract coarse-grained features larger than the local receptive field.

[0030] Optionally, in step S4, the process of band selection enhancing the bidirectional Mamba branch includes:

[0031] The output data X of the heterogeneous spatial convolution block out An input position information encoder is used to add position information to the input data through position information encoding to provide context information of the spatial arrangement of pixels;

[0032] The encoded data is input into the depth-separable convolutional layer for feature extraction, and the spatial features X of the image are extracted. c ; The formula of band selection enhanced bidirectional Mamba branch is expressed as:

[0033]

[0034] Among them, Mamba spaf (X c ) and Mamba spab (X c ) represent the operation process of forward space Mamba and backward space Mamba respectively, Flip(X c ) represents the flip operation of data, X spaf is the output of the forward spatial Mamba, X spab is the output of the backward space Mamba.

[0035] Optionally, in step S5, the processing process of the band selection Mamba model includes:

[0036] Exchange the output of the depthwise separable convolutional layer X c The spatial and spectral dimensions of X r ,

[0037]

[0038] X r After the forward spectral Mamba and the backward spectral Mamba operations, the spectral feature X is obtained by adding them together. b :

[0039]

[0040] For spectral feature X bPerform pooling, fully connected layers, and sigmoid activation function operations to generate a channel importance score S, which represents the importance of different spectral channels:

[0041]

[0042] Multiply the output of the band selection enhanced bidirectional Mamba branch and the channel importance score S to obtain the weighted feature Y spaf and Y spab :

[0043]

[0044] The weighted features are fused through convolutional layers and concatenation operations to obtain the first output Y M :

[0045]

[0046] Y M It is a feature representation that contains the long-range dependency between the forward and backward spectral information and the spatial information in the image.

[0047] Optionally, in step S6, the process of re-parameterizing the Chebyshev graph convolution branch includes:

[0048] The output data X of the heterogeneous spatial convolution block out Perform layer normalization and treat each pixel as a node in the graph structure; use cosine similarity to calculate the similarity between all nodes and obtain the vector similarity matrix M;

[0049] The matching indicator sets a threshold T, filters the vector similarity matrix M, and generates an adjacency matrix A; performs degree mapping on the adjacency matrix to obtain a degree matrix D, and constructs a graph Laplacian matrix L based on the degree matrix D and the adjacency matrix A;

[0050] In order to make the nodes retain their own characteristics, self-loops are added to the graph Laplacian matrix L to obtain the updated graph Laplacian matrix ;

[0051] Use Chebyshev iteration to update the graph Laplacian matrix Perform calculations to obtain Chebyshev polynomials of multiple orders; Represents the k-order Chebyshev polynomial, which is calculated through a recursive relationship; the calculation result is input into the reparameterized weighter to obtain the reparameterized Chebyshev coefficient w k :

[0052] The output of the re-parameterized Chebyshev graph convolution branch, that is, the second output Y is expressed as:

[0053]

[0054] Where X in is the input feature matrix, containing the features of the nodes in the graph; Y∈R N×C , N is the number of pixels, C is the dimension of the output feature, and K is the order of the Chebyshev polynomial.

[0055] Optionally, the step S7 specifically includes:

[0056] The first output Y of the bidirectional Mamba branch is enhanced for band selection M The second output Y of the re-parameterized Chebyshev graph convolution branch is subjected to pooling, first-layer convolution, first activation function, second-layer convolution, and second activation function operations to generate channel attention weights;

[0057] The output feature of each branch is multiplied element-by-element by the corresponding channel attention weight, and then the weighted features are fused to obtain the dual-branch fusion feature;

[0058] The dual-branch fusion features are input into the classifier, the classification task is performed, and the final classification result is obtained.

[0059] In another aspect, an electronic device is provided, the electronic device comprising:

[0060] processor;

[0061] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned hyperspectral image classification method are implemented.

[0062] On the other hand, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores program codes, and the program codes can be called by a processor to execute the steps of the above-mentioned hyperspectral image classification method.

[0063] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0064] 1. The present invention provides a dual-branch structure combining a band-selective enhanced bidirectional Mamba network and a reparameterized Chebyshev graph convolutional network, which can simultaneously process spectral information and spatial information, effectively integrate the characteristics of the two, accurately capture the complex nonlinear relationship between spectral characteristics and spatial characteristics, and comprehensively capture the similarity relationship between image pixels, thereby improving the accuracy of hyperspectral image classification.

[0065] 2. The present invention can effectively process the similarity relationship between pixels in non-Euclidean space through graph structure modeling of graph convolution, and uses the Mamba model to enhance the selection of band information and construct long-distance dependencies of pixels, providing a new solution for exploring complex spatial relationships in hyperspectral images.

[0066] 3. Relying on the linear complexity of the Mamba model and the efficient graph structure modeling advantages of the graph convolutional network, the present invention has better processing efficiency on large-scale data sets than the convolutional neural network and the transformer model, and has wide application potential in environments with limited computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 It is a schematic diagram of the overall process of a hyperspectral image classification method integrating Mamba and Chebyshev graph convolution provided by an embodiment of the present invention;

[0069] Figure 2 is a schematic diagram of a heterogeneous spatial convolution block provided by an embodiment of the present invention;

[0070] Figure 3 is a schematic diagram of a band-selective enhanced bidirectional Mamba branch provided by an embodiment of the present invention;

[0071] Figure 4 is a schematic diagram of a band selection Mamba model provided by an embodiment of the present invention;

[0072] Figure 5 is a schematic diagram of a re-parameterized Chebyshev graph convolution branch provided by an embodiment of the present invention;

[0073] Figure 6 Schematic diagram of a dual-branch integrated module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the word "exemplary" is used to present concepts in a specific way.

[0076] The embodiment of the present invention provides a hyperspectral image classification method integrating Mamba and Chebyshev graph convolution, such as Figure 1 As shown in the figure, the overall process of the method includes: the original hyperspectral image is first divided into image blocks, which are processed by heterogeneous spatial convolution blocks and then divided into two branches. One branch is processed by band selection enhanced bidirectional Mamba branch to obtain the long-range dependency relationship between spatial features and spectral features in the hyperspectral image, and the other branch is processed by re-parameterized Chebyshev graph convolution branch to obtain the similarity relationship between different pixels in the image; finally, the outputs of the two branches are effectively fused through a dual-branch integration module, and then the fused features are processed through pooling operation and classifier to output the final classification result.

[0077] During the training phase, the output and final output prediction error of the band selection enhanced bidirectional Mamba branch and the reparameterized Chebyshev graph convolution branch are measured using the cross entropy loss function. At the same time, in order to ensure the consistency of the decision results of the two branches, the KL divergence loss function (Kullback-Leibler Divergence Loss) is introduced to the output of both branches.

[0078] Specifically, the method comprises the following steps:

[0079] S1. Collect a hyperspectral image dataset and divide it into a training set and a validation set in proportion.

[0080] In this step, four different widely used public hyperspectral image datasets are collected, which differ in the number of spectral bands, coverage, image size, and ground sampling distance.

[0081] The collected hyperspectral image dataset is divided into a training set and a validation set in a ratio of 11:1 to form a complete dataset. The training set is used for model training, and the validation set is used to verify the model effect; the model with the best performance on the validation set is saved as the final model.

[0082] S2. Preprocess the image data and divide each image into a predetermined number of image blocks X with each pixel as the center.

[0083] In this step, each image is divided into a predetermined number of image blocks X with each pixel as the center. Each pixel and the pixels around it together constitute an image block. In this way, the image can be divided into as many image blocks as the number of pixels, which serve as the input of the model.

[0084] Input image patch X∈R B×Cin×H×W , where B represents the batch size, C inis the number of input spectral channels, H and W are the height and width of the image block.

[0085] S3: Input the image block X into the heterogeneous spatial convolution block for processing to obtain output data X with different receptive fields. out .

[0086] In this step, if Figure 2 As shown in Figure 2, the processing of the heterogeneous spatial convolution block includes:

[0087] Use a 1×1 convolution to map the input image block X to a high-dimensional embedding space to generate a high-dimensional feature X emb :

[0088]

[0089] Where X emb ∈R B×Cemb×H×W , C emb Set to 256, indicating the number of spectral channels after embedding.

[0090] This operation maps the spectral dimension of the input data to a higher feature space in preparation for subsequent operations such as convolution.

[0091] For high-dimensional features X emb Perform 1×1 convolution and 3×3 convolution operations, add and fuse the two convolution results element by element to obtain a multi-scale feature combination, and then use a 1×1 convolution to restore the number of spectral channels to the number of spectral channels C of the original input. in , the operation is as follows:

[0092]

[0093] Where X out ∈R B×Cin×H×W , 1×1 convolution operation is used to extract local fine-grained features of each pixel, and 3×3 convolution operation is used to extract coarse-grained features larger than the local receptive field.

[0094] S4, X out It is divided into two branches, one of which is processed by band-selectively enhancing the bidirectional Mamba branch to obtain the long-range dependency between the spatial and spectral features in the image.

[0095] In this step, if Figure 3 As shown, the processing process of band selection enhanced bidirectional Mamba branch includes:

[0096] The output data X of the heterogeneous spatial convolution block out An input position information encoder is used to add position information to the input data through position information encoding to provide context information of the spatial arrangement of pixels;

[0097] The encoded data is input into the depth-separable convolutional layer for feature extraction, and the spatial features X of the image are extracted. c Then X c As input, forward and backward spatial feature information extraction is performed.

[0098] The general state space model equation is:

[0099]

[0100] where h(t)∈R N represents the potential state, x(t)∈R represents the input, y(t)∈R represents the output, and A∈R N×N is the state transfer matrix, B∈R N and C∈R N are projection matrices that map input and latent states to output, respectively. The latent state h(t) encapsulates the internal state of the system, which evolves over time according to the input x(t) and the system dynamics determined by A. To meet the requirements of practical applications, the continuous state space model equation is discretized:

[0101]

[0102] in Indicates the length of the walk. represents the discrete state transfer matrix, represents the discrete input matrix and I is the identity matrix. This discretization can accurately capture the system dynamics at each time step, allowing the state to be updated at discrete intervals. The discretized state space model equation can be expressed as:

[0103]

[0104] where h t represents the potential state at time t. The output at time t is y t Represents, C is the output matrix. This discrete model ensures the same dynamic preservation performance as the original continuous system model and can be applied to forward and backward discrete state space modeling.

[0105] Applied to the Mamba branch, the formula for band-selective enhanced bidirectional Mamba branch is expressed as:

[0106]

[0107] Among them, Mamba spaf (X c ) and Mamba spab (X c ) represent the operation process of forward space Mamba and backward space Mamba respectively, Flip(Xc ) represents the flip operation of data, X spaf is the output of the forward spatial Mamba, and then the input sequence is flipped for the backward spatial Mamba, X spab is the output of the backward spatial Mamba. The forward spatial Mamba extracts the positive dependency from the starting point to the end point of the data, and the backward spatial Mamba extracts the reverse dependency from the end point to the starting point of the data.

[0108] S5. Use the band-selective Mamba model to generate channel importance scores S for different spectral channels, and perform weighted processing with the output of the band-selective enhanced bidirectional Mamba branch to obtain the first output Y M .

[0109] In this step, if Figure 4 As shown, the processing of the band selection Mamba model includes:

[0110] Exchange the output of the depthwise separable convolutional layer X c The spatial and spectral dimensions of X r ,

[0111]

[0112] X r After the forward spectral Mamba and the backward spectral Mamba operations, the spectral feature X is obtained by adding them together. b :

[0113]

[0114] For spectral feature X b Perform pooling, fully connected layers, and sigmoid activation function operations to generate a channel importance score S, which represents the importance of different spectral channels:

[0115]

[0116] Multiply the output of the band selection enhanced bidirectional Mamba branch and the channel importance score S to obtain the weighted feature Y spaf and Y spab :

[0117]

[0118] The weighted features are fused through convolutional layers and concatenation operations to obtain the first output Y M :

[0119]

[0120] Y M It is a feature representation that contains the long-range dependency between the forward and backward spectral information and the spatial information in the image.

[0121] S6. X out The other branch inputs the re-parameterized Chebyshev graph convolution branch for processing to obtain the similarity relationship between different pixels in the image and obtain the second output Y.

[0122] In this step, if Figure 5 As shown in Figure 2, the processing of the reparameterized Chebyshev graph convolution branch includes:

[0123] The output data X of the heterogeneous spatial convolution block out Perform layer normalization, treat each pixel as a node in the graph structure; use cosine similarity to calculate the similarity between all nodes and obtain the vector similarity matrix M, with x i and x j are the feature vectors of pixels i and j respectively, and each element of the vector similarity matrix M is:

[0124]

[0125] The matching indicator sets the threshold T, filters the vector similarity matrix M, and generates the adjacency matrix A. Each element of A is expressed as:

[0126]

[0127] Perform degree mapping on the adjacency matrix to obtain the degree matrix D. , construct the graph Laplacian matrix L according to the degree matrix D and the adjacency matrix A:

[0128]

[0129] In order to make the node retain its own characteristics, add self-loops to obtain the updated graph Laplacian matrix :

[0130]

[0131] Use Chebyshev iteration to update the graph Laplacian matrix Calculate and obtain Chebyshev polynomials of multiple orders:

[0132]

[0133] Where T0 represents the zero-order Chebyshev polynomial, and its value is fixed to 1; represents the first-order Chebyshev polynomial, which is equal to the Laplace matrix ; Represents the k-order Chebyshev polynomial, which is calculated through a recursive relationship; the iterative calculation result is input into the reparameterized weighter to obtain the reparameterized Chebyshev coefficient w k :

[0134]

[0135] where x j is the value of the Chebyshev node calculated during initialization, λ j is a learnable parameter; finally, the output of the reparameterized Chebyshev graph convolution branch is obtained, that is, the second output Y:

[0136]

[0137] Where X in is the input feature matrix, containing the features of the nodes in the graph; Y∈R N×C , N is the number of pixels, and C is the dimension of the output feature. After multi-order feature propagation, Y provides rich feature expressions for the classification task of hyperspectral images.

[0138] S7, using the dual-branch integrated module to the first output Y M It is fused with the second output Y to obtain a dual-branch fusion feature, which is input into the classifier to obtain the final classification result.

[0139] In this step, if Figure 6 As shown, the first output Y of the bidirectional Mamba branch is enhanced for the band selection. M The second output Y of the re-parameterized Chebyshev graph convolution branch is pooled (reducing the dimension of the feature while retaining key feature information), the first layer of convolution (extracting local spatial features), the first activation function (introducing nonlinearity), the second layer of convolution (further refining the feature representation to ensure that the output information is richer), and the second activation function to generate channel attention weights.

[0140] After that, the output features of each branch are multiplied element-by-element by the corresponding channel attention weight, and then the weighted features are fused to obtain the dual-branch fusion features. The fused features have complementary information of the two branches and have stronger classification representation capabilities.

[0141] Finally, the dual-branch fusion features are input into the classifier to perform the classification task and obtain the final classification result.

[0142] In an embodiment of the present invention, the bidirectional Mamba model is enhanced by band selection to capture the long-range dependencies between the spatial dimension and the spectral dimension, and the feature propagation capability of the reparameterized Chebyshev graph convolution in non-Euclidean space is utilized to deeply explore the complementarity of spatial features and spectral features, thereby significantly improving the classification accuracy. In addition, the present invention significantly reduces the computational cost by utilizing the linear complexity of the Mamba model and the efficient graph structure modeling of the reparameterized Chebyshev graph convolution. Compared with the quadratic complexity of the popular transformer model, the Mamba model can effectively process high-resolution data, and at the same time avoids explicit feature decomposition through the recursive approximation of Chebyshev polynomials, greatly reducing resource and time consumption, and improving the efficiency and stability when processing large-scale hyperspectral data.

[0143] Compared with the prior art, the present invention has the following advantages:

[0144] 1. When processing hyperspectral images, existing methods cannot deeply explore the complex relationship between the spatial and spectral features of the images because they process them independently. In addition, the fusion of spatial scale and spectral scale usually adopts a simple linear method, which cannot capture the deep correlation between the two. To address this problem, the present invention proposes a dual-branch structure based on band selection enhanced bidirectional mamba and reparameterized Chebyshev graph convolution to realize the classification method of hyperspectral images. The bidirectional mamba model is enhanced by band selection to model the long-range dependency of spatial and spectral information, and the reparameterized Chebyshev graph convolution model is used to efficiently model non-Euclidean spatial information. The spatial information and spectral features are deeply integrated, the complex nonlinear relationship between the two is effectively captured, and the accuracy of hyperspectral image classification is improved.

[0145] 2. Existing methods are often computationally inefficient and have high computational complexity when processing high-resolution and large-scale hyperspectral image data. To address this problem, the present invention uses the linear complexity of the Mamba model and the efficient graph structure modeling of the reparameterized Chebyshev graph convolution to effectively reduce the computational complexity, improve the computational efficiency, and expand the applicability of the model on large-scale data sets.

[0146] 3. The existing hyperspectral classification model uses a single network architecture, which makes it difficult to capture the similarity relationship between pixels in the image on the basis of processing spectral features and spatial features at the same time, resulting in insufficient fusion ability of spectral information and spatial information, and poor pixel relationship modeling effect. To address this problem, the present invention solves the problems of spectral feature and spatial feature extraction and image pixel similarity relationship modeling by designing a dual-branch structure of band selection enhanced bidirectional Mamba and re-parameterized Chebyshev graph convolution. The dual-branch structure makes full use of the complementarity of the two models in the fusion stage, which can not only extract spectral features and spatial features, but also fully capture the complex relationship between image pixels.

[0147] In an exemplary embodiment, the present invention further provides an electronic device, the electronic device comprising:

[0148] processor;

[0149] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are loaded and executed by the processor, the steps of the above-mentioned hyperspectral image classification method are implemented.

[0150] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is loaded and executed by a processor to implement the steps of the hyperspectral image classification method as described above. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0151] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0152] References in the specification to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments", etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the relevant art to implement such feature, structure, or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0153] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0154] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0155] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0156] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0159] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0160] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A hyperspectral image classification method integrating Mamba and Chebyshev graph convolution, characterized in that: The following steps are involved: S1. Collect a hyperspectral image dataset and divide it into a training set and a validation set in proportion. S2, preprocessing the image data, dividing each image into a predetermined number of image blocks X with each pixel as the center; S3: Input the image block X into the heterogeneous spatial convolution block for processing to obtain output data X with different receptive fields. out ; S4, X out It is divided into two branches, one of which is processed by band-selectively enhancing the bidirectional Mamba branch to obtain the long-range dependency between the spatial and spectral features in the image; S5. Use the band-selective Mamba model to generate channel importance scores S for different spectral channels, and perform weighted processing with the output of the band-selective enhanced bidirectional Mamba branch to obtain the first output Y M ; S6. X out The other branch inputs the re-parameterized Chebyshev graph convolution branch for processing to obtain the similarity relationship between different pixels in the image and obtain the second output Y; S7, using the dual-branch integrated module to the first output Y M It is fused with the second output Y to obtain a dual-branch fusion feature, which is input into the classifier to obtain the final classification result.

2. The hyperspectral image classification method according to claim 1, characterized in that: The step S1 specifically includes: Collect multiple different public hyperspectral image datasets, with different spectral band numbers, coverage, image sizes, and ground sampling distances between different datasets; The collected hyperspectral image dataset is divided into a training set and a validation set in a ratio of 11:1; the training set is used for model training, and the validation set is used to verify the model effect; the model with the best performance on the validation set is saved as the final model.

3. The hyperspectral image classification method according to claim 1, characterized in that: The step S2 specifically includes: Each image is divided into a predetermined number of image blocks X with each pixel as the center, which are used as the input of the model; input image block X∈R B×Cin×H×W , where B represents the batch size, C in is the number of input spectral channels, H and W are the height and width of the image block.

4. The hyperspectral image classification method according to claim 1, characterized in that: In step S3, the processing process of the heterogeneous spatial convolution block includes: Use a 1×1 convolution to map the input image block X to a high-dimensional embedding space to generate a high-dimensional feature X emb : ； Where X emb ∈R B×Cemb×H×W , C emb Set to 256, indicating the number of spectral channels after embedding; For high-dimensional features X emb Perform 1×1 convolution and 3×3 convolution operations, add and fuse the two convolution results element by element to obtain a multi-scale feature combination, and then use a 1×1 convolution to restore the number of spectral channels to the number of spectral channels of the original input C. in , the operation is as follows: ； Where X out ∈R B×Cin×H×W , 1×1 convolution operation is used to extract local fine-grained features of each pixel, and 3×3 convolution operation is used to extract coarse-grained features larger than the local receptive field.

5. The hyperspectral image classification method according to claim 1, characterized in that: In step S4, the process of band selection enhancing bidirectional Mamba branch includes: The output data X of the heterogeneous spatial convolution block out An input position information encoder is used to add position information to the input data through position information encoding to provide context information of the spatial arrangement of pixels; The encoded data is input into the depth-separable convolutional layer for feature extraction, and the spatial features X of the image are extracted. c ; The formula of band selection enhanced bidirectional Mamba branch is expressed as: ； Among them, Mamba spaf (X c ) and Mamba spab (X c ) represent the operation process of forward space Mamba and backward space Mamba respectively, Flip(X c ) represents the data flip operation, X spaf is the output of the forward spatial Mamba, X spab is the output of the backward space Mamba.

6. The hyperspectral image classification method according to claim 1, characterized in that: In step S5, the processing process of the band selection Mamba model includes: Exchange the output of the depthwise separable convolutional layer X c The spatial and spectral dimensions of X r , ； X r After the forward spectral Mamba and the backward spectral Mamba operations, the spectral feature X is obtained by adding them together. b : ； For spectral feature X b Perform pooling, fully connected layers, and sigmoid activation function operations to generate a channel importance score S, which represents the importance of different spectral channels: ； Multiply the output of the band selection enhanced bidirectional Mamba branch and the channel importance score S to obtain the weighted feature Y spaf and Y spab : ； The weighted features are fused through convolutional layers and concatenation operations to obtain the first output Y M : ； Y M It is a feature representation that contains the long-range dependency between the forward and backward spectral information and the spatial information in the image.

7. The hyperspectral image classification method according to claim 1, characterized in that: In step S6, the process of re-parameterizing the Chebyshev graph convolution branch includes: The output data X of the heterogeneous spatial convolution block out Perform layer normalization and treat each pixel as a node in the graph structure; use cosine similarity to calculate the similarity between all nodes and obtain the vector similarity matrix M; The matching indicator sets a threshold T, filters the vector similarity matrix M, and generates an adjacency matrix A; performs degree mapping on the adjacency matrix to obtain a degree matrix D, and constructs a graph Laplacian matrix L based on the degree matrix D and the adjacency matrix A; In order to make the nodes retain their own characteristics, self-loops are added to the graph Laplacian matrix L to obtain the updated graph Laplacian matrix ; Use Chebyshev iteration to update the graph Laplacian matrix Perform calculations to obtain Chebyshev polynomials of multiple orders; Represents the k-order Chebyshev polynomial, which is calculated through a recursive relationship; the calculation result is input into the reparameterized weighter to obtain the reparameterized Chebyshev coefficient w k : The output of the re-parameterized Chebyshev graph convolution branch, that is, the second output Y is expressed as: ； Where X in is the input feature matrix, containing the features of the nodes in the graph; Y∈R N×C , N is the number of pixels, C is the dimension of the output feature, and K is the order of the Chebyshev polynomial.

8. The hyperspectral image classification method according to claim 1, characterized in that: The step S7 specifically includes: The first output Y of the bidirectional Mamba branch is enhanced for band selection M The second output Y of the re-parameterized Chebyshev graph convolution branch is subjected to pooling, first-layer convolution, first activation function, second-layer convolution, and second activation function operations to generate channel attention weights; The output feature of each branch is multiplied element-by-element by the corresponding channel attention weight, and then the weighted features are fused to obtain the dual-branch fusion feature; The dual-branch fusion features are input into the classifier, the classification task is performed, and the final classification result is obtained.

9. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein the computer-readable instructions, when loaded and executed by the processor, implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 8.

Images