Remote sensing image land cover classification method and system based on Mama

Abstract

The invention discloses a Mama-based remote sensing image land cover classification method and system, and the method comprises the steps: obtaining remote sensing image data, carrying out the preprocessing, constructing a Mama-based remote sensing image classification network, fusing the global modeling advantages of a Mama architecture with the local fine representation characteristics of a KAG attention module, and carrying out the classification of the land cover of the remote sensing image. A feature extraction module with a multi-scale cooperative processing capability is constructed, and the limitation of limited receptive field of a traditional CNN and high calculation complexity of a TF network can be effectively dealt with. The multi-granularity analysis of the features is realized by adopting a double-branch parallel structure, and the fine-granularity feature identification efficiency is improved while the global context association capability is guaranteed. According to the inter-class imbalance characteristic of the remote sensing image, a joint optimization strategy of two loss functions is adopted, and learning weights of different ground feature classes are effectively balanced. According to the method provided by the invention, the land cover classification effect in a complex remote sensing scene is remarkably improved, and the method has an important engineering application value.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular, to a method and system for classifying land cover of remote sensing images based on Mamba. Background Art

[0002] With the rapid development of remote sensing technology, the acquisition ability of high-resolution remote sensing images has been significantly improved. As one of the core tasks of remote sensing image interpretation, land cover classification has important application values in fields such as land resource investigation, ecological environment monitoring, and urban planning. Traditional land cover classification methods mainly rely on manually designed features (such as texture, spectral index, etc.) combined with traditional machine learning models (such as support vector machines, random forests), but their feature expression ability is limited and it is difficult to handle the non-linear distribution of complex land object types and the detail differences in high-resolution images.

[0003] With the breakthrough of deep learning technology, models based on convolutional neural networks (CNNs) and Transformers have gradually become the mainstream methods for remote sensing image classification. CNNs extract spatial features through local receptive fields, but their global modeling ability is insufficient and it is difficult to effectively capture the long-range dependencies of widely distributed land objects in remote sensing images; while Transformers achieve global context modeling through self-attention mechanisms, but their computational complexity grows quadratically with the image size, facing huge computational resource consumption and memory bottlenecks when processing high-resolution remote sensing images.

[0004] As a new type of state space model, the Mamba architecture provides a new idea for long-range dependence modeling in high-resolution remote sensing image classification with its selective state mechanism and linear computational complexity advantage. The Kolmogorov-Arnold neural network (KAN) enhances the local feature modeling ability through learnable activation functions and can effectively capture the edge details of land objects. The current research difficulty lies in how to synergistically integrate the global modeling advantage of Mamba and the local refinement representation ability of KAN to accurately model the land object boundaries while maintaining the capture of global information of remote sensing images, which is of great significance for improving the land cover classification accuracy.

[0005] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art. Summary of the Invention

[0006] Therefore, the purpose of the present invention is to provide a method for classifying land cover of remote sensing images based on Mamba, which can more efficiently and accurately identify various land objects through global modeling and local refinement, effectively improve the land object classification accuracy, and is applicable to land cover classification in remote sensing scenarios.

[0007] To achieve the above object, the present invention also provides a method for classifying land cover of remote sensing images based on Mamba, comprising the following steps: S1. Obtain the original remote sensing image; S2. Input the original remote sensing image into the trained Mamba remote sensing image classification network. The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network; the core module connects the visual Mamba module and the KAG attention module in parallel; the Mamba remote sensing image classification network is trained using the established sample set; S3. Use the Mamba remote sensing image classification network to output the classification result of the land cover of the classified remote sensing image.

[0008] Further preferably, the process of establishing the sample set includes the following steps: Obtain the original remote sensing image data and preprocess it; Use ArcGIS software to draw the surface features of the ground, so that the original image data and label data meet the preset size, and obtain the preprocessed remote sensing image data; Divide the preprocessed images and the corresponding labels into a training set, a test set and a validation set according to a preset ratio.

[0009] Further preferably, the Mamba remote sensing image classification network includes: Set the encoder to divide the slices through patch partition, project the slices into a high-dimensional space, and use the Mamba-KAN module for 3 times of downsampling to extract feature information; At each horizontal stage, use skip connections to fuse the high-level semantic features and the shallow features. Finally, linearly interpolate and magnify the feature map by 4 times, and then use the softmax and argmax functions to segment the image pixels into multiple types of ground features.

[0010] Further preferably, the training process of the Mamba remote sensing image classification network includes the following steps: S201. Input the preprocessed remote sensing image data into the land cover classification network. During training, select the SGD optimizer function as the parameter optimizer, use the cross-entropy loss and the Lovász-Softmax loss as the loss functions, with an initial learning rate of 0.01, a training iteration number of 100, and a step size of 4; S202. Extract features through the Mamba-KAN module of the encoder in the land cover classification network, and the decoder uses the feature map generated by upsampling to fuse with the left-side feature map. In the process, each Mamba-KAN module is used to obtain the global and local features of the image; S203. Use a 1×1 convolution to adjust the channels and adjust the number of channels of the final feature layer to the number of categories; S204. Then, through upsampling by a factor of four, the feature map is adjusted to the original image resolution; S205. Through the softmax function and the argmax function, process the upsampled feature map to generate the final land cover result map, whose size is the same as the original Figure 1 one, and the calculation formula (1) is as follows: Formula (1) where F represents the upsampled feature map, represents the ratio, means to first expand the number of channels using a Linear layer, and then magnify the height and width through the rearrange function, with a magnification ratio of 4 to achieve upsampling.

[0011] Further preferably, the parallel connection of the visual Mamba module and the KAG attention module includes: S210. Extract features from the input remote sensing image data to obtain the input feature F; S211. Input the input feature F into the visual Mamba module for dual-branch parallel processing; obtain the first feature element and the second feature element respectively; S212. Input the input feature F into the KAG attention module, divide it into multiple sub-features, and fuse them after normalizing each sub-feature to obtain the third feature element; S213. Multiply the first feature element, the second feature element, and the third feature element, perform feature mapping through a linear layer, and add it to the original input feature to obtain the final output.

[0012] Further preferably, in S211, inputting the input feature F into the visual Mamba module for dual-branch parallel processing includes: Generate the feature F' after normalizing the input feature F; The first branch generates a feature map through a linear layer, and the calculation formula (2) is as follows: Formula (2) is the first feature element; The second branch adjusts the channel dimension through a linear layer, and then processes it through a 3×3 depthwise separable convolution and a SiLU activation function, and inputs the SS2D module to achieve global modeling of cross-channel and spatial dimensions, and the calculation formula (3) is as follows: Formula (3) is the second characteristic element indicating the execution of a separable convolution operation; LN represents a layer normalization operation.

[0013] Further preferably, in S212, the input feature F is input into the KAG attention module, divided into multiple sub-features, and each sub-feature is normalized and then fused to obtain a third characteristic element, including: The input feature is evenly divided into i sub-features along the channel dimension ; The importance of each sub-feature channel is evaluated through an efficient channel attention ECA module; , is the importance evaluation result of each sub-feature; Perform Tanh normalization on each sub-feature to generate a constrained feature in the range [-1,1]; Based on the Gram polynomial, polynomial fusion is performed on the normalized features to generate enhanced features ; ; ; ; Among them, and are both learnable parameters, is about of the th polynomial; the bias function adopts function; represents the result after Gram polynomial fusion; Calculate the attention weight based on the spatial global average feature ; Concatenate the original features and multiply by to form the final third characteristic element.

[0014] Further preferably, the calculation of the attention weight based on the spatial global average feature is calculated using the following formula (4): Formula (4) Among them, H is the height of the overall feature map, h represents the height of each feature, W is the width of the overall feature map, and w is the width of each feature.

[0015] The present invention also provides a remote sensing image land cover classification system based on Mamba, including: a data acquisition module, a sample library, and a Mamba remote sensing image classification network; The data acquisition module is used to acquire the original remote sensing image; The sample library is used to train the constructed Mamba remote sensing image classification network; The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network; the core module connects the visual Mamba module and the KAG attention module in parallel; using the trained network parameters, the classified remote sensing image land cover classification result is output according to the input original remote sensing image.

[0016] The disclosed Mamba-based remote sensing image land cover classification method and system of the present application constructs a Mamba-KAN module by combining Mamba and Kolmogorov-Arnold neural networks, enabling the model to have powerful global modeling and local refinement representation capabilities, effectively addressing the bottlenecks of limited receptive fields of traditional convolutional neural networks and high computational complexity of Transformer models. This module realizes multi-scale collaborative modeling through a parallel architecture, enabling the network to more efficiently and accurately identify various ground objects while maintaining global context awareness. At the same time, the model adopts a combined loss function of cross-entropy and Lovász-Softmax to effectively alleviate the class imbalance problem commonly existing in remote sensing images. The land cover classification method provided by the present invention effectively improves the classification accuracy of ground objects and is applicable to land cover classification in remote sensing scenarios. Description of the Drawings

[0017] Figure 1 It is a flowchart of a Mamba-based remote sensing image land cover classification method provided by the present invention; Figure 2 It is a structural diagram of a Mamba-based remote sensing image land cover classification system provided by the present invention; Figure 3 It is a structural diagram of the land cover classification model constructed in the embodiment of the present invention; Figure 4 It is a structural diagram of the Mamba-KAN module constructed in the embodiment of the present invention. Detailed Embodiments

[0018] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0019] As Figure 1 shown, a Mamba-based remote sensing image land cover classification method provided by an embodiment of the present invention on the one hand includes the following steps: S1. Obtain the original remote sensing image. It should be noted that for the obtained original remote sensing image, if it is real-time remote sensing image data, after inputting it into the trained Mamba remote sensing image classification network, the classification result is output; if it is a past original remote sensing image, it can be used to create a sample library.

[0020] Further preferably, the process of establishing the sample set includes the following steps: Obtain the original remote sensing image data and perform preprocessing on it; Use ArcGIS software to delineate the surface features, make the original image data and label data meet the preset size, and obtain the preprocessed remote sensing image data; Divide the preprocessed images and the corresponding labels into a training set, a test set, and a validation set according to a preset ratio. Specifically, it includes: using the original remote sensing image to construct a sample set for land cover classification, performing manual sample annotation, cropping the image after annotation, and then dividing the samples into a training set, a validation set, and a test set according to the ratio of 7:2:1 to ensure the effectiveness and generalization ability of model training.

[0021] S2. Input the original remote sensing image into the trained Mamba remote sensing image classification network. The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network; the core module connects the visual Mamba module and the KAG attention module in parallel; the Mamba remote sensing image classification network is trained using the established sample set. The specific structure of the Mamba remote sensing image classification network is as Figure 3 shown.

[0022] Further, the Mamba remote sensing image classification network includes: Set the encoder to partition the slices through patch partition and project the slices into a high-dimensional space, and perform 3 times of downsampling using the Mamba-KAN module to extract feature information; At each horizontal stage, use skip connections to fuse the high-level semantic features and the shallow features. Finally, linearly interpolate and magnify the feature map by 4 times, and then use the softmax and argmax functions to segment the image pixels into multiple types of ground features.

[0023] As Figure 4 shown, the parallel connection of the visual Mamba module and the KAG attention module includes: S210. Extract features from the input remote sensing image data to obtain the input feature F; S211. Input the input feature F into the visual Mamba module for dual-branch parallel processing; respectively obtain the first feature element and the second feature element; In S211, the input feature F is input into the Vision Mamba module, and the dual-branch parallel processing includes: The input feature F is normalized to generate the feature F'; The first branch generates a feature map through a linear layer, and the calculation formula (2) is as follows: Formula (2) is the first feature element; After the second branch adjusts the channel dimension through a linear layer, it is processed by a 3×3 depthwise separable convolution and a SiLU activation function, and is input into the SS2D module to achieve global modeling across channels and spatial dimensions. The calculation formula (3) is as follows: Formula (3) is the second feature element, represents the execution of the separable convolution operation; LN represents the layer normalization operation.

[0024] S212. The input feature F is input into the KAG attention module, divided into multiple sub-features, and each sub-feature is normalized and then fused to obtain the third feature element; In S212, the input feature F is input into the KAG attention module, divided into multiple sub-features, and each sub-feature is normalized and then fused to obtain the third feature element, including: The input feature is evenly divided into i sub-features along the channel dimension ; The importance of each sub-feature channel is evaluated through the efficient channel attention (ECA) module; , is the importance evaluation result of each sub-feature; Each sub-feature is subjected to Tanh normalization to generate a constrained feature in the range [-1, 1]; Based on the Gram polynomial, the normalized features are polynomially fused to generate an enhanced feature ; ; ; ; Among them, and are both learnable parameters, is a polynomial of degree about The bias function adopts the Represents the result after Gram polynomial fusion.

[0025] S213. Multiply the first feature element, the second feature element, and the third feature element, perform feature mapping through a linear layer, and add it to the original input feature Obtain the final output.

[0026] S2131. Calculate the attention weight based on the spatial global average feature ; Further preferably, the calculating the attention weight based on the spatial global average feature is calculated by the following formula (4): Formula (4) where H is the height of the overall feature map, h represents the height of each feature, W is the width of the overall feature map, and w is the width of each feature; S2132. Concatenate the original features and multiply by to form the final third feature element. Concatenate the original features and multiply by to form the final output feature, and the calculation formula is as follows: ; ; Multiply the corresponding elements of the features, perform feature mapping through a linear layer, and then add it to the original features to generate the final output of the Mamba - KAN module, and the calculation formula is as follows: .

[0027] Furthermore, the training process of the Mamba remote sensing image classification network includes the following steps: S201. Input the pre - processed remote sensing image data into the land cover classification network. During training, select the SGD optimizer function as the parameter optimizer, use the cross - entropy loss and the Lovász - Softmax loss as the loss functions, with an initial learning rate of 0.01, a training iteration number of 100, and a step size of 4; S202. Extract features through the Mamba - KAN module in the encoder of the land cover classification network. The decoder uses the feature map generated by upsampling to fuse with the left - hand feature map, and each Mamba - KAN module in the process is used to obtain the global and local features of the image; S203. Use a 1×1 convolution for channel adjustment to adjust the number of channels of the final feature layer to the number of classes; S204. After another four - fold upsampling, the feature map Adjust to the original image resolution; S205. Through the softmax function and the argmax function, process the upsampled feature map to generate the final land cover result map, whose size is the same as the original Figure 1 one, and the calculation formula (1) is as follows: Formula (1) where F represents the upsampled feature map, represents the ratio, means first expanding the number of channels using the Linear layer, and then magnifying the height and width through the rearrange function, with a magnification ratio of 4 to achieve upsampling.

[0028] S3. Use the Mamba remote sensing image classification network to output the classified land cover classification result of the remote sensing image.

[0029] For example Figure 2 , the present invention also provides a Mamba-based remote sensing image land cover classification system for performing the steps of the above classification method, including: a data acquisition module, a sample library, and a Mamba remote sensing image classification network; The data acquisition module is used to acquire the original remote sensing image; The sample library is used to train the constructed Mamba remote sensing image classification network; The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network; the core module connects the visual Mamba module and the KAG attention module in parallel; using the trained network parameters, according to the input original remote sensing image, output the classified land cover classification result of the remote sensing image.

[0030] The experiments of the present invention can be implemented in the Ubuntu 22.04 operating system environment. All network models are implemented through the PyTorch framework, and an NVIDIA RTX 4090 24GB GPU is used for accelerated training. In the model training stage, stochastic gradient descent (SGD) is used as the optimizer, and the model parameter update is guided by the combined loss function of cross-entropy and Lovász-Softmax. This combined loss function generates an error signal by quantifying the difference between the predicted value and the true label, calculates the gradient through the backpropagation algorithm, and drives the iterative optimization of the model weights and biases. The specific calculation method is as follows: , where represents the number of categories, is the true label distribution, is the predicted probability output by the model, is the Lovász extension of IoU, representing a piecewise linear function with a global minimum, is the error vector of the class, , is the number of pixels considered.

[0031] The final loss function is expressed as: It also includes classifying by inputting the image to be detected into the Mamba remote sensing image classification network obtained. To detect the performance of the surface cover classification method provided by the present invention, in a certain embodiment, the following formula is used to quantify the prediction result: In the formula, Precision: The precision rate, which characterizes the reliability of the model prediction result, is calculated as the ratio of the number of correctly predicted positive samples to the total number of positive samples predicted by the model; Recall: The recall rate, which reflects the coverage ability of the model for positive class samples, is calculated as the ratio of the number of correctly predicted positive samples to the total number of positive samples actually existing in the dataset; F1-score: A balanced index for comprehensively evaluating the model performance, which integrates precision and recall through the harmonic mean; IoU: Measures the spatial matching degree between the predicted region and the true annotation region, and is calculated as the ratio of the intersection area to the union area of the two; TP: Positive samples correctly predicted as positive; FP: Negative samples wrongly predicted as positive; FN: Positive samples wrongly predicted as negative.

[0032] It should be specifically noted that the method steps involved in this technical solution can be adjusted, including but not limited to reordering the step sequence, adding or deleting steps, or changing the execution manner. Specifically, the technical steps recorded in the present invention can be processed in parallel, executed sequentially, or in any other arbitrary logical order, as long as the expected goal of the technical solution is achieved, and its specific implementation manner is not limited by the order described in this article.

[0033] Obviously, the above embodiments are merely examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or alterations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or alterations derived therefrom still fall within the protection scope of the present invention.

Claims

1. A method for classifying land cover of remote sensing images based on Mamba, characterized in that, It includes the following steps: S1. Obtain the original remote sensing image; S2. Input the original remote sensing image into the trained Mamba remote sensing image classification network. The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network. The core module connects the visual Mamba module and the KAG attention module in parallel. The Mamba remote sensing image classification network is trained using the established sample set; S3. Use the Mamba remote sensing image classification network to output the classified remote sensing image surface cover classification result.

2. The method for classifying land cover of remote sensing images based on Mamba according to claim 1, wherein The establishment process of the sample set includes the following steps: Obtain the original remote sensing image data and preprocess it; Use ArcGIS software to delineate surface features, so that the original image data and label data meet the preset size, and obtain the preprocessed remote sensing image data; Divide the preprocessed images and corresponding labels into a training set, a test set, and a validation set according to a preset ratio.

3. The method for classifying land cover of remote sensing images based on Mamba according to claim 1, wherein The Mamba remote sensing image classification network includes: Set the encoder to divide slices through patch partition, project the slices into a high-dimensional space, and perform 3 times of downsampling using the Mamba-KAN module to extract feature information; Use skip connections to fuse high-level semantic features and shallow features at each horizontal stage; Linearly interpolate and magnify the feature map by 4 times, and then use the softmax and argmax functions to divide the image pixels into multiple types of ground features.

4. The method for classifying land cover of remote sensing images based on Mamba according to claim 1, characterized in that The training process of the Mamba remote sensing image classification network includes the following steps: S201. Input the preprocessed remote sensing image data into the surface cover classification network. During training, select the SGD-optimizer function as the parameter optimizer, use the cross-entropy loss and the Lovász-Softmax loss as the loss functions, with an initial learning rate of 0.01, a training iteration number of 100, and a step size of 4; S202. Extract features through the Mamba-KAN module of the encoder in the surface cover classification network. The decoder uses the feature map generated by upsampling to fuse with the left-side feature map, and each Mamba-KAN module in the process is used to obtain the global and local features of the image; S203. Use a 1×1 convolution to adjust the channels and adjust the number of channels of the final feature layer to the number of classes; S204. After another four-fold upsampling, the feature map is adjusted to the original image resolution; S205. Process the upsampled feature map through the softmax function and the argmax function to generate the final land cover result map, whose size is the same as the original image. The calculation formula (1) is as follows: to generate the final land cover result map, whose size is the same as the original image. The calculation formula (1) is as follows: Formula (1) Among them, F represents the feature map after upsampling. represents the ratio. It means that the number of channels is first expanded using a Linear layer, and then the height and width are enlarged through the rearrange function with an enlargement ratio of 4 to achieve upsampling.

5. The method for classifying land cover of remote sensing images based on Mamba according to claim 1, wherein, Connecting the visual Mamba module and the KAG attention module in parallel includes: S210. Extract features from the input remote sensing image data to obtain the input feature F; S211. Input the input feature F into the visual Mamba module for dual-branch parallel processing, and obtain the first feature element and the second feature element respectively; S212. Input the input feature F into the KAG attention module, divide it into multiple sub-features, normalize each sub-feature and then fuse them to obtain the third feature element; S213. Multiply the first feature element, the second feature element, and the third feature element, perform feature mapping through a linear layer, and add it to the original input feature to obtain the final output.

6. The method for classifying land cover of remote sensing images based on Mamba according to claim 4, characterized in that, In S211, inputting the input feature F into the visual Mamba module for dual-branch parallel processing includes: Generate the feature F' after normalizing the input feature F; The first branch generates a feature map through a linear layer, and the calculation formula (2) is as follows: Formula (2) is the first characteristic element; After the second branch adjusts the channel dimension through a linear layer, it is processed by a 3×3 depthwise separable convolution and a SiLU activation function, and is input into the SS2D module to achieve global modeling across channels and spatial dimensions. The calculation formula (3) is as follows: Formula (3) is the second characteristic element, indicating the execution of a separable convolution operation; LN indicates a layer normalization operation.

7. The method for classifying land cover of remote sensing images based on Mamba according to claim 4, wherein In S212, the input feature F is input into the KAG attention module, divided into multiple sub-features, and each sub-feature is normalized and then fused to obtain the third feature element, including: Divide the input feature evenly into i sub-features along the channel dimension ; Evaluating the channel importance of each sub-feature through the efficient channel attention ECA module; , is the evaluation result of the importance of each sub - feature; Performing Tanh normalization on each sub-feature to generate a constrained feature in the range [-1, 1]; Perform polynomial fusion on the normalized features based on Gram polynomials to generate enhanced features ; ； ； ； Among them, and are both learnable parameters, is a sub-polynomial of ; the bias function adopts the function; represents the result after the fusion of Gram polynomials; Calculate the attention weight based on the global average feature of the space ; Concatenate the original features and multiply by to form the final third feature element.

8. The method for classifying land cover of remote sensing images based on Mamba according to claim 7, wherein Calculating the attention weight based on the global spatial average feature It is calculated using the following formula (4): Formula (4) Among them, H is the height of the overall feature map, h represents the height of each feature, W is the width of the overall feature map, and w is the width of each feature.

9. A remote sensing image land cover classification system based on Mamba, characterized in that , including: a data acquisition module, a sample library, and a Mamba remote sensing image classification network; The data acquisition module is used to acquire the original remote sensing image; The sample library is used to train the constructed Mamba remote sensing image classification network; The Mamba remote sensing image classification network uses the Mamba-KAN module as the core module to form a symmetric encoder-decoder network; the core module connects the visual Mamba module and the KAG attention module in parallel; using the trained network parameters, the classified remote sensing image surface coverage classification result is output according to the input original remote sensing image.

Images