A crop classification method for remote sensing images based on Mamba

Abstract

The present invention belongs to the technical field of image data processing, and in particular is a remote sensing image crop classification method based on Mamba. The method includes an input layer, a preprocessing module, an ENVI embedding module, a 3D convolution layer, an EMA channel attention module, a Mamba Block module, an Uper Net decoder and an output layer. Mamba Block realizes efficient feature extraction and information fusion through a unique structural design, and the NDVI embedding module integrates NDVI information into the image through a unique algorithm design, thereby improving the classification accuracy and computational efficiency of the model. The present invention not only improves the accuracy of crop classification, but also optimizes the computational efficiency of the model, is suitable for large-scale remote sensing image processing tasks, and has important application value.

Description

Technical Field

[0001] The invention relates to the technical field of image data processing, and in particular to a remote sensing image crop classification method based on Mamba. Background Art

[0002] With the acceleration of agricultural modernization, higher requirements are put forward for the effective management and utilization of cultivated land resources. As an effective monitoring method, remote sensing technology plays an important role in the investigation and management of cultivated land resources.

[0003] However, traditional methods face many challenges when processing large-scale, high-resolution remote sensing images, such as inaccurate feature extraction and low classification accuracy. In recent years, deep learning technology has demonstrated its powerful capabilities in image processing, especially convolutional neural networks (CNNs), which have achieved remarkable results in image classification tasks. However, the standard CNN model still has shortcomings such as high computational cost and long training time when processing remote sensing images. The existing Transformer model is also difficult to use for high-resolution remote sensing images due to the influence of secondary computational complexity.

[0004] Recently, the state-space-based model Mamba architecture has received widespread attention. The linear computational complexity of Mamba is particularly suitable for feature extraction of high-resolution remote sensing images. NDVI reflects one of the important parameters of ground vegetation growth. The combination of the two can improve classification accuracy and efficiency.

[0005] Therefore, we propose a remote sensing image crop classification method based on Mamba to solve the above problems. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the deficiencies in the prior art, the present invention provides a remote sensing image crop classification method based on Mamba, which solves the problems raised in the above background technology.

[0008] (II) Technical solution

[0009] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0010] A remote sensing image crop classification method based on Mamba, comprising the following steps:

[0011] S1: Use multispectral remote sensing images with a resolution higher than 20 meters, including red and near-infrared channels, use ENVI software to annotate crop information for the images, and then cut them into small square blocks with a side length of 128 pixels;

[0012] S2: Build the Mamba Block module, which uses Mamba to extract semantic features and uses convolution to assist in extracting local features;

[0013] S3: Build a crop classification model for remote sensing images based on Mamba, use NDVI information to weaken the non-cultivated land features, use the EMA channel attention module to capture the dependencies between channels, use three Mamba Block modules and the EMA channel attention module residual link, and the output of each pair of residuals goes to the next module and the Uper Net decoding head respectively;

[0014] S4: The Uper Net decoding head uses the obtained three-level features to perform crop classification based on the semantic segmentation method.

[0015] Furthermore, the Mamba Block module includes layer normalization, residual connection, 3×3 convolution layer, first GELU activation function, 5×5 convolution layer, average pooling layer and second GELU activation function.

[0016] Furthermore, the Mamba Block module is firstly processed by layer normalization, and then enters Mamba to extract semantic information, the result is residually connected with the original input, and then enters the subsequent convolution step after layer normalization again, and the features extracted by the convolution step are residually connected with the features extracted by Mamba to obtain the Mamba Block module output.

[0017] Furthermore, in the subsequent convolution step of the Mamba Block module, the data is sequentially subjected to 3×3 convolution, the first GELU activation function, 5×5 convolution, average pooling, and the second GELU activation function.

[0018] Furthermore, the Mamba-based remote sensing image crop classification model includes a model structure:

[0019] Input layer, used to receive remote sensing images;

[0020] A preprocessing module, used to standardize the input image;

[0021] NDVI embedding module, used to embed NDVI information and suppress non-cultivated land pixels;

[0022] 3D convolution layer, which uses three-dimensional convolution operation to preliminarily extract local features and channel features of the image;

[0023] ENVI embedding module, which embeds ENVI information into the image, thereby suppressing the information of non-cultivated pixels;

[0024] EMA channel attention module, which strengthens the information exchange between different channels through the exponential moving average EMA mechanism, improves the model's perception of key features, is used to capture inter-channel dependencies, and enhances information interaction between different channels;

[0025] Mamba Block module, Mamba Block achieves efficient feature extraction and information fusion through unique structural design;

[0026] Uper Net decoder, which is responsible for combining high-level semantic information with low-level detail information to generate the final classification result, which is used to fuse high-level semantic information with low-level detail information;

[0027] The output layer outputs the classification results and is used to identify the cultivated land areas in the image.

[0028] Furthermore, the preprocessing module comprises:

[0029] A normalization unit, used for normalizing the input image;

[0030] The cropping unit is used to crop the image to fit the model's input size of 128×128 pixels;

[0031] The convolution kernel size of the 3D convolution layer is 3×3×3.

[0032] Furthermore, the Mamba-based remote sensing image crop classification model first receives the image by the input layer, and the preprocessing module performs standardization and cropping to ensure that the image of the input model meets the requirements; the standardized image enters the NDVI embedding module to fuse the NDVI information, and then enters the 3D convolution layer to preliminarily extract channel and local information; the output of the 3D convolution layer enters the EMA channel attention module to capture the features between channels, and then enters the residual connection of three pairs of Mamba Block modules and the EMA channel attention module, and the output of each pair of residual connections enters the next pair of residual blocks and the Uper Net decoding head respectively; the Uper Net decoding head uses semantic segmentation technology according to the feature maps of three different scales to obtain the classification result.

[0033] Furthermore, the NDVI embedding module first calculates the NDVI matrix based on the red channel and the near infrared channel, where NIR represents the near infrared channel, R represents the red channel, and NDVI represents the calculated NDVI matrix:

[0034] ;

[0035] The matrix is ​​then normalized from [-1,1] to [0,1], where Represents the normalized matrix:

[0036] ;

[0037] The resulting normalized matrix is ​​then used to increase the contrast, where Represents the matrix after contrast enhancement:

[0038] ;

[0039] Finally, the obtained contrast-enhanced matrix is ​​multiplied element-by-element with the input, and the multiplied matrix is ​​added to the input to obtain the embedded output, where OUT represents the final output result. Represents all input, is an arbitrary constant greater than 0, used to correct the participation amount of the original input. The default value is 0.6:

[0040] .

[0041] (III) Beneficial effects

[0042] Compared with the prior art, the present invention provides a remote sensing image crop classification method based on Mamba, which has the following beneficial effects:

[0043] The present invention not only improves the accuracy of crop classification, but also optimizes the computational efficiency of the model, and is suitable for large-scale remote sensing image processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0045] Figure 2 It is a schematic diagram of the structure of the model of the present invention;

[0046] Figure 3 This is a schematic diagram of the Mamba Block module structure of the present invention;

[0047] Figure 4 This is a partial structural diagram of the Mamba Block module of the present invention.

[0048] Figure 5 It is a schematic diagram comparing the present invention with the prior art. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example

[0051] like Figure 1-5 As shown, a remote sensing image crop classification method based on Mamba proposed in one embodiment of the present invention cuts the entire remote sensing image and its annotations into a number of square blocks with a side length of 128 pixels;

[0052] The classification model proposed in this application is used for feature extraction, and at the end of the model, the Uper Net decoding head is used to obtain the classification result based on semantic segmentation technology.

[0053] A remote sensing image crop classification method based on Mamba, comprising the following steps:

[0054] S1: Use multispectral remote sensing images with at least red and near-infrared channels and a resolution of no less than 20 meters. Use ENVI or other similar software to annotate the images with crop information, and then cut them into square blocks with a side length of 128 pixels.

[0055] S2: Build the Mamba Block module, which uses Mamba to extract semantic features and uses convolution to assist in extracting local features;

[0056] S3: Build a crop classification model for remote sensing images based on Mamba, use NDVI information to weaken the non-cultivated land features, use the EMA channel attention module to capture the dependencies between channels, use three Mamba Block modules and the EMA channel attention module residual link, and the output of each pair of residuals goes to the next module and the Uper Net decoding head respectively;

[0057] S4: The Uper Net decoding head uses the obtained three-level features to perform crop classification based on the semantic segmentation method.

[0058] In some embodiments, the Mamba Block module includes layer normalization, residual connection, a 3×3 convolution layer, a first GELU activation function, a 5×5 convolution layer, an average pooling layer, and a second GELU activation function.

[0059] In some embodiments, the Mamba Block module is first processed by layer normalization, and then enters Mamba to extract semantic information. The result is residually connected with the original input, and then enters the subsequent convolution step after layer normalization again. The features extracted by the convolution step are residually connected with the features extracted by Mamba to obtain the Mamba Block module output.

[0060] In some embodiments, in the subsequent convolution step of the Mamba Block module, the data is sequentially subjected to 3×3 convolution, the first GELU activation function, 5×5 convolution, average pooling, and the second GELU activation function.

[0061] Mamba-based crop classification models for remote sensing images include:

[0062] Model structure:

[0063] Input layer, used to receive remote sensing images;

[0064] A preprocessing module, used to standardize the input image;

[0065] NDVI embedding module, used to embed NDVI information and suppress non-cultivated land pixels;

[0066] 3D convolution layer, which uses three-dimensional convolution operation to preliminarily extract local features and channel features of the image;

[0067] ENVI embedding module: This is one of the core innovations of the present invention, which embeds ENVI information into the image, thereby suppressing the information of non-cultivated pixels;

[0068] EMA channel attention module, which strengthens the information exchange between different channels through the exponential moving average (EMA) mechanism, improves the model's perception of key features, is used to capture inter-channel dependencies, and enhances information interaction between different channels;

[0069] Mamba Block module: Mamba Block achieves efficient feature extraction and information fusion through unique structural design;

[0070] Uper Net decoder, which is responsible for combining high-level semantic information with low-level detail information to generate the final classification result, which is used to fuse high-level semantic information with low-level detail information;

[0071] The output layer outputs the classification results and is used to identify the cultivated land areas in the image.

[0072] Mamba Block modules include:

[0073] Layer normalization: normalize the input data to ensure numerical stability during model training;

[0074] Residual connection: Residual connection preserves the original input information, which helps alleviate the gradient vanishing problem in deep networks.

[0075] 3*3 convolution layer: Use small-sized convolution kernels to extract local features;

[0076] The first GELU activation function: Gaussian error linear unit (GELU) is used as the activation function to increase the nonlinear expression ability of the model;

[0077] 5*5 convolution layer: Use larger convolution kernels to capture more extensive context information;

[0078] Average pooling layer: reduces the dimension of feature maps through average pooling operations, reducing the computational burden;

[0079] Second GELU activation function: Use the GELU activation function again to maintain the nonlinear characteristics of the model.

[0080] The Mamba-based remote sensing image crop classification model first receives the image by the input layer, and the preprocessing module standardizes and crops it to ensure that the image input to the model meets the requirements; the standardized image enters the NDVI embedding module to fuse the NDVI information, and then enters the 3D convolution layer to preliminarily extract channel and local information; the output of the 3D convolution layer enters the EMA channel attention module to capture the features between channels, and then enters the residual connection of three pairs of Mamba Block modules and EMA channel attention modules. The output of each pair of residual connections enters the next pair of residual blocks and Uper Net decoding head respectively; The Uper Net decoding head uses semantic segmentation technology based on feature maps of three different scales to obtain the classification result.

[0081] The NDVI embedding module first calculates the NDVI matrix based on the red channel and the near-infrared channel, where NIR represents the near-infrared channel, R represents the red channel, and NDVI represents the calculated NDVI matrix:

[0082] ;

[0083] The matrix is ​​then normalized from [-1,1] to [0,1], where Represents the normalized matrix:

[0084] ;

[0085] The resulting normalized matrix is ​​then used to increase the contrast, where Represents the matrix after contrast enhancement:

[0086] ;

[0087] Finally, the obtained contrast-enhanced matrix is ​​multiplied element-by-element with the input, and the multiplied matrix is ​​added to the input to obtain the embedded output, where OUT represents the final output result. Represents all input, is an arbitrary constant greater than 0, used to correct the participation amount of the original input. The default value is 0.6:

[0088] ;

[0089] The preprocessing module comprises:

[0090] A normalization unit, used for normalizing the input image;

[0091] The cropping unit is used to crop the image to fit the model input size of 128×128 pixels.

[0092] The convolution kernel size of the 3D convolution layer is 3×3×3.

[0093] Figure 5 This is a comparison diagram of the effects produced by the method of the present invention and the method of the prior art. The SVM result shows the classification of some areas, but it looks very messy and discontinuous as a whole;

[0094] The results of ResNet are slightly better than SVM, showing clearer boundaries, but there are still many holes and discontinuous edges;

[0095] The Transformer results further improved the classification results, with clearer boundaries and more uniform color distribution, but similar areas were still misclassified;

[0096] The result of the present invention is the most outstanding, and the channel information and texture information are fully utilized. Not only the boundary is very clear, but also the misclassification phenomenon is the lowest, and it is closest to the label image.

[0097] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote sensing image crop classification method based on Mamba, characterized by: The following steps are involved: S1: Use multispectral remote sensing images with a resolution higher than 20 meters, including red and near-infrared channels, use ENVI software to annotate crop information for the images, and then cut them into small square blocks with a side length of 128 pixels; S2: Build the Mamba Block module, which uses Mamba to extract semantic features and uses convolution to assist in extracting local features; S3: Build a remote sensing image crop classification model based on Mamba, use NDVI information to weaken the non-cultivated land features, use the EMA channel attention module to capture the dependencies between channels, use three Mamba Block modules to connect with the three EMA channel attention modules residuals respectively, and the output of each pair of residuals enters the next module and the Uper Net decoding head respectively; S4: The Uper Net decoding head uses the obtained three-level features to classify crops based on the semantic segmentation method; The Mamba-based remote sensing image crop classification model includes a model structure: Input layer, used to receive remote sensing images; A preprocessing module, used to standardize the input image; NDVI embedding module, used to embed NDVI information and suppress non-cultivated land pixels; 3D convolution layer, which uses three-dimensional convolution operation to preliminarily extract local features and channel features of the image; EMA channel attention module, which strengthens the information exchange between different channels through the exponential moving average EMA mechanism, improves the model's perception of key features, is used to capture inter-channel dependencies, and enhances information interaction between different channels; Mamba Block module, Mamba Block achieves efficient feature extraction and information fusion through unique structural design; Uper Net decoder, which is responsible for combining high-level semantic information with low-level detail information to generate the final classification result; Output layer, which outputs the classification results and is used to identify the cultivated land areas in the image; The preprocessing module comprises: A normalization unit, used for normalizing the input image; The cropping unit is used to crop the image to fit the model's input size of 128×128 pixels; The convolution kernel size of the 3D convolution layer is 3×3×3; The Mamba-based remote sensing image crop classification model first receives an image by an input layer, and performs standardization and cropping by a preprocessing module to ensure that the image of the input model meets the requirements; the standardized image enters the NDVI embedding module to fuse the NDVI information, and then enters the 3D convolution layer to preliminarily extract channel and local information; the output of the 3D convolution layer enters the EMA channel attention module to capture the features between channels, and then enters the residual connection of three pairs of Mamba Block modules and the EMA channel attention module, the outputs of the first two residual connections enter the next pair of residual blocks and the Uper Net decoding head respectively, and the output of the third residual connection enters the Uper Net decoding head; the Uper Net decoding head uses semantic segmentation technology according to feature maps of three different scales to obtain classification results; The NDVI embedding module first calculates the NDVI matrix based on the red channel and the near infrared channel, where NIR represents the near infrared channel, R represents the red channel, and NDVI represents the calculated NDVI matrix: ； The matrix is ​​then normalized from [-1,1] to [0,1], where Represents the normalized matrix: ； The resulting normalized matrix is ​​then used to boost the contrast, where Represents the matrix after contrast enhancement: ； Finally, the obtained contrast-enhanced matrix is ​​multiplied element-by-element with the input, and the multiplied matrix is ​​added to the input to obtain the embedded output, where OUT represents the final output result. Represents all input, is an arbitrary constant greater than 0, used to correct the participation amount of the original input. The default value is 0.6: 。 2. The method for crop classification based on Mamba in remote sensing images according to claim 1, characterized in that: The Mamba Block module includes layer normalization, residual connection, 3×3 convolution layer, first GELU activation function, 5×5 convolution layer, average pooling layer and second GELU activation function.

3. The method for crop classification based on Mamba remote sensing images according to claim 2, characterized in that: The Mamba Block module is first processed by layer normalization, and then enters Mamba to extract semantic information. The result is residually connected with the original input, and then enters the subsequent convolution step after layer normalization again. The features extracted by the convolution step are residually connected with the features extracted by Mamba to obtain the Mamba Block module output.

4. The method for crop classification based on Mamba remote sensing images according to claim 3, characterized in that: In the subsequent convolution steps of the Mamba Block module, the data is sequentially subjected to 3×3 convolution, the first GELU activation function, 5×5 convolution, average pooling, and the second GELU activation function.

Images