A lightweight agricultural land type fine semantic segmentation method based on an improved LightM-UNet model

Abstract

The application discloses a kind of light agricultural land type fine semantic segmentation methods based on improved LightM-UNet model, comprising: obtaining remote sensing image to be segmented;The remote sensing image to be segmented is input to semantic segmentation model, and segmentation result is obtained, wherein the semantic segmentation model is obtained by training set training, the training set is remote sensing image data, the semantic segmentation model is obtained by improved LightM-UNet model construction, the improved LightM-UNet model is obtained by improving the multiscale feature perception and extraction of LightM-UNet model, and the improvement of LightM-UNet model includes: respectively improving encoder and decoder.The present application realizes the high-precision segmentation of farmland multi-target by introducing multiscale attention enhancement module (MSAA) and rectangular self-calibration module (RCM).

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image segmentation technology, and in particular relates to a lightweight, fine semantic segmentation method for agricultural land types based on an improved LightM-UNet model. Background Technology

[0002] With the rapid development of remote sensing technology, high-resolution remote sensing imagery is increasingly widely used in agricultural land monitoring and management. Fine semantic segmentation of agricultural land types is a crucial foundational task in precision agriculture, land resource surveys, and ecological environmental protection. Traditional land type classification methods primarily rely on manual interpretation or pixel-based classification algorithms, such as maximum likelihood classification, support vector machines (SVM), and random forests. However, these methods often struggle to effectively capture the spatial contextual information in remote sensing imagery, resulting in limited classification accuracy, especially in complex agricultural scenarios where issues such as blurred boundaries and category confusion arise.

[0003] In recent years, deep learning technologies, especially encoder-decoder structures such as Convolutional Neural Networks (CNNs) and U-Net, have demonstrated significant advantages in semantic segmentation tasks. U-Net, with its symmetrical structure and skip connections, can effectively combine shallow detail information with deep semantic features, making it suitable for segmenting high-resolution remote sensing images. However, traditional U-Nets have high computational complexity, especially the self-attention mechanism in the Transformer part, which consumes a lot of GPU memory and has a slow inference speed when processing high-resolution images. Furthermore, the feature fusion mechanism relies on skip connections with fixed weights, which is inflexible and difficult to deploy efficiently in computationally limited agricultural applications (such as drones or mobile terminals).

[0004] In addition, the traditional iterative transformer structure (5 CA / SA calculations) results in high computational complexity and limited inference speed; the model design is heavily dependent on multimodal data input, and its performance may degrade in single-modal scenarios; furthermore, the bi-branch prediction architecture increases the difficulty of training and parameter tuning, and does not perform specific geometric feature optimization for rectangular targets (such as fields and crop rows) commonly found in agricultural scenarios, affecting its adaptability in practical applications. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention proposes a lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model. By introducing a multi-scale attention enhancement module (MSAA) and a rectangular self-calibration module (RCM), high-precision segmentation of multiple targets in farmland is achieved.

[0006] To achieve the above objectives, this invention provides a lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model, comprising:

[0007] Acquire the remote sensing image to be segmented;

[0008] The remote sensing image to be segmented is input into the semantic segmentation model to obtain the segmentation result. The semantic segmentation model is obtained by training a training set, which is remote sensing image data. The semantic segmentation model is constructed using an improved LightM-UNet model. The improved LightM-UNet model is obtained by improving the multi-scale feature perception and extraction of the LightM-UNet model. The improvement of the LightM-UNet model includes improving the encoder and decoder respectively.

[0009] Optionally, obtaining the training set includes:

[0010] The remote sensing image data is rotated and scaled to obtain an enhanced image;

[0011] The enhanced images are labeled to obtain the training set.

[0012] Optionally, improvements to the encoder include:

[0013] A rectangular self-calibration module is added after the residual visual Mamba layer in the encoder, and a multi-scale attention enhancement module is added between the encoders.

[0014] Optionally, improving the decoder includes adding a rectangular self-calibration module to the decoder, the output of which is connected to a depthwise separable convolution module.

[0015] Optionally, the rectangular self-calibration module includes a rectangular self-calibration attention unit, a batch normalization unit, and a multilayer perceptron;

[0016] The rectangular self-calibrating attention unit is used to obtain an attention feature map through horizontal pooling and vertical pooling;

[0017] The batch normalization unit is used to normalize the attention feature map;

[0018] The multilayer perceptron is used to perform nonlinear mapping and dimensional adjustment on the features after rectangular self-calibration attention processing, thereby enhancing the expressive power of the features.

[0019] Optionally, the multi-scale attention enhancement module includes: a spatial aggregation unit and a channel aggregation unit;

[0020] The spatial aggregation unit is used to extract spatial features and obtain a spatial feature map;

[0021] The channel aggregation unit is used to obtain the channel attention map.

[0022] Optionally, obtaining the spatial feature map includes:

[0023] Convolutional layers are used to reduce the channel dimension of the input features, and multi-scale fusion is performed on the processed input features to obtain fused features;

[0024] The fused features are aggregated through an average pooling layer and a max pooling layer to obtain aggregated features;

[0025] The spatial features of the aggregated features are extracted using convolutional layers to obtain the spatial feature map.

[0026] Optionally, obtaining the channel attention map includes:

[0027] The spatial dimension of the input features is compressed by a global average pooling layer, and the compressed input features are transformed into channel attention features by a convolutional layer. The channel attention features are then activated by the ReLU activation function to obtain the channel attention map.

[0028] Compared with the prior art, the present invention has the following advantages and technical effects:

[0029] This invention reduces computational resources in agricultural precision segmentation scenarios. It enhances the ability to perceive regional disasters and is used for segmenting drought / flood-prone areas. It improves the segmentation speed of rectangular targets such as farmland, crop rows, and residential areas. It provides a high-precision, low-cost solution for precision agricultural management.

[0030] This invention achieves high-precision segmentation of multiple targets in farmland by introducing a multi-scale attention enhancement module (MSAA) and a rectangular self-calibration module (RCM), including the segmentation and classification of crop growth areas (wheat, corn, etc.), water bodies, forest land, drought / flood disaster areas, and lodged crop areas. Attached Figure Description

[0031] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is a flowchart of a lightweight agricultural land type fine semantic segmentation method based on an improved LightM-UNet model according to an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the improved LightMUnet structure according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the encoder structure according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of the decoder structure according to an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of the MSAA module according to an embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram of the RCM structure according to an embodiment of the present invention;

[0038] Figure 7 This is a comparison diagram of the segmentation results before and after the model improvement in this embodiment of the invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] This embodiment provides a lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model, such as... Figure 1 As shown, the specific steps include:

[0042] Acquire the remote sensing image to be segmented;

[0043] The remote sensing image to be segmented is input into a semantic segmentation model to obtain the segmentation result. The semantic segmentation model is trained using a training set of remote sensing image data, and is constructed using an improved LightM-UNet model. Figure 2 As shown, the improved LightM-UNet model is obtained by improving the multi-scale feature perception and extraction of the LightM-UNet model. The improvements to the LightM-UNet model include improvements to the encoder and decoder respectively.

[0044] Specifically, the LightM-UNet model includes: (1) a shallow feature extraction layer: using depthwise separable convolution (DWConv) to extract initial features. (2) three consecutive encoder modules: extracting depth features layer by layer and reducing resolution. (3) a bottleneck block: establishing long-range spatial dependencies. (4) three consecutive decoder blocks: progressively restoring image resolution. (5) an output layer: using DWConv to map the number of channels to the number of segmentation targets and generating an image mask using SoftMax.

[0045] Furthermore, obtaining the training set includes:

[0046] Rotate and scale remote sensing image data to obtain enhanced images;

[0047] The enhanced images are labeled to obtain the training set.

[0048] Specifically, remote sensing image data is collected and the dataset is divided: the dataset is divided into training set, validation set and test set in a 6:2:2 ratio.

[0049] Data augmentation and annotation: Images in the training, validation, and test sets were rotated by 0°, 90°, 180°, and 270°, and scaled by 0.75x, 1x, and 1.25x, increasing the dataset size by twelve times. The augmented images were annotated using LabelMe annotation software, resulting in a dataset with seven categories of labels. A PNG mask image was output, with each pixel labeled as its category number (1 for background, 2 for buildings, 3 for roads, 4 for water bodies, 5 for bare soil, 6 for woodland, and 7 for farmland).

[0050] Further improvements to the encoder include:

[0051] A rectangular self-calibration module is added after the residual vision Mamba layer in the encoder, and a multi-scale attention enhancement module is added between the encoders.

[0052] Specifically, such as Figure 3 As shown, in the encoder, after embedding the RCM module into the RVM Layer (Residual Vision Mamba Layer), a composite submodule consisting of RVM+RCM+MSAA is constructed. This structure, through multi-scale feature aggregation and channel attention modeling, can effectively distinguish different crop types within the same field, thereby significantly improving the accuracy of multi-crop region segmentation.

[0053] Furthermore, such as Figure 4 As shown, the improvements to the decoder include adding a rectangular self-calibration module to the decoder, the output of which is connected to a depthwise separable convolution module.

[0054] Specifically, the original decoder can be represented by the following formula:

[0055]

[0056] Feature maps from skip connections and the feature map P from the previous block output inFeature fusion is performed by addition, where DWConv represents depthwise convolution, and the adjustment factor s′ is a parameter added to further enhance decoding capabilities. Finally, bilinear interpolation is used to restore the prediction to the original resolution.

[0057] The decoder after RCM insertion can be represented as:

[0058]

[0059] By introducing the RCM module into the decoder, the local attention mechanism can be effectively supplemented. In the wheat lodging region segmentation task, this module can enhance the model's ability to recover the boundaries and morphology of occluded regions, while better capturing texture detail changes, thereby improving segmentation performance.

[0060] Furthermore, the rectangular self-calibration module includes a rectangular self-calibration attention unit, a batch normalization unit, and a multilayer perceptron;

[0061] A rectangular self-calibrating attention unit is used to obtain attention feature maps through horizontal pooling and vertical pooling;

[0062] Batch normalization unit, used to normalize the attention feature map;

[0063] Multilayer perceptrons are used to perform nonlinear mapping and dimensional adjustment on features after rectangular self-calibrated attention processing, thereby enhancing the expressive power of the features.

[0064] Specifically, the RCM module improves the model's focus on the foreground and extracts axial global information from the pyramid context, such as... Figure 6 As shown, the RCM module consists of rectangular self-calibrating attention (RCA), batch normalization (BN), and multilayer perceptron (MLP).

[0065] RCA: Captures the global context of the image through horizontal and vertical pooling. These two pooling operations generate two axis vectors, which are combined using broadcast addition to help the model better focus on the rectangular target region. Then, a shape self-calibration function is used to calibrate the region of interest. Two large-kernel stripe convolutions calibrate the shape of the feature maps in the horizontal and vertical directions respectively, enabling the model to more accurately adjust the shape of the region of interest. Non-linearity is enhanced by batch normalization (BN) and the ReLU function. This function can be expressed by the following formula:

[0066]

[0067] in, This represents the attention feature map obtained in the previous step through rectangular self-calibrated attention (RCA), γ represents large kernel strip convolution, m represents the size of the convolution kernel, φ represents batch normalization followed by the ReLU function, and δ represents the Sigmoid function.

[0068] Furthermore, a feature fusion function was designed to fuse attention features with input features. Fusing attention features with the original input features helps enhance the network's attention to local details. By using 3×3 depthwise convolutions and element-wise multiplications, the two features are combined, thereby further improving the feature representation.

[0069] F(X,Y)=γ 3×3 (X)☉Y

[0070] Where γ 3×3 This represents a depthwise convolution with a 3×3 kernel, Y is the attention feature obtained in the previous step, and ⊙ represents element-wise multiplication.

[0071] Batch normalization and a multilayer perceptron were added after rectangular self-calibrating attention to further refine the features. Finally, residual connections were used.

[0072] F out =ρ(F(X,Y))+X

[0073] Where ρ represents Batch Normalization (BN) and Multilayer Perceptron (MLP).

[0074] Furthermore, the multi-scale attention enhancement module includes: a spatial aggregation unit and a channel aggregation unit;

[0075] Spatial aggregation unit, used to extract spatial features and obtain spatial feature maps;

[0076] The channel aggregation unit is used to obtain the channel attention map.

[0077] Specifically, such as Figure 5 As shown, the MSAA module aggregates and refines features through two sub-paths: spatial path and channel path. It is suitable for tasks that require capturing details and background features at different scales. The input to MSAA is a C×H×W tensor, where C is the number of channels, and H and W are the height and width of the image.

[0078] Furthermore, obtaining the spatial feature map includes:

[0079] Convolutional layers are used to reduce the channel dimension of the input features, and multi-scale fusion is performed on the processed input features to obtain fused features;

[0080] The fused features are aggregated through average pooling and max pooling layers to obtain aggregated features;

[0081] Spatial features of aggregated features are extracted using convolutional layers to obtain spatial feature maps.

[0082] Specifically, spatial aggregation works as follows: First, a 1×1 convolution reduces the number of channels in the input feature map from C to C1, where C1 = C / β, thus reducing the dimensionality of the channels. Then, convolutional kernels of different sizes (e.g., 3×3, 5×5, 7×7, etc.) are used to perform multi-scale fusion of the features, capturing spatial information at different scales. The feature maps are then aggregated using average pooling and max pooling, followed by a 7×7 convolution to further refine the spatial features, and finally activated using the Sigmoid activation function.

[0083] Furthermore, obtaining the channel attention map includes:

[0084] The spatial dimension of the input features is compressed by a global average pooling layer, and the compressed input features are transformed into channel attention features by a convolutional layer. The channel attention features are then activated by the ReLU activation function to obtain the channel attention map.

[0085] Specifically, channel aggregation: The spatial dimension of the feature map is compressed to C×1×1 using global average pooling. Then, a 1×1 convolution is used to transform C into a channel attention map, which is then activated using the ReLU activation function to form the channel attention map. The generated channel attention map is then expanded to the dimension of the input features and combined with the spatial feature maps in the spatial path.

[0086] Adding an MSAA module between each encoder helps improve the model's ability to perceive multi-scale features, thereby significantly enhancing its ability to represent regional disasters at multiple scales. This can be used for segmentation of drought-stricken or flood-prone areas.

[0087] More specifically, the training and evaluation of the improved LightM Unet model: First, the improved model structure is constructed, and then it is trained using training data. During training, after each epoch, the model performance is evaluated using a validation set, and the model parameters are adjusted based on the validation results to continuously optimize the model until optimal performance is achieved.

[0088] The model is deployed on hardware, and the trained model is used to complete the segmentation task. The segmentation result is as follows: Figure 7 As shown.

[0089] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model, characterized in that... include: Acquire the remote sensing image to be segmented; The remote sensing image to be segmented is input into the semantic segmentation model to obtain the segmentation result. The semantic segmentation model is obtained by training a training set, which is remote sensing image data. The semantic segmentation model is constructed by improving the LightM-UNet model. The improved LightM-UNet model is obtained by improving the multi-scale feature perception and extraction of the LightM-UNet model. The improvement of the LightM-UNet model includes improving the encoder and decoder respectively. Improvements to the encoder include: A rectangular self-calibration module is added after the residual vision Mamba layer in the encoder, and a multi-scale attention enhancement module is added between the encoders; The improvement to the decoder includes adding a rectangular self-calibration module to the decoder, the output of which is connected to a depthwise separable convolution module; The rectangular self-calibration module includes a rectangular self-calibration attention unit, a batch normalization unit, and a multilayer perceptron. The rectangular self-calibrating attention unit is used to obtain an attention feature map through horizontal pooling and vertical pooling; The batch normalization unit is used to normalize the attention feature map; The multilayer perceptron is used to perform nonlinear mapping and dimension adjustment on the features after rectangular self-calibration attention processing, thereby enhancing the expressive power of the features. The multi-scale attention enhancement module includes: a spatial aggregation unit and a channel aggregation unit; The spatial aggregation unit is used to extract spatial features and obtain a spatial feature map; The channel aggregation unit is used to obtain the channel attention map.

2. The lightweight agricultural land type fine semantic segmentation method based on the improved LightM-UNet model according to claim 1, characterized in that, Obtaining the training set includes: The remote sensing image data is rotated and scaled to obtain an enhanced image; The enhanced images are labeled to obtain the training set.

3. A lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model, as described in claim 1, is characterized in that... Obtaining the spatial feature map includes: Convolutional layers are used to reduce the channel dimension of the input features, and multi-scale fusion is performed on the processed input features to obtain fused features; The fused features are aggregated through an average pooling layer and a max pooling layer to obtain aggregated features; The spatial features of the aggregated features are extracted using convolutional layers to obtain the spatial feature map.

4. A lightweight, fine-grained semantic segmentation method for agricultural land types based on an improved LightM-UNet model, as described in claim 1, is characterized in that... Obtaining the channel attention map includes: The spatial dimension of the input features is compressed by a global average pooling layer, and the compressed input features are transformed into channel attention features by a convolutional layer. The channel attention features are then activated by the ReLU activation function to obtain the channel attention map.

Images