Ploughing field segmentation method and system based on multispectral SAM model and multitask learning guidance

Abstract

The invention discloses a farmland field segmentation method and system based on multispectral SAM model and multitask learning guidance, and belongs to the field of field segmentation. The problem that in the prior art, channel number matching is conducted in a simple channel compression or deep convolution mode, and the challenges that the boundaries of cultivated land parcels are complex, diversified and subtle are difficult to effectively deal with is solved. The method comprises the following steps: performing spectrum compression on a multispectral remote sensing image through a U-Net structure fused with a CBAM attention mechanism to generate a three-channel pseudo visible light image; the image is input into an SAM-ViT encoder subjected to LoRA fine tuning, and global semantic features are extracted; a task token sequence is constructed, explicit interaction between the token and the image features is realized through a bidirectional Transform module, and high-resolution fusion features are generated; calculating cross-task attention, and outputting a multi-task mask through inner product operation; and performing post-processing by using a watershed algorithm to generate a field boundary. The method is used in the field of cultivated land identification.

Description

Technical Field

[0001] The present invention belongs to the field of field segmentation, and in particular relates to a method for segmenting cultivated land plots based on a multispectral SAM model and multi-task learning guidance. Background Art

[0002] With the rapid development of remote sensing technology, the identification and extraction of cultivated land parcels based on remote sensing imagery has become a crucial foundation for agricultural information management. The task of segmenting individual plots within cultivated land aims to accurately extract the boundaries of plots with independent cultivation attributes from a complex background. This is a key step in supporting applications such as precision cultivated land management, crop type identification, and cropping system monitoring. However, this task still faces numerous technical challenges.

[0003] Currently, mainstream methods mostly adopt a multi-task learning paradigm. By constructing a dual-branch structure of semantic segmentation and edge detection, the model simultaneously learns pixel-level category information and local boundary information. Typically, the model outputs both a semantic segmentation map and an edge map, which are then processed using post-processing (such as the edge-guided watershed algorithm and connected component analysis) to generate complete instance segmentation results for the land parcels. This approach improves the model's ability to perceive boundaries and alleviates the limitation of traditional single-task semantic segmentation, which often fails to distinguish between similar land parcels.

[0004] However, due to the complexity of farmland plot boundaries, the model still has difficulty achieving accurate plot segmentation. On the one hand, different crop types, rotation timings, and management methods can lead to significant differences in spectrum and texture between adjacent plots, forming semantic boundaries. On the other hand, in the context of large-scale monoculture, adjacent plots often show only slight geometric or textural differences due to continuous planting, and the boundaries are not obvious or even indistinguishable. It is difficult to effectively distinguish such instance plots by relying solely on traditional pixel-level semantic segmentation methods. In addition, although edge detection methods can enhance the model's ability to perceive local boundaries to a certain extent, they usually lack global semantic modeling capabilities and are easily disturbed by image noise or changes in texture within the farmland, resulting in false boundaries.

[0005] Furthermore, while existing multi-task learning architectures incorporate multiple task branches into the model structure, they lack explicit interaction mechanisms between different tasks. Information fusion is limited to a shared feature extraction network or a joint loss function. Essentially, the model adjusts the focus of feature representations using supervisory signals from different tasks, but fails to establish a direct connection between semantic and boundary information. Consequently, the spatial localization capabilities of semantic segmentation and the boundary sensitivity of edge detection are difficult to effectively integrate, limiting the model's ability to resolve complex land boundaries.

[0006] Multispectral remote sensing images have richer spectral information than visible light images. They can reveal spectral differences between different crops and even the same crop under different management measures, providing important support for the precise identification of plot boundaries. In recent years, large visual basic models represented by SegmentAnythingModel (SAM) have demonstrated strong generalization and boundary modeling capabilities in segmentation tasks in natural images. SAM has rich pre-training knowledge and can adapt to a variety of segmentation scenarios. However, its design is mainly based on visible light images and cannot be adapted to remote sensing multispectral data. Existing studies have used simple channel compression or deep convolution to match the number of channels, but it is difficult to retain detailed information in multispectral images, resulting in information loss. Summary of the Invention

[0007] In view of this, the present invention aims to propose a method and system for farmland segmentation based on a multispectral SAM model and multi-task learning guidance, to solve the problem that the existing technology uses simple channel compression or deep convolution to match the number of channels, but it is difficult to retain the detailed information in the multispectral image, resulting in information loss, and it is difficult to effectively deal with the complex, diverse and subtle challenges of farmland plot boundaries.

[0008] To achieve the above object, the present invention adopts the following technical solutions: A method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance, the method comprising: The U-Net structure integrating the CBAM attention mechanism is used to perform spectral compression on multispectral remote sensing images to generate three-channel pseudo visible light images. The three-channel pseudo visible light image is input into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Build a task token sequence, which includes: basic mask, semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operations; A weighted joint loss function is used to optimize multi-task masks; The watershed algorithm is used for post-processing to generate field boundaries.

[0009] Furthermore, a preferred method is proposed. The U-Net structure integrating the CBAM attention mechanism adopts a three-layer encoding-decoding structure. The encoding stage contains three CBAM convolution blocks with 64, 128, and 256 channels respectively. Each convolution block is composed of two layers of 3×3 convolution + BN layer + ReLU layer and CBAM module in series; the decoding stage reconstructs features through upsampling and skip connection, and finally outputs a three-channel image through 1×1 convolution. The process is expressed as follows:

[0010] in, is the output feature of the encoding stage, 、 is the upsampling deconvolution layer in the decoding stage, 、 The CBAM decoding module is responsible for decoding. It is a convolution layer with a convolution kernel size of 1×1.

[0011] Furthermore, a preferred embodiment is proposed, wherein the SAM-ViT encoder fine-tuned by LoRA is:

[0012] in, and is a new low-rank learnable matrix, is the rank constraint dimension, is a scaling factor used to adjust the update amplitude, is the standard weight matrix, Encodes the dimension of the feature.

[0013] Furthermore, a preferred embodiment is proposed, wherein the bidirectional Transformer module interaction includes: Phase 1: Using the task token as the query and the image features as the key / value, an attention mechanism is implemented to introduce task semantics into the image representation:

[0014] in, For task sequence, It is the flattened sequence form of the image features output by the ViT encoder; The second stage: using image features as query, As Key / Valuee, reverse interaction with image features is performed again: ; Image features High-resolution features extracted from the ViT intermediate layer Upsampling is performed through transposed convolution and layer normalization modules to obtain high-resolution representation and , the two are added together to form the final fusion feature: .

[0015] Furthermore, a preferred method is proposed, wherein the cross-task attention is specifically: Instance Task For query, represents the instance encoding dimension, Indicates the number of instance tokens, semantics, and edge tasks For Key / Value, perform cross-task attention calculation to obtain the enhanced instance task representation:

[0016] Map all task tokens to the same dimension as the image channel through MLP, denoted as , and with high-resolution image features Perform inner product operations pixel by pixel to generate multi-task masks:

[0017] in, is the token after MLP projection, represents high-resolution image features, 、 is the size of the output mask, Represents the dimension of the instance after projection.

[0018] Furthermore, a preferred method is proposed, wherein the weighted joint loss function includes three types of loss functions: semantic segmentation, edge detection, and instance segmentation. The semantic segmentation loss function is:

[0019] in, is the pixel-level cross entropy loss of the semantic graph, is the softDice loss; and is the weight coefficient; The edge detection loss function is:

[0020] in, is the cross entropy loss for marginal tasks, is the Dice loss for edge tasks, and is the weight hyperparameter of each loss; The instance segmentation loss function is:

[0021] in, is the cross entropy loss for instance segmentation task, is the Dice loss for instance segmentation tasks, To predict the mask, is the true mask, is the number of output masks.

[0022] Furthermore, a preferred method is proposed, and the specific post-processing strategy is: using the instance segmentation mask as the seed point of the watershed algorithm, the semantic segmentation mask as the spatial constraint area, and generating continuous field boundaries through morphological optimization.

[0023] Based on the same inventive concept, the present invention also proposes a farmland segmentation system based on a multispectral SAM model and multi-task learning guidance, the system comprising: The spectral compression unit is used to perform spectral compression on multispectral remote sensing images by integrating the U-Net structure with the CBAM attention mechanism to generate a three-channel pseudo visible light image; A feature extraction unit is used to input the three-channel pseudo visible light image into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Feature fusion unit, used to construct task token sequence, task token sequence includes: basic mask , semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Output unit, used to perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operation; An optimization unit for optimizing multi-task masks using a weighted joint loss function; The post-processing unit is used to generate field boundaries by performing post-processing using a watershed algorithm.

[0024] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance as described in any one of the above items.

[0025] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the steps of a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance as described above.

[0026] Compared with the prior art, the present invention has the following beneficial effects: To address the spectral information loss problem inherent in traditional multispectral compression methods, the proposed U-Net architecture, incorporating the CBAM attention mechanism, dynamically enhances key band responses and spatially significant features through a dual-attention mechanism involving channels and space. Experiments show that compared to simple convolutional compression methods, this architecture improves the overall accuracy (OA) by 0.54% (0.8777 → 0.8831) and the boundary detection Fedge score by 2% (0.6174 → 0.6374). This architecture effectively preserves subtle spectral differences between crops and addresses the SAM model's adaptation bottleneck in multispectral scenarios.

[0027] Traditional multi-task models rely on implicit feature sharing, resulting in a disconnect between semantic and boundary information. This paper uses a token interaction mechanism to explicitly guide instance segmentation using semantic segmentation tokens and edge detection tokens. This approach improves boundary completeness (Com) by 6.38% (0.5221 → 0.5859) and correctness (Corr) by 7.72% (0.7073 → 0.7845), significantly reducing over-segmentation and under-segmentation errors in complex land parcels.

[0028] Through the LoRA fine-tuning mechanism, multispectral feature adaptation is achieved by training only the low-rank matrices A / B while freezing 99% of the SAM parameters. This resolves the conflict between limited computing resources and the cost of deploying large models in agricultural scenarios, ensuring that the efficiency of field extraction from 10-meter-resolution Sentinel-2 imagery meets operational requirements.

[0029] The joint loss function enhances the model's ability to discriminate between weak boundaries (such as continuously planted plots) and heterogeneous regions (such as crop rotation plots) through three-level weighted supervision: semantic segmentation, edge detection, and instance segmentation. Ablation experiments show that the joint loss improves the mean Intersection Over Union (MIOU) by 2.66% (0.7555 → 0.7821) and reduces the segmentation error index (SEI) by 2.64% (0.2901 → 0.2637) compared to single-task training, demonstrating the robustness gains of multi-task collaborative optimization for complex agricultural scenarios. Watershed post-processing, using instance segmentation results as seed points and semantic segmentation masks as constraint regions, bridges the spatial inconsistencies between multi-task predictions. Visualization results demonstrate that this method effectively repairs broken polygons and smoothes jagged boundaries, ensuring that field vector boundaries conform to GIS mapping standards.

[0030] The present invention is applied to the field of farmland plot identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to the present invention; Figure 2 This is a schematic diagram of the structure of the multi-spectral compression encoder according to the present invention; Figure 3 This is a schematic diagram of the CBAM spectral-spatial attention module structure of the present invention, wherein: Figure 3 (a) represents the CBMA module, Figure 3 (b) represents the channel attention module, Figure 3 (c) represents the spatial attention module; Figure 4 This is a schematic diagram of the Token fusion module structure of the present invention; Figure 5 This is a diagram of a visualization of the cultivated land plots extracted by the present invention, where: Figure 5 (a) represents the original image, Figure 5 (b) shows the segmented image. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely explain the technical solutions in the embodiments of the present invention. It should be noted that the embodiments of the present invention and the features therein can be combined with each other in the absence of conflict, and the embodiments described are only part of the embodiments of the present invention, not all of the embodiments.

[0033] Embodiment 1: This embodiment describes a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance, the method comprising: The U-Net structure integrating the CBAM attention mechanism is used to perform spectral compression on multispectral remote sensing images to generate three-channel pseudo visible light images. The three-channel pseudo visible light image is input into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Build a task token sequence, which includes: basic mask , semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operations; A weighted joint loss function is used to optimize multi-task masks; The watershed algorithm is used for post-processing to generate field boundaries.

[0034] This addresses the problem that the SegmentAnythingModel (SAM) is not suitable for multispectral remote sensing data. Existing studies generally use simple channel compression or deep convolution to perform channel matching on multispectral images, but such methods are difficult to effectively retain the fine-grained spectral information in the multispectral image and are prone to loss of key ground features. To this end, this implementation designs a U-Net structure that integrates the CBAM attention mechanism, which can compress high-dimensional multispectral images into pseudo-visible light images and effectively extract key band information. At the same time, by introducing the LoRA (Low-Rank Adaptation) mechanism to lightweight and fine-tune the SAM encoder, the effective injection of spectral information is achieved while maintaining the generalization ability of the original model. Compared with the traditional Depthwise convolution compression method, this method has more obvious advantages in image information retention rate and segmentation accuracy.

[0035] This paper addresses the problem of the lack of explicit interaction mechanisms between multiple tasks in existing farmland segmentation models. Most current mainstream methods only perform implicit fusion through shared encoders or joint loss functions, which makes it difficult to establish semantic alignment and complementary relationships between tasks, resulting in limited model recognition capabilities for complex boundaries and heterogeneous regions. To this end, this embodiment involves a token-based multi-task decoding structure, which introduces an explicit cross-domain attention mechanism to achieve deep interaction between task tokens (including semantic segmentation, edge detection, and instance segmentation tokens) and image features. At the same time, a semantic-boundary explicit guidance mechanism is designed to enable the model to effectively fuse information from different tasks during the prediction process, thereby improving the clarity of boundary characterization and the ability to distinguish instances.

[0036] Implementation method 2. This implementation method further limits the cultivated land segmentation method based on the multispectral SAM model and multi-task learning guidance described in implementation method 1. The U-Net structure integrating the CBAM attention mechanism adopts a three-layer encoding-decoding structure. The encoding stage contains three CBAM convolution blocks with 64, 128, and 256 channels, respectively. Each convolution block is composed of two layers of 3×3 convolution + BN layer + ReLU layer and CBAM module in series; the decoding stage reconstructs features through upsampling and jump connection, and finally outputs a three-channel image through 1×1 convolution. The process is expressed as follows:

[0037] in, is the output feature of the encoding stage, 、 is the upsampling deconvolution layer in the decoding stage, and its upsampling rate is 2. 、 The CBAM decoding module is responsible for decoding. It is a convolution layer with a convolution kernel size of 1×1.

[0038] Implementation method 3: This implementation method further limits the cultivated land segmentation method based on the multispectral SAM model and multi-task learning guidance described in implementation method 1. The SAM-ViT encoder fine-tuned by LoRA is:

[0039] in, and is a new low-rank learnable matrix, is the rank constraint dimension, is a scaling factor used to adjust the update amplitude, is the standard weight matrix, Encodes the dimension of the feature.

[0040] Implementation 4: This implementation further limits the farmland segmentation method based on the multispectral SAM model and multi-task learning guidance described in Implementation 1. The bidirectional Transformer module interaction includes: Phase 1: Using the task token as the query and the image features as the key / value, an attention mechanism is implemented to introduce task semantics into the image representation:

[0041] in, For task sequence, It is the flattened sequence form of the image features output by the ViT encoder; The second stage: using image features as query, As Key / Valuee, reverse interaction with image features is performed again: ; Image features High-resolution features extracted from the ViT intermediate layer Upsampling is performed through transposed convolution and layer normalization modules to obtain high-resolution representation and , the two are added together to form the final fusion feature: .

[0042] Implementation 5: This implementation further limits the farmland segmentation method based on the multispectral SAM model and multi-task learning guidance described in Implementation 1. The cross-task attention is specifically: Instance Task For query, represents the instance encoding dimension, Indicates the number of instance tokens, semantics, and edge tasks For Key / Value, perform cross-task attention calculation to obtain the enhanced instance task representation:

[0043] Map all task tokens to the same dimension as the image channel through MLP, denoted as , and with high-resolution image features Perform inner product operations pixel by pixel to generate multi-task masks:

[0044] in, Represents the token after MLP projection, represents high-resolution image features, 、 represents the size of the output mask, Represents the dimension of the instance after projection.

[0045] Implementation 6. This implementation further limits the method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance described in Implementation 1. The weighted joint loss function includes three types of loss functions: semantic segmentation, edge detection, and instance segmentation. The semantic segmentation loss function is:

[0046] in, is the pixel-level cross entropy loss of the semantic graph, is the softDice loss; and is the weight coefficient; The edge detection loss function is:

[0047] in, is the cross entropy loss for marginal tasks, is the Dice loss for edge tasks, and is the weight hyperparameter of each loss; The instance segmentation loss function is:

[0048] in, is the cross entropy loss for instance segmentation task, is the Dice loss for instance segmentation tasks, To predict the mask, is the true mask, is the number of output masks.

[0049] Implementation method seven. This implementation method further limits the cultivated land segmentation method based on the multispectral SAM model and multi-task learning guidance described in implementation method one. The specific post-processing strategy is: using the instance segmentation mask as the seed point of the watershed algorithm, the semantic segmentation mask as the spatial constraint area, and generating continuous field boundaries through morphological optimization.

[0050] Embodiment 8: This embodiment describes a farmland segmentation system based on a multispectral SAM model and multi-task learning guidance, the system comprising: The spectral compression unit is used to perform spectral compression on multispectral remote sensing images by integrating the U-Net structure with the CBAM attention mechanism to generate a three-channel pseudo visible light image; A feature extraction unit is used to input the three-channel pseudo visible light image into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Feature fusion unit, used to construct task token sequence, task token sequence includes: basic mask , semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Output unit, used to perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operation; An optimization unit for optimizing multi-task masks using a weighted joint loss function; The post-processing unit is used to generate field boundaries by performing post-processing using a watershed algorithm.

[0051] Implementation method nine: A computer device described in this implementation method includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance as described in any one of implementation methods one to seven.

[0052] Implementation method ten, a computer-readable storage medium described in this implementation method, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a method for segmenting cultivated land plots based on a multispectral SAM model and multi-task learning guidance as described in any one of implementation methods one to seven are executed.

[0053] Implementation method 11, see Figures 1 to 5 This embodiment provides a specific example of the method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance described in the first embodiment, and is also used to explain the second to seventh embodiments. Specifically: This implementation proposes a model framework for instance segmentation of cultivated land plots adapted to multispectral data. This framework consists of a multispectral image compression and adaptation module, a SegmentAnything Model (SAM) encoder with LoRa fine-tuning capabilities, a multi-task-guided decoding module, and a joint loss function. This framework, combined with a post-processing strategy, produces high-quality field segmentation results.

[0054] During the multispectral data compression phase, this implementation designed a U-Net architecture that incorporates an attention mechanism to achieve efficient compression and pseudo-color reconstruction of multi-band information. This U-Net architecture uses the Convolutional Block Attention Module (CBAM) as a basic building block, embedded within each convolutional module, to dynamically enhance the response relationships between key bands and the characteristics of spatially salient regions. The CBAM module consists of a channel attention module (CA) and a spatial attention module (SA) connected in series.

[0055] Specifically, the CA module first performs the input feature map Perform global average pooling and maximum pooling operations respectively to obtain two channel descriptors and , through the shared two layers Convolution and ReLU activation to generate channel attention map :

[0056] Then it will Multiply it with the original feature and enter the spatial attention module. The SA module performs average and maximum pooling on the input in the channel dimension, and then inputs a Convolutional layer and Sigmoid activation function to obtain spatial attention map :

[0057] The final CBAM output is: , realizing layer-by-layer dynamic weighting of spatial and channel dimensions.

[0058] The spectral compression module built with CBAM as the unit adopts U-Net encoding-decoding architecture and is divided into three layers. In the encoding path, the input multispectral image First, three CBAM convolution blocks (ConvBlock) are used, with the number of channels being 64, 128, and 256 respectively. Each convolution block consists of two layers. It consists of convolution + BN layer + ReLU layer, and the CBAM module is connected in series at the end for attention enhancement. Max pooling gradually reduces the spatial resolution and improves the semantic abstraction ability. The output features of the encoding stage are recorded as , whose dimensions are , , .

[0059] The decoding stage reconstructs the high-resolution feature map through two upsampling and skip connections. Specifically, first Upsampling by 2 times yields , and the corresponding encoding features After splicing, it is decoded into 128 channels through the CBAM convolution block; then it is upsampled again with Splicing, output 64 channel features through convolution block, and finally through a Convolution compresses the output into three channels to construct a pseudo visible light image The above process can be formally expressed as:

[0060] The feature extractor of the model proposed in this implementation is based on the Vision Transformer (ViT) encoder in the SegmentAnythingModel (SAM). The encoder is efficiently fine-tuned using the LoRA (Low-Rank Adaptation) mechanism to improve its transferability to multispectral remote sensing scenarios and adaptability to downstream tasks. The ViT encoder is composed of multiple layers of self-attention (MHSA) and feed-forward neural networks (FFN). The linear transformation layers (such as query, key, and value generation, and the FFN projection operation) are the main parameter-intensive modules. This implementation introduces LoRA in these linear layers to replace the partially learned weight update path, avoiding the need for large-scale parameter adjustments to the original large model.

[0061] Specifically, LoRA converts the standard weight matrix Replaced with an expression containing a low-rank reconstruction term:

[0062] in, and is a new low-rank learnable matrix, is the rank constraint dimension, is a scaling factor used to adjust the update amplitude, is the standard weight matrix, Encode the dimension for the feature. By training only and With these two matrices, the model can achieve efficient adaptation of task perception while keeping the original parameters frozen, significantly reducing the sample size and computing resource overhead required for fine-tuning.

[0063] The mask generation stage primarily consists of two core submodules: the bidirectional Transformer interaction module and the MaskDecoder prediction module. These modules are designed to introduce an explicit task-guidance mechanism to enhance the interpretability of image features, improve semantic localization capabilities, and accurately depict land boundaries. The bidirectional Transformer module uses a cross-domain attention mechanism to achieve deep interaction between task tokens and image features. The MaskDecoder module performs two key functions: first, it introduces an explicit interaction mechanism for instance tokens to enhance its understanding of the target region's scope and boundary locations; second, it maps the fused semantic representations into multi-task outputs, including base masks, semantic masks, edge masks, and instance masks.

[0064] In order to achieve explicit task guidance, this implementation constructs a set of independent task token representations, denoted as

[0065] in, is the dimension of token, each is a learnable vector parameter. Specifically, it includes a basic mask , a semantic segmentation , an edge detection ,as well as Instance segmentation These tokens automatically learn task-related semantic representations through gradient backpropagation during training, which are used to explicitly guide the decoding process of image features.

[0066] The decoding stage first converts the image features output by the ViT encoder into Flattened to sequence form , and with the token sequence The two-way transformer module is fed into the dataset. This module consists of two interactive phases: In the first phase, the attention mechanism is executed with the token as the query and the image features as the key / value, introducing the task semantics into the image representation:

[0067] Its output is processed by the feedforward network (FFN) and LayerNorm, and then used as the key / value to interact with the image features again in reverse order to strengthen the fusion of task-guided features:

[0068] Then, the image features High-resolution features extracted from the ViT intermediate layer Upsampling is performed through transposed convolution and layer normalization modules to obtain high-resolution representation and , the two are added together to form the final fusion feature: .

[0069] In the MaskDecoder module, first perform an inter-token interaction operation on all tokens: For query, semantics and edge For Key / Value, perform cross-task attention calculation to obtain the enhanced instance token representation:

[0070] Next, all tokens (including and ) is mapped to the same dimension as the image channel through MLP, denoted as , and with high-resolution image features Perform inner product operations pixel by pixel to generate multi-task masks:

[0071] In terms of loss function design, this implementation constructs a joint optimization mechanism for three types of loss functions: semantic segmentation, edge detection, and instance segmentation, based on the multi-task structure of the model output. For the semantic segmentation branch, the loss function uses a weighted combination of cross-entropy loss and Dice loss to simultaneously optimize pixel-level classification accuracy and spatial overlap of region predictions. The loss is defined as follows:

[0072] in, is the pixel-level cross entropy loss of the semantic graph, is the softDice loss; and is the weight coefficient.

[0073] The edge detection branch also uses a loss function with a similar structure to enhance the model's responsiveness in boundary areas. Its loss function is:

[0074] In the instance segmentation branch, the Hungarian matching algorithm is first used to perform the optimal one-to-one matching between the predicted mask and the true mask. Let the matched instance pair be ,in is the number of valid instance pairs. On this basis, cross entropy and Dice loss are calculated for each pair of instances, which are defined as follows:

[0075] This design aims to improve the model's ability to identify semantic boundaries between different instances and ensure that the segmentation results have good regional integrity and discrimination.

[0076] Finally, the losses of all tasks are combined in a weighted manner into a total loss function:

[0077] in, 、 、 is the weight adjustment factor for the three types of task losses, which is determined through experiments to achieve the optimal balance between tasks.

[0078] To further improve the spatial consistency of segmentation results and the integrity of plot outlines, this implementation devised a post-processing strategy that integrates semantic segmentation and instance masking. Specifically, the method uses instance segmentation results as seed points and semantic segmentation masks as constraint regions, introducing a watershed algorithm to achieve spatially refined segmentation. This strategy effectively combines regional priors about semantic ranges with structural information about instance boundaries, bridging the consistency gaps between predictions from multiple task branches, ultimately generating a final plot segmentation map with clear boundaries and complete structure.

[0079] To systematically evaluate the performance improvements of the present invention in the farmland parcel detection task, this implementation conducted comparative and ablation experiments based on the constructed AI for Boundary parcel detection dataset. The processed data is derived from a remote sensing imagery dataset covering seven EU countries (including typical agricultural regions such as Austria and France). This dataset covers a total area of ​​approximately 376,000 km². The core sample consists of 783 256×256 pixel monthly Sentinel-2 synthetic images. Each image accurately contains information from four original spectral bands: B2 (central wavelength approximately 490 nm, blue), B3 (central wavelength approximately 560 nm, green), B4 (central wavelength approximately 665 nm, red), and B8 (central wavelength approximately 842 nm, near-infrared), with a spatial resolution of 10 meters. All image samples have precise geo-referenced information and are accompanied by manually interpreted and verified corresponding farmland parcel boundary vector data as precise labels for model training and evaluation. To ensure the objectivity and rigor of model evaluation, the original samples were randomly divided into training set, validation set and test set in a ratio of 5:1:1.

[0080] Model training and evaluation were performed on an NVIDIA RTX 3090 (24GB of video memory) GPU using the PyTorch 2.4 deep learning framework. Adam was selected as the optimizer, with the base learning rate initially set to 1e-3. During training, the learning rate decayed to 0.1 times its original value after every 10 training epochs. A total of 50 epochs were used for the full training. Due to the constraints of the input size of a single image and the complexity of the model, the batch size for each forward-backward propagation was set to 1. To mitigate the gradient estimation noise introduced by small batches and stabilize the training process, this implementation employed a cumulative gradient descent strategy with a set number of accumulation steps to 16, meaning that the model weight parameter updates were performed only after processing every 16 samples.

[0081] Two sets of control ablation experiments were designed in this embodiment. In the first set of experiments, in order to further evaluate the role of the explicit task interaction mechanism, a multi-task learning framework with Unet as the backbone was constructed in this embodiment as a baseline method. This framework does not include explicit information interaction between tasks, and only achieves task collaboration through shared feature extractors and joint loss functions. Compared with the explicit interaction decoding structure guided by task tokens designed in this invention, it aims to analyze whether the semantic fusion mechanism between multiple tasks can improve the comprehensive ability of the model in semantic consistency and boundary expression.

[0082] In the second set of experiments, to evaluate the performance of the multispectral compression module, the proposed CBAM-Unet-based compression module was replaced with a common depthwise convolutional compression method. Comparisons were conducted using the same dataset and training settings. By comparing the differences in land segmentation accuracy between the two compression methods, the advantage of structured compression in preserving key multispectral information was verified.

[0083] To comprehensively evaluate the segmentation accuracy of the present invention in the farmland plot extraction task, a multi-level and multi-angle evaluation index system was constructed, covering pixel-level, boundary-level, and object-level accuracy indicators, as follows: (1) Pixel-level evaluation indicators For pixel-level classification accuracy, we use four indicators: overall accuracy (OA), mean intersection over union (mIou), F1 score (F1-Score), and Kappa. These indicators can effectively measure the error types and overall performance of the model in plot pixel extraction: Confusion matrix diagram: Table 1 Confusion matrix diagram

[0084] Overall Accuracy (OA):

[0085] Mean intersection-over-union (mIou):

[0086] F1-score:

[0087] Kappa coefficient (Kappa):

[0088] (2) Boundary-level evaluation indicators In order to accurately evaluate the degree of spatial consistency between the extracted land boundary and the true boundary, this implementation adopts three boundary matching-based metrics: completeness (Com), correctness (Corr) and its harmonic mean F1-edge score (Fedge) Completeness (COM): The ratio of successfully detected reference boundaries to all reference boundaries:

[0089] Correctness (Corr): The proportion of accurately detected prediction boundaries to all prediction boundaries:

[0090] F1-Fedge: The harmonic average of the first two, measuring the overall performance of boundary extraction:

[0091] in, and Represent the prediction boundary and the reference boundary respectively; the intersection operation is obtained by setting a 5-meter buffer zone.

[0092] (3) Object-level evaluation indicators Over-segmentation (Sover): The degree to which a single reference plot is covered by multiple prediction plots:

[0093] Under-segmentation (Sunder): The degree to which a single predicted plot covers multiple reference plots:

[0094] SEI Segmentation Evaluation Index: A comprehensive evaluation of Sover and Sunder. The smaller the value, the lower the segmentation error.

[0095] in, represents the set of plots extracted by the model, Represents a set of reference plots, where the area is the number of pixels in the overlapping region.

[0096] The accuracy of different methods on the test set is as follows: Table 2 Comparison of land parcel extraction performance improvement

[0097] Table 3 Comparison of effectiveness of spectrum compression modules

[0098] The experimental results, shown in Tables 2 and 3, comprehensively cover multi-dimensional metrics, including pixel-level (e.g., OA, F1, mIoU, Kappa), boundary-level (Com, Corr, Fedge), and object-level (Sover, Sunder, SEI). Compared with the traditional multi-task Unet model, the proposed method achieves significant improvements across all evaluation metrics. For example, the overall accuracy (OA) increases from 0.8664 to 0.8831, the boundary accuracy (Fedge) increases from 0.5690 to 0.6374, and the comprehensive segmentation error (SEI) decreases from 0.2901 to 0.2637. These improvements demonstrate a significant improvement in the model's semantic representation and spatial boundary understanding capabilities in complex scenes. Furthermore, comparative experiments with the spectral compression module further validate the superiority of the CBAM-Unet architecture. Compared to simple convolutional compression, the proposed CBAM-Unet spectral encoder can compress four-channel remote sensing images into a more semantically-preserving three-channel representation.

[0099] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0100] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not intended to limit its scope of protection. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the disclosed claims.

Claims

1. A method for farmland segmentation based on a multispectral SAM model and multi-task learning guidance, characterized in that: The method comprises: The U-Net structure integrating the CBAM attention mechanism is used to perform spectral compression on multispectral remote sensing images to generate three-channel pseudo visible light images. The three-channel pseudo visible light image is input into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Build a task token sequence, which includes: basic mask , semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operations; A weighted joint loss function is used to optimize multi-task masks; The watershed algorithm is used for post-processing to generate field boundaries.

2. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The U-Net structure integrating the CBAM attention mechanism adopts a three-layer encoding-decoding structure. The encoding stage contains three CBAM convolution blocks with 64, 128, and 256 channels respectively. Each convolution block is composed of two layers of 3×3 convolution + BN layer + ReLU layer and CBAM module in series; the decoding stage reconstructs features through upsampling and skip connection, and finally outputs a three-channel image through 1×1 convolution. The process is expressed as follows: in, is the output feature of the encoding stage, 、 is the upsampling deconvolution layer in the decoding stage, 、 The CBAM decoding module is responsible for decoding. It is a convolution layer with a convolution kernel size of 1×1.

3. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The LoRA-tuned SAM-ViT encoder is: in, and is a new low-rank learnable matrix, is the rank constraint dimension, is a scaling factor used to adjust the update amplitude, is the standard weight matrix, Encodes the dimension of the feature.

4. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The bidirectional Transformer module interaction includes: Phase 1: Using the task token as the query and the image features as the key / value, an attention mechanism is implemented to introduce task semantics into the image representation: in, For task sequence, It is the flattened sequence form of the image features output by the ViT encoder; The second stage: using image features as query, As Key / Valuee, reverse interaction with image features is performed again: ； Image features High-resolution features extracted from the ViT intermediate layer Upsampling is performed through transposed convolution and layer normalization modules to obtain high-resolution representation and , the two are added together to form the final fusion feature: 。 5. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The cross-task attention is specifically: Instance Task For query, represents the instance encoding dimension, Indicates the number of instance tokens, semantics, and edge tasks For Key / Value, perform cross-task attention calculation to obtain the enhanced instance task representation: Map all task tokens to the same dimension as the image channel through MLP, denoted as , and with high-resolution image features Perform inner product operations pixel by pixel to generate multi-task masks: in, is the token after MLP projection, represents high-resolution image features, 、 is the size of the output mask.

6. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The weighted joint loss function includes three types of loss functions: semantic segmentation, edge detection and instance segmentation. The semantic segmentation loss function is: in, is the pixel-level cross entropy loss of the semantic graph, is the softDice loss; and is the weight coefficient; The edge detection loss function is: in, is the cross entropy loss for marginal tasks, is the Dice loss for edge tasks, and is the weight hyperparameter of each loss; The instance segmentation loss function is: in, is the cross entropy loss for instance segmentation task, is the Dice loss for instance segmentation tasks, To predict the mask, is the true mask, is the number of output masks.

7. The method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to claim 1, characterized in that: The specific post-processing strategy is as follows: using instance segmentation mask as the seed point of the watershed algorithm, semantic segmentation mask as the spatial constraint area, and generating continuous field boundaries through morphological optimization.

8. A farmland segmentation system based on a multispectral SAM model and multi-task learning guidance, characterized by: The system comprises: The spectral compression unit is used to perform spectral compression on multispectral remote sensing images by integrating the U-Net structure with the CBAM attention mechanism to generate a three-channel pseudo visible light image; A feature extraction unit is used to input the three-channel pseudo visible light image into the SAM-ViT encoder fine-tuned by LoRA to extract global semantic features; Feature fusion unit, used to construct task token sequence, task token sequence includes: basic mask , semantic segmentation , edge detection token and N instance segmentation , through the bidirectional Transformer module to achieve explicit interaction between task tokens and image features, generating high-resolution fusion features ; Output unit, used to perform cross-task attention calculation in the MaskDecoder module and output multi-task masks through inner product operation; An optimization unit for optimizing multi-task masks using a weighted joint loss function; The post-processing unit is used to generate field boundaries by performing post-processing using a watershed algorithm.

9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a method for segmenting cultivated land based on a multispectral SAM model and multi-task learning guidance as described in any one of claims 1 to 7.

Images