Unmanned aerial vehicle image plot-level crop type multi-task extraction method and system

Abstract

The application provides a plot-level crop type multi-task extraction method and system for unmanned aerial vehicle images, and belongs to the technical field of remote sensing image analysis and target recognition. The system comprises an encoder, a decoder, a multi-task semantic segmentation head and a post-processing module. The encoder is used to receive pre-acquired unmanned aerial vehicle image data of a study area and perform multi-stage feature extraction. The decoder is used to perform multi-stage up-sampling on the feature map extracted by the encoder. Each stage comprises multiple task branches, each branch corresponds to each task in the multi-task, and each branch is sequentially connected by a multi-scale feature fusion Up-Fuse block, an Intrinsic-Mamba block for modeling task internal dependence and a Freq-Harmony Mamba block for modeling task interdependence. The multi-task semantic segmentation head is used to perform multi-task prediction. The post-processing module is used to post-process the multi-task prediction result to generate a plot-level crop recognition result. The system can realize high-precision plot extraction and crop type recognition in the same framework.

Description

Technical Field

[0001] This application relates to the field of remote sensing image analysis and target recognition technology, and in particular to a multi-task method and system for extracting crop types at the plot level from UAV images. Background Technology

[0002] Farmland plots can be subdivided into natural plots and crop-planted plots. Precision agriculture relies on refined farmland information, among which farmland plot and crop type information are the foundation and core of practical activities. Research on farmland plot extraction and plot-scale crop planting information identification is of great significance for promoting the development of precision agriculture and serving related agricultural fields.

[0003] Currently, there are two main technical approaches to crop mapping: one is to use machine learning models or deep learning to classify crops at the pixel level in remote sensing images, merging similar pixels to form plots, resulting in significant salt-and-pepper noise in the final product; the other approach is a two-stage method for crop identification, first extracting plots and then identifying crop types from the extracted results. This method suffers from the problem of error accumulation and propagation at each stage, leading to overall deviation in the results. In other words, errors in the plot extraction stage accumulate in the crop type identification stage, causing the error to expand rapidly. Therefore, while traditional methods can meet the needs of natural plot identification to some extent, when transferred to extracting crop planting plots from centimeter-level ultra-high resolution UAV imagery, problems such as incomplete plots, blurred and overlapping boundaries, and increased salt-and-pepper noise occur, making it difficult to meet the requirements of fine plot-level crop type mapping.

[0004] To address issues such as blurred boundaries, overlapping plots, and false boundaries encountered in farmland planting plot extraction under ultra-high resolution, Chinese patent (CN119360235A, AI-based automatic extraction system and method for farmland planting plots from UAV imagery) proposes a solution. This solution first performs Fourier transform (DCT) on the high-dimensional features obtained from the encoder to decompose them in the frequency domain. The Fourier transform is then used to decouple the high and low frequencies of the encoded features. The high-frequency and low-frequency components are then fed into the boundary and region decoding branches, respectively, to strengthen geometric boundary perception and region consistency constraints at the network structure level. However, this solution cannot support accurate crop classification. Therefore, after plot extraction, a two-stage method is still needed for crop type identification, which makes it difficult to avoid the deviation problem caused by error accumulation.

[0005] An improved technical solution is needed for fully automated plot-level crop type mapping based on ultra-high resolution UAV imagery. Summary of the Invention

[0006] The purpose of this application is to fully utilize the coupling relationship between plot geometry and crop semantics to provide a plot-level crop type multi-task extraction method and system for UAV imagery. By modeling the remote spatial dependency relationship and effectively integrating cross-stage information, the method enhances feature expression capabilities and improves the overall accuracy of crop identification, thereby supporting fully automatic and high-precision plot-level crop type mapping.

[0007] To achieve the above objectives, this application provides the following technical solution:

[0008] This application provides a plot-level crop type multi-task extraction system for UAV imagery. The multi-task includes plot area extraction task, plot boundary extraction task, and crop type prediction task. The system includes: encoder, decoder, multi-task semantic segmentation head, and post-processing module.

[0009] The encoder is used to receive pre-acquired UAV image data of the research area and perform multi-stage feature extraction.

[0010] The decoder is used to perform multi-stage upsampling on the feature map extracted by the encoder. Each stage contains multiple task branches, and each branch corresponds one-to-one with each task in the multi-task. Each branch is formed by sequentially connecting the multi-scale feature fusion Up-Fuse block, the task internal dependency modeling Intrinsic-Mamba block, and the task external dependency modeling Freq-Harmony Mamba block.

[0011] The Up-Fuse block is used to fuse the feature map output from the previous stage of the current task branch with the feature map of the corresponding encoder in the current stage; the Intrinsic-Mamba block is used to perform long-distance dependency modeling and feature enhancement within the current task using Mamba technology based on the fused feature map; the Freq-Harmony Mamba block is used to perform cross-task dependency modeling of the current task using Mamba technology based on the output of the Intrinsic-Mamba block, and obtain the output result of the current task branch;

[0012] The multi-task semantic segmentation head is used to perform multi-task prediction based on the output results of each branch in the last stage of the decoder, and obtain the multi-task prediction result.

[0013] The post-processing module is used to post-process the multi-task prediction results to generate vectorized plot-level crop identification results.

[0014] The technical solution provided in this embodiment belongs to an end-to-end architecture of encoder, decoder, multi-task semantic segmentation head, and post-processing module. The decoder introduces a multi-branch structure corresponding one-to-one with the three main tasks (plot region extraction, plot boundary extraction, and plot-level crop type prediction). Each branch sequentially integrates Up-Fuse, Intrinsic-Mamba, and Freq-Harmony Mamba blocks, realizing a three-level progressive processing mechanism of feature extraction, intra-task modeling, and inter-task collaboration. This improves the geometric integrity, boundary clarity, and semantic consistency of plot-level crop mapping. Specifically, the Up-Fuse block fuses the features upsampled in the decoding stage of the current task branch with the features from the corresponding stage of the encoder, providing a feature foundation with both spatial detail and semantic discriminative power for subsequent task modeling. The core of Mamba technology is the Selective State Space Module (S6). The Intrinsic-Mamba block uses Mamba technology to perform sequence modeling within the current task on the fused feature map, capturing long-distance dependencies within the task. The Freq-Harmony Mamba block, based on Mamba technology, interacts with plot area information, boundary information, and crop type semantics during the decoding stage, achieving collaborative constraints and complementary enhancements between multiple tasks, maintaining plot extraction accuracy, and providing complete and continuous semantic feature support for plot-level crop type prediction. Thus, high-precision plot extraction and crop type identification are achieved within the same framework.

[0015] In conjunction with the first aspect, the Intrinsic-Mamba block preferably includes: a channel extension layer, an SS2D sub-branch, a linear sub-branch, and a task-internal fusion layer;

[0016] The channel expansion layer is used to expand the channels through a linear fully connected layer using the fused feature map as the input feature map to obtain an expanded feature map.

[0017] The SS2D sub-branch and the linear sub-branch are set in parallel. The SS2D sub-branch is used to perform four spatial direction unfolding on the extended feature map, corresponding to four unfolding sequences. The state space model SSM is performed on each sequence to obtain the SSM output feature map. The linear sub-branch is used to perform convolution and SiLU activation on the extended feature map in sequence to obtain the linear branch feature map.

[0018] The task-internal fusion layer is used to perform feature fusion on the SSM output feature map and the linear branch feature map, and then perform a skip connection with the input feature map to obtain the output of the Intrinsic-Mamba block.

[0019] In the above scheme, by setting a channel expansion layer in the Intrinsic-Mamba block and using a linear fully connected layer to expand the channels of the fused feature map, the original features (i.e., input features) obtain a higher-dimensional channel representation, providing a more sufficient feature expression space for subsequent state-space model processing. By implementing long-distance dependencies of features within the task in different spatial directions through the SS2D sub-branch, features in different directions can be modeled and preserved separately, improving the task's ability to model spatial correlation information. Parallel linear sub-branches are set up, and SiLU activation is performed on the expanded feature map to obtain a linear branch feature map, allowing the features to retain their original local linear response information. Feature fusion is then performed by the task's internal fusion layer, thereby enhancing the task's ability to model long-distance dependencies while maintaining the continuity of the feature transfer process.

[0020] In conjunction with the first aspect, preferably, the Freq-Harmony Mamba block includes: a shared task sub-branch and a specific task sub-branch;

[0021] The shared task sub-branch uses a multilayer perceptron (MLP) network to perform cross-task information fusion on the outputs of the Intrinsic-Mamba blocks of other task branches to obtain fused features. Other task branches refer to task branches other than the current task.

[0022] The specific task sub-branch is used based on the fusion features. SSM parameters are generated, and cross-sequence cross-scanning is performed on the fusion features of other task branches and the features of the current task branch based on Mamba technology to obtain cross-task dependency features that enhance the expression of the current task branch.

[0023] In the above scheme, by setting up an MLP network in the shared task sub-branch, the features of other task branches are fused across tasks to generate fused features, effectively integrating complementary information between multiple tasks and enhancing the feature representation ability of each task. Using specific task sub-branches, SSM parameters are generated based on the fused features, and through Mamba technology, cross-sequence cross-scanning is used to inject the fused features of other tasks into the current task, realizing adaptive modeling of dependencies between different tasks. In addition, inter-task information interaction is realized in the decoding stage, avoiding parameter redundancy and increased computational overhead, and improving the overall performance and convergence efficiency of multi-task learning.

[0024] Preferably, the shared task sub-branch further includes: a shared task fusion layer, a first frequency enhancement module (LFEB) block, and a hybrid gated HGU unit;

[0025] The shared task fusion layer is used to perform feature fusion on the output of the Intrinsic-Mamba blocks of other task branches to obtain preliminary inter-task fusion features. ;

[0026] LFEB block, used for Perform a Fourier transform to obtain the frequency domain features. and will Multiply by the weight matrix W, and then perform an inverse Fourier transform to obtain the frequency-enhanced features. ;

[0027] HGU unit, used to perform operations using a two-layer fully connected multilayer perceptron (MLP) network. Perform an adaptive transformation to obtain the adaptive transformation result. At the same time, Perform channel attention processing to obtain the channel attention processing result. Then After sigmoid activation and Summation is performed to obtain the gating weights, and these gating weights are then used to adjust... and The proportion of these proportions is used to obtain the final fusion characteristics. .

[0028] In the above scheme, a shared task sub-branch is set in the Freq-Harmony Mamba block, and the Intrinsic-Mamba outputs of other task branches are sequentially fused and frequency domain enhanced, so that the shared features between tasks can obtain controllable frequency response adjustment capability and enhance the expression of cross-task related features; the HGU unit uses gating weights to adjust the proportion of the two types of features, so that the information interaction process between tasks has adaptability.

[0029] In conjunction with the first aspect, preferably, the specific task sub-branch includes: a TS-SS2D component;

[0030] The TS-SS2D component is used to determine the fusion features. Generate SSM parameters and, based on Mamba technology, use the output of the Intrinsic-Mamba block of the current task branch as the query input. Based on the SSM parameters, the State-Space Model (SSM) processing is performed to obtain the processing result. The processing results A skip join is made with the output of the Intrinsic-Mamba block of the current task branch to obtain cross-task dependency features.

[0031] In the above scheme, the specific task refers to the current task branch. The TS-SS2D component is introduced into the specific task sub-branch, utilizing fusion features. Generate SSM parameters to enable features from other tasks to participate in the state space modeling of the current task in a parameterized manner, achieving deep fusion of information between tasks. Using the features of the current task as query input, cross-sequence interactions are performed in the state space model based on the SSM parameters, dynamically capturing cross-task dependencies and improving the representational ability of the features of the current task and the consistency between tasks.

[0032] In conjunction with the first aspect, preferably, the multi-task semantic segmentation head includes a plot branch, a boundary branch, and a crop type branch that correspond one-to-one with the plot region extraction task, the plot boundary extraction task, and the crop type prediction task;

[0033] The boundary branch includes a second frequency domain enhancement module (PDFR) block, which receives the output of the Freq-Harmony Mamba block, the last stage of the parcel boundary extraction task branch in the decoder. After performing a Fourier transform, the parcel boundary features are enhanced in the high-frequency information part, and the enhanced boundary features are obtained by inverse Fourier transform. .

[0034] In the above scheme, boundary enhancement features are obtained through inverse Fourier transform reconstruction. This allows the model to maintain the enhanced boundary structure in the spatial domain, effectively suppressing noise and texture interference, and improving the robustness of boundary extraction in complex scenes.

[0035] In conjunction with the first aspect, preferably, the post-processing module is further configured to:

[0036] The parcel boundary extraction results from the multi-task prediction are subjected to semantically constrained morphological dilation to obtain semantically enhanced boundaries. ;

[0037] Lee's skeleton algorithm was used to perform... Boundary skeleton extraction is performed to obtain a boundary skeleton graph, and a semantic topology graph is constructed based on the boundary skeleton graph. Each topological segment in the topological network records the crop type attributes on both sides;

[0038] Using the semantic topology graph The crop type is used as semantic information to guide the extension direction of the hanging line. The hanging line segment optimization process is performed on the boundary skeleton diagram to obtain the optimized skeleton diagram.

[0039] Semantic vectorization is performed on the optimized skeleton diagram to obtain vector crop plots with crop type attributes.

[0040] In the above scheme, the boundary continuity is enhanced by morphological expansion and skeleton extraction based on semantic constraints. The crop type attributes recorded by the semantic topology graph guide the extension direction of the hanging line segment, avoid cross-type incorrect connections, and realize the synergistic optimization of semantic information and geometric structure.

[0041] In conjunction with the first aspect, preferably, the parcel boundary extraction results in the multi-task prediction results are subjected to semantically constrained morphological dilation processing to obtain semantically enhanced boundaries. ,include:

[0042] Morphological dilation processing is performed on the plot boundary extraction results in the multi-task prediction results to obtain the boundary dilation result Boundary_3;

[0043] The crop type prediction results in the multi-task prediction results are used to detect crop type changes using an edge detection algorithm, determine the pixel locations where crop types have changed, and generate crop plot edges (Crop_boundary).

[0044] Morphological dilation is performed on the crop plot edge Crop_boundary to obtain the dilated crop boundary Crop_boundary_3;

[0045] Taking the union of the boundary expansion result Crop_boundary and the expanded crop boundary Crop_boundary_3 yields the semantically enhanced boundary. .

[0046] In the above scheme, semantic information is embedded into the geometric expansion process by fusing the predicted boundary after morphological expansion (i.e., Crop_boundary) with the semantic boundary generated based on crop type change detection (Crop_boundary_3), thereby achieving semantic and geometric joint boundary enhancement.

[0047] In conjunction with the first aspect, preferably, the semantic topology graph is used. The crop type is used as semantic information to guide the extension direction of the suspending line. The suspending line segment optimization process is performed on the boundary skeleton diagram to obtain the optimized skeleton diagram, including:

[0048] The semantically enhanced boundary The crop type prediction result and the plot area extraction result in the multi-task prediction results are superimposed to generate an enhanced boundary mask Crop_mask that integrates geometry and semantics;

[0049] Based on the semantic topology graph The suspension line was identified.

[0050] Traverse each dangling line, using the tangent direction of the dangling line as the extension direction, and extend it from the dangling vertex in the extension direction. For each pixel extended, perform geometric collision checks and semantic constraint checks.

[0051] The geometric collision check includes: checking whether the currently extending pixel collides with an existing skeleton line in the boundary skeleton graph; if so, closing the dangling line; if not, performing a semantic constraint check. The semantic constraint check includes: based on the enhanced boundary mask Crop_mask, checking whether the line composed of the extending pixels cuts off a complete crop patch; if so, stopping the extension and removing the dangling line from the boundary skeleton graph and the semantic topology graph. If the line segment is removed, and the semantic constraint check passes, the extension process continues. For each pixel extended, geometric collision and semantic constraint checks are performed until successful closure or the maximum extension length is reached. The newly extended line segment is then drawn in the boundary skeleton graph, and the semantic topology graph is updated. .

[0052] In the above scheme, a geometrically and semantically integrated enhanced boundary mask (Crop_mask) is generated by overlaying semantically enhanced boundaries, crop type prediction results, and plot area extraction results. This provides accurate semantic and geometric constraints for dangling line extension, ensuring that the extension path conforms to both geometric continuity and semantic consistency, thus achieving semantically and geometrically coordinated dangling line repair. A dual mechanism of geometric collision checking and semantic constraint checking is introduced to dynamically detect collisions with existing skeleton lines and whether complete crop plots are cut during the extension process, avoiding topological damage and semantic conflicts caused by erroneous extension. A pixel-by-pixel extension and real-time checking strategy is adopted, enabling the dangling line extension process to adjust the extension path and termination timing in a timely manner according to actual geometric and semantic conditions, ensuring the repair effect.

[0053] This embodiment also provides a plot-level crop type multi-task extraction method for UAV imagery. This method is executed by the plot-level crop type multi-task extraction system for UAV imagery provided in any of the above embodiments, and includes:

[0054] Acquire historical UAV imagery data of the study area and create a sample dataset;

[0055] The plot-level crop type multi-task extraction system for UAV imagery described in any of the above embodiments is trained using the sample dataset.

[0056] The UAV image to be predicted is input into the plot-level crop type multi-task extraction system of the UAV image that has been trained, and the output results are obtained. The output results are then used to create plot-level crop type maps. Attached Figure Description

[0057] Figure 1This is a schematic diagram of the overall structure of a multi-task system for extracting crop types at the plot level from UAV imagery.

[0058] Figure 2 This is a schematic diagram of the Up-Fuse block.

[0059] Figure 3 This is a schematic diagram of the Intrinsic-Mamba block.

[0060] Figure 4 This is a schematic diagram of the Freq-Harmony Mamba block.

[0061] Figure 5 This is a schematic diagram of the scanning process for the TS-SS2D component.

[0062] Figure 6 This is a schematic diagram of the PDFR (Prior-Driven Frequency Refinement Block) structure. Detailed Implementation

[0063] The State-Space Sequence Model (SSM), originating from control theory, is used to describe a linear time-invariant dynamic system. It expresses a one-dimensional continuous input signal through state equations and output equations. Mapped to output signal The process, the expression of the state equation is: The output equation is: ,in, , , , All are SSM parameters. Since it is a feedforward matrix, it can usually be ignored. The hidden state vector. It represents the derivative of the state with respect to time.

[0064] Mamba technology (Chinese: Linear-Time Sequence Modeling with Selective State Spaces, English: Linear-Time Sequence Modeling with Selective State Spaces, also known as the S6 model) is an improved version of the S4 model. The S4 model, through parameterization (including...) The initialization and computational optimization design integrates SSM theory into deep learning. However, the S4 model is time-invariant, meaning the parameters... With all inputs and time steps fixed, it is essentially a structured linear time-invariant system. Mamba improves upon this by introducing input-dependent parameters (i.e.,...) Transform into input The function implements a selective mechanism, enabling its evolution into a selective state-space model (S6 model). The S6 model possesses an exponentially decaying memory mechanism, and its receptive field is determined by… The value is determined by adjusting It can control the model's modeling of long-distance dependencies.

[0065] Natural land parcels refer to land spatial units formed under the influence of natural or human factors, with relatively clear physical boundaries (such as field ridges and ditches), and possessing independent management or operation attributes. Because natural land parcels have clear physical boundaries, traditional methods can still achieve high-precision segmentation and identification.

[0066] A crop planting plot is a specific area within a land boundary where only a single type of crop is grown during each agricultural production cycle.

[0067] Pixel-level crop classification refers to a method that uses the spectral, texture, and training sample features of individual pixels in remote sensing images as input, and uses machine learning or deep learning models to identify the crop type for each pixel. Because it judges each pixel independently, it is prone to problems such as salt-and-pepper noise and irregular boundaries.

[0068] Land parcel extraction / identification refers to the process of identifying the boundaries and internal areas of land parcels within farmland areas and generating land parcel objects with defined geometric shapes.

[0069] Crop type identification / prediction: The process of determining the types of crops planted in a plot of land. Existing methods for crop type identification can employ machine learning to achieve pixel-level classification, grouping similar pixels into plots, and then determining crop types based on plot-level statistical features or deep learning models using features such as spectral texture.

[0070] Object-oriented classification methods first segment remote sensing images according to spectral, textural, and spatial neighborhood relationships to generate several image objects, and then perform feature extraction and category determination for each object. This method is susceptible to the influence of segmentation scale and suffers from the problem of error propagation at each stage. In other words, segmentation or extraction errors generated in the previous stage will be amplified or solidified in the subsequent stage, resulting in an overall deviation in the final result.

[0071] Vector plots refer to farmland plots in vector data format, which include a series of attributes such as the plot's boundary coordinates, area, and unique identifier.

[0072] It should be noted that, unless otherwise specified, all plots of land mentioned in the subsequent technical description of this application refer to crop planting plots, that is, areas within a specific land area where only a single type of crop is planted in each agricultural production cycle.

[0073] Crop planting plots often lack clear physical boundaries, distinguishing only by different crop types. Multiple crops may be grown within the same natural plot. Therefore, accurate extraction of crop planting plots heavily relies on ultra-high-resolution UAV imagery. Research has found that traditional medium-resolution (e.g., 10-30 meter resolution) and high-resolution (1-10 meter) remote sensing imagery are completely incapable of identifying plot boundaries. Therefore, it is necessary to identify these plots based on ultra-high-resolution UAV imagery. When the ground sampling distance (GSD) of the UAV imagery reaches 0.05 meters (ultra-high resolution), ground details are highly preserved. Even without obvious physical boundaries (such as field ridges or ditches), the human eye can still distinguish the planting areas of different crop plots from differences in texture, plant arrangement, and canopy structure. However, using computers to perform traditional methods to identify plots in this type of data yields unsatisfactory results. Analysis of these methods reveals… The reasons may be as follows: Firstly, ultra-high resolution leads to a sharp increase in the complexity of ground textures. 0.05-meter resolution data can clearly reflect microstructures such as plant row spacing, leaf texture, and shadows. These textures are easily misidentified by computers as plot boundaries, resulting in a large number of oversegments and a sharp increase in the number of objects, making it difficult to form complete plot shapes. Secondly, in such images, the boundary differences of crop planting plots come from weak texture variations rather than physical boundaries (such as field ridges), making them difficult to accurately detect using traditional algorithms (such as thresholding, region growing, or edge-based algorithms), leading to blurring and adhesion of plot outlines (boundaries of crop planting areas). In addition, at 0.05-meter resolution, spectral variations between pixels are greater, and noise increases significantly. The higher the resolution, the more severe the salt-and-pepper noise in pixel-level classification becomes, making it difficult to stably recover plot boundaries based solely on pixel classification results. At the same time, ultra-high resolution leads to a surge in image data volume, resulting in huge computational costs and low efficiency for traditional methods.

[0074] Traditional (publication number: CN119360235A) methods use frequency domain decomposition for multi-task farmland planting plots. While this can solve the problems of blurred plot outlines and adhesion to some extent, it cannot support accurate crop classification. The reason for this may be that after the encoder features are decomposed in the frequency domain, smoothed in the region, and geometrically enhanced at the boundaries, the intermediate frequency and cross-channel information required for crop discrimination is partially lost. As a result, the obtained features can no longer support the crop classification requirements, making it difficult to obtain high-precision crop type recognition output.

[0075] In view of this, this application provides a method and system for multi-task extraction of crop types at the plot level from UAV imagery. This scheme constructs a multi-task learning model (Frequency-Augmented Multi-task Mamba Network, FAMM-Net) for plot-level crop type identification. This model collaboratively performs tasks such as plot region extraction, plot boundary extraction, and crop type prediction. It utilizes an encoder, decoder, and multi-task semantic segmentation head to form an end-to-end multi-task joint learning framework. The encoder's end does not perform frequency domain decomposition on high-dimensional features but directly inputs them into a multi-stage decoder built based on Mamba technology. For each stage of the decoder, Up-Fuse, Intrinsic-Mamba, and Freq-Harmony Mamba blocks are set to fully utilize the coupling relationship between plot geometry (processed by plot branches and boundary branches) and crop semantics (processed by crop type branches). This collaboratively performs internal dependency modeling within a single task and external dependency modeling between multiple tasks, obtaining extraction results that better reflect the actual planting area and crop type of the plot.

[0076] The embodiments of this application will now be described with reference to the accompanying drawings.

[0077] This application provides a plot-level crop type extraction system (Frequency-Augmented Multi-task Mamba Network, FAMM-Net) for UAV imagery. The multi-tasks include plot region extraction, plot boundary extraction, and crop type prediction. Figure 1As shown, the system includes: an encoder, a decoder, a multi-task semantic segmentation head, and a post-processing module. The encoder receives pre-acquired UAV imagery data (UHR) of the research area and performs multi-stage feature extraction. The decoder performs multi-stage upsampling on the feature maps extracted by the encoder. Each stage contains multiple task branches, each corresponding one-to-one with each task in the multi-task module. Each branch is composed of a multi-scale feature fusion Up-Fuse block, an intra-task dependency modeling Intrinsic-Mamba block, and an inter-task external dependency modeling Freq-Harmony Mamba block connected sequentially. The Up-Fuse block fuses the feature map output from the previous stage of the current task branch with the feature map of the corresponding encoder in the current stage. The Intrinsic-Mamba block uses Mamba technology to perform long-distance dependency modeling and feature enhancement within the current task based on the fused feature map. The Freq-Harmony Mamba block... The Mamba block is used to model cross-task dependencies of the current task based on the outputs of the Intrinsic-Mamba blocks of the current task branch and other task branches, and obtain the output results of the current task branch; the multi-task semantic segmentation head is used to perform multi-task prediction based on the output results of each branch in the last stage of the decoder, and obtain the multi-task prediction results.

[0078] The task of plot area extraction refers to dividing the input UAV imagery into regions to identify the overall spatial extent of crop planting plots in the imagery. This task aims to generate continuous and complete regional masks to distinguish areas belonging to a certain crop from planting areas that do not belong to that crop, providing basic spatial units for subsequent boundary localization and crop type identification.

[0079] The task of plot boundary extraction refers to the task of precisely locating the boundary lines of plot areas. As mentioned earlier, since plots lack obvious physical boundaries in ultra-high resolution images, the purpose of this task is to capture the true outer contours of plots from subtle features such as weak texture variations and differences in plant arrangement, and generate high-precision plot boundary prediction results to constrain the geometric shape of plots and improve the completeness and accuracy of plot extraction.

[0080] The crop type prediction task refers to the task of classifying crop types within each plot. This task uses spectral information, texture structure, plant morphology features, and regional and boundary constraints within the plot to output a prediction result for each plot belonging to a specific crop category, thereby achieving plot-level semantic recognition of crop types.

[0081] It should be noted that this embodiment achieves the interrelation and collaboration among the three tasks by designing a multi-branch, multi-stage decoder and a multi-task semantic segmentation head. Specifically, the plot area and plot boundary provide geometric constraints, crop type prediction provides semantic information, and the crop type planted in the plot is directly output based on the accurate segmentation of the former two. The parallel modeling of the three tasks can fully share features and establish cross-task dependencies, achieving end-to-end fine extraction of crop planting plots and crop types, avoiding cross-stage error accumulation, and improving the overall accuracy and robustness of fine extraction of crop planting plots.

[0082] In this embodiment, the multi-task crop type extraction system for UAV imagery at the plot level is designed as an encoder, decoder, multi-task semantic segmentation head, and post-processing module. Detailed structural descriptions of each component are as follows:

[0083] I. Encoder.

[0084] like Figure 1 As shown, the encoder is used to receive UAV imagery and then perform multi-stage feature extraction to obtain high-level semantic features and low-level detail information. In this embodiment, the encoder can be a CNN network or a Transformer-based network. Different tasks can share the same encoder, but the network structure of the encoder is not limited.

[0085] In a preferred embodiment, to obtain low-level detailed information and high-level semantic features to support fine-grained semantic segmentation tasks, a Swin Transformer may be optionally used as the encoder. Further, the encoder may include multiple stages, such as Stage 1 to Stage 4, with four stages extracting feature representations at different scales. Each stage consists of a Swin Transformer Block and a local block merging layer. The patch merging layer is used to reduce the spatial feature scale and expand the channel dimension; the Swin Transformer Block is the core computational unit of the Swin Transformer network, primarily used for feature extraction.

[0086] The encoder uses drone imagery as input data and employs... This indicates that the Patch Patition layer will first... The data is divided into non-overlapping local patches. Then, a linear embedding layer projects the pixels of each local patch into a feature vector (also called a token). This feature vector is then processed through four stages of feature extraction by the encoder, outputting feature representations at different scales. The features output from different stages of the encoder are used... , , , express:

[0087] (1)

[0088] in, , , , The shapes are respectively , , and .

[0089] II. Decoder.

[0090] In this embodiment, the decoder performs multi-stage upsampling on the feature map extracted by the encoder to gradually restore spatial resolution and fuse multi-level semantic information. Specifically, the encoder has a total of M stages (e.g., four stages), and the decoder has a total of M-1 stages (e.g., three stages). When the decoder starts performing upsampling, the deep feature map output from the M-th stage of the encoder is input into the Up-Fuse blocks of each branch of the (M-1)-th stage of the decoder, and fused with the feature map output from the (M-1)-th stage of the encoder.

[0091] Each stage of the decoder contains multiple branches, each corresponding one-to-one with a task in the multi-task learning framework. This ensures that different tasks can independently utilize shared high-level semantic and geometric features and perform task-specific feature optimizations. Furthermore, each branch is composed of a multi-scale feature fusion Up-Fuse block, a task-internal dependency modeling Intrinsic-Mamba block, and an inter-task external dependency modeling Freq-Harmony Mamba block connected sequentially.

[0092] The Up-Fuse block is used to progressively recover high-resolution detail information through progressive upsampling and multi-scale feature fusion. For example... Figure 2 As shown, the Up-Fuse block consists of a Linear layer, a rearrange operation layer, and an LN layer connected in sequence. This module takes the feature map output from the previous stage of the current task branch as input, and performs feature upsampling sequentially through the Linear layer and the rearrange operation layer to obtain a temporary upsampling result. This increases the resolution to twice that of the input feature map while compressing the channels to half. Then, the temporary upsampling result is connected to the feature map of the corresponding layer of the encoder (e.g., ...) through residual connections. , , , The fusion process is performed at the channel dimension (concatenation) to achieve multi-scale feature information complementarity and alleviate the spatial detail loss during encoder downsampling. Finally, a convolutional layer (Conv) is used to compress the fused features, thereby enhancing features and reducing computational complexity.

[0093] In other words, the Up-Fuse block is used to fuse the feature map output from the previous stage of the current task branch with the feature map of the corresponding encoder in the current stage. Specifically, starting from the (M-1)th stage of the decoder, the deep feature map output from the Mth stage of the encoder is first upsampled through each Up-Fuse block of different tasks to match the spatial scale, and then fused with the feature map output from the (M-1)th stage of the encoder. This achieves an effective combination of high-level semantic information and mid-level spatial details, and this fusion mechanism is repeated in subsequent stages of the decoder, tracing back up to the shallow features of the encoder.

[0094] Following each Up-Fuse block are sequentially connected Intrinsic-Mamba and Freq-Harmony Mamba blocks. The Intrinsic-Mamba block primarily utilizes Mamba technology to model long-range dependencies and enhance features within the current task, such as... Figure 3 As shown, the Intrinsic-Mamba block includes: a channel expansion layer, an SS2D sub-branch, a linear sub-branch, and a task-internal fusion layer. The channel expansion layer uses the fused feature map as the input feature map and expands the channels through a fully connected linear layer to obtain an expanded feature map. The SS2D sub-branch and the linear sub-branch are set in parallel. The SS2D sub-branch performs four spatial direction unfolding on the expanded feature map, resulting in four unfolding sequences. The State Space Model (SSM) is applied to each sequence to obtain the SSM output feature map. The linear sub-branch performs convolution and SiLU activation on the expanded feature map sequentially to obtain the linear branch feature map. The task-internal fusion layer fuses the SSM output feature map and the linear branch feature map, and then performs a skip connection with the input feature map to obtain the output of the Intrinsic-Mamba block.

[0095] The processing flow of Intrinsic-Mamba blocks is consistent for each individual task in a multitasking environment. Specifically, as... Figure 3 As shown, the fused feature map output by the Up-Fuse block is used as the input to the Intrinsic-Mamba block (denoted as...). ), First, layer normalization (LN) is performed in the channel expansion layer. Then, channel expansion (from C channels to 2C) is performed through a linear fully connected layer to obtain the expanded feature map. The input feature map is then processed... The number of channels is increased from C to 2C, increasing the capacity of the model and allowing the network to learn more complex features.

[0096] Optionally, to compensate for the shortcomings of the state-space model SSM in local feature extraction, before performing SS2D processing, the extended feature map first extracts local features through a convolutional layer (Conv), and uses the SiLU activation function to enhance the nonlinear feature representation. The result processed by the SiLU activation function is then input into the SS2D structure in the SS2D sub-branch.

[0097] In this embodiment, the SS2D structure in the SS2D sub-branch serves as the core component for feature extraction. This structure is an improvement on the one-dimensional SSM to be suitable for two-dimensional image processing. Specifically, SS2D first expands the input two-dimensional feature map (i.e., the expanded feature map after SiLU activation) into four one-dimensional sequences along four preset spatial directions, resulting in four one-dimensional sequences. These four preset spatial directions are: left-to-right (L2R), right-to-left (R2L), top-to-bottom (T2B), and bottom-to-top (B2T). Then, each of the four expanded one-dimensional sequences undergoes SSM processing, i.e., state-space SSM processing based on the S6 model is applied independently. After normalization, the SSM output feature map is obtained, denoted as... This allows for the separate modeling of long-distance context dependencies in each direction. The state-space SSM processing based on the S6 model can be performed using existing techniques and will not be elaborated upon here.

[0098] The linear sub-branch runs in parallel with the SS2D sub-branch. It is used to first perform convolution (Conv) operations on the expanded feature map, and then perform SiLU activation to output the feature map of the linear branch.

[0099] In output After the linear branch feature maps, the data enters the task's internal fusion layer, which uses element-wise multiplication to... Feature fusion is performed with the linear branch feature map to establish long-range dependency modeling within the current task, compensating for the shortcomings of SSM in local feature modeling; then, a compression (reduction) operation is performed through a linear layer to compress the number of channels to C, resulting in the output feature map. Here, channel compression reduces feature redundancy and improves the computational efficiency of subsequent processing. Then, to enhance model performance and training stability, the channel-compressed feature maps are... and Perform a skip join (i.e., add elements one by one) to obtain the output of the Intrinsic-Mamba block. ,Right now .

[0100] It should be noted that in this embodiment, since the decoder is configured with a multi-branch structure, each branch corresponds one-to-one with each task in the multi-task system. Therefore, the Intrinsic-Mamba block of each branch corresponds to an output. The output consists of three branches. For ease of description, we can choose any branch as the current task branch, and other branches are called other task branches.

[0101] After completing the internal dependency modeling of the task using the Intrinsic-Mamba block, the Freq-HarmonyMamba block is used to perform the external dependency modeling between tasks, also known as the cross-task feature interaction phase. In this phase, the current task is also called the specific task, and other tasks are also called shared tasks.

[0102] In this embodiment, the Freq-Harmony Mamba block is used to model the cross-task dependency relationship of the current task based on the output of the Intrinsic-Mamba block of other task branches besides the current task branch, and obtain the output result of the current task branch.

[0103] In other words, the Freq-Harmony Mamba block is the core module for information exchange between different tasks, used to model complex dependencies between features of different tasks. For example... Figure 4 As shown, this module includes: a shared task sub-branch and a specific task sub-branch; wherein, the shared task sub-branch uses a multilayer perceptron (MLP) network to process the output of the Intrinsic-Mamba blocks of other task branches (i.e., the output of other branches). Cross-task information fusion is performed to obtain fusion features. ; Specific task sub-branch, used for based on fusion features SSM parameters are generated, and cross-sequence cross-scanning is performed on the fusion features of other task branches and the features of the current task branch based on Mamba technology to obtain cross-task dependency features that enhance the expression of the current task branch.

[0104] In this context, "other task branches" refers to task branches other than the current task. It should be noted that any one of the three task branches in this application can serve as the current task branch. When any one of these tasks is the current task, the other two tasks are called "other task branches," which are the branches corresponding to the aforementioned shared tasks, while the current task is called the "specific task branch."

[0105] In one specific implementation, the shared task sub-branch further includes: a shared task fusion layer, a first frequency enhancement module (LFEB) block, and a hybrid gated HGU unit; wherein, the shared task fusion layer is used to perform feature fusion on the output of the Intrinsic-Mamba block of other task branches outside the current task to obtain preliminary inter-task fusion features. ;LFEB block, used for Perform a Fourier transform to convert to the frequency domain to obtain the frequency domain features. and will Multiply by the weight matrix W, and then perform an inverse Fourier transform to obtain the frequency-enhanced features. The HGU unit is used to process frequency domain features using a two-layer fully connected multilayer perceptron (MLP) network. Perform an adaptive transformation to obtain the adaptive transformation result. At the same time, Perform channel attention processing to obtain the channel attention processing result. Then After sigmoid activation and Summation is performed to obtain the gating weights, and these gating weights are then used to adjust the system. and The proportion of these proportions is used to obtain the final fusion characteristics. .

[0106] The specific task sub-branch includes: the TS-SS2D component; this component is used to determine the fusion features. Generate SSM parameters and, based on Mamba technology, use the output of the Intrinsic-Mamba block of the current task branch as the query input. Based on the SSM parameters, the state-space model SSM processing is performed to obtain the processing result. The processing results A skip join is made with the output of the Intrinsic-Mamba block of the current task branch to obtain cross-task dependency features.

[0107] The following is combined Figure 4 Describe in detail the processing flow of Freq-Harmony Mamba blocks.

[0108] like Figure 4As shown, the shared task fusion layer of the shared task sub-branch receives the output of Intrinsic-Mamba blocks from other task branches. Feature concatenation is performed at the channel dimension. The process involves feature fusion, followed by normalization (LN) and convolution (Conv) to generate a global feature representation, which is the initial inter-task fusion feature, denoted as . The expression is as follows:

[0109] (2)

[0110] in, This indicates the output of the Intrinsic-Mamba block from other task branches, with the superscript indicating the task number. This represents the total number of other tasks.

[0111] Considering the need to balance global and local information in semantic segmentation tasks, in the frequency domain, high-frequency signals correspond to local information, and low-frequency signals correspond to global information. Therefore, compared to the spatial domain, the frequency domain can better represent global and local features. Based on this, this embodiment designs a first frequency enhancement module, LFEB (also known as a Learned Frequency Enhancement Block), to achieve perceptual enhancement for different tasks. The core design of this module is: based on the two-dimensional Fast Fourier Transform (FFT2D), the input feature map is transformed from the spatial domain to the frequency domain, thereby explicitly separating and manipulating its low-frequency and high-frequency components. Low-frequency signals correspond to global information, and high-frequency signals correspond to local information. In the frequency domain, the LFEB block introduces a set of learnable complex weight parameters (i.e., weight matrix W) to form a frequency domain filter that can be trained end-to-end. This filter can adaptively weight different frequency components according to different task requirements, enhancing the frequency components that are discriminative for the current task while attenuating irrelevant frequency signals.

[0112] In this model, the weight matrix W assigns a complex weight (including amplitude and phase adjustment) to each channel and each frequency point (coordinates (u,v)). If the task is semantic segmentation (which typically requires smooth information), the model automatically learns to increase the weight of low-frequency regions; if the task is edge detection (which requires sharp boundaries), the model automatically learns to increase the weight of high-frequency regions. In other words, different task branches within the LFEB block can dynamically adjust their frequency domain response according to their task characteristics. For example, the task of extracting land parcel boundaries needs to emphasize high-frequency signals, while the task of extracting land parcel regions needs to focus on low-frequency signals. For crop type prediction, weights can be automatically learned based on the texture features of different crops to highlight effective specific frequency signals. Therefore, LFEB not only achieves learnable control of frequency domain information but also significantly enhances the perception ability of different task branches to structures at different scales, thereby effectively improving overall segmentation accuracy and generalization performance.

[0113] Specifically, The input is fed into the LFEB block to perform a Fourier transform to enhance the frequency features, so as to adaptively learn global and local feature representations.

[0114] like Figure 4 As shown, LFEB block pair Perform a 2D Fourier transform to Transforming from the spatial domain to the frequency domain yields the frequency domain characteristics. ( Figure 4 (also known as the original frequency), then... Multiplying the weighted frequency by the learnable weight matrix W yields the weighted frequency, denoted as . ( Figure 4 (Also known as weighted frequency), to achieve frequency-selective enhancement; then... Perform an inverse Fourier transform to obtain the frequency-enhanced features. The above process can be expressed by the following formula:

[0115] (3)

[0116] (4)

[0117] (5)

[0118] In the formula, , These represent the 2D Fourier transform and its inverse transform, respectively. This indicates element-wise multiplication.

[0119] By Multiplying with a learnable weight matrix W, weights of different frequencies are automatically learned, thereby capturing both coarse and detailed spatial features.

[0120] After obtaining frequency enhancement features Next, a Hybrid Gated Unit (HGU) is set up, and a pixel-wise gating method is used to dynamically control the weights of the spatial and frequency domains, so that the shared task features can fully learn the effective information of the spatial and frequency domains (dual domains) and achieve adaptive fusion.

[0121] Specifically, after the inverse Fourier transform, The spatial domain features, enhanced by the frequency domain, are input into the HGU cell. Within the HGU cell, the spatial domain features are first processed... Normalization (LN) is performed, followed by adaptive transformation using a two-layer fully connected network (MLP) (the specific execution steps of the MLP network can be found in existing technologies and will not be elaborated here). The adaptive transformation result... The expression is as follows:

[0122] (6)

[0123] Meanwhile, preliminary inter-task fusion features (This feature is also a spatial feature.) First, a 1×1 convolution is performed to complete the channel mapping. Then, a channel attention module (CAB) is applied to calculate the weights of local feature information to obtain the channel attention processing result. The expression is as follows:

[0124] (7)

[0125] The Channel Attention Module (CAB) can be implemented using the existing Channel Attention Mechanism (CBAM), which will not be elaborated here.

[0126] Then, the sigmoid activation function is used to calculate and The gating weights are scaled after being processed by a 1×1 convolution to calculate pixel-level gating weights. The expression is as follows:

[0127] (8)

[0128] In the formula, It is the sigmoid activation function.

[0129] Finally, use gating weights To adjust and The proportion of each component determines the final fusion feature output. :

[0130] (9)

[0131] because For spatial features enhanced by the frequency domain, Spatial domain features fused for the initial task, using Performing feature fusion on a pixel-by-pixel basis can achieve a balance between frequency domain features and spatial features. The HGU unit controls the weights of the spatial and frequency domains at the pixel level, avoiding the loss of spatial or frequency domain information and reducing information redundancy. The synergy of information from both domains helps to improve feature representation capabilities.

[0132] The above describes the structure and execution flow of shared task sub-branches. Specific task sub-branches within the Freq-Harmony Mamba block are configured to merge features. (Representing shared task features) are injected into the current task to enhance the cross-task dependency feature expression of the current task branch. The structure and process of a specific task sub-branch are described in detail below.

[0133] like Figure 4 As shown, the core structure of the specific task sub-branch is the TS-SS2D component. This component can leverage the characteristics of SSM to fuse shared task features and specific task features, explicitly model cross-task dependencies, and improve the multi-task collaboration effect.

[0134] The overall structure of the TS-SS2D component is constructed based on the SS2D sub-branch and linear sub-branch of the Intrinsic-Mamba block. Like the Intrinsic-Mamba block, this component also includes a channel expansion layer, two branches, and a fusion layer. However, unlike the Intrinsic-Mamba block, the TS-SS2D component uses TS-SS2D instead of the traditional SS2D structure. Specifically:

[0135] In the Intrinsic-Mamba block, the SS2D structure of the SS2D sub-branch is used for feature extraction to achieve long-range dependency modeling within a single task. In the Freq-Harmony Mamba block, the TS-SS2D component is used to achieve cross-sequence interaction of features, dynamically modeling dependencies between tasks.

[0136] Therefore, the TS-SS2D component first uses fusion features. Generate SSM parameters (including time step) and parameter matrix Then, take the output of the Intrinsic-Mamba block of the current task branch. As query input Based on the generated SSM parameters, the state-space model SSM processing is performed to obtain the processing result. The expression is as follows:

[0137] (10)

[0138] (11)

[0139] In the formula, This indicates parameter generation processing. This indicates SSM processing.

[0140] It should be noted that in traditional SS2D, the input sequence is used both to generate parameters and as query input. Formula (10) uses the output of the shared task sub-branch. To generate The parameters leverage shared features from other tasks to provide global semantic guidance for the current task, enabling cross-sequence interaction of features and dynamically modeling dependencies between tasks.

[0141] in, The specific execution details of parameter generation and processing are similar to those of existing Mamba technologies and will not be elaborated here.

[0142] It should also be noted that the SS2D in the Intrinsic-Mamba block performs an unrolling of the current task features along four spatial directions, with each sequence scanned independently. This differs from, for example... Figure 5 As shown, the SSM processing of the TS-SS2D component is performed based on formulas (10) and (11), specifically using specific task features as query input. (That is, the output of the Intrinsic-Mamba block of the current task branch after LN operation, channel expansion, convolution, and SiLU) Figure 5 Task-Specific features (feature A) are used as input sequences, and shared task features (i.e., ...) , Figure 5 The Task-Shared sequence is used as another sequence (feature B, used to generate parameters). The two sequences are cross-sequence scanned in four directions: left to right (L2R), right to left (R2L), top to bottom (T2B), and bottom to top (B2T). They serve as scanning guides for each other instead of scanning independently, thereby achieving the fusion of features from different tasks and completing long-distance dependency modeling.

[0143] The following uses L2R expansion as an example to illustrate the process of cross-sequence scanning. For the i-th row, scanning proceeds from left to right, comparing the previous pixel of feature A (i.e., the (j-1)-th pixel of the i-th row) with feature B (i.e., the pixel above feature A). The current position (i.e., the j-th pixel in the i-th row) is used as input to generate the current parameters according to formula (10). Then, according to formula (11), the generated parameters are used. Update the current position state of feature A by scanning line by line from left to right to generate the result for the current direction. The same process is applied to other directions (R2L, T2B, B2T). Then, the outputs from each direction are fused (e.g., by concatenation) to obtain features that enhance global information.

[0144] In this embodiment, the TS-SS2D component performs SSM processing simultaneously using the characteristics of both the shared task and the specific task as input. By interacting with specific task features, the TS-SS2D component can effectively capture long-range dependencies and semantic interactions between tasks. Compared to SS2D, which only relies on information from a single input, it has a stronger ability to collaboratively model multiple tasks.

[0145] The above describes the structure of the decoder. After multi-stage upsampling by the decoder, in the final stage, the Freq-Harmony Mamba blocks of different branches output their corresponding feature maps, so that the multi-task semantic segmentation head can perform multi-task prediction and obtain the multi-task prediction result.

[0146] III. Multi-task semantic segmentation head.

[0147] In this embodiment, the multi-task semantic segmentation head includes a parcel head, a boundary head, and a crop type head, each corresponding to a parcel region extraction task, a parcel boundary extraction task, and a crop type prediction task, respectively. The boundary head includes a second frequency domain enhancement module (PDFR) block, which receives the output of the Freq-Harmony Mamba block (the last stage of the parcel boundary extraction task branch) in the decoder. After performing a Fourier transform, it enhances the parcel boundary features in the high-frequency information portion, and then obtains the enhanced boundary features through an inverse Fourier transform. .

[0148] In other words, each task has its own semantic segmentation prediction head, and these semantic segmentation prediction heads make predictions separately to obtain the final prediction results.

[0149] Structurally, each semantic segmentation prediction head consists of a cascaded (i.e., sequentially connected) layer of 3×3 convolutions, batch normalization, ReLU activation, convolutional projection layers, and bilinear interpolation upsampling. The outputs of the Freq-Harmony Mamba block in the final stage of the decoder for different task branches are fed into their respective semantic segmentation prediction heads. First, 3×3 convolutions capture local feature details and adjust the channel dimensions. Batch normalization layers stabilize the feature distribution to accelerate convergence. ReLU activation enhances non-linear expressive power. Finally, features are projected to the dimensions required by different task branches, and bilinear interpolation ×4 is used to restore the original resolution. This method has good efficiency and effectively reduces the computational complexity of the model.

[0150] Furthermore, the boundary branch in the multi-task semantic segmentation head includes a second frequency domain enhancement module PDFR block (also known as the Prior-Driven Frequency Refinement Block). This module is set after the Freq-Harmony Mamba block in the last stage of the decoder and is a frequency domain enhancement module dedicated to the edge detection task. It uses known prior knowledge to drive the model to enhance the modeling of parcel boundary features in the high-frequency information part.

[0151] Similar to LFEB, PDFR also extracts frequency domain features through Fast Fourier Transform (FFT). The difference is that PDFR uses the learnable weight parameters from LFEB. Replace it with a high-frequency enhancement filter to highlight the high-frequency components.

[0152] Specifically, such as Figure 6 As shown, PDFR consists of sequentially connected LN layers, FFT2D layers, high-frequency enhancement filters, IFFT2D layers, LN layers, and skip connections after the input features to obtain boundary enhancement features. This module uses the output of the Freq-Harmony Mamba block, the final stage of the decoder, as its input feature, denoted as... First, normalization is performed through the LN layer to obtain... Then, the FFT2D layer performs a 2D Fourier transform on the features. Transforming from the spatial domain to the frequency domain yields The expression is as follows:

[0153] (12)

[0154] Subsequently, in the frequency domain, a high-frequency enhancement filter is introduced, applying an additional amplification factor to the high-frequency region of the spectrum to enhance the contribution of high-frequency components, resulting in the output of the high-frequency enhanced characteristics. The expression is as follows:

[0155] (13)

[0156] in, Represents the high-frequency components in the spectrum. These are the input features Height and width , These are the row and column indices of the frequency components in the frequency domain, respectively.

[0157] Next, the IFFT2D layer... Using the inverse fast Fourier transform, we can map it back to the spatial domain, as shown in the following expression:

[0158] (14)

[0159] To ensure numerical stability, normalization is performed after IFFT2D. Finally, residual connections (skip connections) are used to fuse the input features. and enhanced features This method preserves the original information while enhancing the model's ability to perceive high-frequency details. Compared to attention mechanisms, this method can explicitly enhance features while reducing computational costs and improving model computational efficiency.

[0160] In this embodiment, by introducing PDFR, while preserving global structural information, boundary information is enhanced through explicit high-frequency bias, thereby improving the sensitivity of boundary branches to details. This module is located in the boundary branch of the multi-task framework and follows the last Freq-Harmony Mamba module. It can fully utilize the high-resolution features restored by the decoder upsampling, which are rich in detail, while also utilizing the low-level features output by the backbone network, which helps in the effective extraction of features.

[0161] V. Loss Function Design.

[0162] In this embodiment, the loss function of FAMM-Net is the weighted sum of the losses of the three tasks: plot branch, boundary branch, and crop type branch.

[0163] The loss for the plot branch is represented by binary cross-entropy (BCE), denoted as: The loss for the boundary branches uses negative log-likelihood loss (NLL), denoted as: The loss for the crop type branch is a weighted sum of the Dice loss (weight 0.5) and the cross-entropy loss (CE) (weight 0.5), denoted as: The total losses are as follows:

[0164] (15)

[0165] in, , , For example, the weights can be set as follows: = = 1, = 2. The specific calculation formulas for binary cross-entropy loss, negative logarithmic probability loss, Dice loss, and cross-entropy loss can be referred to existing technologies and will not be elaborated here.

[0166] V. Post-processing module.

[0167] The multi-task prediction result output by FAMM-Net is a raster image, but in practical agricultural applications, vector data (closed polygons with attributes) is required. Furthermore, to achieve accurate vectorized extraction of crop plots, in an optional scheme, the post-processing module of the system provided in this embodiment is further used to: perform semantically constrained morphological dilation processing on the plot boundary extraction results (i.e., the output of the boundary branch) in the multi-task prediction results to obtain semantically enhanced boundaries. Lee's skeleton algorithm was used to... Boundary skeleton extraction is performed to obtain a boundary skeleton graph, and a semantic topology graph is constructed based on the boundary skeleton graph. Each topological segment in the network records the crop type attributes on both sides; a semantic topological graph is used. The crop type is used as semantic information to guide the extension direction of the hanging line. The hanging line segment of the boundary skeleton map is optimized to obtain the optimized skeleton map. Based on the optimized skeleton map, semantic vectorization is performed to obtain vector crop plots with crop type attributes.

[0168] Specifically, the extracted land parcel boundaries from the multi-task prediction results undergo semantically constrained morphological dilation to obtain semantically enhanced boundaries. The process includes: performing morphological dilation on the extracted plot boundaries in the multi-task prediction results to obtain the dilated boundary result Boundary_3; using an edge detection algorithm to detect crop type changes in the crop type prediction results from the multi-task prediction results, determining the pixel locations where crop type changes occur, and generating crop plot edges Crop_boundary; performing morphological dilation on the crop plot edges Crop_boundary to obtain the dilated crop boundary Crop_boundary_3; and taking the union of the dilated boundary result Crop_boundary and the dilated crop boundary Crop_boundary_3 to obtain the semantically enhanced boundary. Using semantic topology graphs The crop type is used as semantic information to guide the extension direction of the dangling line. The boundary skeleton diagram is then optimized by processing the dangling line segments to obtain the optimized skeleton diagram, which includes: semantically enhanced boundaries. The crop type prediction results and plot area extraction results from the multi-task prediction results are overlaid to generate an enhanced boundary mask (Crop_mask) that integrates geometry and semantics; based on the semantic topology graph... The process involves identifying dangling lines; traversing each dangling line, using the tangent direction of the dangling line as the extension direction, and extending from the dangling vertex in the extension direction. For each pixel extended, geometric collision checks and semantic constraint checks are performed. Geometric collision checks include checking if the current extending pixel collides with an existing skeleton line in the boundary skeleton graph; if so, the dangling line is closed; otherwise, a semantic constraint check is performed. Semantic constraint checks include checking, based on the enhanced boundary mask `Crop_mask`, whether the line composed of the extending pixels cuts off a complete crop patch; if so, the extension is stopped and the dangling line is removed from the boundary skeleton graph and semantic topology graph. If the line segment is removed, and the semantic constraint check passes, the extension process continues. For each pixel extended, geometric collision checks and semantic constraint checks are performed until successful closure or the maximum extension length is reached. The newly extended line segment is then drawn in the boundary skeleton graph, and the semantic topology graph is updated. .

[0169] Specifically, the post-processing module is based on the Topo-Semantic Optimized Vectorization for Crop Fields (TSOV-CF) post-processing optimization algorithm, which uses the semantic information of crop types to constrain and repair geometric boundaries, achieving coordinated optimization of geometric accuracy and semantic consistency in the vectorization of farmland plots of multiple crop types, while completing the conversion from raster to high-precision vector plots.

[0170] The TSOV-CF algorithm takes multi-task prediction results as input, including: region_pred (plot region result, each cell value is 0 and 1, a binary graph), Crop_pred (crop type result, each cell value is crop ID, i.e., 1, 2, 3, 4... represent different crops), and Boundary_pred (plot boundary result, a binary graph of 0 and 1).

[0171] As an example, the detailed execution steps of post-processing are as follows:

[0172] Step 1: Perform semantically enhanced boundary processing: Perform semantically constrained morphological dilation processing on the plot boundary extraction results in the multi-task prediction results to obtain semantically enhanced boundaries. Among them, the plot boundary in the multi-task prediction result is the output of the plot boundary extraction task branch in the model, referred to as plot boundary (Boundary_pred). The input plot boundary (Boundary_pred) is subjected to semantic constraints (i.e., constraints are imposed using crop type Crop_pred, where crop type Crop_pred is the output of the crop type prediction task branch). Morphological dilation is performed on the input plot boundary (Boundary_pred). This operation can enhance the boundary connectivity and inherit the information of neighboring crop types to generate an enhanced boundary mask that integrates geometry and semantics. The detailed steps are as follows: (1) Morphological dilation is performed on the plot boundary (Boundary_pred) (the dilation radius is 3 pixels) to obtain the boundary dilation result Boundary_3; (2) Then, crop type change detection is performed on the crop type map (Crop_Pred) using the edge detection algorithm to find the pixel position where the crop type has changed. Specifically, a 3×3 convolution kernel is used to scan the crop type map. If the category ID of the center pixel is different from that of the surrounding pixels, it is marked as an edge to obtain the crop plot edge (Crop_boundary). Morphological dilation is also performed on Crop_boundary (dilation radius is 3 pixels) to obtain the dilated crop boundary Crop_boundary_3; (3) The union of the two dilated boundaries (i.e. Boundary_3, Crop_boundary_3) is calculated, i.e. To obtain the semantic enhancement boundary The above morphological dilation process not only includes the geometric dilation of the plot boundary, but also combines the constraints of crop type to enhance the connectivity of the semantic boundary of crop planting, which can more accurately express the actual crop planting situation. (4) Generate mask (Crop_mask): While enhancing the boundary connectivity, it inherits the neighboring crop type information to generate an enhanced boundary mask that integrates geometry and semantics. The specific steps include: a) Initializing Crop_mask as a copy of Crop_Pred; b) ... (The widened boundary area) The corresponding pixel position is set to 0 in Crop_mask (that is, the widened boundary position may be a crop that has encroached on the interior of the plot); c) Using region_pred, the corresponding pixel position of the non-farmland area in Crop_mask is also set to 0 (the 0 value area in the region_pred diagram is the non-plot area, which may be the boundary area between plots of this planting type and other planting types or non-farmland plots), thereby generating the final mask, in which 1, 2, 3, 4... represent the interior of different crop plots.

[0173] Step 2, semantic topology construction, specifically includes the following sub-steps:

[0174] (1) Using the classic Lee skeleton algorithm (for specific algorithm details, please refer to existing technologies), the semantic enhancement boundary is defined. The boundary is refined to obtain a pixel-width boundary skeleton map, denoted as: ,Right now: .

[0175] (2) Transformation graph data structure ,in It refers to nodes (intersections, endpoints). It is an edge (connecting line). (Diagram) The specific transformation process is as follows: First, traverse the boundary skeleton diagram. For all skeleton pixels, identify nodes: if the current skeleton pixel has only one neighboring skeleton pixel in its 8-neighborhood, mark the skeleton pixel as an endpoint (V_Endpoint); if the previous skeleton pixel has more than two neighboring skeleton pixels in its 8-neighborhood, mark the skeleton pixel as a junction (V_Junction). Then, trace edges: starting from one node, trace along the skeleton path until another node is encountered, and record the coordinates of all pixels on the path as an edge Edge(i) for subsequent identification of dangling lines.

[0176] (3) is a diagram Adding semantic information, that is Each edge (Edges(i)) in the graph is assigned two fields: left_type and right_type, to record the crop semantics (i.e., crop type attributes) of that edge. left_type represents the crop type planted in the left region of the edge, and right_type represents the crop type planted in the right region of the edge, forming a semantic topology graph. The specific implementation is as follows: [Figure] Each edge (Edges(i)) in the graph consists of a series of pixels. For each Edge(i), 10 pixels are probed outward along the normal direction (i.e., the vertical direction) on both sides. The crop ID values ​​of these 10 pixels at the corresponding positions in the Crop_mask are counted. The crop ID value with the highest frequency (non-zero) is the crop type on both sides of the boundary. If the left and right sides of the boundary have the same crop ID, it means that this is a boundary of the same type of crop. This constructs a semantic topology graph. Each topological line segment (i.e., edge) records the crop type attributes on both sides of it (e.g., corn on the left and rice on the right).

[0177] Step 3, Optimization and Repair of Suspended Lines. Suspended lines refer to lines that are not connected to other lines or whose endpoints are suspended. This step is used to resolve boundary breakage issues. Specifically, it includes the following sub-steps:

[0178] (1) Identify the suspension line.

[0179] The semantic topology graph generated based on the aforementioned steps Filter out the endpoint (V_Endpoint) and the edges (Edges(i) connected to the endpoint, and define Edges(i) as dangling lines. That is, one end of the dangling line belongs to the endpoint (this endpoint is also called the dangling vertex, indicating that the point is not connected to other lines and is disconnected), and the other end connects to the intersection point.

[0180] (2) Calculate the extension direction of the suspension line. In order to naturally complete the boundary, the extension direction should be along the tangent direction of the suspension line. The specific steps are as follows: a) Extract several pixels close to the suspension vertex (e.g., the 10 points closest to the suspension vertex), use the least squares method to fit a straight line based on the coordinates of these pixels, and calculate the unit direction vector. .

[0181] (3) Semantic guidance.

[0182] Starting from the dangling vertex, i.e., the endpoint V_Endpoint, along the direction Extending pixel by pixel, two checks are performed after each pixel extension: a) Geometric collision check: Checking if the current point collides with an existing skeleton line (i.e., a point with a value of 1 in Skeleton_img). If yes, the dangling line is successfully closed and stops extending; otherwise, a semantic constraint check is performed. b) Semantic constraint check: Checking if the extended line cuts off a complete crop patch. Specifically, this includes using a semantic topology graph. The algorithm determines whether the crop types on both sides of the suspending line are consistent. If they are inconsistent, it uses the Crop_mask crop type map to determine the crop types on both sides of the newly added extended pixel. If the pixels within a 5×5 area around the extended pixel are of the same crop type (for example, if the suspending line originally demarcated corn and rice, but the extended pixel (extended line) is inserted into the center of a cornfield, meaning the 5×5 area at the endpoint is all corn), then it is determined to be an illegal extension, the extension is stopped, and the pixel value of this suspending line is set from 1 to 0. The suspending line is also removed from the semantic topology map. Remove from the middle. Otherwise, if the pixels in the 5×5 area around the extended pixel are different crop types, the semantic constraint check is considered passed, and the extension continues, entering the two-check process for the next extended pixel, until successful closure or the maximum extension length is reached (e.g., maximum extension length max_length = 50), after which topology update is performed.

[0183] (4) Topology update.

[0184] If the new extended pixel collides during the geometric collision check, the closure is successful, the new line segment is drawn on Skeleton_img, and added to the semantic topology graph. Add new edges and nodes. Also, if a collision occurs in the middle of another edge, the edge that is being collided with needs to be broken into two parts and a new node inserted.

[0185] Traverse all hanging lines and execute the above hanging line optimization and repair process until the process is completed, and obtain the repaired closed skeleton image, denoted as optimized_Skeleton_img. This image is a raster image, where a pixel value of 1 represents a skeleton pixel, i.e., the boundary of the plot, and 0 represents a non-skeleton pixel, i.e., the internal area of ​​the plot.

[0186] Step 4, semantic vectorization, converts the repaired closed skeleton map optimized_Skeleton_img from raster format to the final GIS vector data (polygons with attributes), and performs geometric simplification to remove jagged edges. This includes the following sub-steps:

[0187] (1) Region extraction. Invert the optimized_Skeleton_img to change the pixel value of the plot area from 0 to 1, i.e., to become the foreground color, and change the skeleton to the background color, with a pixel value of 0. Then, use connected component analysis to assign a unique code crop_parcel_id to each independent plot, and the code is the same within the same plot.

[0188] (2) Attribute assignment. Overlay optimized_Skeleton_img and Crop_mask, and extract each encoded crop_parcel_id in optimized_Skeleton_img as the current plot. Perform a majority vote based on the crop ID of the pixel at the corresponding position in Crop_mask of the current plot, that is, count the crop category of all pixels in the current plot, and take the crop ID that appears most frequently as the crop attribute (CROP_TYPE) of the current plot. After the traversal processing is completed, generate the raster outline map of the crop plot, which is recorded as the first raster outline map. Simultaneously, the optimized_Skeleton_img and region_pred are overlaid, and each encoded crop_parcel_id in optimized_Skeleton_img is extracted as the current plot. Based on the crop ID of the pixel at the corresponding position of the current plot in region_pred, a majority vote is performed, and the category with the most occurrences (0 or 1) is taken as the plot attribute of the current plot. After the traversal processing is completed, a raster outline map of the plot is generated, which is denoted as the second raster outline map.

[0189] (3) Vectorization and adaptive simplification.

[0190] The plot edges in the first and second raster contour maps are converted into point sequences (at this point, the points are very dense, resulting in a jagged appearance). The Douglas-Peucker algorithm is then used to adaptively simplify the polygons formed by the two point sequences, yielding two high-precision vector data results: crop plot vectors and plot region vectors. The Douglas-Peucker algorithm requires setting a parameter to allow a maximum deviation distance (i.e., a threshold). Its adaptive selection method is as follows: based on the semantic topology graph. Determine if the current edge is the boundary between two different crops. If so, set the threshold. Set the threshold to 0.5 meters to retain high precision; otherwise, it indicates that the current edge is merely the boundary between the plot and the background, or that the crop type is the same, and the threshold should be adjusted accordingly. The measurement was set to 1.2 meters to simplify calculations and improve efficiency.

[0191] VI. Construction of the training dataset.

[0192] In existing technologies, plot-level datasets are mainly constructed for plot extraction purposes, and mostly consist of natural farmland plots. Limited by the resolution of remote sensing imagery, it is difficult to achieve fine-grained annotation at the plot level, and there is a lack of publicly available large-scale farmland plot-level datasets, especially those with detailed crop type information. To train FAMM-Net, this embodiment also includes a plot-level dataset construction module, used to acquire historical UAV imagery data of the study area and create a sample dataset. This dataset is also called: Large-Scale Ultra-High Resolution UAV Farmland Plot Crop Type Recognition Dataset (UHR-based Parcel-level Crop Type Dataset, UHR-PCD), and its construction process is as follows:

[0193] The UHR-PCD dataset, a refined dataset of drone imagery collected across a province, was created through manual annotation by experts with specialized knowledge. UHR-PCD boasts advantages such as diverse plot scenes, wide data distribution, large scale, and high spatial resolution. Its design aims to improve effectiveness in complex tasks such as farmland plot extraction and plot-level crop type identification, and to enhance the accuracy of precision agriculture mapping. Specifically, the dataset includes 1612 drone images collected in July 2023, with a ground sampling distance (GSD) of 0.05 meters. Experts manually annotated 102,663 planting plots, covering an area of ​​approximately 370 square kilometers. The images were resampled to a resolution of 0.1 meters, resulting in 17,937 image-label pairs. The training, validation, and test sets are in a 6:2:2 ratio.

[0194] The FAMM-Net is trained based on this dataset to obtain the trained FAMM-Net. The UAV image to be predicted is then input into the plot-level crop type multi-task extraction system of the trained UAV image to obtain the output results. The output results are then used to create plot-level crop type maps.

[0195] Based on the same inventive concept, this embodiment also provides a plot-level crop type multi-task extraction method for UAV imagery. This method is executed by the plot-level crop type multi-task extraction system for UAV imagery provided in any of the above embodiments, and includes:

[0196] Historical UAV imagery data of the study area is acquired and a sample dataset is created. The sample dataset is used to train the plot-level crop type multi-task extraction system for UAV imagery provided in any of the above embodiments. The UAV imagery to be predicted is input into the trained plot-level crop type multi-task extraction system for UAV imagery to obtain the output results. The output results are then used to create plot-level crop type maps.

[0197] To verify the excellent performance of the technical solution provided in this embodiment, the accuracy of the system provided in this application is compared with that of various existing single-task and multi-task learning methods (existing models) based on the UHR-PCD dataset. The existing comparison models are shown in Table 1:

[0198] Table 1. Introduction to the Comparison Model

[0199]

[0200] The comparison results are shown in Table 2, which is as follows:

[0201] Table 2. Plot-level accuracy evaluation results for crop type identification

[0202]

[0203] In Table 2, GOC (Global Over-Classification) represents the global over-classification error, GUC (Global Under-Classification) represents the global under-classification error, and GTC (Global Total-Classification) represents the global total classification error. GOC, GUC, and GTC are used to evaluate the geometric accuracy of the land parcel identification results and are defined as follows:

[0204] (16)

[0205] (17)

[0206] (18)

[0207] In the formula, For the first One predicted plot of land, The total number of land parcels. , , The calculation formulas for oversegmentation, undersegmentation, and total segmentation error are as follows:

[0208] (19)

[0209] (20)

[0210] (twenty one)

[0211] in, for Reference plots.

[0212] Table 2 presents the accuracy evaluation results of different models at the object level. For the two target crop types, maize and rice, FAMM-Net achieved the best accuracy in both GUC and GTC metrics. Compared to the second-best results, the GTC metric for rice and maize was reduced by 2.85% and 1.56% respectively. Compared to BsiNet, the model reduced the GTC error by 4.32% and 2.78% for rice and maize, and the GUC error by 4.23% and 5% respectively. Furthermore, the InvPT++ model performed second only to FAMM-Net in multi-task model evaluation. Based on the above analysis, FAMM-Net demonstrates superior capabilities in object-level accuracy evaluation compared to other state-of-the-art models.

[0213] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-task system for extracting crop types at the plot level from UAV imagery, characterized in that, The multi-tasks include the task of extracting the plot area of ​​crop planting plots, the task of extracting plot boundaries, and the task of predicting crop type. The system includes: an encoder, a decoder, a multi-task semantic segmentation head, and a post-processing module. The encoder is used to receive pre-acquired UAV image data of the research area and perform multi-stage feature extraction. The decoder is used to perform multi-stage upsampling on the feature map extracted by the encoder. Each stage contains multiple task branches, and each branch corresponds one-to-one with each task in the multi-task. Each branch is formed by sequentially connecting the multi-scale feature fusion Up-Fuse block, the task internal dependency modeling Intrinsic-Mamba block, and the task external dependency modeling Freq-Harmony Mamba block. The Up-Fuse block is used to fuse the feature map output from the previous stage of the current task branch with the feature map of the corresponding encoder in the current stage; the Intrinsic-Mamba block is used to perform long-distance dependency modeling and feature enhancement within the current task using Mamba technology based on the fused feature map; the Freq-Harmony Mamba block is used to perform cross-task dependency modeling of the current task using Mamba technology based on the output of the Intrinsic-Mamba block, and obtain the output result of the current task branch; The multi-task semantic segmentation head is used to perform multi-task prediction based on the output results of each branch in the last stage of the decoder, and obtain the multi-task prediction result. The post-processing module is used to post-process the multi-task prediction results to generate vectorized plot-level crop identification results. The Intrinsic-Mamba block includes: a channel extension layer, an SS2D sub-branch, a linear sub-branch, and a task-internal fusion layer; The channel expansion layer is used to expand the channels through a linear fully connected layer using the fused feature map as the input feature map to obtain an expanded feature map. The SS2D sub-branch and the linear sub-branch are set in parallel. The SS2D sub-branch is used to perform four spatial direction unfolding on the extended feature map, corresponding to four unfolding sequences. The state space model SSM is performed on each sequence to obtain the SSM output feature map. The linear sub-branch is used to perform convolution and SiLU activation on the extended feature map in sequence to obtain the linear branch feature map. The task-internal fusion layer is used to perform feature fusion on the SSM output feature map and the linear branch feature map, and then perform a skip connection with the input feature map to obtain the output of the Intrinsic-Mamba block. The Freq-Harmony Mamba block includes: shared task sub-branches and specific task sub-branches; The shared task sub-branch uses a multilayer perceptron (MLP) network to perform cross-task information fusion on the outputs of the Intrinsic-Mamba blocks of other task branches to obtain fused features. Other task branches refer to task branches other than the current task. The specific task sub-branch is used based on the fusion features. SSM parameters are generated, and cross-sequence cross-scanning is performed on the fusion features of other task branches and the features of the current task branch based on Mamba technology to obtain cross-task dependency features that enhance the expression of the current task branch.

2. The system according to claim 1, characterized in that, The shared task sub-branch further includes: a shared task fusion layer, a first frequency enhancement module LFEB block, and a hybrid gated HGU unit; The shared task fusion layer is used to perform feature fusion on the output of the Intrinsic-Mamba blocks of other task branches to obtain preliminary inter-task fusion features. ; LFEB block, used for Perform a Fourier transform to obtain the frequency domain features. and will Multiply by the weight matrix W, and then perform an inverse Fourier transform to obtain the frequency-enhanced features. ; HGU unit, used to perform operations using a two-layer fully connected multilayer perceptron (MLP) network. Perform an adaptive transformation to obtain the adaptive transformation result. At the same time, Perform channel attention processing to obtain the channel attention processing result. Then After sigmoid activation and Summation is performed to obtain the gating weights, and these gating weights are then used to adjust... and The proportion of these proportions is used to obtain the final fusion characteristics. .

3. The system according to claim 1, characterized in that, The specific task sub-branch includes: the TS-SS2D component; The TS-SS2D component is used to determine the fusion features. Generate SSM parameters and, based on Mamba technology, use the output of the Intrinsic-Mamba block of the current task branch as the query input. Based on the SSM parameters, the State-Space Model (SSM) processing is performed to obtain the processing result. The processing results A skip join is made with the output of the Intrinsic-Mamba block of the current task branch to obtain cross-task dependency features.

4. The system according to claim 3, characterized in that, The multi-task semantic segmentation head includes a plot branch, a boundary branch, and a crop type branch, which correspond one-to-one with the plot area extraction task, the plot boundary extraction task, and the crop type prediction task. The boundary branch includes a second frequency domain enhancement module (PDFR) block, which receives the output of the Freq-Harmony Mamba block, the last stage of the parcel boundary extraction task branch in the decoder. After performing a Fourier transform, the parcel boundary features are enhanced in the high-frequency information part, and the enhanced boundary features are obtained by inverse Fourier transform. .

5. The system according to claim 1, characterized in that, The post-processing module is further used for: The parcel boundary extraction results from the multi-task prediction are subjected to semantically constrained morphological dilation to obtain semantically enhanced boundaries. ; Lee's skeleton algorithm was used to perform... Boundary skeleton extraction is performed to obtain a boundary skeleton graph, and a semantic topology graph is constructed based on the boundary skeleton graph. Each topological segment in the topological network records the crop type attributes on both sides; Using the semantic topology graph The crop type is used as semantic information to guide the extension direction of the hanging line. The hanging line segment optimization process is performed on the boundary skeleton diagram to obtain the optimized skeleton diagram. Semantic vectorization is performed on the optimized skeleton diagram to obtain vector crop plots with crop type attributes.

6. The system according to claim 5, characterized in that, The parcel boundary extraction results from the multi-task prediction are subjected to semantically constrained morphological dilation to obtain semantically enhanced boundaries. ,include: Morphological dilation processing is performed on the plot boundary extraction results in the multi-task prediction results to obtain the boundary dilation result Boundary_3; The crop type prediction results in the multi-task prediction results are used to detect crop type changes using an edge detection algorithm, determine the pixel locations where crop types have changed, and generate crop plot edges (Crop_boundary). Morphological dilation is performed on the crop plot edge Crop_boundary to obtain the dilated crop boundary Crop_boundary_3; The semantically enhanced boundary is obtained by taking the union of the boundary dilation result Crop_boundary and the dilated crop boundary Crop_boundary_3. .

7. The system according to claim 5, characterized in that, Using the semantic topology graph The crop type is used as semantic information to guide the extension direction of the suspending line. The suspending line segment optimization process is performed on the boundary skeleton diagram to obtain the optimized skeleton diagram, including: The semantically enhanced boundary The crop type prediction result and the plot area extraction result in the multi-task prediction results are superimposed to generate an enhanced boundary mask Crop_mask that integrates geometry and semantics; Based on the semantic topology graph The suspension line was identified. Traverse each dangling line, using the tangent direction of the dangling line as the extension direction, and extend it from the dangling vertex in the extension direction. For each pixel extended, perform geometric collision checks and semantic constraint checks. The geometric collision check includes: checking whether the currently extending pixel collides with an existing skeleton line in the boundary skeleton graph; if so, closing the dangling line; if not, performing a semantic constraint check. The semantic constraint check includes: based on the enhanced boundary mask Crop_mask, checking whether the line composed of the extending pixels cuts off a complete crop patch; if so, stopping the extension and removing the dangling line from the boundary skeleton graph and the semantic topology graph. If the line segment is removed, and the semantic constraint check passes, the extension process continues. For each pixel extended, geometric collision and semantic constraint checks are performed until successful closure or the maximum extension length is reached. The newly extended line segment is then drawn in the boundary skeleton graph, and the semantic topology graph is updated. .

8. A multi-task method for extracting crop types at the plot level from UAV imagery, characterized in that, The method is executed by a plot-level crop type multi-task extraction system for UAV imagery according to any one of claims 1 to 7, including: Acquire historical UAV imagery data of the study area and create a sample dataset; The sample dataset is used to train a plot-level crop type multi-task extraction system for UAV imagery as described in any one of claims 1 to 7; The UAV image to be predicted is input into the plot-level crop type multi-task extraction system of the UAV image that has been trained, and the output results are obtained. The output results are then used to create plot-level crop type maps.

Images