Crop weed segmentation method and system based on semantic detail decoupling

Abstract

The application provides a crop weed segmentation method and system based on semantic detail decoupling, wherein the method comprises: acquiring a farmland image, performing label mapping and preprocessing on the annotation data corresponding to the farmland image to form input data under a unified category space; extracting multi-level features by using a backbone network and obtaining pyramid features with high-level semantic information and high-resolution detail information by top-down fusion; performing semantic-detail decoupling enhancement and adaptive fusion at the high-resolution feature layer to obtain enhanced features; outputting a crop weed segmentation result based on the enhanced features, and automatically generating boundary supervision information based on semantic annotation in the training stage to assist in supervising the boundary prediction branch; in the training stage, the segmentation model is trained by using foreground perception sampling and a multi-loss joint optimization mechanism; and in the inference stage, the segmentation result is output; and the application can improve the fine weed segmentation effect in the farmland scene of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for crop and weed segmentation based on semantic detail decoupling. Background Technology

[0002] With the development of precision agriculture, crop and weed identification based on drone aerial images has become an important technical means for variable-rate pesticide application and refined management in the field. Existing technologies typically use semantic segmentation networks to perform pixel-level classification of farmland images to distinguish between crops, weeds, and background.

[0003] However, in real-world farmland scenarios, crops and weeds are often quite similar in color, texture, and morphology, and are easily affected by factors such as seedling growth stage, shading and adhesion, and changes in light intensity. This leads to class confusion and missed weed detection in existing methods. Furthermore, existing segmentation networks do not adequately utilize the combined high-level semantic information and low-level detail information; small targets such as weeds, leaf margins, and boundaries of adhered regions are easily over-smoothed, affecting segmentation accuracy. In addition, inconsistent label definitions and encoding methods across different datasets limit the ability of existing methods to uniformly utilize multi-source data, which also affects model training performance and cross-scenario adaptability.

[0004] Therefore, there is an urgent need for a crop and weed segmentation method and system that can take into account overall semantic expression, detailed structure restoration, boundary clarity, and adaptability to multiple datasets, so as to improve the fine segmentation effect of weeds in UAV farmland scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a crop and weed segmentation method and system based on semantic detail decoupling, which aims to solve the technical problems of poor small target recognition, unclear boundaries, and insufficient cross-dataset adaptability in crop and weed segmentation in UAV farmland scenarios.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a crop weed segmentation method based on semantic detail decoupling, the steps of which include:

[0007] S1. Obtain farmland images, and perform label mapping and preprocessing on the labeled data corresponding to the farmland images to form input data under a unified category space; S2. Input the input data into the multi-level feature extraction backbone network to extract multi-level features; S3. Perform top-down fusion on the multi-level features to obtain pyramid features containing high-level semantic information and high-resolution detail information; S4. Perform semantic-detail decoupling enhancement processing on the high-resolution feature layer to obtain semantic components and detail components respectively, and adaptively fuse the semantic components and detail components to obtain enhanced features; S5. Output crop and weed segmentation results based on the enhanced features, and automatically generate boundary supervision information based on semantic annotation during the training phase to assist in supervising the boundary prediction branch. S6. During the training phase, a foreground-aware sampling and multi-loss joint optimization mechanism are used to train the segmentation model; S7. Output the segmentation results during the inference phase.

[0008] As a further improvement to the above technical solution, the unified category space includes at least background, crop, and weed categories; the label mapping includes converting the annotations of different encoding methods in different data sources into category labels under the unified category space.

[0009] As a further improvement to the above technical solution, the multi-level feature extraction backbone network is a convolutional neural network backbone or a visual Transformer backbone capable of outputting multi-level features; preferably, the multi-level feature extraction backbone network is a ConvNeXt backbone network.

[0010] As a further improvement to the above technical solution, the top-down fusion adopts a feature pyramid fusion method, which upsamples the features of the previous level and fuses them with the features of the current level to form multi-scale pyramid features.

[0011] As a further improvement to the above technical solution, the feature pyramid fusion satisfies the following relationship: ; in, This indicates the output of the backbone network. Level features, Indicates the fused first Features of a pyramid This represents the channel mapping operation for the current layer features. Indicates an upsampling operation. and These represent the learnable fusion weights corresponding to the features of the current layer and the features of the previous layer pyramid, respectively. This indicates a smoothing term to prevent the denominator from being zero.

[0012] As a further improvement to the above technical solution, the semantic-detail decoupling enhancement process includes: performing semantic branching processing on the input high-resolution features to obtain semantic components; performing detail branching processing on the difference between the high-resolution features and their local smooth features to obtain detail components; and adaptively fusing the semantic components and detail components based on a gating mechanism to obtain the enhanced features.

[0013] As a further improvement to the above technical solution, the semantic-detail decoupling enhancement process satisfies the following relationship: ; ; in, This indicates that the input is a high-resolution feature. This represents semantic guidance information derived from the features of the previous level of the pyramid. This indicates the average pooling operation. This represents the high-frequency detail response in the input features. This represents a semantic branch mapping function. Represents semantic components, This represents the detail branch mapping function. Indicates detail components, This represents the gated generation function. This represents the Sigmoid activation function. Indicates the gating weight, This indicates the characteristics of the fused output.

[0014] As a further improvement to the above technical solution, the boundary supervision information is automatically generated in the following manner: ; ; in, This represents a semantic annotation mask. Represents the characteristic function, Indicates the foreground union region. This indicates an expansion operation. This indicates a corrosion operation. This represents the automatically generated boundary truth value.

[0015] As a further improvement to the above technical solution, the boundary prediction branch only participates in auxiliary supervision during the training phase and does not perform explicit post-processing on the segmentation results during the inference phase.

[0016] As a further improvement to the above technical solution, the foreground perception sampling includes: preferentially determining the cropping center around the foreground pixels of crops or weeds; using a higher target foreground proportion threshold in the early stage of training, and gradually reducing the target foreground proportion threshold as the number of training rounds increases; when a cropping block that meets the target foreground proportion threshold is not obtained within a limited number of attempts, it degenerates into random cropping.

[0017] As a further improvement to the above technical solution, the multi-loss joint optimization mechanism includes main segmentation loss, boundary auxiliary loss, and consistency constraint loss, and satisfies the following relationship:

[0018] ; in, This represents the overall optimization objective. Indicates the primary segmentation loss. Indicates the boundary auxiliary loss. This represents the loss due to consistency constraints. , and This represents the weighting coefficient of each loss term.

[0019] As a further improvement to the above technical solution, the main segmentation loss includes at least two of the following: region overlap loss, hard sample focusing loss, and boundary constraint loss; the boundary auxiliary loss includes boundary binary cross-entropy loss and boundary Dice loss.

[0020] As a further improvement to the above technical solution, the consistency constraint loss is obtained by comparing the difference between the predicted distributions of the original view and the enhanced view, and is calculated only at pixel locations that meet the uncertainty screening conditions.

[0021] As a further improvement to the above technical solution, the consistency weight The following segmentation relationship is satisfied: ; in, Indicates the current training round number. This indicates the number of Warm-up rounds in which the consistency regularization was enabled. This represents the consistency baseline weight after the Warm-up phase ends.

[0022] Secondly, the present invention also provides a crop weed segmentation system based on semantic detail decoupling, employing the crop weed segmentation method based on semantic detail decoupling as described in the first aspect, including: The data processing module is used to acquire farmland images and perform label mapping and preprocessing; The feature extraction module is used to extract multi-level features; The feature fusion module is used to perform top-down fusion of the multi-level features; The decoupling enhancement module is used to perform semantic-detail decoupling enhancement processing on the high-resolution feature layer; The segmentation output module is used to output the crop and weed segmentation results; The boundary supervision module is used to automatically generate boundary supervision information and provide auxiliary supervision for the boundary prediction branch during the training phase. The training optimization module is used to train the segmentation model using a foreground-aware sampling and multi-loss joint optimization mechanism.

[0023] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it is used to implement the steps of the crop weed segmentation method based on semantic detail decoupling described in the first aspect.

[0024] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the crop weed segmentation method based on semantic detail decoupling described in the first aspect.

[0025] Because the present invention adopts the above technical solutions, the beneficial effects of the present invention are as follows: This invention provides a crop weed segmentation method based on semantic detail decoupling. Firstly, it performs label mapping and preprocessing on the labeled data corresponding to farmland images, unifying data from different sources and with different encoding methods into the same category space. This reduces interference caused by inconsistent label definitions during joint training of multiple datasets, improving the model's adaptability to different farmland data and the consistency of segmentation results. Furthermore, this invention extracts features at different levels through a multi-level feature extraction backbone network and performs top-down fusion on these features. This allows high-level semantic information and high-resolution detail information to work synergistically, maintaining overall semantic discriminative ability while also preserving local texture, edge, and small-region structural information. This improves the recognition of small-target weeds, fine weeds, and partially occluded areas in UAV-based farmland scenarios.

[0026] This invention further performs semantic-detail decoupling enhancement processing on the high-resolution feature layer, separating and modeling semantic and detail components before adaptive fusion. Since the semantic component maintains the stability of category discrimination, and the detail component compensates for local high-frequency structural information, the two, after adaptive fusion, jointly form enhanced features. This improves the detail representation of leaf edges, narrow slits, sharp corners, and adherent regions while ensuring overall segmentation stability, thereby improving the segmentation accuracy of easily confused crop and weed areas. Simultaneously, this invention automatically generates boundary supervision information based on semantic annotation during the training phase and implements auxiliary supervision using a boundary prediction branch. This enhances the model's attention to contour regions and category boundaries without requiring additional manual boundary annotation, thus improving the boundary clarity and contour integrity of the segmentation results and mitigating the problem of blurred boundaries in areas where crops and weeds adhere.

[0027] Furthermore, this invention employs foreground-aware sampling and a multi-loss joint optimization mechanism to train the segmentation model. Foreground-aware sampling increases the effective proportion of crop and weed foreground regions in the training input, reducing the impact of large background areas on the training process, thus improving the model's learning efficiency for foreground targets, especially small weed areas. The multi-loss joint optimization mechanism simultaneously constrains the model from aspects such as region segmentation results, boundary information, and prediction stability, thus better balancing overall region segmentation performance, local boundary quality, and model training stability. Moreover, this invention directly outputs the segmentation results during the inference stage, eliminating the need for additional explicit post-processing steps. Therefore, while improving segmentation accuracy, it maintains a simple inference process, facilitating practical deployment in drone-based farmland scenarios. In summary, this invention effectively balances overall semantic representation, detailed structure recovery, boundary clarity, and adaptability to multiple datasets, thereby improving the fine segmentation effect of crops and weeds in drone-based farmland scenarios. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a crop weed segmentation method based on semantic detail decoupling disclosed in this invention. Figure 2 This is a schematic diagram of the network structure and prediction supervision relationship of a crop weed segmentation system based on semantic detail decoupling disclosed in this invention; Figure 3 This is a schematic diagram of the training and application process disclosed in this invention, which employs foreground-aware sampling, multi-task joint loss, consistency constraints, evaluation metrics, and deployment inference. Figure 4 This is a schematic diagram of the structure and processing flow of the semantic-details decoupling enhancement module disclosed in this invention; Figure 5 This is a schematic diagram of the boundary truth value generation and boundary auxiliary branch supervision process of the boundary supervision module disclosed in this invention. Figure 6 This is a schematic diagram of the multi-loss joint optimization mechanism consisting of the main segmentation loss, boundary auxiliary loss, and consistency loss disclosed in this invention.

[0030] The objectives, features, and advantages of the invention will be further explained in conjunction with the implementation methods and with reference to the accompanying drawings. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] It should be noted that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0033] Example 1

[0034] See Figures 1-6 This invention provides a crop and weed segmentation method based on semantic detail decoupling, which is applicable to pixel-level segmentation of three types of targets: background, crops and weeds in UAV farmland scenarios.

[0035] S1. Acquire farmland images, and perform label mapping and preprocessing on the labeled data corresponding to the farmland images to form input data under a unified category space: In this embodiment, the first step is to establish a standardized multi-source dataset and unify the labels. Specifically, Tobacco aerial farmland data, PhenoBench farmland semantic segmentation data, Sorghum drone farmland data, and CoFly farmland weed data are uniformly constructed into a standardized multi-source dataset without changing the original data content. This dataset is then organized into training, validation, and test sets according to a unified data directory structure. The training set contains 4347 images, the validation set contains 420 images, and the test set contains 1499 images. The test set further includes 360 images in the test_tobacco dataset, 772 images in the test_phenobench dataset, 100 images in the test_sorghum_sharp dataset, 100 images in the test_sorghum_blur dataset, 100 images in the test_sorghum_deblurred dataset, and 67 images in the test_cofly_weedonly dataset.

[0036] To address the differences in annotation rules across different data sources, a unified label mapping rule was established, standardizing all sample labels to the standard label space corresponding to background, crops, weeds, and ignored regions. The background label is denoted as 0, the crop label as 1, the weed label as 2, and the ignored label as 255. For Tobacco data, the background, crop, and weed labels are retained and uniformly mapped to the standard label space. For PhenoBench data, labels 0, 1, and 2 are mapped to background, crops, and weeds respectively, and labels 3 and 4 corresponding to partially incomplete labels are uniformly mapped to the ignored label. For Sorghum data, the image and mask are extracted based on the preset color coding relationship in its composite annotation map and mapped to the background, crop, and weed labels. For CoFly data, due to the lack of reliable crop labels, all non-zero labels are uniformly mapped to the weed category and used only as weed-only external test data. By using the above methods, data from different sources and with different annotation systems can be unified into a consistent supervision space, thereby reducing semantic ambiguity when multiple data sources are mixed and improving the consistency of model training and cross-scenario evaluation.

[0037] Furthermore, to avoid leakage of homologous samples and improve cross-scenario generalization ability, this embodiment adopts a source-aware data partitioning method. For Tobacco data, it is partitioned according to the collection campaign, with campaigns 1 to 4 used for training, campaign 5 for validation, and campaigns 6 to 8 for testing; for PhenoBench data, train is included in the training set, and val is included in the external test set; for Sorghum data, its official train / val / test partition is retained, and additional blur and deblurred test subsets are constructed; for CoFly data, the test_cofly_weedonly test set is constructed according to its official test list. This forms a unified, traceable, and source-specific multi-source farmland semantic segmentation data organization method suitable for joint training and source-specific evaluation.

[0038] After completing the multi-source data standardization and label unification, the farmland image to be processed is input into the segmentation model. The segmentation model performs pixel-level semantic discrimination and outputs three segmentation results: background, crops, and weeds. The process can be represented as follows: ; ; X represents the input farmland image. Let θ represent the segmentation model mapping function characterized by the parameter θ, and P represent the pixel-level class prediction result output by the model. This represents the predicted value of pixel (i,j) belonging to class c. Represents pixels The final category label, where c represents the category index.

[0039] Specifically, firstly, the preprocessed farmland image is input into the segmentation model; then, the segmentation model performs feature extraction, semantic modeling, and category prediction on the image to obtain the predicted value of each pixel location for each category; then, the pixel locations are further processed... The predicted values ​​for each category are compared, and the category with the largest predicted value is selected as the output category for that pixel. Finally, the category determination results of all pixels are aggregated to form the three-class semantic segmentation result of the entire image.

[0040] The previously mentioned "ignore label 255" in the unified label space is mainly used to mask incompletely labeled regions during the training or evaluation phases. These regions are not included in the final output category of the model for the three-class classification. In other words, the established unified label rules provide a unified semantic benchmark for subsequent pixel-level classification, and this step completes pixel-by-pixel identification of background, crop, and weed regions under this unified semantic benchmark. This "unify labels first, then perform segmentation" approach allows multi-source farmland data to participate in model learning and testing within the same category space, thereby improving the consistency and comparability of crop and weed segmentation results.

[0041] Preferably, during the training phase, the input image can be subjected to random horizontal flipping, random vertical flipping, random rotation, affine transformation, blurring, noise reduction, brightness and contrast changes, hue and saturation changes, and CLAHE enhancement. The image is then uniformly scaled to 480×480 and normalized before being input into the network. During the validation phase, training enhancement and foreground course sampling are disabled, retaining only the uniform preprocessing to ensure the stability of the evaluation results. This preprocessing improves the diversity of training samples and provides a uniform scale and statistical distribution for subsequent network inputs.

[0042] S2. Input the input data into the multi-level feature extraction backbone network to extract multi-level features: In this embodiment, the multi-level feature extraction backbone network adopts the ConvNeXt backbone network. After the preprocessed farmland image is input into the ConvNeXt backbone network, it outputs four-level hierarchical features. , , and .in, These are high-resolution low-level features, mainly containing local details such as edges and textures; Features of the intermediate layer; and These are high-level semantic features, primarily reflecting the category discrimination information among crops, weeds, and the background. This hierarchical feature extraction method can simultaneously preserve high-resolution detail information and high-level semantic information, providing an input foundation for subsequent pyramid fusion and high-resolution feature enhancement. This helps to improve the problems of easily missed detection of small-target weeds and easy confusion between crops and weeds in UAV-based farmland scenarios.

[0043] S3. Perform top-down fusion of the multi-level features to obtain pyramid features containing high-level semantic information and high-resolution detail information: The multi-level features extracted in step S2 are input into the FPN module for top-down fusion. Specifically, the highest-level features are fused first. Perform channel mapping to obtain the initial pyramid features. Then, the features from the previous pyramid are upsampled to the current scale and fused with the channel mapping results of the current layer features to generate... , and Preferably, during the fusion process, learnable fusion weights are assigned to the features of the current layer and the features of the previous layer pyramid, and weighted fusion is performed after normalization to adaptively balance high-level semantic information and high-resolution detail information.

[0044] By introducing learnable fusion weights, the contribution ratio of features at different scales can be adaptively adjusted according to the training process, which is beneficial to improving the segmentation ability of targets at different scales in complex farmland scenarios, especially fine weeds and small areas.

[0045] S4. Perform semantic-detail decoupling enhancement processing on the high-resolution feature layer to obtain semantic components and detail components separately, and then adaptively fuse the semantic components and detail components to obtain enhanced features: In high-resolution feature layer and / or A DSDF (Decoupled Semantic-Detail Fusion) enhancement module is implemented. This DSDF module includes a semantic branch, a detail branch, and a gated fusion branch. Considering that crops and weeds in drone-based farmland scenarios are similar in color, texture, and morphology, and that weed targets are typically small with fragmented edges, relying solely on conventional high-resolution features for segmentation can easily lead to insufficient overall category discrimination or inadequate local detail recovery. Therefore, this embodiment separates and models the semantic and detail information from the high-resolution features, and then performs adaptive fusion through a gating mechanism. This compensates for local high-frequency structural information while maintaining category discrimination stability, thereby providing enhanced features for subsequent segmentation heads that balance overall semantic expression and local detail preservation.

[0046] Specifically, for the high-resolution features of the input Semantic-detail decoupling enhancement is performed. First, semantic guidance information is extracted from the features of the previous pyramid level and upsampled to the current scale to obtain the semantic guidance quantity. Then, for the input high-resolution features Local smoothing is performed, and the difference between the original feature and the locally smoothed feature is calculated to obtain the high-frequency detail response. Then input high-resolution features With semantic guidance They are fed into the semantic branch together to obtain semantic components. and respond to high-frequency details Feed into the detail branch to obtain the detail components. Finally, the semantic components , details and semantic guidance volume Input gating generates branches, and the gating weights are obtained. The detailed components are adaptively compensated based on the gating weights to form fused output features. This processing method allows high-level semantic information to constrain local detail compensation in high-resolution features, thus helping to suppress noise textures irrelevant to classification and enhance detail responses related to crop, weed boundaries, and small target recognition.

[0047] S5. Based on the enhanced features obtained in step S4, output the crop and weed segmentation results, and automatically generate boundary supervision information based on semantic annotation during the training phase to assist in supervising the boundary prediction branch: The boundary supervision information is the foreground boundary region annotation information automatically extracted by the semantic annotation mask, and serves as the supervision ground truth for the boundary auxiliary branch. Specifically, the enhanced features obtained in step S4 are input into the main segmentation head, which outputs three categories of semantic prediction results: background, crops, and weeds. Simultaneously, a boundary auxiliary branch is set on the enhanced features to output single-channel boundary prediction results.

[0048] The process of generating the boundary supervision information is as follows: First, read the semantic annotation mask corresponding to the input farmland image, and extract the crop and weed pixels through the indicator function to form the foreground union region; then, perform dilation and erosion on the foreground union region to construct a contour band with a certain width; finally, subtract the dilation result from the erosion result to obtain the boundary ground truth, and input it into the boundary auxiliary branch for training.

[0049] In drone-based farmland scenarios, crops and weeds often overlap, intersect, and have unclear boundaries. If the main segmentation result is supervised solely based on region semantic annotation, the model typically focuses more on the consistency within the region, while providing insufficient constraints on the contours at the class boundaries. This can easily lead to blurred boundaries or overlap in local areas. Therefore, this embodiment directly utilizes existing semantic annotation masks to automatically extract foreground boundary regions, generating the boundary supervision information, which is then used to train the boundary auxiliary branch. On one hand, this eliminates the need for additional manual boundary annotation, reducing annotation costs; on the other hand, it allows the model to explicitly focus on the foreground contour position during training, thereby improving the boundary clarity and contour integrity of the crop-weed interface.

[0050] S6. During the training phase, a foreground-aware sampling and multi-loss joint optimization mechanism are used to train the segmentation model: During the training phase, a foreground-aware curriculum sampling strategy (FAC) is first used to generate training image patches. Specifically, a cropping center is selected around the foreground pixels of crops or weeds with a preset probability to increase the effective proportion of the foreground region in the training samples. A higher foreground proportion threshold is used in the early stages of training, and this threshold is gradually reduced as the number of training rounds increases, so that the sampling distribution gradually transitions from emphasizing foreground information to a distribution closer to the real data. When an image patch that meets the foreground proportion condition cannot be obtained within a limited number of attempts, it degenerates into random cropping to balance training efficiency and sample diversity.

[0051] In terms of loss design, the overall optimization objective is constructed using main segmentation loss, boundary auxiliary loss, and consistency constraint loss. Preferably, the main segmentation loss consists of DiceLoss, FocalLoss, and BoundaryLoss, used to constrain the degree of region overlap, hard-to-classify pixels, and boundary geometric consistency, respectively; the boundary auxiliary loss consists of binary cross-entropy loss and Dice loss, used to enhance the supervision ability of the boundary branch on thin contours and small-area edge regions; the consistency constraint loss is activated after reaching a preset number of warm-up rounds, constructing an original view and a strongly enhanced view of the same image, and comparing the difference in the predicted distribution of the two only at high-confidence pixel locations, in order to improve the prediction stability of the model under conditions of brightness changes, noise perturbation, and flipping.

[0052] S7. Output the segmentation results during the inference phase: During the inference phase, only the backbone network, FPN fusion, DSDF enhancement module, and main segmentation head are retained. The system directly outputs a three-class semantic segmentation result of the same size as the original image from the input farmland image. The boundary auxiliary branch does not participate in explicit post-processing, and the output result can be directly used for weed monitoring, precision pesticide application, and intelligent farmland operations.

[0053] Since the inference stage does not add any additional post-processing steps, it can maintain a relatively simple deployment path while improving the quality of the segmentation results boundary, making it easy to implement in drone farmland application scenarios.

[0054] In a preferred embodiment, the feature pyramid fusion satisfies the following relationship: ; ; in, This indicates the output of the backbone network. Level features; Indicates the fused first Features of a pyramid; This represents the channel mapping operation for the current layer features, used to map the current layer features to a predetermined channel dimension; This indicates an upsampling operation, used to restore the features of the previous pyramid level to the scale of the current layer; and These represent the learnable fusion weights corresponding to the features of the current layer and the features of the previous layer pyramid, respectively. This indicates a smoothing term to prevent the denominator from being zero; This indicates the feature level index.

[0055] Based on the above relationships, the highest-level features are considered first. The initial pyramid features are obtained through channel mapping. Then for the first Current layer features Features of the previous pyramid upsampling results After assigning normalized learnable weights to each, they are weighted and fused to generate the following results sequentially. , and By setting learnable fusion weights, the contribution ratio of current layer detail information and upper layer semantic information during fusion can be adaptively adjusted according to the training process. This maintains the overall semantic expressive power while taking into account local texture, edge, and small region structural information, providing a relatively stable multi-scale feature foundation for subsequent semantic-detail decoupling enhancement and segmentation output.

[0056] Furthermore, in this embodiment, the top-down fusion is not limited to simple addition, but achieves inter-layer feature transfer through a combination of upsampling and weighted fusion. This allows for the use of higher-level features to supplement the semantic discriminative information of lower-level features, while not completely weakening the original spatial resolution advantage of lower-level features. Therefore, it is more suitable for processing crop and weed targets with large scale differences in farmland images. Especially for fine weeds, leaf margins, and areas where crops and weeds are intertwined, this fusion method can provide input features with both semantic stability and detail preservation capabilities for subsequent high-resolution enhancement branches.

[0057] In some preferred embodiments, the feature pyramid fusion can also be applied to other convolutional neural network backbones or visual Transformer backbones capable of outputting multi-level features, as long as the multi-scale pyramid features are still constructed by progressively passing high-level semantics to low-level layers and fusing them between adjacent layers. Using this approach, different computing power conditions and application scenarios can be adapted while maintaining the overall processing logic of this invention.

[0058] In a preferred embodiment, the semantic-detail decoupling enhancement process satisfies the following relationship: ; ; in, Indicates high-resolution input features; This represents semantic guidance information derived from the features of the previous level of the pyramid; Indicates an upsampling operation; Indicates the characteristics of the next higher level of the pyramid; This indicates an average pooling operation; This represents the high-frequency detail response in the input features; This represents the concatenation result of input high-resolution features and semantic guidance information; This represents a semantic branch mapping function; Represents semantic components; This represents the function that maps detailed branches; Indicates detail components; This represents the concatenation result of semantic components, detail components, and semantic guidance information; Indicates the gated generation function; This represents the Sigmoid activation function; Indicates the gating weight; Indicates the fused output features; This indicates the current pyramid feature level index.

[0059] Where, formula The representation first performs local smoothing on the input high-resolution features, then subtracts the smoothed result from the original features to extract high-frequency detail responses. These high-frequency detail responses primarily reflect edge and texture variations and narrow structural information, making them suitable for describing local structures such as small weed targets, leaf margins, and crop-weed boundaries. The representation inputs the current high-resolution features along with semantic guidance information from the upper layer into the semantic branch to obtain semantic components with strong category discrimination capabilities. The representation maps the high-frequency detail response to the detail branches to obtain the detail components. (Equation) The representation generates gating weights jointly based on semantic components, detail components, and semantic guidance information. (Equation) The representation incorporates detail components in a gated manner based on semantic components to obtain the final enhanced features. This fusion is not a simple superposition of semantic and detail components, but rather adaptively adjusts the detail compensation intensity according to the semantic discrimination requirements of the current position, thus improving effective detail representation while reducing noise interference.

[0060] Through the aforementioned semantic-detail decoupling enhancement process, on the one hand, semantic branches can maintain the stability of class discrimination of high-resolution features, avoiding class confusion caused by excessive detail information; on the other hand, detail branches and gating fusion mechanisms can compensate for local high-frequency structural information, enhancing the ability to express details of fine weeds, small targets, leaf edges and sharp corners, and adhered areas. Therefore, this processing method is specifically targeted at solving the problems of poor small target recognition and unclear boundaries that this invention aims to address, and can provide the subsequent main segmentation head with input features more suitable for fine segmentation of farmland scenes.

[0061] In some preferred embodiments, the semantic-detail decoupling enhancement process can also be arranged on multiple high-resolution feature layers, or the specific implementation of the semantic branch, detail branch and gating generation branch can be adaptively adjusted according to actual needs, as long as the processing method of semantic component extraction, detail component extraction and adaptive fusion based on gating mechanism for high-resolution features is still satisfied.

[0062] In a preferred embodiment, the boundary supervision information is automatically generated as follows: ; ; in, Indicates a semantic annotation mask; Indicative functions are used to filter crop and weed pixels from a semantically labeled mask; This indicates the foreground union region, which is the area composed of both crops and weeds; This indicates an expansion operation, used to expand the boundary of the foreground region outwards; This indicates an erosion operation, used to shrink the boundary of the foreground region inwards; This represents the automatically generated boundary truth value. In the above relationship, the equation... This indicates that the foreground union region is first extracted based on whether the pixel values ​​in the semantically labeled mask belong to the crop or weed category; Then, dilation and erosion operations are performed on the foreground union region, and the difference between the two results is calculated to obtain the boundary region located between the outer and inner edges of the foreground. This boundary region can stably represent the pixel distribution near the target contour and is suitable as the supervised ground truth for the boundary auxiliary branch.

[0063] Since the boundary ground truth is directly derived from the semantic annotation mask, there is no need to create separate boundary annotations. At the same time, since the boundary ground truth reflects the area near the foreground contour, the boundary auxiliary branch can more effectively learn the boundary information of the boundary between crops and weeds, the leaf edge area, and the small target contour area, thereby effectively supplementing the main segmentation branch.

[0064] Furthermore, the boundary supervision information is only used to assist in optimizing the boundary prediction branch during the training phase, and no explicit post-processing is performed on the main segmentation results during the inference phase. This approach integrates boundary constraints into the training process without increasing the processing complexity during deployment. Therefore, it not only helps improve the problems of blurred boundaries and unclear segmentation of contiguous regions, but also maintains a relatively simple inference process, making it suitable for practical applications in UAV-based farmland scenarios.

[0065] In a preferred embodiment, the foreground-aware sampling uses a curriculum-based sampling method to generate training image patches. Considering that the background area typically occupies a large proportion in drone-based farmland scenarios, if purely random cropping is used directly during training, the training samples are easily dominated by a large area of ​​background for a long time, making it difficult for the model to learn the foreground features of crops and weeds in a timely manner, thus affecting the recognition performance of small targets such as weeds and minority regions. Therefore, this embodiment prioritizes determining the cropping center around the foreground pixels of crops or weeds during the training phase, and increases the proportion of the foreground region in the training image patches in the early stages of training, allowing the model to access effective foreground information earlier. As the number of training rounds increases, the foreground proportion requirement is gradually reduced, allowing the sampling distribution to smoothly transition from emphasizing foreground information to a distribution close to the real data distribution, thereby balancing training efficiency, sample diversity, and model generalization ability.

[0066] Specifically, in each training round, candidate cropping blocks are first generated from the input image. Crop centers are then preferentially selected around foreground pixels of crops or weeds with a preset probability to obtain candidate cropping blocks containing a higher foreground proportion. Next, a target foreground threshold is determined based on the current training round number, and the foreground proportion in the candidate cropping blocks is calculated. If a candidate cropping block meets the target foreground proportion condition, it is used as the final training input. If no candidate cropping block meets the condition within a limited number of attempts, random cropping is degraded to avoid affecting training efficiency due to over-selection. This approach reduces the interference of background regions on the optimization direction in the early stages of training, improves the model's learning efficiency for foreground regions, especially weed areas, and maintains basic sample distribution diversity through a failure fallback mechanism.

[0067] Preferably, the process is characterized by target foreground threshold scheduling and adaptive sampling probability. Correspondingly, the curricular foreground perception sampling process can be summarized as: according to the first... Round training to determine the target foreground threshold ; For candidate clipping blocks Calculate its foreground proportion ;in accordance with and The difference between them adaptively adjusts the foreground sampling probability. ; and in the maximum number of attempts Search for candidate cropping blocks that meet the conditions; if a candidate cropping block meets the conditions, it is used as the final training block. Otherwise, it degenerates into a random clipping block. .

[0068] Specifically, the formula for calculating the target foreground threshold is as follows: ; The formula for calculating the foreground proportion in the candidate cropping block is as follows: ; The formula for calculating the foreground sampling probability is as follows: ; The final cropped block used for training can be represented by the following formula: ; in, Indicates the first The target foreground threshold corresponding to each round of training; Indicates the current training round number; Indicates the dividing point between the number of Warm-up rounds; Indicates the number of subsequent threshold switching rounds; Indicates a candidate clipping block. This indicates the proportion of foreground elements in the candidate cropping block. Represents the characteristic function, The true label category of pixel position (i,j) is represented by 255, where 255 indicates that the label is ignored, and ε indicates a smoothing term to prevent the denominator from being zero. This indicates the foreground sampling probability for the candidate cropping block in the current round; Indicates the basic foreground sampling probability; Indicates the maximum sampling probability; Indicates the probability adjustment coefficient; Indicates the maximum number of attempts; This represents the cropped block ultimately used for training; Indicates at most In the next candidate cropping attempt, the candidate cropping block whose foreground proportion first meets the threshold condition is selected. This represents the sequence number of the first attempt within the maximum number of attempts T to satisfy the condition that the proportion of the candidate cropped block foreground is not less than the target foreground threshold; This indicates a randomly cropped block.

[0069] In this embodiment, the target foreground threshold A scheduling method that varies with the number of training rounds is adopted: In the early stages of training, a higher threshold is used to increase the effective proportion of foreground regions in the training input, allowing the model to prioritize learning crop and weed targets; in the later stages of training, the threshold is gradually decreased, causing the training samples to gradually transition from a distribution "biased towards foreground information" to a distribution closer to that of actual farmland images. Correspondingly, the foreground sampling probability... The sampling strategy is adjusted based on the difference between the current foreground proportion of the candidate cropped block and the target foreground threshold: when the foreground proportion in the candidate cropped block is low, its foreground-guided sampling probability is increased; when the foreground proportion in the candidate cropped block is close to or reaches the target threshold, the additional guidance intensity is maintained or reduced. This approach allows the sampling strategy to emphasize foreground learning in the early stages of training, while retaining more background and complex scene information in the later stages, thus achieving a better balance between foreground recall and overall generalization ability.

[0070] Furthermore, in this embodiment, when the maximum number of attempts is reached... If no candidate cropping block that meets the target foreground proportion threshold is found, a random cropping block is used. Instead of continuously searching for candidate blocks that meet the conditions, the samples are used as training input. This failure backoff mechanism avoids significantly increasing training overhead due to excessive pursuit of high foreground proportions, and also prevents training samples from being concentrated in a small number of high-foreground regions for a long time, thus maintaining the basic diversity of data distribution. For the UAV farmland scenario targeted by this invention, this mechanism can improve the learning efficiency of small targets such as weeds while reducing the risk of overfitting caused by excessive sampling bias.

[0071] Therefore, by using the processing chain of "prioritizing the determination of the cropping center around the foreground pixels of crops or weeds - using a higher target foreground ratio threshold in the early training stage and gradually reducing it - degenerating to random cropping when failing within a limited number of attempts", this embodiment can effectively solve the problem that random cropping is easily dominated by a large area of ​​background. It provides more effective foreground sample input for subsequent main segmentation, boundary-assisted supervision and consistency constraint training, thereby helping to improve the problems of poor recognition of small target weeds and insufficient recall of minority classes in UAV farmland scenarios.

[0072] As a preferred embodiment, a multi-loss joint optimization mechanism is adopted, which enables different supervision signals to work synergistically during the same training process, thereby improving the model's adaptability to complex farmland scenarios.

[0073] The multi-loss joint optimization mechanism satisfies the following relationship: ; in, This represents the overall optimization objective, used to characterize the comprehensive loss that ultimately participates in backpropagation during the training phase; This represents the principal segmentation loss, used to constrain the segmentation results for the three types of regions: background, crops, and weeds. This represents the boundary-aided loss, used to enhance the model's attention to contour regions and class boundaries; This represents the consistency constraint loss, used to constrain the consistency of the predicted distribution of the same image under different view perturbations; , and These represent the weight coefficients corresponding to the main segmentation loss, boundary auxiliary loss, and consistency constraint loss, respectively. These are dynamic weights that change with the number of training epochs. By setting these parameters, region supervision, boundary supervision, and stability supervision can work together under the same optimization objective, avoiding reliance on only one type of loss and resulting in a singular training objective.

[0074] In this embodiment, the main segmentation loss includes at least two of the following: region overlap loss, hard sample focusing loss, and boundary constraint loss, preferably including Dice Loss, Focal Loss, and Boundary Loss. Specifically, the main segmentation loss is shown in the following formula: ; in, , and This represents the corresponding weighting coefficient.

[0075] This represents the region overlap loss, used to improve the overall overlap consistency between the predicted region and the true region, thereby mitigating the region bias problem caused by the small area of ​​the foreground target; the formula for calculating the region overlap loss is as follows: ; in, This represents the set of prospect categories participating in the statistics. Indicates the number of foreground categories. This represents the predicted probability that pixel position (i,j) belongs to class c. Let ε represent the true label of pixel position (i,j) belonging to class c, and let ε represent the smoothing term to prevent the denominator from being zero. Specifically, the calculation steps include: First, determining the set of foreground categories participating in the Dice statistics. And for each foreground category, obtain the predicted probability at each pixel location. With real labeling Then, the predicted probabilities and ground truth labels are multiplied and summed pixel by pixel to obtain the intersection term of the corresponding categories; the predicted probabilities and ground truth labels are summed pixel by pixel and combined with the smoothing term ε to form the denominator; finally, the Dice values ​​of all foreground categories are averaged and subtracted to obtain the region overlap loss in the main segmentation.

[0076] This represents the hard sample focusing loss, used to improve the contribution of hard-to-classify pixels and minority class pixels during training. It is suitable for improving the recognition performance of small targets such as weeds and easily confused regions. The calculation formula for the hard sample focusing loss is as follows: ; in, This represents the class weight of class c. This represents the predicted probability of the current pixel corresponding to the true class. Represents the modulation coefficient. Represents logarithmic operations; Specifically, the calculation steps include: First, determining the predicted probability of the current pixel in the true category. Then, assign appropriate category weights based on the category they belong to. Then utilize the modulation coefficient The loss contribution of easily classified samples is suppressed, while the loss contribution of difficult-to-classify samples is amplified. Finally, the weighted results of all pixels are accumulated to obtain the focusing loss value of difficult samples.

[0077] The boundary constraint loss is used to compare the geometric consistency between the predicted boundary and the true boundary, thereby improving contour sharpness and boundary integrity. Specifically, the formula for calculating the boundary constraint loss is as follows: ; in, Indicates pixel position The predicted prospect probability is obtained by summing the predicted probability of crop type and the predicted probability of weed type; This represents the ground truth boundary map extracted from the foreground union region in the ground truth annotations; This represents the differentiable boundary approximation constructed from the predicted foreground probability map; This indicates an average pooling operation; This indicates a smoothing term to prevent the denominator from being zero; This represents the boundary constraint loss.

[0078] Specifically, the calculation steps include: First, constructing a predicted foreground probability map based on the predicted probabilities of crop and weed classes; then, extracting the foreground union region from the ground truth semantic annotations, and performing dilation and erosion operations on the foreground union region to obtain the ground truth boundary map; next, constructing a differentiable boundary approximation using the difference between the predicted foreground probability map and its local smoothing result; finally, calculating the overlap between the predicted boundary and the ground truth boundary using the Dice form to obtain the boundary constraint loss, and using the obtained boundary constraint loss as part of the main segmentation loss. The project participated in joint optimization.

[0079] Furthermore, the boundary auxiliary loss includes the boundary binary cross-entropy loss and the boundary Dice loss. Specifically, the calculation formula for the boundary auxiliary loss is shown below: ; in, Indicates the boundary auxiliary loss. This represents the boundary binary cross-entropy loss. This represents the boundary Dice loss. It is used to enhance the supervision of boundary branches for thin contours and small edge regions.

[0080] Specifically, the calculation steps include: First, calculating the binary cross-entropy loss based on the boundary branch output and the boundary truth value. Then, calculate the Dice loss for the boundary region. This is used to characterize the regional overlap between the boundary prediction and the ground truth boundary value; finally, the two parts of the loss are summed to form the boundary auxiliary loss. Boundary binary cross-entropy loss is used to constrain the pixel-by-pixel difference between the boundary prediction branch output and the automatically generated boundary ground truth, while boundary Dice loss is used to constrain the overall overlap of the boundary region. Since the boundary supervision signal comes directly from the automatically generated boundary ground truth based on semantic annotation, contour constraints can be introduced into the training process without increasing the cost of manual boundary annotation. Combining the above two types of boundary losses allows the boundary branch to simultaneously consider local pixel accuracy and overall contour consistency, thus being more conducive to improving the prediction quality of areas where crops and weeds are stuck together, leaf margins, and thin contour areas.

[0081] For the consistency constraint loss, this embodiment obtains it by comparing the difference between the predicted distributions of the original view and the enhanced view, and calculates it only at pixel locations that meet the uncertainty screening criteria. Specifically, the original view prediction result is constructed for the same input image. Compared with augmented view prediction results First, both are probability normalized, and then the pixel-level prediction uncertainty is calculated. With uncertainty threshold Generate high-confidence filter mask The difference between the two predicted distributions is calculated only at pixel locations that meet the high confidence condition, and the resulting difference is normalized and averaged to form the consistency constraint loss. .in, This indicates the predicted result of the original view; This indicates the prediction result of the augmented view; Indicates pixel position The uncertainty of prediction; Indicates the uncertainty threshold; This represents the high-confidence mask obtained through uncertainty filtering. This approach avoids including low-confidence, noisy regions in the consistency constraints early in training, thereby mitigating the interference of unstable predictions on training and improving the model's stability under conditions of brightness perturbations, noise contamination, and flipping changes.

[0082] Specifically, the formula for calculating the prediction uncertainty is as follows: ; Consistency constraint loss As shown in the following formula: ; As a preferred embodiment, the consistency weight The following segmentation relationship is satisfied: ; in, Indicates the current training round number; Indicates the number of Warm-up rounds in which the consistency regularization was enabled; This represents the consistency baseline weights after the Warm-up phase. This segmented relationship indicates that consistency constraints are not enabled in the early stages of training, when the model is still building its basic region discrimination capabilities; consistency constraints are then introduced into the overall optimization objective after the preset Warm-up phase is reached. This setting allows the model to first learn its basic semantic segmentation capabilities and then gradually enhance cross-view stability constraints, thereby reducing consistency misleading issues caused by unstable predictions in the early stages of training.

[0083] In the specific training process, the main segmentation loss is first calculated based on the input image and its annotations. and boundary auxiliary loss Then, based on the current training round number Determine whether to enable consistency constraints; if ,but At this point, the overall optimization objective consists only of the main segmentation loss and the boundary auxiliary loss; if This introduces a consistency constraint loss. and according to the benchmark weight Together with other losses, they participate in a weighted summation to form the overall optimization objective. This training strategy ensures that the model prioritizes learning region segmentation and boundary representation capabilities in the early stages of training, while further improving prediction stability under augmented views in the later stages of training, thus achieving a good balance between segmentation accuracy, boundary quality, and generalization ability.

[0084] Therefore, this embodiment combines region-level supervision, boundary-level supervision, and cross-view stability constraints through a joint optimization structure of "main segmentation loss, boundary auxiliary loss, and consistency constraint loss," as well as a training mechanism that delays the activation of consistency constraints and selects high-confidence pixels. Compared to training methods that rely solely on a single segmentation loss, this design is more suitable for the UAV farmland scenario targeted by this invention. It helps to improve the problems of insufficient identification of small target weeds, unclear boundaries between crops and weeds, and unstable prediction under enhanced perturbations. Furthermore, it improves the overall quality of the final segmentation results without increasing explicit post-processing during the inference stage.

[0085] Example 2

[0086] This invention also provides a crop and weed segmentation system based on semantic detail decoupling, used to segment crop regions, weed regions, and background regions in farmland images. The system can be deployed on servers, edge computing devices, industrial control computers, or other computing devices with processors and memory.

[0087] The system includes a data processing module, a feature extraction module, a feature fusion module, a decoupling enhancement module, a segmentation output module, a boundary supervision module, and a training optimization module.

[0088] The data processing module is used to acquire farmland images and perform label mapping and preprocessing. Specifically, it can uniformly map the original labels from different datasets to three label spaces: background, crops, and weeds, and perform size adjustment, normalization, and data augmentation on the farmland images. By unifying the label encoding methods of different data sources, the impact of differences in labeling standards on the training process can be reduced, thereby improving the model's ability to utilize multi-source data and adapt across datasets. Preprocessing the input images also helps to improve the stability of subsequent feature extraction and model training.

[0089] The feature extraction module is used to extract multi-level features. Specifically, the preprocessed farmland image is input into the backbone network, and feature maps of different levels are output. Shallower features contain more spatial location information and local texture information, while deeper features contain stronger category semantic information. By extracting multi-level features, both the overall semantics and local details in the farmland image can be preserved simultaneously, providing a foundation for subsequent fine segmentation of crops and weeds.

[0090] The feature fusion module is used to perform top-down fusion of the multi-level features. Specifically, high-level semantic information from deep features is progressively passed to the high-resolution feature layer and fused with detailed information from shallow features to obtain fused features that balance semantic expression and spatial resolution. This approach enhances the model's ability to represent small targets such as weeds, leaf margins, and adhered regions, mitigating the detail loss problem caused by relying solely on single-layer features.

[0091] The decoupling enhancement module is used to perform semantic-detail decoupling enhancement processing on the high-resolution feature layer. Specifically, semantic information and detail information in the high-resolution fused features can be modeled separately, with the semantic branch focusing on category discrimination and the detail branch focusing on edge texture and local structure recovery, and then the two are enhanced and fused. By decoupling semantic information and detail information, mutual interference between the two can be reduced. While maintaining the ability to distinguish between crop and weed categories, the ability to recover the boundary structure of small weeds, narrow areas, and target boundaries is enhanced, thereby improving the precision of the segmentation results.

[0092] The segmentation output module is used to output the crop and weed segmentation results. Specifically, the decoupled and enhanced features are input into the segmentation head, and the category prediction results corresponding to each pixel in the farmland image are output to form a segmentation map of the background, crops, and weeds. Since the features input to the segmentation head have both strong semantic expressive power and rich detailed information, it helps to improve the overall segmentation accuracy and the recognition effect of complex regions.

[0093] The boundary supervision module is used to automatically generate boundary supervision information and provide auxiliary supervision for the boundary prediction branch during the training phase. Specifically, it can automatically extract boundary information between different categories of regions based on semantic annotation results to generate boundary supervision labels; simultaneously, it sets a boundary prediction branch in the network to learn the distribution characteristics of the target boundary region, and uses the boundary supervision labels to provide auxiliary supervision for this boundary prediction branch. By introducing boundary supervision, the model can pay more attention to the boundary region between crops and weeds and the region where the target contour changes during training, thereby improving the problems of blurred boundaries and difficulty in separating adjacent targets, and improving the quality of boundary localization.

[0094] The training optimization module employs a foreground-aware sampling and multi-loss joint optimization mechanism to train the segmentation model. Specifically, during the construction of training samples, image patches containing foreground regions of crops or weeds are preferentially selected for training to avoid excessively large background areas that could weaken the model's learning of foreground targets. Simultaneously, the main segmentation task corresponding to the segmentation output module and the boundary auxiliary task corresponding to the boundary prediction branch are jointly optimized. Foreground-aware sampling increases the effective proportion of crop and weed foreground regions in the training samples, helping to enhance the model's ability to recognize minority targets, especially weed targets. Multi-loss joint optimization balances class discrimination and boundary recovery, improving the stability of model training and the completeness of segmentation results.

[0095] This system, through the coordinated efforts of unified label processing, multi-level feature extraction, top-down fusion, semantic and detail decoupling enhancement, boundary-assisted supervision, and foreground perception training optimization, can effectively improve the problems of poor small target recognition, unclear boundaries, and insufficient cross-dataset adaptability in crop and weed segmentation in UAV farmland scenarios, thereby improving the precision of segmentation results and practical application effects.

[0096] Example 3

[0097] The present invention also provides an electronic device, including a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute some or all of the steps in Embodiment 1; A processor may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0098] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0099] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces processor waiting time, and thus improves system efficiency.

[0100] Example 4

[0101] The present invention also provides a storage medium having a computer program stored thereon, wherein, when the program is executed, it controls the device where the storage medium is located to perform some or all of the steps in Embodiment 1.

[0102] The storage medium may include high-speed RAM memory, and may also include nonvolatile memory, such as at least one disk storage device. It is understood that the storage medium can be any machine-readable medium capable of storing program code, such as random access memory (RAM), magnetic disk, hard disk, solid state disk (SSD), or nonvolatile memory.

[0103] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or storage media. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. All equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A crop weed segmentation method based on semantic detail decoupling, characterized in that, The steps include: S1. Obtain farmland images, and perform label mapping and preprocessing on the labeled data corresponding to the farmland images to form input data under a unified category space; S2. Input the input data into the multi-level feature extraction backbone network to extract multi-level features; S3. Perform top-down fusion on the multi-level features to obtain pyramid features containing high-level semantic information and high-resolution detail information; S4. Perform semantic-detail decoupling enhancement processing on the high-resolution feature layer to obtain semantic components and detail components respectively, and adaptively fuse the semantic components and detail components to obtain enhanced features; S5. Output crop and weed segmentation results based on the enhanced features, and automatically generate boundary supervision information based on semantic annotation during the training phase to assist in supervising the boundary prediction branch. S6. During the training phase, a foreground-aware sampling and multi-loss joint optimization mechanism are used to train the segmentation model; S7. Output the segmentation results during the inference phase.

2. The crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The top-down fusion adopts a feature pyramid fusion method, which upsamples the features of the previous level and fuses them with the features of the current level to form multi-scale pyramid features; The feature pyramid fusion satisfies the following relationship: ； in, This indicates the output of the backbone network. Level features, Indicates the fused first Features of a pyramid This represents the channel mapping operation for the current layer features. Indicates an upsampling operation. and These represent the learnable fusion weights corresponding to the features of the current layer and the features of the previous layer pyramid, respectively. This indicates a smoothing term to prevent the denominator from being zero.

3. The crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The semantic-detail decoupling enhancement process includes: Semantic branching is performed on the high-resolution input features to obtain semantic components; The difference between the high-resolution feature and its local smoothing feature is processed by detail branching to obtain detail components; The semantic components and detail components are adaptively fused based on a gating mechanism to obtain the enhanced features.

4. The crop weed segmentation method based on semantic detail decoupling according to claim 3, characterized in that, The semantic-detail decoupling enhancement process satisfies the following relationship: ； ； in, This indicates that the input is a high-resolution feature. This represents semantic guidance information derived from the features of the previous level of the pyramid. This indicates the average pooling operation. This represents the high-frequency detail response in the input features. This represents a semantic branch mapping function. Represents semantic components, This represents the detail branch mapping function. Indicates detail components, This represents the gated generation function. This represents the Sigmoid activation function. Indicates the gating weight, This indicates the characteristics of the fused output.

5. The crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The boundary monitoring information is automatically generated in the following manner: ； ； in, This represents a semantic annotation mask. Represents the characteristic function, Indicates the foreground union region. This indicates an expansion operation. This indicates a corrosion operation. This represents the automatically generated boundary truth value.

6. The crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The boundary prediction branch only participates in auxiliary supervision during the training phase and does not perform explicit post-processing on the segmentation results during the inference phase.

7. The crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The foreground perception sampling includes: preferentially determining the cropping center around the foreground pixels of crops or weeds; using a higher target foreground proportion threshold in the early stages of training, and gradually decreasing the target foreground proportion threshold as the number of training rounds increases; when a cropping block that meets the target foreground proportion threshold is not obtained within a limited number of attempts, it degenerates into random cropping.

8. A crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The multi-loss joint optimization mechanism includes main segmentation loss, boundary auxiliary loss, and consistency constraint loss, and satisfies the following relationship: ； in, This represents the overall optimization objective. Indicates the primary segmentation loss. Indicates the boundary auxiliary loss. This represents the loss due to consistency constraints. , and The weight coefficients of each loss term are represented; the main segmentation loss includes at least two of the following: region overlap loss, hard sample focusing loss, and boundary constraint loss; the boundary auxiliary loss includes boundary binary cross-entropy loss and boundary Dice loss; the consistency constraint loss is obtained by comparing the difference between the predicted distributions of the original view and the enhanced view, and is calculated only at pixel locations that meet the uncertainty screening conditions.

9. A crop weed segmentation method based on semantic detail decoupling according to claim 1, characterized in that, The consistency weight The following segmentation relationship is satisfied: ； in, Indicates the current training round number. This indicates the number of Warm-up rounds in which the consistency regularization was enabled. This represents the consistency baseline weight after the Warm-up phase ends.

10. A crop weed segmentation system based on semantic detail decoupling, employing the crop weed segmentation method as described in any one of claims 1-9, characterized in that, include: The data processing module is used to acquire farmland images and perform label mapping and preprocessing; The feature extraction module is used to extract multi-level features; The feature fusion module is used to perform top-down fusion of the multi-level features; The decoupling enhancement module is used to perform semantic-detail decoupling enhancement processing on high-resolution feature layers. The segmentation output module is used to output the crop and weed segmentation results; The boundary supervision module is used to automatically generate boundary supervision information and provide auxiliary supervision for the boundary prediction branch during the training phase. The training optimization module is used to train the segmentation model using a foreground-aware sampling and multi-loss joint optimization mechanism.

Images