A method for detecting occluded broccoli florets in a field based on improved YOLOv11

Abstract

The application belongs to the technical field of agricultural automation and computer vision, and discloses a field occluded broccoli flower ball detection method based on an improved YOLOv11. The feature extraction capability and detection precision of the model are enhanced in the manner of improving the attention mechanism, the convolution method of the feature extraction layer and the neck network structure, the detection effect of the model in the complex field environment is effectively improved, the calculation amount of the algorithm increases less, the model is beneficial to deployment and application on the edge device, and has strong practical significance.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of agricultural automation and computer vision, specifically involving a field-based method for detecting occluded broccoli heads based on an improved YOLOv11. This method is suitable for crop target recognition in field environments and provides a visual perception foundation for automated harvesting robots. Background Technology

[0002] In recent years, the industrialization of broccoli cultivation has promoted large-scale production, but the high cost and low efficiency of manual harvesting have also become increasingly prominent. Therefore, promoting the transformation of broccoli harvesting towards intelligent and mechanized processes has become an urgent need for agricultural modernization. The core of achieving intelligent harvesting lies in the accurate identification and positioning of mature florets in the field, and the prerequisite for accurate positioning is the reliable identification of mature florets.

[0003] Currently, the identification and location of mature broccoli heads is mainly done manually, which suffers from inherent drawbacks such as high labor intensity and difficulty in guaranteeing accuracy, constituting a technological bottleneck for industrial upgrading. To solve this problem, using computer vision technology to replace manual labor and achieve accurate and stable identification of broccoli heads in complex environments has become an important research topic in the field of agricultural automation.

[0004] With the continuous development of deep learning technology and hardware, object detection algorithms have been increasingly applied in various fields in recent years, providing a new direction for the detection of broccoli heads. Object detection models based on convolutional neural networks can automatically extract features from images to achieve accurate identification of the target, and have been widely used in various fields. However, most existing deep learning-based broccoli head detection methods are still in the verification stage under laboratory or ideal conditions, and their training data is usually collected from scenes with simple backgrounds and uniform lighting. When these models are applied to complex field environments, they are often affected by many practical factors, such as uneven lighting due to weather changes, differences in crop growth morphology, partial leaf occlusion, soil background interference, and changes in near and far scales. These uncertainties significantly affect the detection accuracy and stability of the model, leading to failures in subsequent positioning and mechanical harvesting. Therefore, for high-precision detection of broccoli heads in complex field environments, there is an urgent need for a detection method with stronger feature representation capabilities, better robustness, and real-time performance, providing a reliable visual perception foundation for intelligent harvesting equipment. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a field-based method for detecting shaded broccoli heads based on an improved YOLOv11. By optimizing the model structure, the feature extraction capability of the model is significantly improved with a limited increase in the number of parameters, thereby enhancing the accuracy and efficiency of identifying broccoli heads in the field environment.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A field-based method for detecting shaded broccoli heads based on an improved YOLOv11 includes the following steps:

[0008] Broccoli images were acquired in the actual broccoli growing environment, preprocessed and labeled to obtain the broccoli training dataset;

[0009] An improved YOLOv11 model was constructed by replacing the C3k2 module in the backbone network of the basic YOLOv11 model with the C3k2_PC module based on partial convolution, introducing the SCSA attention mechanism unit into the C2PSA module in the backbone network of the basic YOLOv11 model, and replacing the feature extraction pyramid structure in the neck network of the basic YOLOv11 model with the DFF-RFPN network structure that introduces a dynamic feature fusion module.

[0010] The improved YOLOv11 model was trained using the broccoli training dataset until the training iterations were completed.

[0011] Using the trained and improved YOLOv11 model, the detection results of broccoli flower heads are output for real-time acquired broccoli images.

[0012] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0013] Preferably, the C3k2_PC module includes a splitting unit, a main branch information flow and a shortcut branch information flow connected to and parallel to the splitting unit, a first fusion unit connected to the main branch information flow and the shortcut branch information flow, and a first convolution module connected to the first fusion unit.

[0014] The main branch information flow includes a second convolutional module, a first feature segmentation module, a bottleneck convolutional module with two fused partial convolutions, a second fusion unit, and a third convolutional module connected in sequence. The output of the second convolutional module is connected to the second fusion unit, and the output of the first feature segmentation module is connected to the second fusion unit.

[0015] The shortcut branch information flow includes a fourth convolutional module, a second feature segmentation module, a C3k module with two fused partial convolutions, a third fusion unit, and a fifth convolutional module connected in sequence. The output of the fourth convolutional module is connected to the third fusion unit, and the output of the second feature segmentation module is connected to the third fusion unit.

[0016] Preferably, the C3k module of the fused partial convolution includes a sixth convolution module, two bottleneck convolution modules of the fused partial convolution, a fourth fusion unit and a seventh convolution module connected in sequence, and the output of the sixth convolution module is connected to the fourth fusion unit through an eighth convolution module.

[0017] Preferably, the bottleneck convolution module of the fused partial convolution includes a ninth convolution module, a partial convolution, and a fifth fusion unit connected in sequence, and the input of the bottleneck convolution module of the fused partial convolution is connected to the fifth fusion unit.

[0018] Preferably, the C2PSA module in the backbone network of the basic YOLOv11 model introduces the SCSA attention mechanism unit to obtain the C2PSCSA module. The C2PSCSA module includes a tenth convolutional module, two PSCSA modules, a sixth fusion unit and an eleventh convolutional module connected in sequence, wherein the output of the tenth convolutional module is connected to the sixth fusion unit.

[0019] The PSCSA module includes an SCSA attention mechanism unit, a seventh fusion unit, a feedforward network unit, and an eighth fusion unit connected in sequence. The inputs of the PSCSA module are connected to the seventh fusion unit and the eighth fusion unit, respectively.

[0020] Preferably, the processing flow of the DFF-RFPN network structure is as follows:

[0021] The P3 feature output by the backbone network is connected to the tenth fusion unit through the twelfth convolution module. The P4 feature output by the backbone network is connected to the tenth fusion unit. The P4 feature output by the backbone network is connected to the ninth fusion unit through the thirteenth convolution module. The P5 feature output by the backbone network is connected to the ninth fusion unit through the fourteenth convolution module. The output of the ninth fusion unit is connected to the first DFFCk unit. The output of the first DFFCk unit is connected to the tenth fusion unit through the first upsampling unit.

[0022] The P3 feature output by the backbone network is connected to the eleventh fusion unit. The output of the tenth fusion unit is connected to the eleventh fusion unit after passing through the second DFFCk unit and the second upsampling unit. The output of the eleventh fusion unit is used as the input of the small-scale head in the detection head after passing through the third DFFCk unit.

[0023] The output of the second DFFCk unit is connected to the thirteenth fusion unit through the fifteenth convolution module. The output of the second DFFCk unit is connected to the thirteenth fusion unit. The output of the thirteenth fusion unit is used as the input of the mesoscale head in the detection head after passing through the fourth DFFCk unit.

[0024] The output of the second DFFCk unit is connected to the twelfth fusion unit via the sixteenth convolution module. The output of the first DFFCk unit is connected to the twelfth fusion unit. The output of the fourth DFFCk unit is connected to the twelfth fusion unit via the seventeenth convolution module. The output of the twelfth fusion unit is then used as the input of the large-scale head in the detection head after passing through the fifth DFFCk unit.

[0025] Preferably, the DFFCk unit includes an eighteenth convolutional module, a third feature segmentation module, a bottleneck convolutional module that fuses two DFF modules, a fourteenth fusion unit, and a nineteenth convolutional module connected in sequence. The output of the eighteenth convolutional module is connected to the fourteenth fusion unit, and the output of the third feature segmentation module is connected to the fourteenth fusion unit.

[0026] Preferably, the bottleneck convolutional module of the fused DFF module includes a twentieth convolutional module, a DFF module and a fifteenth fusion unit connected in sequence;

[0027] The DFF module includes an uplink semantic enhancement branch and a downlink detail compensation branch set in parallel. The output of the uplink semantic enhancement branch and the output of the downlink detail compensation branch are multiplied element-wise and then used as the output of the DFF module.

[0028] The uplink semantic enhancement branch includes a parallel 21st convolutional module and a 22nd convolutional module. The outputs of the 21st and 22nd convolutional modules are input into the first Sigmoid activation function after element-wise addition. The output of the first Sigmoid activation function is used as the output of the uplink semantic enhancement branch.

[0029] The downlink detail compensation branch concatenates the inputs of the uplink semantic enhancement branch and the downlink detail compensation branch. The concatenated features are then passed through the average pooling module, the 23rd convolution module, and the 2nd Sigmoid activation function, and then multiplied element-wise with the concatenated features. The multiplied features are then passed through the 24th convolution module as the output of the downlink detail compensation branch.

[0030] The field broccoli head detection method based on the improved YOLOv11 provided by this invention has the following advantages compared with the prior art:

[0031] (1) The present invention uses the C2PSCSA module to replace the original C2PSA module, making full use of multiple semantic information, improving the model's feature extraction ability in complex field environments, and enhancing the robustness of the model under uncertain factors such as leaf occlusion.

[0032] (2) This invention uses PartialConv (Partial Convolution) to replace the convolution module in the original model C3k2 module, which solves the problem that traditional convolution will introduce meaningless calculations and cause feature pollution when there are occlusions, defects or invalid regions in the input image. It realizes adaptive convolution operation on effective pixel regions and ensures the reliability and stability of the feature generation process.

[0033] (3) The present invention adopts an improved DFF-RFPN network structure to replace the feature extraction pyramid structure of the original neck network, which enhances the cross-scale feature fusion capability of the model and optimizes the overall running efficiency of the model. It can capture broccoli florets of different sizes more accurately, realize the adaptive fusion of high-dimensional abstract semantic features and local detail information, and improve the representation quality of the target region.

[0034] (4) The improvement measures adopted in this invention not only improve the performance of the model, but also control the growth of the number of parameters, which is conducive to the deployment of the model on edge devices and provides a foundation for intelligent harvesting. Attached Figure Description

[0035] Figure 1 This is a flowchart of the field broccoli head detection method based on the improved YOLOv11 of the present invention;

[0036] Figure 2 This is a schematic diagram of the SDRP-YOLOv11 model structure of the present invention;

[0037] Figure 3 for Figure 2 Schematic diagram of the C3k2_PC module structure;

[0038] Figure 4 for Figure 2 Schematic diagram of the C2PSCSA module structure;

[0039] Figure 5 This is a schematic diagram of the SCSA attention mechanism module structure in the C2PSCSA module of the present invention;

[0040] Figure 6 for Figure 2 Schematic diagram of the DFF-RFPN network structure;

[0041] Figure 7This is a schematic diagram comparing the detection precision results of the conventional YOLOv11 network and the detection precision results of the SDRP-YOLOv11 network of the present invention.

[0042] Figure 8 This is a schematic diagram comparing the detection results of the conventional YOLOv11 network and the detection recall results of the SDRP-YOLOv11 network of the present invention.

[0043] Figure 9 This is a schematic diagram comparing the detection results of a conventional YOLOv11 network and the detection mAP@50 values ​​of the SDRP-YOLOv11 network of this invention.

[0044] Figure 10 This is a schematic diagram comparing the detection results of a conventional YOLOv11 network and the detection mAP@95 value results of the SDRP-YOLOv11 network of the present invention.

[0045] Figure 11 A heatmap comparison of the training process of a conventional YOLOv11 network and the SDRP-YOLOv11 network of this invention;

[0046] Figure 12 This diagram illustrates a comparison of the detection performance of a conventional YOLOv11 network and the SDRP-YOLOv11 network of this invention under actual detection tasks. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0049] Example 1:

[0050] like Figure 1 As shown, this embodiment provides a field-based method for detecting shaded broccoli heads based on an improved YOLOv11, including the following steps:

[0051] Step 1: Obtain broccoli images in the actual broccoli growing environment, preprocess and label them to obtain the broccoli training dataset.

[0052] This embodiment captures images of broccoli in a broccoli field at different times, under different weather conditions, and with varying leaf occlusion. After filtering and labeling, a dataset for flower head detection is constructed. In this embodiment, the dataset is primarily collected from broccoli growing bases, encompassing field images of broccoli at different times (morning, noon, evening, and night) and from different shooting angles, including occlusion. The image acquisition device is a mobile phone camera (or a drone camera, etc.), and the acquired images have a resolution of 1920×1080. After selecting suitable images for training, the labelimg software is used for dataset labeling, generating a txt file containing specific categories and corresponding bounding box coordinates, enabling the model to accurately locate the flower head during recognition.

[0053] After labeling, the dataset was divided into training, validation, and test sets in an 8:1:1 ratio. To further improve the model's detection accuracy and generalization performance, data augmentation was performed on the validation set. This involved enhancing image quality, random cropping, contrast adjustment, and introducing noise to improve the dataset's quality. Additionally, some images were supplemented from open-source datasets to provide more comprehensive information. These enhancements improved the model's adaptability to different scenarios.

[0054] Step 2: Construct an improved YOLOv11 model to obtain the SDRP-YOLOv11n model for broccoli head detection in the field. The improvement involves replacing the C3k2 module in the backbone network of the basic YOLOv11 model with a C3k2_PC module based on partial convolutions, replacing the C2PSA module in the backbone network of the basic YOLOv11 model with a C2PSCSA module that introduces the SCSA attention mechanism unit, and replacing the feature extraction pyramid structure in the neck network of the basic YOLOv11 model with a DFF-RFPN network structure that introduces a dynamic feature fusion module.

[0055] The existing YOLOv11n basic model includes: a backbone network for extracting features from the input image; a neck network for fusing and enhancing the extracted features; and a detection head for performing detection on multi-scale feature maps and outputting detection results. This embodiment introduces a lightweight, high-speed inference convolutional part into the backbone network of the basic YOLOv11n model to replace the original convolutional operation in the C3k2 module, thereby significantly improving the model's inference speed and reducing the number of model parameters. In the terminal feature extraction layer of the basic YOLOv11n model's backbone network, a spatial-channel collaborative attention mechanism module (C2PSCSA) is introduced to replace the self-attention mechanism in the original module, thereby enhancing the model's ability to represent flower head features. The structure of the neck network of the basic YOLOv11n model is improved, specifically by performing three-way downsampling on the features output by the backbone network, concatenating the downsampled multi-way features, and introducing a dynamic feature fusion module (DFF) to further improve the representation quality of the target region. A DFF-RFPN network structure is constructed to improve the model's overall detection performance for flower head targets.

[0056] For details, see attached. Figure 2 As shown, the backbone network consists of 5 Conv (Convolution) modules, 4 C3k2_PC modules, 1 SPFF module, and 1 C2PSCSA module. When an image is input into the backbone network, progressive downsampling is used, and feature extraction is performed in the following order:

[0057] The system extracts features from shallow, medium-resolution small and medium-sized targets using two Conv modules and one C3k2_PC module. Then, it extracts features from medium-resolution, medium-medium-scale targets using one Conv module and one C3k2_PC module, outputting P3 features. Next, it extracts features from deep, low-resolution large targets using one Conv module and one C3k2_PC module, outputting P4 features. Finally, it outputs the attention-enhanced final feature information, namely P4 features (also known as low-level features), using one Conv module, one C3k2_PC module, one SPFF module, and one C2PSCSA module.

[0058] After feature information is extracted by the backbone network, feature fusion is performed through an improved neck network. Three Conv modules downsample the feature information extracted by the backbone network at various levels. After multiple rounds of feature fusion and upsampling, multi-level semantic information is efficiently integrated. The P3 detection layer outputs spatial localization information from low-level features, which is used for regression branch prediction of bounding box position and size. The P4 detection layer outputs semantic classification information that integrates mid-to-high-level features, which is used for classification branch to determine target category. Finally, the P5 detection layer outputs context and cross-scale information that integrates global features, improving mid-scale target detection and reducing fuzzy boundary errors.

[0059] (1) The structure of the C3k2_PC module is shown in the attached figure. Figure 3 As shown, it includes a splitting unit, a main branch information flow and a shortcut branch information flow connected to and running in parallel with the splitting unit, a first fusion unit connected to the main branch information flow and the shortcut branch information flow, and a first convolutional module connected to the first fusion unit; the main branch information flow includes a second convolutional module, a first feature segmentation module, a bottleneck convolutional module with two fusion parts convolutions, a second fusion unit, and a third convolutional module connected in sequence, the output of the second convolutional module is connected to the second fusion unit, and the output of the first feature segmentation module is connected to the second fusion unit; the shortcut branch information flow includes a fourth convolutional module, a second feature segmentation module, a C3k module with two fusion parts convolutions, a third fusion unit, and a fifth convolutional module connected in sequence, the output of the fourth convolutional module is connected to the third fusion unit, and the output of the second feature segmentation module is connected to the third fusion unit.

[0060] The C3k module for fusion of partial convolutions includes a sixth convolution module, two bottleneck convolution modules for fusion of partial convolutions, a fourth fusion unit, and a seventh convolution module connected in sequence. The output of the sixth convolution module is connected to the fourth fusion unit via an eighth convolution module. The bottleneck convolution module for fusion of partial convolutions includes a ninth convolution module, partial convolutions, and a fifth fusion unit connected in sequence. The input of the bottleneck convolution module for fusion of partial convolutions is connected to the fifth fusion unit.

[0061] The processing flow of the C3k2_PC module in this embodiment is as follows:

[0062] After the feature information is input, it first enters a splitting unit. In this unit, the feature information is divided into two parallel information streams along the channel dimension: the main branch information stream and the shortcut branch information stream. The main branch information stream enters a feature enhancement sub-network. This sub-network first performs primary spatial feature extraction through a Conv module (preferably a 3×3 convolution) to capture the local details and spatial context information of the features; subsequently, the preliminarily extracted features are fed into a sequence containing two PartialConv sub-modules. Each submodule performs a series of nonlinear transformations on the input information, carrying out deep nonlinear transformations and hierarchical progressive processing to gradually abstract and refine high-level semantic features, thereby enhancing the model's representational capabilities. The shortcut branch information flow enters the information fidelity path, selectively performing identity mapping or a 1×1 Conv on the input information to retain the detailed information of the original input features to the maximum extent, and establishing a short path directly to the fusion point to ensure the effective flow of gradients during backpropagation and alleviate the gradient decay problem in deep networks. The two information flows are guided to a feature convergence and fusion unit: at the splicing node, the two information flows are spliced ​​along the channel dimension to form a composite feature map that simultaneously contains the deepened high-level semantic information and the fidelity low-level detailed information. Subsequently, this composite feature map undergoes channel fusion and dimension adjustment through a Conv module (preferably a 1×1 convolution), integrating complementary information from the two branches and adjusting the number of channels to match the expected output dimension, ultimately generating the output feature map.

[0063] The operating principle of some convolutional layers is as follows:

[0064] First, the input feature map Extracting binary validity mask , The number of feature map channels. These represent the height and width of the feature map, respectively. A value of 1 indicates that the current pixel position is valid, while a value of 0 indicates a missing or meaningless pixel. During the convolution operation, only pixels within the receptive field of the current convolution kernel will be processed. Perform calculations on the following parts:

[0065]

[0066] In the formula, This represents the output feature value. For the convolution kernel within the receptive field The weights corresponding to each position. Represents the input feature map Experiencing the Nouchi The feature values ​​corresponding to each position This indicates the effectiveness of the binary mask within the receptive field. Validity markers corresponding to each position This represents the current receptive field of the convolution, and the denominator is used to ensure that the feature values ​​are independent of the number of effective pixels, thus achieving adaptive normalization. This is the location number within the convolution receptive field.

[0067] To enable the network to recover the missing regions layer by layer, PartialConv also generates new masks. :

[0068]

[0069] If at least one valid pixel exists within the current convolution window, the output at that location is considered repaired and marked as valid; otherwise, it remains invalid. This process can propagate layer by layer with the network depth, gradually filling in the missing areas with valid predictions.

[0070] Standard convolution assigns equal weight to all pixels, failing to distinguish between valid and invalid regions. This leads to invalid regions participating in feature calculation, easily introducing noise and distribution shifts during training, and potentially amplifying erroneous features from missing regions through cascading. PartialConv, on the other hand, uses only reliable regions for calculation, automatically normalizing the output value to the number of valid pixels. Its mask is automatically updated as features propagate, and its simple module structure allows it to be directly replaced by ordinary convolution. This characteristic offers significant advantages for the recognition and segmentation of crops (such as broccoli florets), effectively improving the model's recognition accuracy and stability under complex environmental factors such as occlusion and lighting conditions.

[0071] In this embodiment, a partial convolution module is introduced into the feature extraction module of the model backbone network. This solves the problem that traditional convolution introduces meaningless calculations and leads to feature contamination when the input image has occlusion, defects, or invalid regions. The present invention uses a partial convolution strategy for feature extraction, which can achieve adaptive convolution operation on effective pixel regions, thereby ensuring the reliability and robustness of the feature generation process.

[0072] Compared to the traditional C3k2 module structure, the C3k2_PC module of this invention can make the output closer to the real structure without significantly increasing the computational cost. It is not affected by the interference of pixels in the missing area, thus effectively reducing feature noise. In complex scenes with occlusion, insufficient lighting, and incomplete targets, the feature representation is more stable. This module has good versatility and can be embedded as a basic feature extraction unit in various convolutional neural networks.

[0073] (2) The C2PSCSA module structure is as follows Figure 4As shown, it includes a tenth convolutional module, two PSCSA modules, a sixth fusion unit, and an eleventh convolutional module connected in sequence, wherein the output of the tenth convolutional module is connected to the sixth fusion unit; the PSCSA module includes an SCSA attention mechanism unit (Spatial and Channel Synergistic Attention), a seventh fusion unit, a feed-forward network (FFN) unit, and an eighth fusion unit connected in sequence, and the input of the PSCSA module is connected to the seventh fusion unit and the eighth fusion unit, respectively.

[0074] The processing logic of the C2PSCSA module is as follows:

[0075] After the feature information is input, the input feature map is first expanded and transformed in the channel dimension by a Conv module, and then split into two branches along the channel dimension. One branch retains the original feature information, while the other branch is fed into the core SCSA attention mechanism unit. In this unit, the features first undergo a spatial attention-first computation stage, which performs global average pooling on the feature map in the height and width dimensions, and extracts multi-scale spatial context information using grouped depthwise convolution to generate a direction-sensitive spatial attention weight map, performing spatial adaptive recalibration on the input features. Then, it enters the channel attention stage based on window downsampling, which obtains the global dependencies between channels through the generation of Q, K, and V vectors and multi-head self-attention computation, and generates channel weight vectors. The above spatial and channel attention outputs are fused by multiplication, and then feature reconstruction is performed through a feedforward neural network and residual connections. Finally, the processed branch is concatenated with the original branch and restored to the target output dimension through one-dimensional convolution, thereby significantly enhancing the model's ability to extract and represent key features while reducing computational overhead.

[0076] The main principles of the SCSA attention mechanism module are as follows: Figure 5 As shown:

[0077] When feature map After being extracted from the backbone network, The batch size is input into the SCSA attention mechanism unit to achieve adaptive enhancement of feature information. This mechanism includes the Multi-Semantic Space Attention Submodule (SMSA) and the Progressive Channel Self-Attention Submodule (PCSA), with the signal flow order as follows:

[0078] First, input features The input features are fed into the SMSA submodule to extract spatial structural features at different semantic levels. This submodule decomposes the input features into two sets of one-dimensional representations through bidirectional average pooling along the spatial dimension:

[0079]

[0080] in This represents the characteristic after averaging along the height direction. This represents the characteristic after averaging along the width direction. This represents the average pooling operation along the height direction. This indicates the average pooling operation along the width direction.

[0081] Subsequently, the channel dimension was divided into 4 sub-feature groups, with each group containing 10 channels. Each subgroup can be separated into one-dimensional convolutions using different kernel sizes to extract multi-scale spatial semantic information.

[0082]

[0083] Among them, the kernel size The values ​​3, 5, 7, and 9 are selected sequentially to simultaneously capture local details and global structural features. Indicates the first Sub-features in the height direction, Indicates the kernel size as Depth-separable one-dimensional convolution, Indicates the first Semantic information in the height direction, Indicates the first Sub-features in the width direction, Indicates the first Semantic information along the width direction. All sub-features are concatenated via channels, and then grouped, normalized, and activated using a sigmoid activation function to obtain spatial attention maps along the height and width directions:

[0084]

[0085]

[0086] In the formula, represents the sub-features of each channel after division. This represents the level space dependency model. Represents the vertical spatial dependency model. Represents the Sigmoid activation function. This represents the GroupNormal normalization operation.

[0087] The attention maps in both directions are broadcast and then multiplied to obtain the final spatial weight matrix, which is then applied to the input features to obtain the features. :

[0088]

[0089] In the formula, the symbol ⊙ represents element-wise multiplication. After this step, the model can enhance the response of key locations based on multi-scale spatial information, thereby achieving spatial semantic enhancement.

[0090] Features after spatial augmentation by the SMSA module The data will be fed into the PCSA module to model the semantic relationships between channels. This module first performs an adaptive downsampling operation on the spatial dimension using average pooling.

[0091]

[0092] In the formula, This is represented as an average pooling operation. The downsampling features are then represented by three 1×1 convolutional layers, which are used to generate query, key, and value tensors, respectively.

[0093]

[0094] In the formula, Represents a linear projection based on 1D convolution. For querying the matrix, The key matrix, It is a value matrix.

[0095] The similarity between channels is calculated using channel self-attention, and specific channel weights are generated.

[0096]

[0097] The generated channel weights are globally pooled, and then multiplied channel by channel to add the weights to the output features of the SMSA module. The final output features of the module are obtained above. :

[0098]

[0099] In the formula, This is a channel weight feature map. This indicates element-wise multiplication. The output is the sum of the elements. enter Maintaining the same spatial dimensions, it achieves multi-semantic selective response in the spatial domain and cross-semantic adaptive fusion in the channel domain, thereby significantly improving the network's feature representation ability and target recognition performance.

[0100] (3) DFF-RFPN network structure and its auxiliary structures, such as Figure 6As shown, it includes the twelfth convolution module, the thirteenth convolution module, the fourteenth convolution module, the fifteenth convolution module, the sixteenth convolution module, the seventeenth convolution module, the ninth fusion unit, the tenth fusion unit, the eleventh fusion unit, the twelfth fusion unit, the thirteenth fusion unit, the first upsampling unit, the second upsampling unit, the first DFFCk unit, the second DFFCk unit, the third DFFCk unit, the fourth DFFCk unit, and the fifth DFFCk unit.

[0101] The P3 feature output from the backbone network is connected to the tenth fusion unit via the twelfth convolutional module. The P4 feature output from the backbone network is also connected to the tenth fusion unit. The P4 feature output from the backbone network is then connected to the ninth fusion unit via the thirteenth convolutional module. The P5 feature output from the backbone network is connected to the ninth fusion unit via the fourteenth convolutional module. The output of the ninth fusion unit is connected to the first DFFCk unit. The output of the first DFFCk unit is then connected to the tenth fusion unit via the first upsampling unit. The P3 feature output from the backbone network is connected to the eleventh fusion unit. The output of the tenth fusion unit is then connected to the eleventh fusion unit via the second DFFCk unit and the second upsampling unit. The output of the eleventh fusion unit... After passing through the third DFFCk unit, it serves as the input to the small-scale head in the detection head; the output of the second DFFCk unit is connected to the thirteenth fusion unit through the fifteenth convolution module, and the output of the second DFFCk unit is connected to the thirteenth fusion unit. The output of the thirteenth fusion unit is connected to the fourth DFFCk unit and serves as the input to the medium-scale head in the detection head; the output of the second DFFCk unit is connected to the twelfth fusion unit through the sixteenth convolution module, the output of the first DFFCk unit is connected to the twelfth fusion unit, the output of the fourth DFFCk unit is connected to the twelfth fusion unit through the seventeenth convolution module, and the output of the twelfth fusion unit is connected to the fifth DFFCk unit and serves as the input to the large-scale head in the detection head.

[0102] The DFFCk unit includes an eighteenth convolutional module, a third feature segmentation module, a bottleneck convolutional module that fuses two DFF modules, a fourteenth fusion unit, and a nineteenth convolutional module connected in sequence. The output of the eighteenth convolutional module is connected to the fourteenth fusion unit, and the output of the third feature segmentation module is connected to the fourteenth fusion unit.

[0103] The bottleneck convolutional module of the DFF module includes a 20th convolutional module, a DFF module, and a 15th fusion unit connected in sequence. The DFF module includes an uplink semantic enhancement branch and a downlink detail compensation branch set in parallel. The output of the uplink semantic enhancement branch and the output of the downlink detail compensation branch are multiplied element-wise and then used as the output of the DFF module. The uplink semantic enhancement branch contains a 21st and a 22nd convolutional module in parallel. The outputs of the 21st and 22nd convolutional modules are added element-wise and then input into the first Sigmoid activation function. The output of the first Sigmoid activation function is used as the output of the uplink semantic enhancement branch. The downlink detail compensation branch concatenates the inputs of the uplink semantic enhancement branch and the downlink detail compensation branch. The concatenated features are passed through an average pooling module, a 23rd convolutional module, and a second Sigmoid activation function in sequence and then multiplied element-wise with the concatenated features. The multiplied features are passed through a 24th convolutional module and then used as the output of the downlink detail compensation branch.

[0104] The processing logic for the DFF-RFPN network structure is as follows:

[0105] When feature information flows into the neck network, dimensionality-adaptive convolutions are first applied to the features at different levels for channel mapping, ensuring they have a unified data dimension representation capability before entering the fusion region. Subsequently, downsampling operations are performed on the features output from multiple feature extraction layers of the backbone network, adjusting features at different scales to the same spatial scale, forming three feature representations with consistent resolution. Unlike traditional structures, this invention eliminates the upsampling branch, which contributes little to performance, in the feature pyramid structure, retaining only the downsampling path, which significantly improves performance, to reduce parameter overhead and increase fusion efficiency. The downsampled multi-path features converge into the fusion node along multiple paths. The three features achieve cross-scale feature complementarity through concatenation or element-wise fusion. The fused features enter an efficient aggregation structure built with the concept of reparameterization. During the inference phase, all convolutional branches are parameter-merged, and the output fused features flow into the DFFCk feature enhancement unit, which is an improvement on the DFF feature fusion module. Within this module, fine-grained and high-level semantic information is further aggregated, ultimately yielding fused output features with both semantic depth and detail sensitivity, which are then sent to the detection head for final target recognition and localization. The above processing steps enable efficient merging and enhancement of broccoli head features at different scales.

[0106] The working principle of the dual-branch feature fusion module (DFF) in the DFFCk module is as follows: When different input features enter the DFF module, they first flow into two parallel processing branches: an uplink semantic enhancement branch and a downlink detail compensation branch. In the uplink branch, because deep semantic features have a small spatial size but high semantic concentration, these features are first adjusted to the same spatial size as shallow features through upsampling or size restoration operations. After upsampling, they are then re-expressed through convolution, improving the structural description ability of deep semantics in high-resolution space. Features from shallow layers have more complete texture contours and target boundaries. These features are fed into the convolution enhancement sequence of the downlink branch, using residual structures to enhance detail expression and maintain stable gradient propagation. Subsequently, branch features are fused through feature concatenation. The resulting fused features combine multi-layer semantic expression and local structural details, serving as the output of the improved feature pyramid, which is then fed into subsequent detection heads or further feature enhancement modules to obtain the final detection result.

[0107] The improved model, through the aforementioned feature flow method and dual-branch fusion mechanism, achieves more efficient multi-scale feature integration. By using full-process downsampling and three-way feature aggregation, it achieves collaborative integration of high-level semantics and shallow details at a unified scale, which is beneficial for improving the recognition ability of broccoli heads of different sizes. It also reduces computational cost while increasing network inference speed. Multiple cross-scale fusions enable shallow features to acquire additional contextual semantic information, maintaining effective response even when the target size is small, the distance is far, or there is strong occlusion interference in the scene, thereby improving the small target detection performance. After introducing the DFF module to enhance the feature extraction module, it can achieve bidirectional compensation of deep and shallow semantics, enabling the model to better recognize the outline of the flower head in a similar color background, improving the detection reliability in cluttered field environments, effectively improving the accurate segmentation and localization performance of broccoli heads in agricultural visual target detection scenarios, and further enhancing the overall recognition performance of the system.

[0108] Step 3: Train the improved YOLOv11 model using the broccoli training dataset until the training iterations are complete.

[0109] The improved SDRP-YOLOv11 detection model was trained on the training set created in step 1 and tested using images from the validation set. Various training evaluation metrics were obtained, including accuracy. Recall rate and mean precision as well as The specific calculation formula is as follows:

[0110]

[0111]

[0112]

[0113]

[0114] In the formula, This indicates the number of targets that were correctly identified; Indicates the number of negative samples that are false positives; This indicates the number of incorrectly identified samples. This represents the total number of categories detected.

[0115] In this embodiment, the dataset used for model training was divided into a training set of 1936 images, a test set of 242 images, and a validation set of 242 images. The training batch size was set to 16, the total number of training epochs was set to 150, the maximum learning rate was set to 0.01, the learning momentum was set to 0.91, and the optimizer selected was stochastic gradient descent.

[0116] Appendix Figures 7-10 The paper compares the changes in various metrics during training between the initial conventional YOLOv11 model and the SDRP-YOLOv11 proposed in this invention. It can be observed that the improved model outperforms the original model in all metrics, and the speed at which the accuracy reaches the convergence value during model training is significantly improved, while the growth process of the metrics is also more stable.

[0117] Table 1 presents the ablation experiment results of the model. YOLOv11n is the basic YOLOv11n model; +C2PSCSA is the basic YOLOv11n model with the C2PSCSA module added; +C3k2_PC is the +C2PSCSA model with the C3k2_PC module added; +RFPN is the +C3k2_PC model with the RFPN module added; and +DFFCk is the +RFPN model with the DFFCk module added (i.e., the SDRP-YOLOv11 model of this invention). As shown in Table 1, the improved model proposed in this invention achieves higher accuracy, recall, mAP50, and mAP95 compared to the original model by 2.4%, 2.5%, 0.012, and 0.032, respectively.

[0118] Table 1 Comparison of ablation experiments using the improved model

[0119]

[0120] Appendix Figure 11The heatmaps generated by the model before and after feature extraction are shown. By observing the color intensity of the region of interest in the heatmap, the model's feature extraction performance in the corresponding region can be intuitively reflected. It can be seen that the improved model has significantly improved the feature extraction effect of broccoli flower heads, and can more accurately place the region of interest on the broccoli flower heads. Moreover, the effect is significantly improved for both normal state and broccoli affected by occlusion, which enhances the model's learning efficiency and detection accuracy.

[0121] Step 4: Using the trained improved YOLOv11 model, output the broccoli head detection results for the real-time acquired broccoli images.

[0122] Images of broccoli from actual field conditions were input into the trained network for flower head detection. (Attached) Figure 12 The paper compares the detection performance of the initial YOLOv11 model and the proposed SDRP-YOLOv11 model under different conditions. As shown in the attached figures, the improved model generates more accurate bounding boxes and reduces the potential for missed detections in the original model.

[0123] The method of this invention enhances the feature extraction capability and detection accuracy of the model by improving the attention mechanism, the convolution method of the feature extraction layer, and the neck network structure. It effectively improves the detection performance of the model in complex field environments, and the computational cost of the algorithm increases only slightly, which is conducive to the deployment and application of the model on edge devices and has strong practical significance.

[0124] Example 2:

[0125] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the field shaded broccoli head detection method based on the improved YOLOv11 in Embodiment 1.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0127] Example 3:

[0128] This embodiment provides a field broccoli head detection device based on an improved YOLOv11, including a processor and a memory storing several computer instructions. When the computer instructions are executed by the processor, they implement the steps of the field broccoli head detection method based on the improved YOLOv11.

[0129] For specific limitations regarding the field shaded broccoli head detection device based on the improved YOLOv11, please refer to the limitations of the field shaded broccoli head detection method based on the improved YOLOv11 mentioned above, which will not be repeated here.

[0130] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the method of the present invention by running the computer program stored in the memory.

[0131] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0132] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A field method for detecting shaded broccoli heads based on an improved YOLOv11, characterized in that, Includes the following steps: Broccoli images were acquired in the actual broccoli growing environment, preprocessed and labeled to obtain the broccoli training dataset; An improved YOLOv11 model was constructed by replacing the C3k2 module in the backbone network of the basic YOLOv11 model with the C3k2_PC module based on partial convolution, introducing the SCSA attention mechanism unit into the C2PSA module in the backbone network of the basic YOLOv11 model, and replacing the feature extraction pyramid structure in the neck network of the basic YOLOv11 model with the DFF-RFPN network structure that introduces a dynamic feature fusion module. The improved YOLOv11 model was trained using the broccoli training dataset until the training iterations were completed. Using the trained and improved YOLOv11 model, the broccoli head detection results are output for real-time acquired broccoli images; The C3k2_PC module includes a splitting unit, a main branch information flow and a shortcut branch information flow connected to and running in parallel with the splitting unit, a first fusion unit connected to the main branch information flow and the shortcut branch information flow, and a first convolution module connected to the first fusion unit. The main branch information flow includes a second convolutional module, a first feature segmentation module, a bottleneck convolutional module with two fused partial convolutions, a second fusion unit, and a third convolutional module connected in sequence. The output of the second convolutional module is connected to the second fusion unit, and the output of the first feature segmentation module is connected to the second fusion unit. The shortcut branch information flow includes a fourth convolutional module, a second feature segmentation module, a C3k module with two fused partial convolutions, a third fusion unit, and a fifth convolutional module connected in sequence. The output of the fourth convolutional module is connected to the third fusion unit, and the output of the second feature segmentation module is connected to the third fusion unit. Specifically, the C2PSA module in the backbone network of the basic YOLOv11 model introduces an SCSA attention mechanism unit to obtain a C2PSCSA module. The C2PSCSA module includes a tenth convolutional module, two PSCSA modules, a sixth fusion unit, and an eleventh convolutional module connected in sequence, wherein the output of the tenth convolutional module is connected to the sixth fusion unit. The PSCSA module includes an SCSA attention mechanism unit, a seventh fusion unit, a feedforward network unit, and an eighth fusion unit connected in sequence, wherein the input of the PSCSA module is connected to the seventh fusion unit and the eighth fusion unit, respectively. The processing flow of the DFF-RFPN network structure is as follows: The P3 feature output by the backbone network is connected to the tenth fusion unit through the twelfth convolution module. The P4 feature output by the backbone network is connected to the tenth fusion unit. The P4 feature output by the backbone network is connected to the ninth fusion unit through the thirteenth convolution module. The P5 feature output by the backbone network is connected to the ninth fusion unit through the fourteenth convolution module. The output of the ninth fusion unit is connected to the first DFFCk unit. The output of the first DFFCk unit is connected to the tenth fusion unit through the first upsampling unit. The P3 feature output by the backbone network is connected to the eleventh fusion unit. The output of the tenth fusion unit is connected to the eleventh fusion unit after passing through the second DFFCk unit and the second upsampling unit. The output of the eleventh fusion unit is used as the input of the small-scale head in the detection head after passing through the third DFFCk unit. The output of the second DFFCk unit is connected to the thirteenth fusion unit through the fifteenth convolution module. The output of the second DFFCk unit is connected to the thirteenth fusion unit. The output of the thirteenth fusion unit is used as the input of the mesoscale head in the detection head after passing through the fourth DFFCk unit. The output of the second DFFCk unit is connected to the twelfth fusion unit via the sixteenth convolution module. The output of the first DFFCk unit is connected to the twelfth fusion unit. The output of the fourth DFFCk unit is connected to the twelfth fusion unit via the seventeenth convolution module. The output of the twelfth fusion unit is used as the input of the large-scale head in the detection head after passing through the fifth DFFCk unit. The DFFCk unit includes the eighteenth convolution module, the third feature segmentation module, the bottleneck convolution module of the two fused DFF modules, the fourteenth fusion unit, and the nineteenth convolution module connected in sequence. The output of the eighteenth convolution module is connected to the fourteenth fusion unit, and the output of the third feature segmentation module is connected to the fourteenth fusion unit.

2. The field broccoli head detection method based on improved YOLOv11 according to claim 1, characterized in that, The C3k module of the fusion partial convolution includes a sixth convolution module, two bottleneck convolution modules of the fusion partial convolution, a fourth fusion unit, and a seventh convolution module connected in sequence. The output of the sixth convolution module is connected to the fourth fusion unit through an eighth convolution module.

3. The field broccoli head detection method based on improved YOLOv11 according to claim 1 or 2, characterized in that, The bottleneck convolution module of the fused partial convolution includes a ninth convolution module, a partial convolution, and a fifth fusion unit connected in sequence, and the input of the bottleneck convolution module of the fused partial convolution is connected to the fifth fusion unit.

4. The field broccoli head detection method based on improved YOLOv11 according to claim 1, characterized in that, The bottleneck convolutional module of the fused DFF module includes a twentieth convolutional module, a DFF module and a fifteenth fusion unit connected in sequence; The DFF module includes an uplink semantic enhancement branch and a downlink detail compensation branch set in parallel. The output of the uplink semantic enhancement branch and the output of the downlink detail compensation branch are multiplied element-wise and then used as the output of the DFF module. The uplink semantic enhancement branch includes a parallel 21st convolutional module and a 22nd convolutional module. The outputs of the 21st and 22nd convolutional modules are input into the first Sigmoid activation function after element-wise addition. The output of the first Sigmoid activation function is used as the output of the uplink semantic enhancement branch. The downlink detail compensation branch concatenates the inputs of the uplink semantic enhancement branch and the downlink detail compensation branch. The concatenated features are then passed through the average pooling module, the 23rd convolution module, and the 2nd Sigmoid activation function, and then multiplied element-wise with the concatenated features. The multiplied features are then passed through the 24th convolution module as the output of the downlink detail compensation branch.

Images