Weed recognition method and system based on hierarchical multi-scale feature fusion

Abstract

The present invention discloses a weed identification method and system based on hierarchical multi-scale feature fusion. The method includes the following steps: constructing an image dataset; constructing an HDMS multi-weed target detection model; the HDMS multi-weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module; and replacing the detection module in the detection network of the YOLO11 model with the IntergraDet module. The constructed dataset is input into the HDMS multi-weed target detection model for training, and the trained HDMS multi-weed target detection model is used to identify weeds in farmland. By replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, the HDMS multi-weed target detection model significantly enhances multi-scale target perception capabilities.

Description

Technical Field

[0001] The present invention belongs to the technical field of farmland management and maintenance, and specifically relates to a weed identification method and system based on hierarchical multi-scale feature fusion. Background Art

[0002] Weeds are considered environmentally unfriendly. They compete with crops for water, nutrients, sunlight, and space, slowing crop growth and reducing yields. They often serve as intermediate hosts for diseases and pests, exacerbating the occurrence and spread of crop pests and diseases. Furthermore, dense weeds can hinder mechanical tillage, harvesting, and irrigation, increasing the difficulty and cost of field management. Therefore, early intervention and treatment of weeds is an urgent issue in agricultural production. Manual weed control requires significant manpower and material resources. While pesticides are the most effective, their extensive use can also cause environmental pollution and pesticide residues on crops, impacting human health. Therefore, improving pesticide efficiency and precisely spraying weeds has become an urgent need. Precision spraying robots enable large-scale application of pesticides on weeds, effectively preventing the problems of chemical dust and pesticide residue. Accurate weed detection is the first step toward precise weed control. With the advancement of artificial intelligence, the application of visual image recognition methods can rapidly detect and identify weeds and crops, significantly facilitating precise spraying.

[0003] In recent years, the application of artificial intelligence in farmland management has become increasingly widespread. However, in complex field environments, the accuracy of machine weed recognition in images can significantly decrease due to factors such as lighting variations, shadows, and dew reflections. Furthermore, existing algorithms struggle to accurately distinguish weeds from crops in their early growth stages due to their morphological similarities. These technical bottlenecks have hindered the large-scale application of intelligent weed detection systems. To address these issues, this model first reconstructs the backbone network through hierarchical feature processing to significantly improve the model's ability to extract image features. Second, a multi-scale feature fusion module enables the model to more accurately detect weeds of varying sizes and shapes under complex conditions such as lighting variations. Finally, the detection head can more accurately identify and locate crops and weeds in complex farmland environments. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a weed identification method and system based on hierarchical multi-scale feature fusion, which aims to solve the problems in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a weed identification method based on hierarchical multi-scale feature fusion, comprising the following steps:

[0006] Step S1: Construct an image dataset; the image dataset is obtained by dividing the Crop and Weed dataset;

[0007] Step S2: constructing an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module;

[0008] Step S3: inputting the image dataset into the constructed HDMS multiple weed target detection model for training to obtain a trained HDMS multiple weed target detection model;

[0009] Step S4: using the trained HDMS multiple weed target detection model to identify weeds in the farmland;

[0010] The feature extraction network includes an SRFD module, and the output end of the SRFD module is sequentially connected to the first PC-MSFA module, the first DRFD module, the second PC-MSFA module, the second DRFD module, the third PC-MSFA module, the third DRFD module, the fourth PC-MSFA module, the SPPF module, and the C2PSA module;

[0011] The second PC-MSFA module includes global average pooling, global maximum pooling, and a third ordinary convolution; the output end of the first DRFD module is connected to the global average pooling, the global maximum pooling, and the third ordinary convolution respectively;

[0012] The output end of the global average pooling and the output end of the global maximum pooling are connected to the seventh fusion module, the output end of the seventh fusion module is connected to the Sigmoid activation function, the output end of the Sigmoid activation function and the output end of the first DRFD module are connected to the eighth fusion module, and the output end of the eighth fusion module is connected to the tenth ordinary convolution;

[0013] The output end of the third ordinary convolution is connected to the fourth ordinary convolution and the fifth fusion module respectively, the output end of the fourth ordinary convolution is connected to the fifth ordinary convolution and the fifth fusion module respectively, the output end of the fifth ordinary convolution is connected to the fifth fusion module, and the output end of the fifth fusion module is connected to the sixth ordinary convolution;

[0014] The output end of the sixth ordinary convolution and the output end of the tenth ordinary convolution are connected to the sixth fusion module, and the output end of the sixth fusion module is connected to the eleventh ordinary convolution.

[0015] Furthermore, the IntergraDet module includes a first shared convolutional layer, the output end of the first shared convolutional layer is sequentially connected to the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module, and the output end of the feature fusion module is respectively connected to the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module;

[0016] The output of the mask and offset generation module and the output of the first task decomposition module are both connected to the deformable convolution V2 module;

[0017] The output end of the second task decomposition module and the output end of the first activation function module are both connected to the feature multiplication and fusion module;

[0018] The output ends of the deformable convolution V2 module are connected to the first classification convolution module, the second classification convolution module, and the third classification convolution module respectively;

[0019] The output end of the feature multiplication and fusion module is connected to the first regression convolution module, the second regression convolution module, and the third regression convolution module respectively. The output end of the first regression convolution module is connected to the first scale adjustment module, the output end of the second regression convolution module is connected to the second scale adjustment module, and the output end of the third regression convolution module is connected to the third scale adjustment module.

[0020] Furthermore, the HDMS multiple weed target detection model includes a feature fusion network and a detection network;

[0021] The feature fusion network includes a first upsampling module, the output end of the C2PSA module is connected to the first upsampling module, the output end of the first upsampling module and the output end of the fourth PC-MSFA module are connected to the first fusion module, the output end of the first fusion module is sequentially connected to the fifth PC-MSFA module and the second upsampling module, the output end of the second upsampling module and the output end of the second PC-MSFA module are connected to the second fusion module, the output end of the second fusion module is connected to the sixth PC-MSFA module, the output end of the sixth PC-MSFA module is connected to the first ordinary convolution, the output end of the first ordinary convolution and the output end of the fifth PC-MSFA module are connected to the third fusion module, the output end of the third fusion module is sequentially connected to the seventh PC-MSFA module and the second ordinary convolution, the output end of the second ordinary convolution and the output end of the first upsampling module are connected to the fourth fusion module, and the output end of the fourth fusion module is connected to the eighth PC-MSFA module;

[0022] The output end of the sixth PC-MSFA module, the output end of the seventh PC-MSFA module, and the output end of the eighth PC-MSFA module are all connected to the Intergradet module;

[0023] Among them, the first PC-MSFA module, the second PC-MSFA module, the third PC-MSFA module, the fourth PC-MSFA module, the fifth PC-MSFA module, the sixth PC-MSFA module, the seventh PC-MSFA module, and the eighth PC-MSFA module have the same structure.

[0024] Furthermore, the processing flow of the HDMS multiple weed target detection model is as follows:

[0025] First, the image data set is input into the SRFD module for processing to obtain the output of the SRFD module, and the output of the SRFD module is sequentially input into the first PC-MSFA module, the first DRFD module, and the second PC-MSFA module for processing to obtain the output F1 of the second PC-MSFA module;

[0026] The output F1 of the second PC-MSFA module is sequentially input into the second DRFD module, the third PC-MSFA module, the third DRFD module, and the fourth PC-MSFA module for processing to obtain the output F2 of the fourth PC-MSFA module;

[0027] The output F2 of the fourth PC-MSFA module is sequentially input into the SPPF module and the C2PSA module for processing to obtain the output F3 of the C2PSA module;

[0028] Input the output F3 of the C2PSA module into the first upsampling module for processing to obtain the output of the first upsampling module, input the output of the first upsampling module and the output F2 of the fourth PC-MSFA module into the first fusion module for processing to obtain the output C1 of the first fusion module;

[0029] Input the output C1 of the first fusion module into the fifth PC-MSFA module for processing to obtain the output C2 of the fifth PC-MSFA module;

[0030] Inputting the output C2 of the fifth PC-MSFA module into the second upsampling module for processing to obtain the output of the second upsampling module, inputting the output of the second upsampling module and the output F1 of the second PC-MSFA module together into the second fusion module to obtain the output C3 of the second fusion module;

[0031] Input the output C3 of the second fusion module into the sixth PC-MSFA module for processing to obtain the output C4 of the sixth PC-MSFA module;

[0032] Input the output C4 of the sixth PC-MSFA module into the first ordinary convolution for processing to obtain the output of the first ordinary convolution. Input the output of the first ordinary convolution and the output C2 of the fifth PC-MSFA module into the third fusion module for processing to obtain the output C5 of the third fusion module.

[0033] Input the output C5 of the third fusion module into the seventh PC-MSFA module for processing to obtain the output C6 of the seventh PC-MSFA module;

[0034] Input the output C6 of the seventh PC-MSFA module into the second ordinary convolution for processing to obtain the output of the second ordinary convolution, input the output of the second ordinary convolution together with the output of the first upsampling module into the fourth fusion module for processing to obtain the output C7 of the fourth fusion module;

[0035] Input the output C7 of the fourth fusion module into the eighth PC-MSFA module for processing to obtain the output C8 of the eighth PC-MSFA module;

[0036] The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are input into the IntergraDet module for processing to obtain the output of the IntergraDet module. The output of the IntergraDet module corresponds to the detection of large, medium, and small scales respectively.

[0037] Further, the processing flow of the IntergraDet module:

[0038] The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are sequentially input into the first shared convolutional layer, the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module in the IntergraDet module for processing to obtain joint features;

[0039] The joint features are respectively input into the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module for processing, and the output of the mask and offset generation module, the output of the first task decomposition module, the output of the second task decomposition module, and the output of the first activation function module are respectively obtained. The output of the mask and offset generation module and the output of the first task decomposition module are input into the deformable convolution V2 module together for processing to obtain the output of the deformable convolution V2 module;

[0040] Input the output of the second task decomposition module and the output of the first activation function module into the feature multiplication and fusion module for processing to obtain the output of the feature multiplication and fusion module;

[0041] The output of the deformable convolution V2 module is respectively input into the first classification convolution module, the second classification convolution module, and the third classification convolution module for processing to obtain the output of the first classification convolution module, the output of the second classification convolution module, and the output of the third classification convolution module respectively;

[0042] The output of the feature multiplication and fusion module is respectively input into the first regression convolution module, the second regression convolution module, and the third regression convolution module for processing to obtain the output of the first regression convolution module, the output of the second regression convolution module, and the output of the third regression convolution module. The output of the first regression convolution module is input into the first scale adjustment module for processing to obtain the output of the first scale adjustment module; the output of the second regression convolution module is input into the second scale adjustment module for processing to obtain the output of the second scale adjustment module; the output of the third regression convolution module is input into the third scale adjustment module for processing to obtain the output of the third scale adjustment module;

[0043] The output of the first classification convolution module and the output of the first scale adjustment module are used as the target position and category of large-scale detection of the IntergraDet module, i.e., large-scale detection;

[0044] The output of the second classification convolution module and the output of the second scale adjustment module are used as the target position and category of the mesoscale detection of the IntergraDet module, i.e., mesoscale detection;

[0045] The output of the third classification convolution module and the output of the third scale adjustment module are used as the target position and category of the small-scale detection of the IntergraDet module, that is, small-scale detection.

[0046] Furthermore, the processing flow of the second PC-MSFA module is as follows:

[0047] The output of the first DRFD module is filtered through a binary gating matrix mechanism to obtain a filtered output; the filtered output is input into global average pooling, global maximum pooling, and the third ordinary convolution for processing to obtain the output of global average pooling, the output of global maximum pooling, and the output of the third ordinary convolution;

[0048] The output of the global average pooling and the output of the global maximum pooling are input into the seventh fusion module for processing to obtain the output of the seventh fusion module, the output of the seventh fusion module is input into the Sigmoid activation function for processing to obtain the output of the Sigmoid activation function, the filtered output and the output of the Sigmoid activation function are input into the eighth fusion module for processing to obtain the output of the eighth fusion module, the output of the eighth fusion module is input into the tenth ordinary convolution for processing to obtain the output of the tenth ordinary convolution;

[0049] Input the output of the third ordinary convolution into the fourth ordinary convolution for processing to obtain the output of the fourth ordinary convolution, input the output of the fourth ordinary convolution into the fifth ordinary convolution for processing to obtain the output of the fifth ordinary convolution;

[0050] The output of the third ordinary convolution, the output of the fourth ordinary convolution, and the output of the fifth ordinary convolution are input together into the fifth fusion module for processing to obtain the output of the fifth fusion module. The output of the fifth fusion module is input into the sixth ordinary convolution for processing to obtain the output of the sixth ordinary convolution. The output of the sixth ordinary convolution and the output of the tenth ordinary convolution are input into the sixth fusion module for processing to obtain the output of the sixth fusion module. The output of the sixth fusion module is input into the eleventh ordinary convolution for processing to obtain the output of the eleventh ordinary convolution, which is the output F1 of the second PC-MSFA module.

[0051] Furthermore, the binary gating matrix is ​​represented as:

[0052] ;

[0053] Where, It represents the gating value of the i-th channel, and the filtered output is obtained by filtering the output of the first DRFD module through the binary gating matrix mechanism; Represents weight The i-th feature in the output of the first DRFD module The dot product of Represents the threshold parameter.

[0054] Furthermore, the C2PSA module includes a first CBS module, the output end of the first CBS module is connected to the feature segmentation module, the output end of the feature segmentation module is connected to multiple superimposed point-level spatial attention modules, the output end of the point-level spatial attention module and the output end of the feature segmentation module are connected to the feature splicing module, and the output end of the feature splicing module is connected to the second CBS module;

[0055] The processing flow of the C2PSA module is as follows:

[0056] The output of the SPPF module is first input into the first CBS module and the feature segmentation module in the C2PSA module for processing in sequence to obtain the output of the feature segmentation module. The output of the feature segmentation module is input into multiple superimposed point-level spatial attention modules for stacking processing to obtain the output of the point-level spatial attention module. The output of the point-level spatial attention module and the output of the feature segmentation module are input into the feature splicing module together for processing to obtain the output of the feature splicing module. The output of the feature splicing module is input into the second CBS module for processing to obtain the output of the second CBS module, which is the output F3 of the C2PSA module.

[0057] Furthermore, the first activation function module is composed of a seventh ordinary convolution, a nonlinear activation function module, an eighth ordinary convolution and a Sigmoid activation function connected in sequence;

[0058] The processing flow of the first activation function module:

[0059] The output of the feature fusion module is sequentially input into the seventh ordinary convolution, the nonlinear activation function module, the eighth ordinary convolution and the Sigmoid activation function in the first activation function module for processing to obtain the output of the Sigmoid activation function, which is the output of the first activation function module.

[0060] A weed identification system based on hierarchical multi-scale feature fusion is used to implement a weed identification method based on hierarchical multi-scale feature fusion, comprising:

[0061] Data construction module, used to construct image dataset; the image dataset is obtained by dividing the Cropandweed dataset;

[0062] A basic model building module for building an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module;

[0063] A training module is used to input the image dataset into the constructed HDMS multiple weed target detection model for training, thereby obtaining a trained HDMS multiple weed target detection model;

[0064] The recognition module is used to identify weeds in farmland using the trained HDMS multiple weed target detection model.

[0065] Compared with existing technologies, the present invention has the following advantages: First, the present invention builds an HDMS multiple weed target detection model, and improves the specific task of weed identification in farmland through three modules:

[0066] First: The present invention replaces the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, which significantly enhances the multi-scale target perception capability.

[0067] Second: The PC-MSFA module is used to improve the feature extraction network and feature fusion network. The PC-MSFA module uses a binary gated matrix processing method to learn diverse feature representations and enhance features, effectively enhancing the HDMS multiple weed target detection model's ability to detect weeds in farmland.

[0068] Third: The detection network learns task interaction features from multiple convolutional layers to obtain joint features, thereby improving the detection network's ability to identify and locate targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Flow chart of the method of the present invention.

[0070] Figure 2 This is a flow chart of the HDMS multiple weed target detection model of the present invention.

[0071] Figure 3 Schematic diagram of the PC-MSFA module of the present invention.

[0072] Figure 4 This is the C2PSA module flow chart of the present invention.

[0073] Figure 5 This is a flow chart of the first activation function module of the present invention.

[0074] Figure 6 This is the flow chart of the IntergraDet module of the present invention.

[0075] Figure 7 This is a flow chart of the second CBS module of the present invention. DETAILED DESCRIPTION

[0076] like Figure 1 As shown, the present invention provides a technical solution: a weed identification method based on hierarchical multi-scale feature fusion, comprising the following steps:

[0077] Step S1: Construct an image dataset; the image dataset is obtained by dividing the Crop and Weed dataset;

[0078] Step S2: constructing an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module;

[0079] Step S3: inputting the image dataset into the constructed HDMS multiple weed target detection model for training to obtain a trained HDMS multiple weed target detection model;

[0080] Step S4: Use the trained HDMS multiple weed target detection model to identify weeds in the farmland.

[0081] The specific process of step S1 is as follows:

[0082] The image dataset is divided into training set, validation set and test set; the training set contains 5393 images; the validation set contains 770 images; and the test set contains 1542 images. The HDMS multiple weed target detection model training rounds are set to 300 rounds, the batch training size is 16, and the optimizer uses the SGD optimizer. To prevent the HDMS multiple weed target detection model from overfitting, the weight decay is set to 0.0005, the confidence of the HDMS multiple weed target detection model is set to 0.7, and the initial learning rate is set to 0.01.

[0083] like Figure 2 As shown in the figure, the HDMS multiple weed target detection model consists of a feature extraction network, a feature fusion network, and a detection network;

[0084] The feature extraction network includes an SRFD module, and the output end of the SRFD module is sequentially connected to the first PC-MSFA module, the first DRFD module, the second PC-MSFA module, the second DRFD module, the third PC-MSFA module, the third DRFD module, the fourth PC-MSFA module, the SPPF module, and the C2PSA module;

[0085] Among them, the SRFD module is (Shallow Robust Feature Downsampling Shallow Robust Feature Downsampling module);

[0086] Among them, the DRFD module is (Deep Robust Feature Downsampling deep robust feature downsampling module);

[0087] Wherein, the first DRFD module, the second DRFD module and the third DRFD module have the same structure;

[0088] Among them, the PC-MSFA module is (Partial Convolution based Multi-Scale Feature Aggregation based on partial convolution multi-scale feature aggregation module);

[0089] The feature fusion network includes a first upsampling module, the output end of the C2PSA module is connected to the first upsampling module, the output end of the first upsampling module and the output end of the fourth PC-MSFA module are connected to the first fusion module, the output end of the first fusion module is sequentially connected to the fifth PC-MSFA module and the second upsampling module, the output end of the second upsampling module and the output end of the second PC-MSFA module are connected to the second fusion module, the output end of the second fusion module is connected to the sixth PC-MSFA module, the output end of the sixth PC-MSFA module is connected to the first ordinary convolution, the output end of the first ordinary convolution and the output end of the fifth PC-MSFA module are connected to the third fusion module, the output end of the third fusion module is sequentially connected to the seventh PC-MSFA module and the second ordinary convolution, the output end of the second ordinary convolution and the output end of the first upsampling module are connected to the fourth fusion module, and the output end of the fourth fusion module is connected to the eighth PC-MSFA module;

[0090] The IntergraDet module is (Integration-based Detection Head based on integration detection head);

[0091] The output end of the sixth PC-MSFA module, the output end of the seventh PC-MSFA module, and the output end of the eighth PC-MSFA module are all connected to the Intergradet module.

[0092] Among them, the processing flow of the HDMS multiple weed target detection model is as follows:

[0093] First, the image data set is input into the SRFD module for processing to obtain the output of the SRFD module, and the output of the SRFD module is sequentially input into the first PC-MSFA module, the first DRFD module, and the second PC-MSFA module for processing to obtain the output F1 of the second PC-MSFA module;

[0094] The output F1 of the second PC-MSFA module is sequentially input into the second DRFD module, the third PC-MSFA module, the third DRFD module, and the fourth PC-MSFA module for processing to obtain the output F2 of the fourth PC-MSFA module;

[0095] The output F2 of the fourth PC-MSFA module is sequentially input into the SPPF module and the C2PSA module for processing to obtain the output F3 of the C2PSA module;

[0096] Input the output F3 of the C2PSA module into the first upsampling module for processing to obtain the output of the first upsampling module, input the output of the first upsampling module and the output F2 of the fourth PC-MSFA module into the first fusion module for processing to obtain the output C1 of the first fusion module;

[0097] Input the output C1 of the first fusion module into the fifth PC-MSFA module for processing to obtain the output C2 of the fifth PC-MSFA module;

[0098] Inputting the output C2 of the fifth PC-MSFA module into the second upsampling module for processing to obtain the output of the second upsampling module, inputting the output of the second upsampling module and the output F1 of the second PC-MSFA module together into the second fusion module to obtain the output C3 of the second fusion module;

[0099] Input the output C3 of the second fusion module into the sixth PC-MSFA module for processing to obtain the output C4 of the sixth PC-MSFA module;

[0100] Input the output C4 of the sixth PC-MSFA module into the first ordinary convolution for processing to obtain the output of the first ordinary convolution. Input the output of the first ordinary convolution and the output C2 of the fifth PC-MSFA module into the third fusion module for processing to obtain the output C5 of the third fusion module.

[0101] Input the output C5 of the third fusion module into the seventh PC-MSFA module for processing to obtain the output C6 of the seventh PC-MSFA module;

[0102] Input the output C6 of the seventh PC-MSFA module into the second ordinary convolution for processing to obtain the output of the second ordinary convolution, input the output of the second ordinary convolution together with the output of the first upsampling module into the fourth fusion module for processing to obtain the output C7 of the fourth fusion module;

[0103] Input the output C7 of the fourth fusion module into the eighth PC-MSFA module for processing to obtain the output C8 of the eighth PC-MSFA module;

[0104] The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are input into the IntergraDet module for processing to obtain the output of the IntergraDet module. The output of the IntergraDet module corresponds to the detection of large, medium, and small scales respectively.

[0105] like Figure 3 As shown, the first PC-MSFA module, the second PC-MSFA module, the third PC-MSFA module, the fourth PC-MSFA module, the fifth PC-MSFA module, the sixth PC-MSFA module, the seventh PC-MSFA module, and the eighth PC-MSFA module have the same structure;

[0106] The second PC-MSFA module includes global average pooling, global maximum pooling, and a third ordinary convolution; the output end of the first DRFD module is connected to the global average pooling, the global maximum pooling, and the third ordinary convolution respectively;

[0107] The output end of the global average pooling and the output end of the global maximum pooling are connected to the seventh fusion module, the output end of the seventh fusion module is connected to the Sigmoid activation function, the output end of the Sigmoid activation function and the output end of the first DRFD module are connected to the eighth fusion module, and the output end of the eighth fusion module is connected to the tenth ordinary convolution;

[0108] The output end of the third ordinary convolution is connected to the fourth ordinary convolution and the fifth fusion module respectively, the output end of the fourth ordinary convolution is connected to the fifth ordinary convolution and the fifth fusion module respectively, the output end of the fifth ordinary convolution is connected to the fifth fusion module, and the output end of the fifth fusion module is connected to the sixth ordinary convolution;

[0109] The output end of the sixth ordinary convolution and the output end of the tenth ordinary convolution are connected to the sixth fusion module, and the output end of the sixth fusion module is connected to the eleventh ordinary convolution;

[0110] The third, fourth, and fifth ordinary convolutions are all k×k ordinary convolutions, where k is the size of the convolution kernel.

[0111] The sixth ordinary convolution, the tenth ordinary convolution, and the eleventh ordinary convolution are 1×1 ordinary convolutions;

[0112] The sixth fusion module is the feature splicing module;

[0113] Processing flow of the second PC-MSFA module:

[0114] The output of the first DRFD module is filtered through a binary gating matrix mechanism to obtain a filtered output; the filtered output is input into global average pooling, global maximum pooling, and the third ordinary convolution for processing to obtain the output of global average pooling, the output of global maximum pooling, and the output of the third ordinary convolution;

[0115] The output of the global average pooling and the output of the global maximum pooling are input into the seventh fusion module for processing to obtain the output of the seventh fusion module, the output of the seventh fusion module is input into the Sigmoid activation function for processing to obtain the output of the Sigmoid activation function, the filtered output and the output of the Sigmoid activation function are input into the eighth fusion module for processing to obtain the output of the eighth fusion module, the output of the eighth fusion module is input into the tenth ordinary convolution for processing to obtain the output of the tenth ordinary convolution;

[0116] Input the output of the third ordinary convolution into the fourth ordinary convolution for processing to obtain the output of the fourth ordinary convolution, input the output of the fourth ordinary convolution into the fifth ordinary convolution for processing to obtain the output of the fifth ordinary convolution;

[0117] The output of the third ordinary convolution, the output of the fourth ordinary convolution, and the output of the fifth ordinary convolution are input together into the fifth fusion module for processing to obtain the output of the fifth fusion module. The output of the fifth fusion module is input into the sixth ordinary convolution for processing to obtain the output of the sixth ordinary convolution. The output of the sixth ordinary convolution and the output of the tenth ordinary convolution are input into the sixth fusion module for processing to obtain the output of the sixth fusion module. The output of the sixth fusion module is input into the eleventh ordinary convolution for processing to obtain the output of the eleventh ordinary convolution, which is the output F1 of the second PC-MSFA module.

[0118] like Figure 4 As shown, the C2PSA module includes a first CBS module, the output end of the first CBS module is connected to the feature segmentation module, the output end of the feature segmentation module is connected to multiple superimposed point-level spatial attention modules, the output end of the point-level spatial attention module and the output end of the feature segmentation module are connected to the feature splicing module, and the output end of the feature splicing module is connected to the second CBS module;

[0119] The processing flow of the C2PSA module is as follows:

[0120] The output of the SPPF module is first input into the first CBS module and the feature segmentation module in the C2PSA module for processing in sequence to obtain the output of the feature segmentation module. The output of the feature segmentation module is input into multiple superimposed point-level spatial attention modules for stacking processing to obtain the output of the point-level spatial attention module. The output of the point-level spatial attention module and the output of the feature segmentation module are input into the feature splicing module together for processing to obtain the output of the feature splicing module. The output of the feature splicing module is input into the second CBS module for processing to obtain the output of the second CBS module, which is the output F3 of the C2PSA module.

[0121] like Figure 5 As shown, the first activation function module is composed of the seventh ordinary convolution, the nonlinear activation function module, the eighth ordinary convolution and the Sigmoid activation function connected in sequence;

[0122] The processing flow of the first activation function module:

[0123] The output of the feature fusion module is sequentially input into the seventh ordinary convolution, the nonlinear activation function module, the eighth ordinary convolution and the Sigmoid activation function in the first activation function module for processing to obtain the output of the Sigmoid activation function, which is the output of the first activation function module.

[0124] Among them, the binary gating matrix represents:

[0125] ;

[0126] Where, It represents the gating value of the i-th channel, and the filtered output is obtained by filtering the output of the first DRFD module through the binary gating matrix mechanism; Represents weight The i-th feature in the output of the first DRFD module The dot product of Represents the threshold parameter.

[0127] like Figure 6 As shown, the IntergraDet module includes a first shared convolutional layer, the output end of the first shared convolutional layer is sequentially connected to the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module, and the output end of the feature fusion module is respectively connected to the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module;

[0128] The output of the mask and offset generation module and the output of the first task decomposition module are both connected to the deformable convolution V2 module;

[0129] The output end of the second task decomposition module and the output end of the first activation function module are both connected to the feature multiplication and fusion module;

[0130] The output ends of the deformable convolution V2 module are connected to the first classification convolution module, the second classification convolution module, and the third classification convolution module respectively;

[0131] The output end of the feature multiplication and fusion module is connected to the first regression convolution module, the second regression convolution module, and the third regression convolution module respectively. The output end of the first regression convolution module is connected to the first scale adjustment module, the output end of the second regression convolution module is connected to the second scale adjustment module, and the output end of the third regression convolution module is connected to the third scale adjustment module.

[0132] The processing flow of the IntergraDet module:

[0133] The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are sequentially input into the first shared convolutional layer, the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module in the IntergraDet module for processing to obtain joint features;

[0134] The joint features are respectively input into the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module for processing, and the output of the mask and offset generation module, the output of the first task decomposition module, the output of the second task decomposition module, and the output of the first activation function module are respectively obtained. The output of the mask and offset generation module and the output of the first task decomposition module are input into the deformable convolution V2 module together for processing to obtain the output of the deformable convolution V2 module;

[0135] Input the output of the second task decomposition module and the output of the first activation function module into the feature multiplication and fusion module for processing to obtain the output of the feature multiplication and fusion module;

[0136] The output of the deformable convolution V2 module is respectively input into the first classification convolution module, the second classification convolution module, and the third classification convolution module for processing to obtain the output of the first classification convolution module, the output of the second classification convolution module, and the output of the third classification convolution module respectively;

[0137] The output of the feature multiplication and fusion module is respectively input into the first regression convolution module, the second regression convolution module, and the third regression convolution module for processing to obtain the output of the first regression convolution module, the output of the second regression convolution module, and the output of the third regression convolution module. The output of the first regression convolution module is input into the first scale adjustment module for processing to obtain the output of the first scale adjustment module; the output of the second regression convolution module is input into the second scale adjustment module for processing to obtain the output of the second scale adjustment module; the output of the third regression convolution module is input into the third scale adjustment module for processing to obtain the output of the third scale adjustment module;

[0138] The output of the first classification convolution module and the output of the first scale adjustment module are used as the target position and category of large-scale detection of the IntergraDet module, i.e., large-scale detection;

[0139] The output of the second classification convolution module and the output of the second scale adjustment module are used as the target position and category of the mesoscale detection of the IntergraDet module, i.e., mesoscale detection;

[0140] The output of the third classification convolution module and the output of the third scale adjustment module are used as the target position and category of the small-scale detection of the IntergraDet module, that is, small-scale detection.

[0141] like Figure 7 As shown, the second CBS module includes a ninth ordinary convolution, a batch normalization, and a second activation function module connected in sequence;

[0142] Wherein, the first CBS module and the second CBS module have the same structure;

[0143] Wherein, the first activation function module and the second activation function module have the same structure;

[0144] The processing flow of the second CBS module:

[0145] The output of the feature splicing module is sequentially input into the ninth ordinary convolution, batch normalization, and the second activation function module in the second CBS module for processing to obtain the output of the second activation function module, which is the output of the second CBS module.

[0146] The deep learning environment used in this example is: NVIDIA GeFore RTX 2080 Ti graphics card with 11.2G video memory;

[0147] Visual Studio Code 2019 is used as the development environment; CUDA 12.2 is used as the deep learning computing framework; Pytorch 2.2.2 is used as the deep learning framework, and the assembly language is Python;

[0148] The trained HDMS multi-weed target detection model was evaluated using the mean average precision (mAP), a common target detection model evaluation metric, and compared with the YOLOv11 model, as shown in Table 1. In Table 1, the HRFN module stands for Hierarchical Robust Feature Network; a check mark indicates that this module is selected.

[0149] Table 1 Comparison of the basic model yolo11 model and the trained HDMS multiple weed target detection model

[0150]

[0151] As can be seen from the results in Table 1 above, compared with the YOLO11 model, the HDMS multiple weed target detection model of the present invention has a relatively significant improvement in detection accuracy. The HDMS multiple weed target detection model has fewer parameters, which greatly improves the algorithm recognition efficiency.

[0152] A weed identification system based on hierarchical multi-scale feature fusion is used to implement a weed identification method based on hierarchical multi-scale feature fusion, comprising:

[0153] Data construction module, used to construct image dataset; the image dataset is obtained by dividing the Cropandweed dataset;

[0154] A basic model building module for building an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module;

[0155] A training module is used to input the image dataset into the constructed HDMS multiple weed target detection model for training, thereby obtaining a trained HDMS multiple weed target detection model;

[0156] The recognition module is used to identify weeds in farmland using the trained HDMS multiple weed target detection model.

[0157] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A weed recognition method based on hierarchical multi-scale feature fusion, characterized in that: The following steps are involved: Step S1: construct an image dataset; The image dataset is obtained by dividing the Cropandweed dataset; Step S2: constructing an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module; Step S3: inputting the image dataset into the constructed HDMS multiple weed target detection model for training to obtain a trained HDMS multiple weed target detection model; Step S4: using the trained HDMS multiple weed target detection model to identify weeds in the farmland; The feature extraction network includes an SRFD module, and the output end of the SRFD module is sequentially connected to the first PC-MSFA module, the first DRFD module, the second PC-MSFA module, the second DRFD module, the third PC-MSFA module, the third DRFD module, the fourth PC-MSFA module, the SPPF module, and the C2PSA module; The second PC-MSFA module includes global average pooling, global maximum pooling, and a third ordinary convolution; the output end of the first DRFD module is connected to the global average pooling, the global maximum pooling, and the third ordinary convolution respectively; The output end of the global average pooling and the output end of the global maximum pooling are connected to the seventh fusion module, the output end of the seventh fusion module is connected to the Sigmoid activation function, the output end of the Sigmoid activation function and the output end of the first DRFD module are connected to the eighth fusion module, and the output end of the eighth fusion module is connected to the tenth ordinary convolution; The output end of the third ordinary convolution is connected to the fourth ordinary convolution and the fifth fusion module respectively, the output end of the fourth ordinary convolution is connected to the fifth ordinary convolution and the fifth fusion module respectively, the output end of the fifth ordinary convolution is connected to the fifth fusion module, and the output end of the fifth fusion module is connected to the sixth ordinary convolution; The output end of the sixth ordinary convolution and the output end of the tenth ordinary convolution are connected to the sixth fusion module, and the output end of the sixth fusion module is connected to the eleventh ordinary convolution.

2. The weed identification method based on hierarchical multi-scale feature fusion according to claim 1, characterized in that: The IntergraDet module includes a first shared convolutional layer, the output end of the first shared convolutional layer is sequentially connected to the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module, and the output end of the feature fusion module is respectively connected to the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module; The output of the mask and offset generation module and the output of the first task decomposition module are both connected to the deformable convolution V2 module; The output end of the second task decomposition module and the output end of the first activation function module are both connected to the feature multiplication and fusion module; The output ends of the deformable convolution V2 module are connected to the first classification convolution module, the second classification convolution module, and the third classification convolution module respectively; The output end of the feature multiplication and fusion module is connected to the first regression convolution module, the second regression convolution module, and the third regression convolution module respectively. The output end of the first regression convolution module is connected to the first scale adjustment module, the output end of the second regression convolution module is connected to the second scale adjustment module, and the output end of the third regression convolution module is connected to the third scale adjustment module.

3. The weed identification method based on hierarchical multi-scale feature fusion according to claim 2, characterized in that: The HDMS multiple weed target detection model includes a feature fusion network and a detection network; The feature fusion network includes a first upsampling module, the output end of the C2PSA module is connected to the first upsampling module, the output end of the first upsampling module and the output end of the fourth PC-MSFA module are connected to the first fusion module, the output end of the first fusion module is sequentially connected to the fifth PC-MSFA module and the second upsampling module, the output end of the second upsampling module and the output end of the second PC-MSFA module are connected to the second fusion module, the output end of the second fusion module is connected to the sixth PC-MSFA module, the output end of the sixth PC-MSFA module is connected to the first ordinary convolution, the output end of the first ordinary convolution and the output end of the fifth PC-MSFA module are connected to the third fusion module, the output end of the third fusion module is sequentially connected to the seventh PC-MSFA module and the second ordinary convolution, the output end of the second ordinary convolution and the output end of the first upsampling module are connected to the fourth fusion module, and the output end of the fourth fusion module is connected to the eighth PC-MSFA module; The output end of the sixth PC-MSFA module, the output end of the seventh PC-MSFA module, and the output end of the eighth PC-MSFA module are all connected to the Intergradet module; Among them, the first PC-MSFA module, the second PC-MSFA module, the third PC-MSFA module, the fourth PC-MSFA module, the fifth PC-MSFA module, the sixth PC-MSFA module, the seventh PC-MSFA module, and the eighth PC-MSFA module have the same structure.

4. The weed identification method based on hierarchical multi-scale feature fusion according to claim 3, characterized in that: The processing flow of the HDMS multiple weed target detection model: First, the image data set is input into the SRFD module for processing to obtain the output of the SRFD module, and the output of the SRFD module is sequentially input into the first PC-MSFA module, the first DRFD module, and the second PC-MSFA module for processing to obtain the output F1 of the second PC-MSFA module; The output F1 of the second PC-MSFA module is sequentially input into the second DRFD module, the third PC-MSFA module, the third DRFD module, and the fourth PC-MSFA module for processing to obtain the output F2 of the fourth PC-MSFA module; The output F2 of the fourth PC-MSFA module is sequentially input into the SPPF module and the C2PSA module for processing to obtain the output F3 of the C2PSA module; Input the output F3 of the C2PSA module into the first upsampling module for processing to obtain the output of the first upsampling module, input the output of the first upsampling module and the output F2 of the fourth PC-MSFA module into the first fusion module for processing to obtain the output C1 of the first fusion module; Input the output C1 of the first fusion module into the fifth PC-MSFA module for processing to obtain the output C2 of the fifth PC-MSFA module; Inputting the output C2 of the fifth PC-MSFA module into the second upsampling module for processing to obtain the output of the second upsampling module, inputting the output of the second upsampling module and the output F1 of the second PC-MSFA module together into the second fusion module to obtain the output C3 of the second fusion module; Input the output C3 of the second fusion module into the sixth PC-MSFA module for processing to obtain the output C4 of the sixth PC-MSFA module; Input the output C4 of the sixth PC-MSFA module into the first ordinary convolution for processing to obtain the output of the first ordinary convolution. Input the output of the first ordinary convolution and the output C2 of the fifth PC-MSFA module into the third fusion module for processing to obtain the output C5 of the third fusion module. Input the output C5 of the third fusion module into the seventh PC-MSFA module for processing to obtain the output C6 of the seventh PC-MSFA module; Input the output C6 of the seventh PC-MSFA module into the second ordinary convolution for processing to obtain the output of the second ordinary convolution, input the output of the second ordinary convolution together with the output of the first upsampling module into the fourth fusion module for processing to obtain the output C7 of the fourth fusion module; Input the output C7 of the fourth fusion module into the eighth PC-MSFA module for processing to obtain the output C8 of the eighth PC-MSFA module; The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are input into the IntergraDet module for processing to obtain the output of the IntergraDet module. The output of the IntergraDet module corresponds to the detection of large, medium, and small scales respectively.

5. The weed identification method based on hierarchical multi-scale feature fusion according to claim 4, characterized in that: The processing flow of the IntergraDet module: The output C4 of the sixth PC-MSFA module, the output C6 of the seventh PC-MSFA module, and the output C8 of the eighth PC-MSFA module are sequentially input into the first shared convolutional layer, the first normalization layer, the second shared convolutional layer, the second normalization layer, and the feature fusion module in the IntergraDet module for processing to obtain joint features; The joint features are respectively input into the mask and offset generation module, the first task decomposition module, the second task decomposition module, and the first activation function module for processing, and the output of the mask and offset generation module, the output of the first task decomposition module, the output of the second task decomposition module, and the output of the first activation function module are respectively obtained. The output of the mask and offset generation module and the output of the first task decomposition module are input into the deformable convolution V2 module together for processing to obtain the output of the deformable convolution V2 module; Input the output of the second task decomposition module and the output of the first activation function module into the feature multiplication and fusion module for processing to obtain the output of the feature multiplication and fusion module; The output of the deformable convolution V2 module is respectively input into the first classification convolution module, the second classification convolution module, and the third classification convolution module for processing to obtain the output of the first classification convolution module, the output of the second classification convolution module, and the output of the third classification convolution module respectively; The output of the feature multiplication and fusion module is respectively input into the first regression convolution module, the second regression convolution module, and the third regression convolution module for processing to obtain the output of the first regression convolution module, the output of the second regression convolution module, and the output of the third regression convolution module. The output of the first regression convolution module is input into the first scale adjustment module for processing to obtain the output of the first scale adjustment module; the output of the second regression convolution module is input into the second scale adjustment module for processing to obtain the output of the second scale adjustment module; the output of the third regression convolution module is input into the third scale adjustment module for processing to obtain the output of the third scale adjustment module; The output of the first classification convolution module and the output of the first scale adjustment module are used as the target position and category of large-scale detection of the IntergraDet module, i.e., large-scale detection; The output of the second classification convolution module and the output of the second scale adjustment module are used as the target position and category of the mesoscale detection of the IntergraDet module, i.e., mesoscale detection; The output of the third classification convolution module and the output of the third scale adjustment module are used as the target position and category of the small-scale detection of the IntergraDet module, that is, small-scale detection.

6. The weed identification method based on hierarchical multi-scale feature fusion according to claim 5, characterized in that: Processing flow of the second PC-MSFA module: The output of the first DRFD module is filtered through a binary gating matrix mechanism to obtain a filtered output; the filtered output is input into global average pooling, global maximum pooling, and the third ordinary convolution for processing to obtain the output of global average pooling, the output of global maximum pooling, and the output of the third ordinary convolution; The output of the global average pooling and the output of the global maximum pooling are input into the seventh fusion module for processing to obtain the output of the seventh fusion module, the output of the seventh fusion module is input into the Sigmoid activation function for processing to obtain the output of the Sigmoid activation function, the filtered output and the output of the Sigmoid activation function are input into the eighth fusion module for processing to obtain the output of the eighth fusion module, the output of the eighth fusion module is input into the tenth ordinary convolution for processing to obtain the output of the tenth ordinary convolution; Input the output of the third ordinary convolution into the fourth ordinary convolution for processing to obtain the output of the fourth ordinary convolution, input the output of the fourth ordinary convolution into the fifth ordinary convolution for processing to obtain the output of the fifth ordinary convolution; The output of the third ordinary convolution, the output of the fourth ordinary convolution, and the output of the fifth ordinary convolution are input together into the fifth fusion module for processing to obtain the output of the fifth fusion module. The output of the fifth fusion module is input into the sixth ordinary convolution for processing to obtain the output of the sixth ordinary convolution. The output of the sixth ordinary convolution and the output of the tenth ordinary convolution are input into the sixth fusion module for processing to obtain the output of the sixth fusion module. The output of the sixth fusion module is input into the eleventh ordinary convolution for processing to obtain the output of the eleventh ordinary convolution, which is the output F1 of the second PC-MSFA module.

7. The weed identification method based on hierarchical multi-scale feature fusion according to claim 6, characterized in that: A binary gating matrix, representing: ； Where, It represents the gating value of the i-th channel, and the filtered output is obtained by filtering the output of the first DRFD module through the binary gating matrix mechanism; Represents weight The i-th feature in the output of the first DRFD module The dot product of Represents the threshold parameter.

8. The weed identification method based on hierarchical multi-scale feature fusion according to claim 7, characterized in that: The C2PSA module includes a first CBS module, the output end of the first CBS module is connected to the feature segmentation module, the output end of the feature segmentation module is connected to multiple superimposed point-level spatial attention modules, the output end of the point-level spatial attention module and the output end of the feature segmentation module are connected to the feature splicing module, and the output end of the feature splicing module is connected to the second CBS module; The processing flow of the C2PSA module is as follows: The output of the SPPF module is first input into the first CBS module and the feature segmentation module in the C2PSA module for processing in sequence to obtain the output of the feature segmentation module. The output of the feature segmentation module is input into multiple superimposed point-level spatial attention modules for stacking processing to obtain the output of the point-level spatial attention module. The output of the point-level spatial attention module and the output of the feature segmentation module are input into the feature splicing module together for processing to obtain the output of the feature splicing module. The output of the feature splicing module is input into the second CBS module for processing to obtain the output of the second CBS module, which is the output F3 of the C2PSA module.

9. The weed identification method based on hierarchical multi-scale feature fusion according to claim 8, characterized in that: The first activation function module is composed of the seventh ordinary convolution, the nonlinear activation function module, the eighth ordinary convolution and the Sigmoid activation function connected in sequence; The processing flow of the first activation function module: The output of the feature fusion module is sequentially input into the seventh ordinary convolution, the nonlinear activation function module, the eighth ordinary convolution and the Sigmoid activation function in the first activation function module for processing to obtain the output of the Sigmoid activation function, which is the output of the first activation function module.

10. A weed identification system based on hierarchical multi-scale feature fusion, used to implement the weed identification method based on hierarchical multi-scale feature fusion according to any one of claims 1 to 9, characterized in that: include: Data construction module, used to construct image datasets; The image dataset is obtained by dividing the Cropandweed dataset; A basic model building module for building an HDMS multiple weed target detection model; the HDMS multiple weed target detection model is obtained by replacing the C3k2 module in the feature extraction network of the YOLO11 model with the PC-MSFA module, and replacing the detection module of the detection network of the YOLO11 model with the IntergraDet module; A training module is used to input the image dataset into the constructed HDMS multiple weed target detection model for training, thereby obtaining a trained HDMS multiple weed target detection model; The recognition module is used to identify weeds in farmland using the trained HDMS multiple weed target detection model.

Images