Green attention and joint feature fused field weed detection method and system

Abstract

The invention discloses a green attention and joint feature fusion field weed detection method and device, and the method comprises the steps: designing a green attention mechanism which is used for enhancing the attention of a weed part, and weakening a non-target region; designing a self-adaptive joint feature fusion mode for weighting a bidirectional feature pyramid fusion architecture, combining a weed color and a texture feature of a bottom layer with high-level semantic information, and extracting individual features of weeds; designing a decoupling head by using a dyhead module, and unifying scale perception attention, space perception attention and task perception attention; using feature content to perceive, recombine and aggregate context information in a large receptive field, and dynamically generating an adaptive kernel according to instance content; predicting a regression frame and positive sample matching of weeds by using a dynamic task alignment strategy and distribution focus loss; and field weeds are detected. The device comprises a processor and a memory. The method is more suitable for real field weed conditions and has higher detection precision.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a field weed detection method and system combining green attention and joint feature fusion. Background Art

[0002] With the rapid development of computer vision and deep learning technologies, weed detection technology has been widely used in smart agriculture, which is of great significance in reducing the use of pesticides, saving resources, protecting the environment and increasing yields. The management of weeds in smart agriculture mainly depends on the detection and identification of weeds. The field environment in real scenes has dynamic, unstructured and various conflicting noise characteristics, which affects the accuracy of weed detection algorithms. In addition, the similarity of weeds between classes, the diversity of morphology of weeds in different growth stages within a class, and the complexity of field soil background, lighting conditions and weather increase the difficulty of weed detection.

[0003] Existing weed detection methods still have the problem of high recognition errors for different types of weeds with similar colors and shapes in complex environments. Summary of the invention

[0004] The present invention provides a field weed detection method and system based on green attention and joint feature fusion. The present invention optimizes and improves YOLOv5 to make it more suitable for the weed conditions in real fields, so that it has higher detection accuracy, better generalization and robustness, and can effectively cope with the challenges of poor prediction effect of similar weeds in complex backgrounds, easy misjudgment of non-detected weeds in the background, inaccurate bounding box regression, and low confidence. See the following description for details:

[0005] In a first aspect, a field weed detection method using green attention and joint feature fusion, the method comprising:

[0006] Design a green attention mechanism to enhance the focus on weeds and weaken non-target areas; design an adaptive joint feature fusion method for a weighted bidirectional feature pyramid fusion architecture to combine the underlying weed color and texture features with high-level semantic information to extract personalized features of weeds;

[0007] The decoupling head is designed using the dyhead module to unify scale-aware attention, space-aware attention and task-aware attention. Feature content-aware reorganization is used to aggregate contextual information within a large receptive field, and an adaptive kernel is dynamically generated based on the instance content. Dynamic task alignment strategy and distribution focus loss are used to predict the regression box and positive sample matching of weeds. Field weeds are detected.

[0008] Among them, the green attention mechanism is designed based on the weed dataset RGB. By introducing two trainable and optimized parameters, the feature maps of the green channel and the red channel are calculated to generate a mask; the mask is multiplied by the input feature map to enhance the green features of the weed area.

[0009] The adaptive joint feature fusion method is as follows: adaptive feature fusion, different input weighted fusion and weighted direct fusion are performed on the input feature map respectively; the three fusion results are spliced ​​to fuse the global and local features of the weeds.

[0010] The decoupling head designed using the dyhead module is composed of two dyhead modules and some convolution blocks. The dyhead module is divided into a scale-aware attention module, a space-aware attention module and a task-aware attention module. The self-attention mechanism is used to respectively detect scale-aware attention, space-aware attention and task-aware attention:

[0011] W(F)=π C (π S (π L (F)·F)·F)·F

[0012] Among them, π L (·),π S (·),π C (·) are the scale-aware attention module, the space-aware attention module, and the task-aware attention module, respectively. Three different attention functions are applied to dimensions L, S, and C.

[0013] The feature content-aware reorganization includes: predicting a reorganization kernel according to the content of each target position, and reorganizing the features using the predicted kernel;

[0014] W l' =ψ(N(c l ,k encoder ))

[0015] X′ l′ =φ(N(c l ,k up ),W l' )

[0016] Among them, the upsampling kernel prediction module ψ is based on χ l The positional upsampling kernel W is predicted for each position l' by using the neighbor l’ , φ is a content-aware reorganization module that transforms adjacent l With kernel W l’ Reorganization.

[0017] In a second aspect, a field weed detection device using green attention and joint feature fusion is provided, the device comprising: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods described in the first aspect.

[0018] A third aspect, a computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes any one of the methods described in the third aspect.

[0019] The beneficial effects of the technical solution provided by the present invention are:

[0020] 1. The green attention mechanism designed by the present invention adjusts the weights adaptively to enhance the attention of the green area (weed part); a new adaptive joint feature fusion method designed by the present invention is used for the weighted bidirectional feature pyramid fusion architecture, which combines the local detail features such as the color and texture of the underlying weeds with the high-level semantic information, and can better extract the personalized features of the weeds;

[0021] 2. The present invention uses the dyhead module to design a decoupling head to unify scale-aware attention, space-aware attention and task-aware attention, so as to better classify and locate weeds; the present invention uses a feature content-aware recombinant upsampling operator to aggregate contextual information within a large receptive field, and dynamically generates an adaptive kernel according to the instance content, so as to better capture the characteristics of weeds;

[0022] 3. The dynamic task alignment strategy and distribution focus loss used in the present invention can more accurately predict the regression box and positive sample matching of weeds. Not only ablation and comparison experiments are carried out on the self-built dataset, but also algorithm comparison is carried out on the larger sample CottonWeedDet12 and the smaller sample 4WEED DATASET dataset. The experimental results and visualization analysis show that compared with the comparison algorithm, the algorithm has better prediction effect on similar weeds in complex backgrounds, more accurate bounding box regression, higher confidence, and better performance in generalization and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The overall flow chart of a field weed detection method based on green attention and joint feature fusion;

[0024] Figure 2 This is a display of 6 kinds of weeds in the self-made dataset;

[0025] Figure 3 A bar chart showing the number of instances of different categories in the self-made dataset;

[0026] Figure 4 This is the network structure diagram of WeedsDetectNet;

[0027] Figure 5 This is the network structure diagram of the green attention module;

[0028] Figure 6 This is the network structure diagram of the feature fusion architecture;

[0029] Figure 7 The network structure diagrams of different fusion methods are shown below;

[0030] Figure 8 The network structure diagram of the feature content-aware reorganization upsampling operator;

[0031] Fig. 9 This is the network structure diagram of the dyhead module;

[0032] Fig.10 The network structure diagram of the designed decoupling head module;

[0033] Fig.11 Schematic diagram of the improved loss function;

[0034] Fig.12 A visual comparison chart of the prediction effects of different algorithms;

[0035] Fig.13 This is a visual comparison of the prediction effects of different heat maps.

[0036] Table 1 shows the comparative experimental results of different algorithms on the self-built dataset;

[0037] Table 2 shows the comparative experimental results of different algorithms on the public dataset CottonWeedDet12;

[0038] Table 3 shows the comparative experimental results of different algorithms on the public dataset 4WEED DATASET. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0040] YOLOv5 is a lightweight, fast and accurate target detection algorithm. It significantly reduces the consumption of computing resources while maintaining high accuracy, achieving a perfect balance between real-time and high efficiency. To further improve the detection effect of field weeds, the attention mechanism is introduced to it. By enhancing the model's attention to different weed feature areas, it helps to more accurately extract the unique features of weeds with similar appearance. In addition, the feature information fusion technology combines low-level local detail information with high-level global semantic information to comprehensively refine the features of different weeds, thereby enhancing the model's ability to learn and identify weed feature information.

[0041] Based on the existing technology, there is still a problem of high recognition error for different types of weeds with similar colors and shapes in complex environments. To address this problem, the embodiment of the present invention improves and designs a new weed detection model WeedsDetectNet based on YOLOv5n. Ablation and comparison experiments are carried out on the self-built dataset, and the algorithm is compared on the larger sample CottonWeedDet12 and the smaller sample 4WEED DATASET dataset. The experimental results and visualization analysis show that compared with the comparison algorithm, the embodiment of the present invention is more accurate in identifying similar weeds and performs better in generalization and robustness.

[0042] Example 1

[0043] like Figure 1 As shown, an embodiment of the present invention provides an overall flow chart of a new method for detecting weeds in farmland in a real scene by integrating green attention and adaptive joint features. The method includes the following steps:

[0044] Step 101: Collect weed pictures, label them using the Labelimg labeling tool, and build a weed dataset with 6 types;

[0045] Among them, the labeling tool is used for model training, and this is used as a basis for judging the performance of the model.

[0046] Step 102: Design a green attention mechanism to enhance the attention of the green area (weed part) and weaken the non-target area, so as to better learn the weed information features;

[0047] Step 103: Design an adaptive joint feature fusion method for a weighted bidirectional feature pyramid fusion architecture, combining the bottom-level local detail features such as weed color and texture with high-level semantic information, so as to better extract personalized features of weeds;

[0048] Step 104: Use the dyhead module to design a decoupling head to unify scale-aware attention, space-aware attention, and task-aware attention, so as to better classify and locate weeds;

[0049] Step 105: Use feature content-aware reorganization to aggregate context information within a large receptive field, and dynamically generate an adaptive kernel based on instance content to better capture different weed features;

[0050] Step 106: Use the dynamic task alignment strategy (TAA) and distribution focus loss to more accurately predict the regression box and positive sample matching of the weeds;

[0051] Step 107: Propose a weed detection network WeedsDetectNet based on YOLOv5;

[0052] Among them, the network improves the recognition accuracy and robustness of weeds by optimizing feature extraction, fusion strategy and target detection head design.

[0053] Step 108: Perform ablation and comparison experiments on the self-built dataset;

[0054] Step 109: Conduct comparative experiments on the CottonWeedDet12 dataset with a larger public sample and the 4WEED DATASET dataset with a smaller public sample;

[0055] Step 110: Model detection effect and visual analysis.

[0056] Among them, the experimental results and visual analysis show that compared with the comparison algorithm, the algorithm of the embodiment of the present invention has better prediction effect on similar weeds in complex backgrounds, more accurate bounding box regression, higher confidence, and better performance in generalization and robustness.

[0057] In summary, the embodiment of the present invention designs a new real-scene farmland weed detection method that combines green attention and adaptive joint features through the above steps 101 to 110. This is aimed at solving the problem that the existing technology still has high recognition errors for different types of weeds with similar colors and shapes in complex environments. Ablation and comparison experiments were performed on a self-built dataset, and algorithm comparisons were also performed on the larger sample CottonWeedDet12 and the smaller sample 4WEED DATASET datasets. The experimental results and visual analysis show that compared with the comparison algorithm, the method of the embodiment of the present invention is more accurate in identifying similar weeds, and performs better in generalization and robustness.

[0058] Example 2

[0059] The scheme in Example 1 is further introduced below, as described below:

[0060] Step 201: Collect weed pictures, label them using the Labelimg labeling tool, and construct a weed dataset with 6 types;

[0061] Among them, in view of the shortage of weed datasets with similar characteristics in real scenes, weed images were collected at Huaxing Farm in Changji City, Changji Hui Autonomous Prefecture, Xinjiang Uygur Autonomous Region, China. The data images are RGB (red-green-blue) color images collected using a mobile phone vivoZ5, with an aspect ratio of 4:3 and a resolution of 4096×3027. Various functions such as exposure and focus were turned off before shooting to ensure that the originality of the image data was not destroyed by the device's own algorithm. In order to ensure the diversity and variability of the images in this study and to improve the robustness of the model, the data sets in this study collected data images in the morning, noon, and afternoon at different growth stages of weeds and different field locations under natural lighting conditions and different weather conditions (e.g., sunny, cloudy, and overcast). The final data set consists of six weeds (Amaranthus retroflexus, Gray Ligustrum lucidum, Solanum nigrum, Camel thorn, Portulaca oleracea, and Field Bindweed), with a total of 1,990 images. The seedlings of Amaranthus retroflexus, Gray Ligustrum lucidum, and Solanum nigrum have similar appearances, and Camel thorn is a weed unique to Xinjiang. This can provide data support for future scientific research on weed similarities. Figure 2 Annotated image examples are shown, where only one weed class is highlighted in each row of images. The collected weed images show diversity between and within classes in terms of shape, texture, color, soil background, and field lighting conditions. This diversity helps the dataset to be closer to real scenes, which is important for building models that are robust to imaging conditions.

[0062] The collected images were annotated by the open source la belimg tool by personnel who have a solid grasp of the marking tools and can clearly distinguish the characteristics of weeds. The marking format used is PascalVOC. When naming the six weeds, the scientific names are used, and the abbreviations are used for the longer ones. In order, they are named 'Amaranthus', 'Chenopodium', 'Solanum nigrum', 'Alhagi sparsifolia', 'Purslane' and 'Convolvulus'. Figure 3 A histogram representing the weeds dataset, which contains 1990 images and 2352 instance bounding boxes for 6 categories.

[0063] Step 202: Designing a green attention mechanism can enhance the attention of the green area (weed part) and weaken the non-target area, so as to better learn the weed information features.

[0064] Among them, the designed green attention mechanism is as follows Figure 5As shown in the figure, the green attention module is designed based on the RGB characteristics of the weed dataset. It introduces two trainable and optimized parameters to calculate the feature maps of the green channel and the red channel to generate a mask. It is then multiplied with the input feature map to enhance the green features of the weed area.

[0065]

[0066] mask=β·H(GRa)+(1-H(GRa))

[0067] y = act(bn(conv(x×mask)))

[0068] Among them, the input feature is represented by x, mask represents the mask, y represents the output feature, w represents the intermediate variable, β and α represent two trainable hyperparameters, G represents the feature map of the green channel, R represents the feature map of the red channel, conv is convolution, bn is batch normalization, and act is the activation function.

[0069] Step 203: Design an adaptive joint feature fusion method (such as Figure 7 ) is used for the weighted bidirectional feature pyramid fusion architecture (as shown in Figure 6 As shown in the figure, combining the local detail features such as the color and texture of the underlying weeds with the high-level semantic information can better extract the personalized features of the weeds.

[0070] Among them, the adaptive joint feature fusion is improved on the basis of the existing fusion method. First, the input feature map is subjected to adaptive feature fusion, different input weighted fusion and weighted direct fusion. Then, the above three parts of results are spliced ​​to better integrate the global and local features of weeds in multiple aspects.

[0071] Among them, the adaptive fusion method, specifically, assumes that the input size can be expressed as (bs, C, H, W), and the embodiment of the present invention can obtain the spatial adaptive weight of (bs, 3, H, W) by performing convolution splicing and softmax operations; different input weighted fusion, specifically, set several parameters as the adjustment weights of each input, and reweight each input feature after ReLu and softmax, and perform weight direct fusion.

[0072] Reweighting means reweighting the input features. In this process, the network first generates a set of weight parameters based on each input feature map. These weight parameters are processed by ReLU and softmax functions to ensure that the weights are non-negative and sum to 1. Then, these weights are applied to the corresponding input feature maps to adjust the contribution of different inputs, thereby achieving more accurate feature fusion. This reweighting method can enhance the focus on important features and reduce the interference on unimportant features, thereby improving the detection effect of the model.

[0073] Specifically, weight direct fusion means: first perform a convolution operation on each input to adjust the feature map channel and size of each input to be consistent, and then directly add the corresponding positions of different inputs.

[0074] Step 204: Use the dyhead module (such as Fig. 9 Design a decoupling head (as shown) Fig.10 As shown in the figure, the scale-aware attention, space-aware attention and task-aware attention are unified to better classify and locate weeds.

[0075] The design of the decoupling head is composed of two dyhead modules and some convolution blocks. The dyhead module is divided into a scale-aware attention module, a space-aware attention module, and a task-aware attention module. The self-attention mechanism is used to unify the scale-aware attention, space-aware attention, and task-aware attention.

[0076] W(F)=π C (π S (π L (F)·F)·F)·F

[0077] Among them, π L (·),π S (·),π C (·) are scale-aware attention module, space-aware attention module and task-aware attention module respectively. Three different attention functions are applied to dimensions L, S and C.

[0078] Scale-aware attention π L : Dynamically fuse features of different scales according to the semantic importance of different scales.

[0079]

[0080] Here, f(·) is a linear function approximated by a 1×1 convolutional layer, and σ(x) is a hard-sigmoid function.

[0081] Spatial Perception Attention π S: Another spatially aware attention module based on fused features is applied to focus on discriminative regions that coexist consistently between spatial locations and feature levels. Considering the high dimensionality in S, the present embodiment decomposes this module into two steps: first, attention learning is made sparse by using deformable convolutions, and then cross-level features are aggregated at the same spatial location.

[0082]

[0083] Where K is the number of sparse sampling locations, is a shifted self-learning space shift to focus on the discrimination area, is the position p k The self-learned importance scalar at . Both are re-learned from the input features at the median level of F.

[0084] Task-aware attention π C : In order to achieve joint learning and promote different representations of objects, the embodiment of the present invention deploys task-aware attention at the end. It dynamically switches on and off channels to support different tasks.

[0085] π C (F)·F=max(α 1 (F)·F c +β 1 (F),α 2 (F)·F c +β 2 (F)

[0086] Where Fc is the feature slice of the cth channel, [α 1 , α 2 , β 1 , β 2 ] T = θ(·) is a hyperfunction that learns to control the activation threshold. The implementation process of θ(·) is as follows: it first performs global average pooling on L×S dimensions to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies a shifted sigmoid function to normalize the output to [-1, 1].

[0087] Step 205: using feature content-aware reorganization to aggregate contextual information within a large receptive field, and dynamically generating an adaptive kernel based on instance content, so as to better capture different weed features;

[0088] Among them, feature content-aware reconstruction consists of two steps (such as Figure 8 ). The first step is to predict a reorganization kernel based on the content of each target location, and the second step is to reorganize the features with the predicted kernel.

[0089] W l′ =ψ(N(c l ,k encoder ))

[0090] X′ l′ =φ(N(c l ,k up ),W l' )

[0091] Among them, the upsampling kernel prediction module ψ is based on χ l The positional upsampling kernel W is predicted for each position l' by using the neighbor l’ , as shown in the following formula. φ is the content-aware reorganization module, which transforms the adjacent χ l With kernel W l’ Reorganization.

[0092] Step 206: Use a dynamic task alignment strategy (TAA) and distribution focus loss to more accurately predict the regression box and positive sample matching of the weeds.

[0093] Among them, YOLOv5 mainly adopts a static allocation strategy to determine whether it is a positive sample or a negative sample based on the IoU of the predicted box and the Ground Truth. The original yolov5 loss function uses BCE classification loss, CIoU positioning loss and BCE confidence loss. The bounding box loss of the embodiment of the present invention uses CIoU positioning loss and distribution focus loss in combination (such as Fig.11 As shown). The embodiment of the present invention uses the task alignment allocation strategy of TOOD. The matching strategy of TaskAlignedAssi gner is to select positive samples according to the scores of classification and regression. That is, it needs to calculate a t.

[0094] t=s a ×u β

[0095] Among them, α and β are weight hyperparameters, s is the prediction score corresponding to the labeled category, and u is the iou of the predicted box and the gt box. The two are multiplied to measure the degree of alignment. For each GT (true value), all predicted boxes are weighted based on the classification scores corresponding to the GT category, and the IoU between the predicted box and the GT is weighted to obtain an associated classification and regression alignment score alignment_metrics, that is, t; for each GT, the top-K largest ones are directly selected as positive samples based on the alignment_metrics alignment score. Among them, in the target detection task, GT stands for Ground Truth, that is, "true value" or "labeled value". This is the actual location and category information of the target object accurately labeled by humans or other means. When training the model, the predicted results will be compared with these real labeled GTs to calculate the loss function for optimization.

[0096] The distribution focus loss is relatively vague for some weed boundary boxes due to occlusion, so its specific position is not very certain. The previous regression box predicts a specific value. For such a vague boundary, the embodiment of the present invention adopts a probability distribution, which is equivalent to the embodiment of the present invention integrating this variable to obtain a value and estimate a When programming, it is quantified, for example: its value range is from y0 to y n In the embodiment of the present invention, it can be quantified, and when estimating after quantization, a weighted summation can be performed on these values ​​and their distribution.

[0097] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))

[0098] The goal of DFL is to focus on expanding the probability of values ​​around the target y (i.e., yi and yi+1). The global minimum solution of DFL, i.e., It can ensure that the estimated regression target ^y is infinitely close to the corresponding label y, that is, This also ensures its correctness as a loss function.

[0099] Step 207: Based on the above improvements of YOLOv5, a weed detection network WeedsDetectNet (such as Figure 4 shown).

[0100] WeedsDetectNet is an improved weed detection network based on YOLOv5n. The proposed green attention (GA) can enhance the attention of green areas (weed parts) and weaken non-target areas. The new adaptive joint feature fusion method (AJFF) is used in the weighted bidirectional feature pyramid (BiFPN) fusion architecture to combine the underlying local detail features such as weed color and texture with high-level semantic information, which can better extract the personalized features of weeds. The introduction of the dyhead module design decoupling head achieves the unification of scale-aware attention, spatial-aware attention and task-aware attention, and better classifies and locates weeds. The introduced feature content-aware recombination (CARAFE) gathers contextual information within a large receptive field and dynamically generates adaptive kernels according to the instance content, so as to better capture different weed features. The introduced dynamic task alignment strategy (TAA) and distribution focus loss (DFL) more accurately predict the regression box and positive sample matching of weeds.

[0101] In the study of target detection algorithms, the evaluation of algorithm performance is mainly divided into detection accuracy, detection speed, and model training and deployment requirements. For the comparison of accuracy, the embodiment of the present invention selects mAP@0.5 (the average of the average accuracy of each category when the IoU threshold is 0.5), mAP@[0.5:0.95] (the average of mAP calculated under multiple thresholds with IoU thresholds from 0.5 to 0.95 (with a step size of 0.05)), and F1-score indicators for comparison. For the comparison of real-time performance, the embodiment of the present invention selects the FPS indicator for comparison. For the comparison of model training and deployment requirements, the embodiment of the present invention selects the Params, FLOPs, and Size (model size) indicators for comparison.

[0102] In order to address the problem that weeds with similar appearance are easily misjudged, a self-built dataset contains three weeds with very similar colors and appearances, such as Amaranthus retroflexus, Amaranthus grisea, and Solanum nigrum seedlings. Ablation and comparison experiments are performed on the self-built dataset. In order to further verify the robustness and generalization of the WeedsDetectNet method proposed in the embodiment of the present invention. The embodiment of the present invention also uses the public datasets CottonWeedDet12 and 4WEED DATASET for comparative experiments. A total of 17 target detection algorithms were selected for comparative experiments with the WeedsDetectNet proposed in the present invention to comprehensively evaluate the performance of the WeedsDetectNet proposed in the embodiment of the present invention. These include the classic target detection algorithms Sparse R-CNN-r50, YOLOX-s, PP-YOLOE+-s, YOLOv5-m r7.0, YOLOv6-s v3.0, YOLOv7-tiny, YOLOv8-s, Y OLOv9-s, Gold-YOLO-s, RTMDet-s, DINO-r50-4scale, CO-DETR-r50, DDQ DETR-4scale-r50, RT-DETR-r34, YOLOv10-s and the latest weed detection research in the field of YOLOv8s-CBAM-C3Gh ost and W-YOLOv5. By comparing with these advanced methods, the embodiment of the present invention aims to verify the effectiveness of WeedsDetectNet in different detection tasks and demonstrate its advantages in processing weeds with similar appearance and improving detection accuracy.

[0103] Step 208: Conduct comparative experiments on the self-built data set;

[0104] First, a comparative experiment was conducted on a self-built dataset, as shown in Table 1 below. The data shows that the WeedsDetectNet proposed in the embodiment of the present invention is leading in all performance indicators, and the algorithm of the embodiment of the present invention can better distinguish between similar weeds between classes and weeds of different growth stages within a class in real scenes. Compared with the most cutting-edge target detection algorithm YOLOv10s, the real-time performance is poor, with mAP@0.5 increased by 2.73%, mAP@[0.5:0.95] increased by 2.45%, F1-score increased by 2.61%, Params decreased by 58.49%, FLOPs decreased by 57.66%, Size decreased by 57.03%, and FPS decreased by 39.19%.

[0105] Step 209: Perform comparative experiments on the CottonWeedDet12 dataset with a larger public sample and the 4WEED DATASET dataset with a smaller public sample;

[0106] In order to further verify the robustness and generalization of the WeedsDetectNet method proposed in the embodiment of the present invention. The embodiment of the present invention selects a larger sample data set CottonWeedDet12 (a total of 5648 images, 12 categories) and a smaller sample data set 4WEED DATASET (a total of 619 images, 4 categories) for further algorithm comparison experiments relative to its own data set, as shown in Table 2 and Table 3. Through a comprehensive analysis of the experimental results, the detection effect of WeedsDetectNet for larger sample data sets is still very good, and the F1-score index for smaller sample data sets is not good. Cause analysis shows that the algorithm complexity of the embodiment of the present invention is much smaller than that of other algorithms. For larger sample data sets, different feature information of weeds can be well learned through attention and feature fusion. For smaller sample data sets, due to data limitations and the lightweight nature of its own algorithm, the features of weeds cannot be well extracted.

[0107] Step 210: Model detection effect and visual analysis. The experimental results and visual analysis show that, compared with the comparison algorithm, the algorithm of the embodiment of the present invention has better prediction effect on similar weeds in complex backgrounds, more accurate bounding box regression, higher confidence, and better generalization and robustness.

[0108] Among them, the embodiment of the present invention selects 12 pictures from the test set, and the prediction results under the baseline YOLOv5n model are as follows: Fig.12 As shown in the second row of the graph. a) is an inter-class misjudgment, and the gray lily is misjudged as Amaranthus retroflexus; b) is a misjudgment of a background non-detected weed target; c) is a misjudgment of a background non-detected weed target; d) is a low confidence; e) is an inter-class misjudgment, and Solanum nigrum is misjudged as gray lily, and one Solanum nigrum target is missed; f) is an inaccurate bounding box regression, and the confidence is low; g) is an inter-class misjudgment, Amaranthus retroflexus is misjudged as Solanum nigrum, and a background non-detected weed target is misjudged; h) is a background non-detected weed target misjudgment; i) is a low confidence; j) is an inter-class misjudgment, and Purslane is misjudged as gray lily with low confidence; k) is a low confidence, and multiple targets are predicted as a single target; l) is a Purslane weed target that is not detected. Through these 12 typical weed images, we can intuitively compare the prediction effects of different algorithms on similar weeds, bounding box regression effects, and confidence effects.

[0109] According to the previous comparative experimental results, the present invention selects YOLOv5m, YOLOv8s, YOLOv9s, YOLOv10s, and YOLOv8s-CBAM-C3Ghost for prediction and makes a visual comparison with the WeedsDetectNet proposed in the present invention. Fig.12As shown, by comparing the labels, it can be seen that WeedsDetectNet has better results, especially the similarity weed prediction effect is obvious, the misjudgment rate of background non-detected weeds is low, the bounding box regression is more accurate, and the confidence is higher.

[0110] In order to better demonstrate the effect of the heat map, the heat map generation method selected by the embodiment of the present invention is Grad-cam++, and the prediction box is displayed on the heat map and the heat map is recalibrated, such as Fig.13 As shown. Using the 12 test set images used for prediction above, the embodiment of the present invention still selects the models YOLOv5m, YOLOv8s, YOLOv9s, YOLOv10s, and YOLOv8s-CBAM-C3Ghost with better results in the comparison experiment to generate a heat map, and makes a visual comparison with the WeedsDetectNet proposed in the embodiment of the present invention. As shown in the figure below, it can be seen from the mutual comparison that the WeedsDetectNet model proposed in the embodiment of the present invention has the best generated heat map effect, which can follow the outline of the weeds and mainly focus on the weed targets.

[0111] Table 1 Comparative experimental results of different algorithms on self-built datasets

[0112]

[0113] Table 2 Comparative experimental results of different algorithms on the public dataset CottonWeedDet12

[0114]

[0115] Table 3 Comparative experimental results of different algorithms on the public dataset 4WEED DATASET

[0116]

[0117]

[0118] Example 3

[0119] A field weed detection device with green attention and joint feature fusion includes a microprocessor and a memory connected to each other; the memory stores program instructions, and the processor calls the program instructions stored in the memory to enable the device to execute the following method steps in Example 1:

[0120] Perform the following method steps in Example 1:

[0121] Collect weed pictures and use Labelimg to create a dataset. Design a green attention mechanism to increase the attention of green areas (weed parts) and weaken non-target areas.

[0122] An adaptive joint feature fusion method is designed for the weighted bidirectional feature pyramid fusion architecture, which combines the underlying local detail features such as weed color and texture with high-level semantic information to better extract the personalized features of weeds.

[0123] The decoupling head designed using the dyhead module achieves the unification of scale-aware attention, space-aware attention and task-aware attention, so as to better classify and locate weeds.

[0124] Feature content-aware recombination is used to aggregate contextual information within a large receptive field, and adaptive kernels are dynamically generated according to instance content to better capture different weed features.

[0125] A dynamic task alignment strategy (TAA) and distribution focus loss are used to more accurately predict the regression box and positive sample matches of the weeds. A dynamic task alignment strategy (TAA) and distribution focus loss are used to more accurately predict the regression box and positive sample matches of the weeds.

[0126] Based on the above improvements of YOLOv5, a weed detection method WeedsDetectNet is proposed. Ablation and comparison experiments are performed on a self-built dataset.

[0127] Comparative experiments on CottonWeedDet12, a larger sample, and 4WEED DATASET, a smaller sample, are performed. Model detection results and visual analysis are performed.

[0128] The above operations are specifically as follows:

[0129] Design a green attention mechanism to enhance the focus on weeds and weaken non-target areas; design an adaptive joint feature fusion method for a weighted bidirectional feature pyramid fusion architecture to combine the underlying weed color and texture features with high-level semantic information to extract personalized features of weeds;

[0130] The decoupling head is designed using the dyhead module to unify scale-aware attention, space-aware attention and task-aware attention. Feature content-aware reorganization is used to aggregate contextual information within a large receptive field, and an adaptive kernel is dynamically generated based on the instance content. Dynamic task alignment strategy and distribution focus loss are used to predict the regression box and positive sample matching of weeds. Field weeds are detected.

[0131] Among them, the green attention mechanism is designed based on the RGB weed dataset. By introducing two trainable and optimized parameters, the feature maps of the green channel and the red channel are calculated to generate a mask; the mask is multiplied by the input feature map to enhance the green features of the weed area.

[0132] Among them, the adaptive joint feature fusion method is: adaptive feature fusion, weighted fusion of different inputs and direct fusion of weights are performed on the input feature map respectively; the three parts of the fusion results are spliced ​​to fuse the global and local features of the weeds.

[0133] Among them, the decoupling head designed using the dyhead module is composed of two dyhead modules and some convolution blocks. The dyhead module is divided into a scale-aware attention module, a space-aware attention module, and a task-aware attention module. The self-attention mechanism is used to respectively detect scale-aware attention, space-aware attention, and task-aware attention:

[0134] W(F)=π C (π S (π L (F)·F)·F)·F

[0135] Among them, π L (·),π S (·),π C (·) are the scale-aware attention module, the space-aware attention module, and the task-aware attention module, respectively. Three different attention functions are applied to dimensions L, S, and C.

[0136] The feature content-aware reorganization includes: predicting a reorganization kernel according to the content of each target position, and reorganizing the features using the predicted kernel;

[0137] W l' =ψ(N(c l ,k encoder ))

[0138] X′ l′ =φ(N(c l ,k up ),W l' )

[0139] Among them, the upsampling kernel prediction module ψ is based on χ l The positional upsampling kernel W is predicted for each position l' by using the neighbor l’ , φ is a content-aware reorganization module that transforms adjacent l With kernel W l’ Reorganization.

[0140] It should be pointed out here that the device description in the above embodiment corresponds to the method description in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0141] The execution subjects of the above-mentioned processor and memory can be devices with computing functions such as computers, single-chip microcomputers, and microcontrollers. In specific implementation, the embodiments of the present invention do not limit the execution subjects and are selected according to the needs of actual applications.

[0142] The data signal is transmitted between the memory and the processor via a bus, which is not described in detail in the embodiment of the present invention.

[0143] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium includes a stored program, and when the program is running, the device where the storage medium is located is controlled to execute the method steps in the above embodiment.

[0144] The computer-readable storage medium includes but is not limited to a flash memory, a hard disk, a solid-state drive, and the like.

[0145] It should be pointed out here that the description of the readable storage medium in the above embodiment corresponds to the description of the method in the embodiment, and the embodiment of the present invention will not be described in detail here.

[0146] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated.

[0147] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted via a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by the computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium or a semiconductor medium, etc.

[0148] Unless otherwise specified, the models of the components in the embodiments of the present invention are not limited, and any device that can perform the above functions may be used.

[0149] Those skilled in the art will appreciate that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A field weed detection method based on green attention and joint feature fusion, characterized in that: The method comprises: Design a green attention mechanism to enhance the focus on weeds and weaken non-target areas; design an adaptive joint feature fusion method for a weighted bidirectional feature pyramid fusion architecture to combine the underlying weed color and texture features with high-level semantic information to extract personalized features of weeds; The decoupling head is designed using the dyhead module to unify scale-aware attention, space-aware attention and task-aware attention. Feature content-aware reorganization is used to aggregate contextual information within a large receptive field, and an adaptive kernel is dynamically generated based on the instance content. Dynamic task alignment strategy and distribution focus loss are used to predict the regression box and positive sample matching of weeds. Field weeds are detected.

2. The field weed detection method based on green attention and joint feature fusion according to claim 1, characterized in that: The green attention mechanism is designed based on the weed dataset RGB. It introduces two trainable and optimized parameters to calculate the feature maps of the green channel and the red channel to generate a mask. Multiply the mask with the input feature map to enhance the green features of the weed area.

3. The field weed detection method based on green attention and joint feature fusion according to claim 1, characterized in that: The adaptive joint feature fusion method is as follows: adaptive feature fusion, different input weighted fusion and weighted direct fusion are performed on the input feature map respectively; the three fusion results are spliced ​​to fuse the global and local features of the weeds.

4. The field weed detection method based on green attention and joint feature fusion according to claim 1, characterized in that: The decoupling head designed using the dyhead module is composed of two dyhead modules and some convolution blocks. The dyhead module is divided into a scale-aware attention module, a space-aware attention module, and a task-aware attention module. The self-attention mechanism is used to respectively detect scale-aware attention, space-aware attention, and task-aware attention: W(F)=π C (p S (p L (F)·F)·F)·F Among them, π L (·),π S (·),π C (·) are the scale-aware attention module, the space-aware attention module, and the task-aware attention module, respectively. Three different attention functions are applied to dimensions L, S, and C.

5. The field weed detection method based on green attention and joint feature fusion according to claim 1, characterized in that: The feature content-aware reorganization includes: predicting a reorganization kernel according to the content of each target position, and reorganizing the features using the predicted kernel; W l' =ψ(N(χ l ,k encoder )) X l ”=φ(N(χ l ,k up ),W l' ) Among them, the upsampling kernel prediction module ψ is based on χ l The positional upsampling kernel W is predicted for each position l' by using the neighbor l’ , φ is a content-aware reorganization module that transforms adjacent l With kernel W l’ Reorganization.

6. A field weed detection device based on green attention and joint feature fusion, characterized in that: The device comprises: a processor and a memory, wherein program instructions are stored in the memory, and the processor calls the program instructions stored in the memory to enable the device to execute any one of the methods of claims 1-5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 5.

Images