A wheat ear detection method based on improved YOLOv8s

Abstract

The application discloses a wheat ear detection method based on an improved YOLOv8s, which is suitable for improving wheat ear detection problems in a complex field environment, so as to realize rapid identification and effective counting of wheat ears. First, the application replaces a C2f module in a Backbone network by using a FasterBiNetBlock, which can reduce model parameters, reduce the number of calculation redundancies and memory accesses, and refine features of the model, so that more attention is paid to the wheat ears during detection, and interference of background factors such as wheat stems and wheat leaves is inhibited. Secondly, a new network structure EQFPN is proposed to strengthen a Neck network, so that the model has the characteristics of being friendly to quantization and hardware. Finally, a triplet attention module is introduced to capture cross-dimension dependency and wheat ear features, and meanwhile, problems such as false detection and missed detection during wheat ear detection are reduced.

Description

Technical Field

[0001] This invention belongs to the field of wheat yield estimation technology and small target detection, specifically involving wheat ear detection technology based on improved YOLOv8s. Background Technology

[0002] Food security is the foundation of a nation's economic stability and sustainable development. Wheat, as one of the world's most important food crops, is widely cultivated globally. Stable wheat yields are crucial for maintaining global food security and also provide valuable insights for agricultural production and management decisions, impacting national economic prosperity and social stability. Therefore, this invention explores methods for wheat ear detection and counting, which are closely related to food production. In recent years, with the rapid improvement of computer hardware performance, artificial intelligence methods employing advanced technologies such as deep learning have made significant progress in the field of wheat ear detection. Deep learning models can automatically learn feature representations from data without the need for manually designing feature extractors, thus better adapting to different types of wheat ears and varying environmental conditions. Furthermore, wheat ear detection often needs to address complex farmland backgrounds; traditional methods may be affected by background interference, while deep learning models can, to some extent, learn how to distinguish wheat ears from the background. It does not rely on manual feature extraction and possesses powerful learning capabilities, thereby improving the accuracy and robustness of wheat ear detection. The application of these new technologies provides more efficient and accurate tools for wheat yield assessment and agricultural management, which is of great significance for ensuring food security and sustainable agricultural development.

[0003] In current deep learning, YOLOv8s is widely used due to its excellent performance and the balance between real-time applications and accuracy. However, despite its near-perfect performance in many aspects, YOLOv8s still faces some challenges in recognizing wheat ears in complex scenes. For example, in field environments, wheat ears are densely distributed and heavily overlapped, leading to numerous false positives and false negatives. Furthermore, the small scale of the wheat ears and the complex background make them difficult to distinguish and locate in the image, also affecting the detection speed. Summary of the Invention

[0004] To address the aforementioned problems, this invention discloses a wheat ear detection method based on an improved YOLOv8s. While ensuring the accuracy of wheat ear detection, it can effectively reduce model parameters and optimize the model inference process, thereby solving the defects of occlusion, overlap, and inaccurate detection and counting in wheat images.

[0005] This invention is a method for detecting wheat ears based on an improved YOLOv8s, comprising the following steps:

[0006] Step 1: Collect wheat ear images and label them to obtain a wheat ear image dataset. Divide the wheat ear image dataset into a training set, a validation set, and a test set.

[0007] Step 2: Construct the YOLOv8s wheat ear detection network model;

[0008] Step 3: Construct the improved YOLOv8s wheat detection network model;

[0009] Step 4: Input the training set into the improved YOLOv8s wheat detection network model for training;

[0010] Step 5: Evaluate and test the improved YOLOv8s wheat ear detection network model to obtain wheat ear image detection results;

[0011] The beneficial effects of this invention are: 1. Improved feature extraction module: This invention introduces the FasterBiNet module to replace the original C2f module. This structure reduces computational redundancy and the number of memory accesses, and also allows the model to refine features, improving feature extraction capabilities. This enables the detection to focus more on wheat ears and suppress interference from background factors such as wheat stalks and leaves. The replaced model retains more semantic information about small targets, maintaining strong feature representation capabilities while reducing the number of model parameters and computational load, and does not affect the accuracy of wheat ear detection for various visual tasks.

[0012] 2. This invention proposes a novel network structure, EQFPN, which enhances the Neck network. The improved Neck network is both quantization-friendly and hardware-friendly, solving the problem that wheat ear detection cannot be deployed due to poor quantization performance. It also solves the problem that the previous network could not achieve a suitable balance between accuracy and speed on the deployed hardware. It better aggregates the target context information of wheat ears to effectively improve the detection accuracy and speed of dense wheat ears.

[0013] 3. This invention introduces a triplet attention module in YOLOv8s to capture cross-dimensional dependencies and wheat ear features, which helps improve the model's ability to represent wheat ear features, better understand the appearance and shape features of wheat ears, reduce the impact of background noise on detection results, and improve the robustness and accuracy of wheat ear detection. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention. Figure 2 This is a diagram of the YOLOv8s network structure. Figure 3 The diagram shows the improved YOLOv8s network structure. Figure 4Here is a structural diagram of the FasterNet Block and FasterCANet Block. Figure 5 Here is the network structure diagram of EQFPN. Figure 6 This is a network structure diagram of the triplet attention mechanism. Figure 7 This is a graph showing the detection results of an embodiment of the present invention. Figure 8 Detecting heatmaps for YOLOv8s models Figure 9 To improve the model's detection heatmap, Figure 10 PR curves of different algorithms for improving the model. Implementation

[0015] This invention is a method for detecting wheat ears based on an improved YOLOv8s, comprising the following steps:

[0016] Step 1: Collect wheat ear images and label them to obtain a wheat ear image dataset. Divide the wheat ear image dataset into a training set, a validation set, and a test set.

[0017] Step 2: Construct the YOLOv8s wheat ear detection network model;

[0018] Step 3: Construct the improved YOLOv8s wheat detection network model;

[0019] Step 4: Input the training set into the improved YOLOv8s wheat detection network model for training;

[0020] Step 5: Evaluate and test the improved YOLOv8s wheat ear detection network model to obtain wheat ear image detection results;

[0021] Step 1 specifically refers to: collecting wheat ear images using field photography to create a wheat ear dataset; collecting images at different stages—post-flowering, grain-filling, grain-filling-maturity, and maturity—under three weather conditions: sunny, cloudy, and overcast; collecting a total of 700 wheat ear images without sampling frames, with an image resolution of 4000 pixels × 2600 pixels; randomly cropping the original images into 2737 wheat ear images of 1024 pixels × 1024 pixels; the dataset expansion in this paper mainly uses horizontal flipping and mirror flipping to expand the training set, thereby increasing the diversity of the dataset; manually labeling the corner points and wheat ears within the sampling frames using the open-source labeling software LabelImg, and storing the labeled data in a format conforming to the PASCAL VOC dataset.

[0022] Step 2 specifically refers to: constructing a YOLOv8s wheat detection network model, whose overall structure consists of three parts connected sequentially to complete the object detection task; first is the Backbone network, which is responsible for extracting features from the input image, including the Conv module, C2f module, and SPPF module; next is the Neck network, which is used to fuse features at different levels to enhance detection performance, and it adopts the FPN+PAN structure, where the FPN structure is used to pass strong semantic feature information from top to bottom; the PAN structure adds an upward feature pyramid after the FPN structure to pass strong localization information from bottom to top; finally, the Head network is used to generate the prediction results of object detection.

[0023] In step 3, this invention proposes to replace all C2f modules in the original Backbone network with the FasterBiNet module, and add a triplet attention module before the SPPF module; finally, the EQFPN module is introduced to enhance the Neck network.

[0024] Step 4 specifically includes: inputting the training set into the improved YOLOv8s wheat detection network model for training; and setting the model training parameters as follows: learning rate = 0.001, momentum = 0.9, decay = 0.0005, batch size = 16, epochs = 100. Adaptive momentum estimation is used to optimize the trained model.

[0025] Step 5 specifically includes the following evaluation metrics: Precision, Recall, and mAP@0.5. Precision represents the probability that the detected region contains a real wheat ear sample; Recall represents how many wheat ear targets are detected in the image; AP (Average Accuracy) measures how well the model learns on each class; mAP (Mean Average Accuracy) measures how well the trained model performs on all classes. mAP is essentially the average of AP across all classes, and the specific calculation formula is as follows:

[0026]

[0027]

[0028]

[0029]

[0030] In the formula, P(r) represents the precision of the recall measurement, N represents the number of target classifications; TP represents the number of wheat ears correctly predicted by the model, FP represents the number of samples that identified non-wheat ear regions as wheat ears, FN represents the number of samples that identified wheat ears as non-wheat ear regions; mAP@0.5 represents the average accuracy of target detection with a confidence level of 0.5.

[0031] The invention will now be further described in conjunction with the accompanying drawings. Figure 1 As shown, this invention is a wheat ear detection method based on an improved YOLOv8s, comprising the following steps:

[0032] Step 1: Collect wheat ear images and label them to obtain a wheat ear image dataset. Divide the wheat ear image dataset into a training set, a validation set, and a test set.

[0033] Step 2: Construct the YOLOv8s wheat ear detection network model;

[0034] Step 3: Construct the improved YOLOv8s wheat detection network model;

[0035] Step 4: Input the training set into the improved YOLOv8s wheat detection network model for training;

[0036] Step 5: Evaluate and test the improved YOLOv8s wheat ear detection network model to obtain wheat ear image detection results.

[0037] Step 1 specifically refers to: collecting wheat ear images using field photography to create a wheat ear dataset; collecting images at different stages—post-flowering, grain-filling, grain-filling-maturity, and maturity—under three weather conditions: sunny, cloudy, and overcast; collecting a total of 700 wheat ear images without sampling frames, with an image resolution of 4000 pixels × 2600 pixels; randomly cropping the original images into 2737 wheat ear images of 1024 pixels × 1024 pixels; the dataset expansion in this paper mainly uses horizontal flipping and mirror flipping to expand the training set, thereby increasing the diversity of the dataset; manually labeling the corner points and wheat ears within the sampling frames using the open-source labeling software LabelImg, and storing the labeled data in a format conforming to the PASCAL VOC dataset.

[0038] like Figure 2As shown, in step 2, the overall structure of the YOLOv8s network model consists of three parts, which are connected sequentially to complete the object detection task. First is the Backbone network, which is responsible for extracting features from the input image, including the Conv module, C2f module, and SPPF module. Next is the Neck network, which is used to fuse features at different levels to enhance detection performance. It adopts an FPN+PAN structure, where the FPN structure is used to pass strong semantic feature information from top to bottom; the PAN structure adds an upward feature pyramid after the FPN structure to pass strong localization information from bottom to top. Finally, there is the Head network, which is used to generate the prediction results of object detection.

[0039] Despite its near-perfect performance in many aspects, YOLOv8s still faces some challenges in recognizing wheat ears in complex scenes. For example, in open field environments, wheat ears are densely distributed and overlap significantly, leading to numerous false positives and false negatives. Furthermore, the small scale of the wheat ears and the complex background also affect the detection speed.

[0040] like Figure 3 As shown, to address the above problems, this invention proposes an improved model based on YOLOv8s, which achieves higher running speed while maintaining accuracy. Step 3 specifically includes the following steps:

[0041] First step, such as Figure 4 As shown in the figure, this is a structural diagram of the FasterNet Block and the FasterBiNet Block. Specifically, the Bi-Level Routing attention mechanism is placed after the last Conv of the FasterNet Block, forming the FasterBiNet Block. Then, the FasterBiNet Block is used to replace the C2f module in YOLOv8s.

[0042] The second step of this invention proposes a novel network structure, EQFPN, to enhance the Neck network, such as... Figure 5 As shown.

[0043] The third step is to introduce a triplet attention module into the YOLOv8s Backbone network, such as... Figure 6 It should be noted that triplet attention is a novel method for cross-dimensional interactive computation of attention weights based on a three-branch structure. Specifically, it captures the dependencies between (C,H), (C,W), and (H,W) of the input tensor through three branches. For a given input of size C×H×W, the specific implementation process is as follows:

[0044] (1) In the first branch, an interaction is established between the H and C dimensions. Through this process, the input features are transformed into W×H×C dimensions. Next, a Z-Pool operation is performed on the W dimension to produce 2×H×C features. Then, these features are passed through a 7×7 convolutional layer and a batch normalization layer, and finally, through cross-dimensional interaction, they are transformed into C×H×W dimensional features.

[0045] (2) In the second branch, an interaction is established between the C and W dimensions. First, the input features are transformed into H×C×W dimensions through cross-dimensional interaction. Then, a Z-Pool operation is performed on the H dimension, followed by processing through a 7×7 convolutional layer and a batch normalization layer. Finally, through cross-dimensional interaction, they are transformed into C×H×W dimensions.

[0046] (3) In the third branch, an interaction is established between the H and W dimensions. First, the input features are channel pooled, then passed through a 7×7 convolutional layer and a batch normalization layer. Finally, this branch uses the sigmoid activation function to generate values ​​representing the spatial attention weights.

[0047] (4) Finally, the average value of all output features of the three branches is calculated.

[0048] Where Z-Pool represents max pooling and average pooling, C represents the channel dimension, H represents the height dimension, and W represents the width dimension.

[0049] Step 4 specifically includes: inputting the training set into the improved YOLOv8s wheat detection network model for training; and setting the model training parameters as follows: learning rate = 0.001, momentum = 0.9, decay = 0.0005, batch size = 16, epochs = 100. Adaptive momentum estimation is used to optimize the trained model.

[0050] Step 5 specifically includes the following evaluation metrics: Precision, Recall, and mAP@0.5. Precision represents the probability that the detected region contains a real wheat ear sample; Recall represents how many wheat ear targets are detected in the image; AP measures how well the model learns in each class; and mAP measures how well the trained model performs across all classes. mAP is essentially the average of the AP across all classes, and the specific calculation formula is as follows:

[0051]

[0052]

[0053]

[0054]

[0055] In the formula, P(r) represents the precision of the recall measurement, N represents the number of target classifications; TP represents the number of correctly predicted wheat ears, FP represents the number of samples that identified non-wheat ear regions as wheat ears, and FN represents the number of samples that identified wheat ears as non-wheat ear regions; mAP@0.5 represents the average accuracy of target detection with a confidence level of 0.5. Furthermore, commonly used metrics for evaluating the performance of target detection models are Param and FLOPs. Param typically refers to the learnable weights and biases in the network, and its quantity can be used to assess the model's complexity and storage requirements; FLOPs refers to the number of floating-point operations, used to assess the model's computational complexity and runtime efficiency.

[0056] Finally, the test set is input into the trained, improved YOLOv8s network model for testing.

[0057] Figure 7 The above figure shows the detection results of the embodiment of the present invention. It can be seen from the figure that the improved model can detect image targets relatively completely and has excellent wheat ear target detection capabilities. Figure 8 Detecting heatmaps for YOLOv8s models Figure 9 To improve the model's detection heatmap, the comparison images above show that the improved model can more accurately identify and locate targets. Table 1 below shows the evaluation metrics of the improved YOLOv8s network model during training.

[0058] Table 1

[0059] YOLOv8s 0.903 0.863 0.914 13.79 30.3 Improved YOLOv8s 0.925 0.893 0.945 9.68 23.5

[0060] Based on the testing results, the wheat detection model based on the improved YOLOv8s achieved good results. The accuracy reached 92.5%, the recall reached 89.3%, and the mAP@0.5 reached 94.5% on the test set. Accuracy, recall, and mAP@0.5 were improved by 2.2%, 3.0%, and 3.1%, respectively, while the number of model parameters and computational cost were significantly reduced, demonstrating that the present invention can achieve a balance between speed and accuracy. Overall, the improved model performs best in both detection speed and accuracy.

[0061] like Figure 9As shown in the figure, A, B, and C represent the performance profiles (PR) curves of different algorithms for YOLOv8s with the FasterBiNet Block, EQFPN, and triplet attention mechanisms added sequentially. The area enclosed by the PR curve and the two coordinate axes represents the mean accuracy (mAP) of the training. The figure shows that adding the three improved methods sequentially to the YOLOv8s model significantly improves the model's performance, fully reflecting the advantages of the improved algorithms and proving the feasibility of this invention.

Claims

1. A method for detecting wheat ears based on an improved YOLOv8s, characterized in that, Includes the following steps: Step 1: Collect and label wheat ear images to obtain a wheat ear image dataset. Divide the wheat ear image dataset into a training set, a validation set, and a test set. Step 2: Construct the YOLOv8s wheat ear detection network model; Step 3: Construct the improved YOLOv8s wheat detection network model; Step 4: Input the training set into the improved YOLOv8s wheat detection network model for training; Step 5: Evaluate and test the improved YOLOv8s wheat detection network model to obtain wheat image detection results; Step 1 specifically refers to: collecting wheat ear images using field photography to create a wheat ear dataset; collecting images at different stages—post-flowering, grain-filling, grain-filling-maturity, and maturity—under three weather conditions: sunny, cloudy, and overcast; collecting a total of 700 original wheat ear images with a resolution of 4000 pixels × 2600 pixels; and randomly cropping the original images into 2737 sub-images of 1024 pixels × 1024 pixels. The dataset was expanded primarily by using horizontal and vertical flipping to increase the diversity of the training set; the open-source annotation tool LabelImg was used to manually annotate the sampling box corners and wheat ear bounding boxes, and then exported them as PASCAL VOC format for storage. Step 2 specifically refers to constructing a YOLOv8s wheat detection network model, whose overall structure consists of three parts connected sequentially to complete the object detection task. First, the Backbone network is responsible for extracting features from the input image, including the Conv module, C2f module, and SPPF module. Next, the Neck network is used to fuse features from different levels to enhance detection performance; it adopts an FPN+PAN structure, where the FPN structure is used to pass strong semantic feature information from top to bottom; the PAN structure adds an upward feature pyramid after the FPN structure to pass strong localization information from bottom to top. Finally, the Head network is used to generate the predicted results of object detection. In step 3, the Backbone is reconstructed using the FasterBiNet module: the FasterBiNet module completely replaces the C2f modules in the original Backbone network; the FasterBiNet structure places the Bi-Level Routing attention mechanism after the last Conv module of FasterNet; a Triplet Attention module is embedded at the end of the Backbone and before the SPPF module to establish a three-dimensional cross-dimensional interaction of channel-height-width; the EQFPN module is introduced to strengthen the Neck network, replacing all Upsample modules in the original Neck network with Simconv modules, and adding Transpose modules after SimConv modules in the FPN structure, which are then concatenated with Concat; all C2f modules in the original Neck network are replaced with QARepNeXt modules, and all Conv modules in the original Neck network are replaced with SimConv modules.

2. The wheat ear detection method based on improved YOLOv8s according to claim 1, characterized in that, Step 4 specifically includes: inputting the training set into the improved YOLOv8s wheat detection network model for training; and setting the model training parameters as follows: learning rate = 0.001, momentum = 0.9, decay = 0.0005, batch size = 16, epochs = 100; adaptive momentum estimation is used to optimize the training model.

3. The wheat ear detection method based on improved YOLOv8s according to claim 1, characterized in that, Step 5 specifically includes the following evaluation metrics: precision, recall, and mAP@0.

5. Precision represents the probability that the detected region contains a true wheat ear sample; recall represents how many wheat ear targets are detected in the image; AP measures how well the model learns in each class; and mAP measures how well the trained model performs across all classes. mAP is essentially the average of the AP across all classes, and the specific calculation formula is as follows: ； ； ； ； In the formula, P(r) represents the precision of the recall measurement, N represents the number of target classifications; TP represents the number of wheat ears correctly predicted by the model, FP represents the number of samples that identified non-wheat ear regions as wheat ears, FN represents the number of samples that identified wheat ears as non-wheat ear regions; mAP@0.5 represents the average accuracy of target detection with a confidence level of 0.5.

Images