A field wheat ear detection method based on YOLOv8

Abstract

The present invention relates to a field wheat ear detection method based on YOLOv8, comprising the following steps: obtaining a wheat ear dataset; improving a YOLOv8 network model to obtain an improved YOLOv8 network model; inputting a training set into the improved YOLOv8 network model for training to obtain a trained YOLOv8 network model; evaluating the trained YOLOv8 network model; inputting an image of wheat ears to be detected into the trained YOLOv8 network model, and having the trained YOLOv8 network model output a final detection result. The present invention introduces AFPN into the YOLOv8 network model, first initiating a fusion process by combining two low-level features of different resolutions, then incorporating high-level features into the fusion process, and finally fusing the top-level features of the backbone, thereby avoiding large semantic gaps between non-adjacent layers. Furthermore, by introducing a depthwise separable convolution module, the number of model parameters and model size can be reduced, thereby accelerating detection speed.

Description

Technical Field

[0001] The present invention relates to the field of modern intelligent agricultural large-scale production technology, and in particular to a field wheat ear detection method based on YOLOv8. Background Art

[0002] Wheat is an important staple crop in my country and one of the most important crops worldwide. Counting wheat ears is crucial for predicting and evaluating wheat yield. However, interference from field factors such as leaves, stalks, shadows, inter-ear occlusion and adhesion, soil, and weeds, as well as meteorological constraints such as wind and rain, severely hamper accurate ear counting, leading to reduced yield estimates. Therefore, timely and accurate determination of wheat ear counts has long been a key focus in wheat breeding and cultivation research.

[0003] Traditional wheat ear counting relies on randomly selecting plots in the field, manually counting within a unit area, and then estimating the number of wheat ears in the field through statistical analysis and averaging. Although manual counting within a plot is highly accurate, random selection, point-based counting, and visual inspections affect the accuracy of the overall wheat field estimate. It is subject to significant subjectivity, low efficiency, and time-consuming. The rapid development of remote sensing, computer vision, image processing, machine learning, and deep learning technologies has provided data and technical support for accurate wheat ear counting. However, existing object detection algorithms suffer from large models, high computing power requirements, and long computation times, making them unsuitable for deployment on portable devices for real-time field counting. To achieve real-time field detection and counting, it is crucial to ensure model detection accuracy while reducing the number of parameters and computation time. Summary of the Invention

[0004] In order to solve the problems of existing detection methods such as large models, low detection speed, and difficulty in embedding into mobile devices, the purpose of the present invention is to provide a field wheat ear detection method based on YOLOv8 that reduces model size and speeds up detection speed without reducing accuracy.

[0005] To achieve the above object, the present invention adopts the following technical solution: a field wheat ear detection method based on YOLOv8, the method comprising the following steps in sequence:

[0006] (1) Obtaining a wheat ear dataset: Take photos of wheat, crop the photos into a uniform size and label them to obtain a wheat ear dataset, which is then divided into a training set, a validation set, and a test set;

[0007] (2) Improve the YOLOv8 network model to obtain an improved YOLOv8 network model;

[0008] (3) Input the training set into the improved YOLOv8 network model for training to obtain the trained YOLOv8 network model;

[0009] (4) Evaluate the trained YOLOv8 network model;

[0010] (5) The wheat ear image to be detected is input into the trained YOLOv8 network model, and the trained YOLOv8 network model outputs the final detection result.

[0011] The step (1) specifically includes the following steps:

[0012] (1a) When taking pictures, keep the lens 1.5m away from the ground and collect 200 pictures, each with a pixel size of 9616×6912;

[0013] (1b) Each image is uniformly cropped to a size of 1024 × 1024, and the wheat ears in each image are labeled to obtain a txt file containing the coordinates and length and width information of the wheat ears in the image, namely the wheat ear dataset;

[0014] (1c) The wheat ear dataset is divided into training set, validation set and test set in a ratio of 6:3:1.

[0015] The step (2) specifically includes the following steps:

[0016] (2a) The feature fusion module in the YOLOv8 network model is replaced by the progressive feature pyramid network AFPN. In the bottom-up feature extraction process of the backbone network of the YOLOv8 network model, the fusion process is first started by combining two low-level features of different resolutions, and then the high-level features are incorporated into the fusion process, and finally the top features of the backbone network are fused. In this process, the semantic information of the low-level features is fused with the high-level features, and the detail information of the high-level features is fused with the low-level features; that is, before feature fusion, the last layer of features is extracted from each feature layer of the backbone network to obtain a set of features of different scales, represented as {C3, C4, C5}, where C3 is the low-level feature, C4 is the high-level feature, and C5 is the top-level feature. To perform feature fusion, the low-level feature C3 is first input into the progressive feature pyramid network AFPN, and then the high-level feature C4 is added, and finally the top-level feature C5 is added; after the feature fusion steps, a set of multi-scale features {P3, P4, P5} is obtained; let Represents the feature vector from layer n to layer l (i, j), and the feature vector obtained by multi-level feature adaptive spatial fusion is recorded as By the eigenvector and The linear combination of is defined as:

[0017]

[0018] Where, Represent the spatial weights of the three hierarchical features of the lth layer,

[0019] (2b) The standard convolution module in the YOLOv8 network model is replaced by a depthwise separable convolution module, which includes a depthwise convolution module and a pointwise convolution module. One convolution kernel of the depthwise convolution module is responsible for one input channel, and the number of convolution kernels is the same as the number of input channels and the number of output channels. In order to effectively utilize the feature information of different channels at the same spatial position and generate a new feature map, the input of the pointwise convolution module is used as the output of the depthwise convolution module, and the convolution size is 1×1. Assume that the convolution kernel size of the standard convolution module and the depthwise convolution module is D K ×D K , D F is the spatial height and width of the square input feature map, M is the number of input channels of the square input feature map, N is the number of output channels of the square input feature map, and the number of parameters of the standard convolution module is:

[0020] D K ·D K ·M·N·D F ·D F

[0021] Depthwise separable convolution first undergoes depthwise convolution, and the parameter amount of depthwise convolution is D K ·D K ·M·D F ·D F , and then after point-by-point convolution, the number of parameters of point-by-point convolution is M·N·D F ·D F , the number of parameters of depth-wise separable convolution is D K ·D K ·M·D F ·D F +M·N·D F ·D F , then the number of parameters of depth-wise separable convolution and standard convolution is compared:

[0022]

[0023] It can be seen that depth-wise separable convolution has the effect of significantly reducing computation and model size.

[0024] The step (4) specifically refers to: using the accuracy P, recall R, average precision mAP, frames per second FPS and floating-point operations GFLOPS to evaluate the trained YOLOv8 network model, the formula is as follows:

[0025]

[0026]

[0027]

[0028] Among them, TP is the true positive sample, which indicates the number of correctly identified wheat ears; FP is the false positive sample, that is, the number of incorrectly identified wheat ears; FN is the false negative sample, that is, the number of unidentified wheat ears; M is the type of wheat ears, and P(R) is the relationship between precision and recall.

[0029] It can be seen from the above technical solution that the beneficial effects of the present invention are: first, the present invention takes photos in the field, labels the wheat ears after cropping, and constructs a wheat ear dataset; second, the present invention introduces AFPN into the YOLOv8 network model to solve the problem that wheat images have overlapping ears and complex backgrounds. First, the fusion process is started by combining two low-level features of different resolutions, and then the high-level features are incorporated into the fusion process. Finally, the top-level features of the backbone are fused, which can avoid large semantic gaps between non-adjacent layers; third, the introduction of the depthwise separable convolution module can reduce the number of model parameters and model size, and speed up the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the method of the present invention;

[0031] Figure 2 This is the improved YOLOv8 network model diagram in the present invention;

[0032] Figure 3 This is the structural diagram of AFPN;

[0033] Figure 4 Schematic diagram of adaptive spatial fusion operation in AFPN;

[0034] Figure 5 It is a structural diagram of the deep convolution module;

[0035] Figure 6 It is a structural diagram of the point-by-point convolution module;

[0036] Figure 7 A schematic diagram of the final test results. DETAILED DESCRIPTION

[0037] like Figure 1 As shown, a field wheat ear detection method based on YOLOv8 includes the following steps in sequence:

[0038] (1) Obtaining a wheat ear dataset: Take photos of wheat, crop the photos into a uniform size and label them to obtain a wheat ear dataset, which is then divided into a training set, a validation set, and a test set;

[0039] (2) Improve the YOLOv8 network model to obtain an improved YOLOv8 network model;

[0040] (3) Input the training set into the improved YOLOv8 network model for training to obtain the trained YOLOv8 network model;

[0041] (4) Evaluate the trained YOLOv8 network model;

[0042] (5) Input the wheat ear image to be detected into the trained YOLOv8 network model, and the trained YOLOv8 network model outputs the final detection result, such as Figure 7 shown.

[0043] The step (1) specifically includes the following steps:

[0044] (1a) Wheat was photographed in the Agricultural Science and Technology Demonstration Park of Lujiang County, Hefei City, Anhui Province. A Huawei nova8 mobile phone was used to hold a camera pole. When taking pictures, the distance between the lens and the ground was kept at 1.5m. 200 pictures were collected, each with a pixel size of 9616×6912.

[0045] (1b) Each image is uniformly cropped to a size of 1024 × 1024, and the wheat ears in each image are labeled to obtain a txt file containing the coordinates and length and width information of the wheat ears in the image, namely the wheat ear dataset;

[0046] (1c) The wheat ear dataset is divided into training set, validation set and test set in a ratio of 6:3:1.

[0047] The YOLOv8 network model is:

[0048] The input images were uniformly resized to 640×640×3. After loading the images and label information, Mosaic augmentation was applied. The images were then translated, rotated, scaled, and cropped to improve the model's generalization capabilities. Mosaic was disabled for the last 10 epochs of training.

[0049] The Backbone architecture uses the Darknet-53 structure and the c2f structure with richer gradient flow. The number of channels in the first convolution is doubled, and the convolution result is split in half across the channels, reducing the number of convolutions and speeding up the network. This ensures a lightweight model while also obtaining richer gradient flow information. Feature extraction is performed, and the extracted features are called feature layers, which are the feature sets of the input image. In the backbone, three feature layers are obtained for the next step of network construction, which are called effective feature layers.

[0050] Neck fuses feature maps at multiple scales and passes these features to the prediction layer, which helps extract better features. It uses the Panet structure to sample features from top to bottom to achieve feature fusion.

[0051] The head determines the feature points and determines whether there is an object corresponding to the prior bounding box on the feature points. The current mainstream Decoupled-Head is adopted to separate the classification and detection heads and use Anchor-Free.

[0052] like Figure 2 As shown, the step (2) specifically includes the following steps:

[0053] (2a) The feature fusion module in the YOLOv8 network model is replaced by the progressive feature pyramid network (AFPN). In the bottom-up feature extraction process of the backbone network of the YOLOv8 network model, the fusion process is first started by combining two low-level features of different resolutions, and then the high-level features are incorporated into the fusion process, and finally the top-level features of the backbone network are fused. In this process, the semantic information of the low-level features is fused with the high-level features, and the detail information of the high-level features is fused with the low-level features; due to the direct interaction between them, the loss or degradation of information during multi-level transmission is avoided.

[0054] That is, before feature fusion, the last layer of features is extracted from each feature layer of the backbone network to obtain a set of features of different scales, expressed as {C3, C4, C5}, where C3 is a low-level feature, C4 is a high-level feature, and C5 is a top-level feature. To perform feature fusion, the low-level feature C3 is first input into the progressive feature pyramid network AFPN, then the high-level feature C4 is added, and finally the top-level feature C5 is added; after the feature fusion steps, a set of multi-scale features {P3, P4, P5} is obtained.

[0055] In the multi-level feature fusion process, different spatial weights are assigned to features at different levels to enhance the importance of key levels and reduce the impact of conflicting information from different objects. Represents the feature vector from layer n to layer l (i, j), and the feature vector obtained by multi-level feature adaptive spatial fusion is recorded as By the eigenvector and The linear combination of is defined as:

[0056]

[0057] Where, Represent the spatial weights of the three hierarchical features of the lth layer,

[0058] (2b) The standard convolution module in the YOLOv8 network model is replaced by a depthwise separable convolution module, which includes a depthwise convolution module and a pointwise convolution module. One convolution kernel of the depthwise convolution module is responsible for one input channel, and the number of convolution kernels is the same as the number of input channels and the number of output channels. In order to effectively utilize the feature information of different channels at the same spatial position and generate a new feature map, the input of the pointwise convolution module is used as the output of the depthwise convolution module, and the convolution size is 1×1. Assume that the convolution kernel size of the standard convolution module and the depthwise convolution module is D K ×D K , D F is the spatial height and width of the square input feature map, M is the number of input channels of the square input feature map, N is the number of output channels of the square input feature map, and the number of parameters of the standard convolution module is:

[0059] D K ·D K ·M·N·D F ·D F

[0060] Depthwise separable convolution first undergoes depthwise convolution, and the parameter amount of depthwise convolution is D K ·D K ·M·D F ·D F , and then after point-by-point convolution, the number of parameters of point-by-point convolution is M·N·D F ·D F , the number of parameters of depth-wise separable convolution is D K ·D K ·M·D F ·D F +M·N·D F ·D F , then the number of parameters of depth-wise separable convolution and standard convolution is compared:

[0061]

[0062] It can be seen that depth-wise separable convolution has the effect of significantly reducing computation and model size.

[0063] The step (3) specifically refers to:

[0064] (3a) The experimental environment is selected as Windows 10 system, NVIDIA Quadro P4000 graphics card, Intel Xeon Gold 6248R CPU, and the model training is implemented under Pytorch 1.12.1 and CUDA 11.3;

[0065] (3b) The training batch size is 8, the learning rate is set to 0.01, the model is iterated 300 times, and the standard image size is designed to be 640×640×3.

[0066] The step (4) specifically refers to: using the accuracy P, recall R, average precision mAP, frames per second FPS and floating-point operations GFLOPS to evaluate the trained YOLOv8 network model, the formula is as follows:

[0067]

[0068]

[0069]

[0070] Among them, TP is the true positive sample, which indicates the number of correctly identified wheat ears; FP is the false positive sample, that is, the number of incorrectly identified wheat ears; FN is the false negative sample, that is, the number of unidentified wheat ears; M is the type of wheat ears, and P(R) is the relationship between precision and recall.

[0071] like Figure 3 As shown in the figure, AFPN fuses low-level features, high-level features, and top-level features in the backbone network from bottom to top, and then outputs the prediction.

[0072] like Figure 4 As shown in Figure 2, the three adaptive spatial ASFs are fused one after another.

[0073] like Figure 5 As shown in the figure, the 3-channel input passes through three 3*3 convolution kernel filters to obtain 3 feature maps.

[0074] like Figure 6 As shown in the figure, three feature maps are passed through 1*1*3 convolution kernel filters to obtain the corresponding number of feature maps.

[0075] In summary, the present invention uses wheat ears as research objects, reduces the model size, speeds up the detection speed without reducing the accuracy, and provides the possibility of embedding into mobile devices. The present invention constructs a wheat ear dataset by taking photos in the field, labeling the wheat ears after cropping. The present invention addresses the problem of wheat images with overlapping ears and complex backgrounds by introducing AFPN into the YOLOv8 network model. First, the fusion process is started by combining two low-level features of different resolutions, and then the high-level features are incorporated into the fusion process. Finally, the top-level features of the backbone are fused, which can avoid large semantic gaps between non-adjacent layers. The introduction of the depthwise separable convolution module can reduce the number of model parameters and model size and speed up the detection speed.

Claims

1. A field wheat ear detection method based on YOLOv8, characterized by: The method comprises the following steps in sequence: (1) Obtaining a wheat ear dataset: Take photos of wheat, crop the photos into a uniform size and label them to obtain a wheat ear dataset, which is then divided into a training set, a validation set, and a test set; (2) Improve the YOLOv8 network model to obtain an improved YOLOv8 network model; (3) Input the training set into the improved YOLOv8 network model for training to obtain the trained YOLOv8 network model; (4) Evaluate the trained YOLOv8 network model; (5) Input the wheat ear image to be detected into the trained YOLOv8 network model, and the trained YOLOv8 network model outputs the final detection result; The step (2) specifically includes the following steps: (2a) The feature fusion module in the YOLOv8 network model is replaced with the progressive feature pyramid network AFPN. In the bottom-up feature extraction process of the backbone network of the YOLOv8 network model, the fusion process is first started by combining two low-level features of different resolutions, and then the high-level features are incorporated into the fusion process, and finally the top features of the backbone network are fused. In this process, the semantic information of the low-level features is fused with the high-level features, and the detail information of the high-level features is fused with the low-level features; that is, before feature fusion, the last layer of features is extracted from each feature layer of the backbone network to obtain a set of features of different scales, represented as {C3, C4, C5}, where C3 is the low-level feature, C4 is the high-level feature, and C5 is the top-level feature. To perform feature fusion, the low-level feature C3 is first input into the progressive feature pyramid network AFPN, and then the high-level feature C4 is added, and finally the top-level feature C5 is added; After the feature fusion step, a set of multi-scale features {P3, P4, P5} is obtained; let Represents the feature vector from layer n to layer l (i, j), and the feature vector obtained by multi-level feature adaptive spatial fusion is recorded as By the eigenvector and The linear combination of is defined as: (2b) The standard convolution module in the YOLOv8 network model is replaced by a depthwise separable convolution module, which includes a depthwise convolution module and a pointwise convolution module. One convolution kernel of the depthwise convolution module is responsible for one input channel, and the number of convolution kernels is the same as the number of input channels and the number of output channels. In order to effectively utilize the feature information of different channels at the same spatial position and generate a new feature map, the input of the pointwise convolution module is used as the output of the depthwise convolution module, and the convolution size is 1×1. Assume that the convolution kernel size of the standard convolution module and the depthwise convolution module is D K ×D K , D F is the spatial height and width of the square input feature map, M is the number of input channels of the square input feature map, N is the number of output channels of the square input feature map, and the number of parameters of the standard convolution module is: D K ·D K ·M·N·D F ·D F Depthwise separable convolution first undergoes depthwise convolution, and the parameter amount of depthwise convolution is D K ·D K ·M·D F ·D F , and then after point-by-point convolution, the number of parameters of point-by-point convolution is M·N·D F ·D F , the number of parameters of depth-wise separable convolution is D K ·D K ·M·D F ·D F +M·N·D F ·D F , then the number of parameters of depth-wise separable convolution and standard convolution is compared: It can be seen that depth-wise separable convolution has the effect of significantly reducing computation and model size.

2. The field wheat ear detection method based on YOLOv8 according to claim 1, characterized in that: The step (1) specifically includes the following steps: (1a) When taking pictures, keep the lens 1.5m away from the ground and collect 200 pictures, each with a pixel size of 9616×6912; (1b) Each image is uniformly cropped to a size of 1024 × 1024, and the wheat ears in each image are labeled to obtain a txt file containing the coordinates and length and width information of the wheat ears in the image, namely the wheat ear dataset; (1c) Divide the wheat ear dataset into training set, validation set, and test set in a ratio of 6:3:

1.

3. The field wheat ear detection method based on YOLOv8 according to claim 1, characterized in that: The step (4) specifically refers to: using the accuracy P, recall R, average precision mAP, frames per second FPS and floating-point operations GFLOPS to evaluate the trained YOLOv8 network model, the formula is as follows: Among them, TP is the true positive sample, which indicates the number of correctly identified wheat ears; FP is the false positive sample, that is, the number of incorrectly identified wheat ears; FN is the false negative sample, that is, the number of unidentified wheat ears; M is the type of wheat ears, and P(R) is the relationship between precision and recall.

Images