A method for detecting alfalfa seed pod maturity based on improved YOLO v8 unmanned aerial vehicle aerial image

Abstract

The application discloses a method for detecting alfalfa seed pod maturity based on improved YOLO v8 from a UAV aerial image, and belongs to the technical field of deep learning and target detection. The method comprises the following steps: collecting field alfalfa pictures by using a UAV aerial image to construct a data set, classifying and labeling the images, and dividing the image data set after processing and having class labels into a training set and a test set; inputting the training set into an improved YOLO v8 network model for training to obtain a maturity prediction network model; inputting the verification set into the maturity prediction network model for verification to output alfalfa seed pod maturity. The method improves the original YOLO v8 precision, improves the YOLO v8 backbone network architecture, makes the network model lightweight, and improves the inference speed, so that the alfalfa seed pod maturity can be quickly and accurately detected from a UAV aerial image.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and object detection technology, and more specifically, it relates to a method for detecting the maturity of alfalfa seed pods based on UAV aerial images using an improved YOLO v8. Background Technology

[0002] Intelligent perception in drones can extract ground feature information from drone imagery, thereby expanding their scene understanding capabilities and providing technical support for autonomous drone detection and flight. Target detection is one of the key technologies for improving intelligent perception in drone imagery. However, aerial images are characterized by complex backgrounds, dense target distribution, small scale, and large angular differences among targets of the same category. Traditional target detection algorithms based on manual feature extraction and classifiers can no longer meet the accuracy requirements of detection in complex environments and at multiple scales. With the continuous development of the efficiency and practicality of deep learning in solving image processing problems, using deep learning to extract depth variation features in aerial images can effectively avoid the shortcomings of traditional methods.

[0003] Currently, with the development of deep neural networks, research in object detection is basically divided into two directions: one is the Two-Stage method, which first generates a set of candidate boxes (bounding boxes) and then performs object classification and precise localization on these candidate boxes. A representative Two-Stage method is Faster R-CNN; the other is the One-Stage method, which attempts to complete object detection in one step, that is, simultaneously generating candidate boxes and performing object classification and precise localization. One-Stage methods are generally simpler in design and faster, but may sacrifice some accuracy in some cases. Representative One-Stage methods include YOLO. YOLO v8 is suitable for real-time applications with high speed requirements and performs well in small object detection; while Faster R-CNN is suitable for applications with high object detection accuracy requirements and performs better in large object detection and object localization.

[0004] Existing target detection methods based on YOLO v8 face two challenges in the unique environment of drone aerial photography. First, targets captured by drones from high altitudes are typically small, occupying a small percentage of pixels, making detection difficult. Second, drones have limited payloads and power supplies, necessitating increased network inference speed to improve efficiency per flight. Therefore, improving the YOLO v8 backbone network architecture, lightweighting the network model, and increasing inference speed, while maintaining existing accuracy, is crucial for applying YOLO v8 to target detection in drone aerial images. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detecting alfalfa seed pod maturity based on UAV aerial images using an improved YOLO v8, in order to overcome the shortcomings of existing technologies, such as high detection difficulty due to the clustering of detection targets into small targets, complex backbone networks, and insufficient real-time performance.

[0006] This invention is achieved using the following technical solution:

[0007] A method for detecting alfalfa seed pod maturity based on improved YOLO v8 drone aerial images includes the following steps:

[0008] (1) Use drones to collect alfalfa images in the field, classify and label the collected images using annotation tools to obtain a dataset with category labels, and divide the processed image dataset with category labels into training set and test set;

[0009] (2) The obtained image dataset with category labels is preprocessed and the preprocessed images are input into the improved YOLO v8 network model to obtain drone aerial image feature maps at different scales. The improved YOLO v8 network model replaces VanillaNet with the backbone network.

[0010] (3) Input the obtained drone aerial image feature map into the Head part of the improved YOLO v8 network. The Head part upsamples and fuses the drone aerial image feature maps of different scales to obtain data of different scales.

[0011] (4) Input the data of different scales into the prediction layer of the improved YOLO v8 network model, remove the large detection head in the prediction layer, and finally obtain the detection box of alfalfa seed pods taken by drone aerial photography. Output the detection map of alfalfa seed pod maturity, and the improved YOLO v8 network model is trained to obtain the maturity prediction network model.

[0012] (5) Input the test set images into the maturity prediction model and output the maturity detection results. The alfalfa seed pod maturity detection results are obtained. The results are evaluated using three parameters: MPDIoU, average precision, and inference speed.

[0013] Furthermore, the image acquisition method in step (1) is as follows:

[0014] Five cell regions were established in the alfalfa field using the five-point method, with 24 cells in each region, totaling 120 cells. Each cell is 5.4 meters long and 4.2 meters wide. The drone flew from the bottom left corner of each cell region to the right, collecting images of each cell in sequence from 1 to 5. Images of these five cell regions were collected for five consecutive days to facilitate recording the growth changes of the alfalfa pods in each cell. The small squares 1 to 5 are squares with the same side length of 1.2 meters. The small squares 1 to 4 are 0.5 meters from the edge. The center of the small square 5 coincides with the center of the cell.

[0015] Furthermore, in step (1), the processed image dataset with category labels is divided into a training set and a test set according to an 8:2 ratio.

[0016] Furthermore, in step (1), alfalfa seed pods are divided into three categories based on their maturity: green, brown, and black.

[0017] Furthermore, in step (2), the backbone network VanillaNet uses a 4×4×3×C convolutional layer with a stride of 4 in the first convolutional layer, following popular settings, to map an image with 3 channels to features with C channels. In the second, third, and fourth convolutional layers, max pooling layers with a stride of 2 are used to reduce the size and feature map, and the number of channels is doubled sequentially. In the fifth convolutional layer, an average pooling layer is used, and the last layer is a fully connected layer that outputs the classification result. In the second to fifth convolutional layers, each convolutional kernel is 1×1 in size, and an activation function is applied after each convolutional layer, and batch normalization is added, where batch normalization is defined as follows:

[0018] X i+1 =γ(x i -μb) / σ B +β

[0019] In the formula x i+1 μ represents the data after batch normalization. b σ represents the mean of this batch of samples. B Indicates the sample variance;

[0020] The VanillaNet activation function is defined as follows:

[0021] A(x)=n·max(0,1-|x|)+max(0,x)

[0022] In the formula, max(0,x) is the ReLU function, n is an adjustable parameter, and max(0,1-|x|) equals 1-|x|.

[0023] Furthermore, the improved YOLO v8 network feature fusion layer described in steps (2) and (3) is an AFPN structure. AFPN first extracts the last layer features from each feature layer of the backbone network to generate a set of features at different scales, represented as {C2, C3, C4, C5}. Then, AFPN initially fuses the low-level features, then fuses the deep-level features, and finally fuses the highest-level features, i.e., the most abstract features.

[0024] Furthermore, in step (4), the maturity of the alfalfa seed pods is calculated using the following formula:

[0025]

[0026] In the formula, A represents the maturity of alfalfa seed pods, C1 represents the number of green seed pods identified in the test results, C2 represents the number of gray seed pods identified in the test results, and C3 represents the number of black seed pods identified in the test results.

[0027] In step (5), MPDIoU is the target box loss function, and its calculation formula is as follows:

[0028]

[0029]

[0030] In the formula, IoU represents the intersection-union ratio, A and B represent any two object detection boxes, d1 and d2 represent the shape cost between the two bounding boxes, and h 2 +w 2 It refers to the sum of the squares of the height and width of the input image.

[0031] Furthermore, in step (5), the average accuracy is calculated by plotting the PR curve, and the inference speed is defined as the number of images that can be detected in one second.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] The method of this invention improves the accuracy of the original YOLO v8 while improving the YOLO v8 backbone network architecture, making its network model lightweight and improving the inference speed, thereby realizing fast and accurate UAV aerial photography detection of alfalfa seed pod maturity. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the process of the present invention;

[0035] Figure 2 A top view of an alfalfa plant;

[0036] Figure 3 This is a schematic diagram of the cell area in this invention;

[0037] Figure 4 This is a flight sequence diagram for the drone;

[0038] Figure 5 A color classification chart for alfalfa seed pods;

[0039] Figure 6 This is a partial structural diagram of the VanillaNet backbone network of the improved YOLO v8 network in this invention;

[0040] Figure 7 The MPDIoU loss function parameter curve of the improved YOLO v8 network in this invention;

[0041] Figure 8 This is the average accuracy parameter curve of the improved YOLO v8 network in this invention. Detailed Implementation

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, a method for detecting alfalfa seed pod maturity based on UAV aerial images using an improved YOLO v8 is described, with the following specific steps:

[0044] (1) Use drones to take aerial photos of alfalfa in the field;

[0045] Image acquisition method: Five cell regions were established in the alfalfa field using the five-point method, with 24 cells in each region, totaling 120 cells. Each cell is 5.4 meters long and 4.2 meters wide. The alfalfa plants are viewed from above as follows: Figure 2 As shown, a single cell range is as follows Figure 3 As shown;

[0046] See Figure 4 drones from Figure 3 Starting from the bottom left corner, the camera flies sequentially from left to right across each cell area. Images are collected from each cell in order from 1 to 5. Images are collected from these five cell areas for five consecutive days to facilitate recording the growth changes of the alfalfa seed pods in each cell. The sides of the small squares 1 to 5 are the same, forming a square with sides of 1.2 meters. The small squares 1 to 4 are 0.5 meters from the edge. The center of the small square 5 coincides with the center of the cell.

[0047] After screening the drone aerial images, they were classified and labeled using the LabelImg image annotation tool. Based on the color of the alfalfa seed pods at different maturity levels, they were divided into three categories: green, brown, and black. (See [link to related documentation]). Figure 5 ;

[0048] The 2000 images with class labels after processing were divided into a training set of 1600 images and a test set of 400 images in an 8:2 ratio. The image size was 2000×1500.

[0049] (2) The image dataset with category labels obtained in step (1) is preprocessed, and the preprocessed images are input into the improved YOLO v8 network model to obtain drone aerial image feature maps at different scales. The improved YOLO v8 network replaces VanillaNet with the backbone network.

[0050] See Figure 6 VanillaNet uses a 4×4×3×C convolutional layer with a stride of 4 in the first convolutional layer, following popular settings, to map an image with 3 channels to features with C channels. In the 2nd, 3rd, and 4th convolutional layers, max pooling layers with a stride of 2 are used to reduce the size and feature map, and the number of channels is doubled accordingly. In the 5th convolutional layer, an average pooling layer is used, and the last layer is a fully connected layer that outputs the classification result. In convolutional layers 2-5, each convolutional kernel is 1×1 in size, and an activation function is applied after each convolutional layer, followed by batch normalization, which is defined as follows:

[0051] X i+1 =γ(x i -μb) / σ B +β

[0052] In the formula x i+1 μ represents the data after batch normalization. b σ represents the mean of this batch of samples. B Indicates the sample variance;

[0053] The VanillaNet activation function is defined as follows:

[0054] A(x)=n·max(0,1-|x|)+max(0,x)

[0055] In the formula, max(0,x) is the ReLU function, n is an adjustable parameter, and max(0,1-|x|) equals 1-|x|.

[0056] (3) Input the drone aerial image feature map obtained in step (2) into the Head part of the improved YOLO v8 network. The Head part upsamples and fuses the drone aerial image feature maps of different scales to obtain data of different scales.

[0057] Specifically, the improved YOLO v8 network feature fusion layer is an AFPN structure. AFPN first extracts the last layer features from each feature layer of the backbone network to generate a set of features at different scales, denoted as {C2, C3, C4, C5}. Then, AFPN initially fuses the low-level features, then the deep-level features, and finally the highest-level features, i.e. the most abstract features.

[0058] (4) Input the tensor data of different scales obtained in step (3) into the prediction layer of the improved YOLO v8 network model, remove the large detection head in the prediction layer, and finally obtain the detection box of alfalfa seed pods taken by drone aerial photography, output the detection map of alfalfa seed pod maturity, and the improved YOLO v8 network model is trained to obtain the maturity prediction network model.

[0059] (5) Input the test set images into the maturity prediction model and output the maturity detection results to obtain the alfalfa seed pod maturity detection results. Use three parameters, MPDIoU, average precision, and inference speed, to evaluate the results.

[0060] The maturity of alfalfa seed pods can be calculated using the following formula:

[0061]

[0062] In the formula, A represents the maturity of alfalfa seed pods, C1 represents the number of green seed pods identified in the test results, C2 represents the number of gray seed pods identified in the test results, and C3 represents the number of black seed pods identified in the test results.

[0063] MPDIoU is the target box loss function, and its calculation formula is as follows:

[0064]

[0065]

[0066] In the formula, IoU represents the intersection-union ratio, A and B represent any two object detection boxes, d1 and d2 represent the shape cost between the two bounding boxes, and h 2 +w 2 It refers to the sum of the squares of the height and width of the input image.

[0067] Average precision is calculated by plotting a PR curve, where precision is the accuracy, recall is the recall rate, and inference speed is defined as the number of images that can be detected per second.

[0068] Simulation experiment:

[0069] The implementation platform was Ubuntu 22.10 operating system. Model training was completed in an experimental environment with an NVIDIA 3090 (24GB VRAM) GPU and CUDA 12.2. The specific experimental steps are as follows:

[0070] (1) Dataset selection: After classifying and labeling the alfalfa field images taken by drones, 1600 images were used as the training set and 400 images were used as the test set.

[0071] (2) Improved YOLO v8 training parameter settings: 300 training rounds, step size of 16, and initial learning rate of 0.001.

[0072] (3) Analysis of training results: The detection results of alfalfa seed pods by UAV aerial photography based on the improved YOLO v8 network model are evaluated by MPDIoU, average precision and inference speed. The smaller the MPDIoU ratio, the higher the accuracy of the target box output. The larger the average precision, the higher the accuracy of the detection result. The faster the inference speed, the better the real-time performance of the detection network.

[0073] Figure 7 To improve the MPDIoU loss function parameter curve of the YOLO v8 network, the vertical axis represents the loss value and the horizontal axis represents the number of training epochs. Figure 8 To improve the average accuracy parameter curve of the YOLO v8 network, the vertical axis represents the average accuracy value, and the horizontal axis represents the number of training epochs. The improved YOLO v8 network can detect 100 images per second, while the unimproved YOLO v8 network can detect 80 images per second. The improved YOLO v8 algorithm improves the detection speed by 25% and has better real-time performance.

[0074] (4) The maturity of alfalfa pods is obtained by calculating the proportion of black and gray pods based on the test results.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting alfalfa seed pod maturity based on UAV aerial images using an improved YOLO v8, characterized in that: Includes the following steps: (1) Use drones to collect alfalfa images in the field, classify and label the collected images using annotation tools to obtain a dataset with category labels, and divide the processed image dataset with category labels into training set and test set; (2) The obtained image dataset with category labels is preprocessed and the preprocessed images are input into the improved YOLO v8 network model to obtain drone aerial image feature maps at different scales. The improved YOLO v8 network model replaces VanillaNet with the backbone network. (3) Input the obtained UAV aerial image feature map into the Head part of the improved YOLO v8 network. The Head part upsamples and fuses the UAV aerial image feature maps of different scales to obtain data of different scales. (4) Input the data of different scales into the prediction layer of the improved YOLO v8 network model, remove the large detection head in the prediction layer, and finally obtain the detection box of alfalfa seed pods taken by drone aerial photography, output the detection map of alfalfa seed pod maturity, and the improved YOLO v8 network model is trained to obtain the maturity prediction network model. (5) Input the test set images into the maturity prediction model and output the maturity detection results. The alfalfa seed pod maturity detection results will be obtained. The results will be evaluated using three parameters: MPDIoU, average precision, and inference speed. In step (2), the backbone network VanillaNet uses a 4×4×3×C convolutional layer with a stride of 4 in the first convolutional layer, following popular settings, to map an image with 3 channels to features with C channels. In the second, third, and fourth convolutional layers, max pooling layers with a stride of 2 are used to reduce the size and feature map, and the number of channels is doubled sequentially. In the fifth convolutional layer, an average pooling layer is used, and the last layer is a fully connected layer that outputs the classification result. In the second to fifth convolutional layers, each convolutional kernel is 1×1 in size, and an activation function is applied after each convolutional layer, and batch normalization is added. The batch normalization is defined as follows: In the formula x i + 1 μ represents the data after batch normalization. b σ represents the mean of this batch of samples. B Indicates the sample variance; The VanillaNet activation function is defined as follows: In the formula, max(0, x) is the ReLU function, n is an adjustable parameter, and max(0, 1 - |x|) equals 1 - |x|; The improved YOLO v8 network feature fusion layer described in steps (2), (3), and (4) is an AFPN structure. AFPN first extracts the last layer features from each feature layer of the backbone network to generate a set of features at different scales, denoted as {C2, C3, C4, C5}. Then, AFPN initially fuses the low-level features, then fuses the deep-level features, and finally fuses the highest-level features, i.e., the most abstract features.

2. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 1, characterized in that: The image acquisition method in step (1) is as follows: Five cell regions were established in the alfalfa field using the five-point method, with 24 cells in each region, for a total of 120 cells. Each cell is 5.4 meters long and 4.2 meters wide. The drone flies from the bottom left corner of each cell area sequentially from left to right, collecting images of each cell in order from 1 to 5. This process is repeated for five consecutive days to record the growth and changes of the alfalfa pods in each cell. The sides of the small squares 1 to 5 are the same, each being a 1.2-meter square. The small squares 1 to 4 are 0.5 meters from the edge, and the center of the small square 5 coincides with the center of the cell.

3. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 1, characterized in that: In step (1), the processed image dataset with category labels is divided into a training set and a test set according to an 8:2 ratio.

4. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 1, characterized in that: In step (1), alfalfa seed pods are classified into three categories based on their maturity: green, brown, and black.

5. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 1, characterized in that: In step (5), the maturity of alfalfa seed pods is calculated using the following formula: In the formula, A represents the maturity of alfalfa seed pods, C1 represents the number of green seed pods identified in the test results, C2 represents the number of gray seed pods identified in the test results, and C3 represents the number of black seed pods identified in the test results.

6. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 1, characterized in that: In step (5), MPDIoU is the target box loss function, and its calculation formula is as follows: In the formula, IoU represents the intersection-union ratio, A and B represent any two object detection boxes, d1 and d2 represent the shape cost between the two bounding boxes, and h 2 +w 2 It refers to the sum of the squares of the height and width of the input image.

7. The method for detecting alfalfa seed pod maturity based on improved YOLO v8 UAV aerial images according to claim 6, characterized in that: In step (5), the average accuracy is calculated by plotting the PR curve, and the inference speed is defined as the number of images that can be detected in one second.

Images