A green mandarin orange detection method based on improved YOLOv8 network model
By improving the YOLOv8 network model and using DCNv2 and Deformable-LKA attention mechanisms, the missed detection and false detection problems in green Wogan detection are solved, the detection accuracy and speed are improved, and more efficient Wogan target detection is achieved.
Patent Information
- Application Number
- CN202411202539.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-29
AI Technical Summary
In the prior art, the missed detection rate and false detection rate of green wool perilla detection are relatively high, making it difficult to accurately distinguish between wool perilla and its leaves and surrounding green plants, resulting in low detection accuracy.
Using the improved YOLOv8 network model, replace the Conv in the backbone network and feature enhancement network to DCNv2, build the Bottleneck_Dcnv2 structure, and introduce the Deformable-LKA attention mechanism, and use the C2f_DCNv2 module to replace the C2f module to enhance the receptive field and feature extraction capabilities of the model, and reduce the False detection rate.
The accuracy and speed of green wool detection have been significantly improved, the missed detection rate is reduced, the error detection rate is reduced, the accuracy rate is increased by 2.7%, the recall rate is increased by 2.2%, and the mAP 0.5 is increased by 1.4%.
Smart Images

Figure CN118710925B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a green mandarin orange detection method, in particular to a green mandarin orange detection method based on an improved YOLOv8 network model, and belongs to the technical field of target detection. Background Art
[0002] With the continuous development of image recognition and artificial intelligence technology, target detection technology has been widely used in agricultural production. As an important variety of citrus fruits, green mandarin oranges are widely demanded in domestic and foreign markets. However, due to the influence of various factors such as the growth environment and picking methods, the quality and yield of green mandarin oranges are difficult to guarantee. Therefore, the research on detection technology of green mandarin oranges can provide strong support for fruit quality control and yield prediction. Due to the large number and density of mandarin oranges, there are problems of fruit overlap and occlusion, and the green mandarin oranges are relatively small and similar in color to their leaves and surrounding green plants, making them difficult to distinguish and capture. Therefore, the missed detection rate and false detection rate are high.
[0003] Therefore, a green mandarin orange detection method based on an improved YOLOv8 network model is proposed to address the above problems. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention provides a green mandarin orange detection method based on an improved YOLOv8 network model, which solves the problems of high missed detection rate and high false detection rate in green mandarin orange detection in the prior art, so as to improve the accuracy of green mandarin orange detection.
[0005] Technical solution of the present invention: The green mandarin orange detection method based on the improved YOLOv8 network model of the present invention specifically comprises the following steps:
[0006] The Step 1 is specifically as follows:
[0007] Step 1.1: The shooting tool is a drone, which is equipped with an image sensor, an aperture of F2.8, and the resolution of the photos taken is 8K, and the resolution of the videos taken is 4K / 60P;
[0008] Step 1.2: The original data captured are all videos;
[0009] Step 1.3: There are two ways to shoot:
[0010] The first method was to move and shoot in four directions, front, back, left, and right, above the Wogan tree until the green Wogan fruits in the tree disappeared from the camera, ensuring that all the green Wogan fruits of a Wogan tree were photographed. This method selected 30 Wogan trees for sampling.
[0011] The second method is to scan all the fruits from top to bottom on both the front and back sides of a Wogan fruit tree, which also ensures that all the green Wogan fruits of a Wogan fruit tree are photographed. This method selected 10 Wogan fruit trees for sampling.
[0012] The Step 2 is specifically as follows:
[0013] Step 2.1: Take one frame of each video at an interval of 30 to avoid overfitting of the model during training due to too many repetitions of the same picture. Finally, 3,500 green mandarin orange pictures were obtained.
[0014] Step 2.2: Use the labelimg annotation tool to annotate the bounding boxes of 3,500 green mandarin orange images:
[0015] First, create a folder named orange_data, and create two folders named images and labels under it to store 3,500 images and their corresponding labels.
[0016] Then, determine that the only target in the image is orange, and add a classes.txt file to the labels folder to store the orange category;
[0017] Finally, after importing 3,500 pictures, change the save format to YOLO, and create rectangular boxes to mark the oranges in the pictures.
[0018] The Step 3 is specifically as follows:
[0019] The labeled Wogan data set is divided into training set, validation set and test set in a ratio of 5:1:1, of which 2500 images are used as training set, 500 images are used as validation set and 500 images are used as test set.
[0020] The Step 4 is specifically as follows:
[0021] Step 4.1: The improved YOLOv8 network model replaces Conv in the backbone network and feature enhancement network with DCNv2; wherein, the DCNv2 is an improvement on the deformable convolution DCN. Compared with DCN, DCNv2 introduces learnable deformation parameters while increasing the learnable offset, which not only has a stronger receptive field adjustment capability, but also enables the convolution kernel to adaptively adjust its position on the input feature map. Due to the large number and density of Wogan, there are problems of overlap and occlusion, resulting in complex changes in the shapes and sizes of the detected targets at different angles. The introduction of DCNv2 can better capture the tiny details of the target and adapt to the shape of the target, reduce the missed detection rate, and effectively improve the detection effect.
[0022] Step 4.2: The improved YOLOv8 network model replaces the Conv in the CBS convolution structure in the bottleneck with DCNv2 to construct the DBS structure, thereby realizing the construction of the Bottleneck_Dcnv2 structure; wherein, the bottleneck is an important component of C2f, and the CBS convolution structure is used internally. The CBS convolution structure consists of a layer of standard convolution (Conv), a BN layer, and a Silu layer. Standard convolution has considerable limitations in the green mandarin orange scene with dense targets, resulting in inaccurate feature learning by C2f. The construction of the Bottleneck_Dcnv2 structure improves the model's ability to extract target features.
[0023] Step 4.3: The improved YOLOv8 network model introduces the Deformable-LKA attention mechanism in Bottleneck_Dcnv2; the Deformable-LKA attention mechanism includes: standard 2D convolution (Conv2D), deformable convolution with offset (Deformable Convolution, Deform-DW Conv2D), calculation of offset field (Offsets Field), activation function GELU. Since green mandarin oranges are generally similar in color to their leaves and surrounding green plants, the traditional YOLOv8 algorithm will have the problem of false detection when detecting them. The introduction of the Deformable-LKA attention mechanism has greatly reduced the false detection rate.
[0024] Step 4.4: The improved YOLOv8 network model replaces the stacked ordinary Bottleneck in the C2f structure with Bottleneck_Dcnv2, thereby completing the construction of the C2f_DCNv2 module. The C2f_DCNv2 module is used to replace the C2f module in the backbone network and the feature enhancement network; wherein the C2f_DCNv2 module first uses 1*1 convolution to change the number of channels for the input features, and then uses the Split operation to replace the 1*1 convolution to split the features, and at the same time, stacks multiple DCNv2 modules to increase the network receptive field, and uses more jump connections, so that the network has a richer structure with gradient flow while reducing the number of parameters, and can extract richer multi-scale features, improving the expressiveness of the feature map.
[0025] Step 4.5: Input the images in the training set into the improved YOLOv8 network model for training to obtain a trained improved YOLOv8 network model.
[0026] The Step 5 is specifically as follows:
[0027] The green mandarin orange images in the divided test set are input into the trained improved YOLOv8 network model for detection. First, the input green mandarin orange images are used by the network model to extract green mandarin orange features, and then the network model fuses the extracted green mandarin orange features and predicts the bounding box of the green mandarin orange. At the same time, all bounding boxes are subjected to non-maximum suppression operations to ensure that each green mandarin orange is accurately marked by only one bounding box, and then the marked bounding box is given a category and confidence score, and finally the detection result of the green mandarin orange is output. According to the experimental results, the improved YOLOv8 network model is 2.7 percentage points higher in precision, 1.4 percentage points higher in mAP0.5, and 2.2 percentage points higher in recall than the original YOLOv8 network model.
[0028] Beneficial effects of the present invention:
[0029] Compared with the prior art, the present invention replaces the Conv in the backbone network and the feature enhancement network with DCNv2, which effectively solves the problems of green mandarin orange overlap and occlusion. The construction of Bottleneck_Dcnv2 improves the model's ability to extract target features of green mandarin oranges. The introduction of the Deformable-LKA attention mechanism reduces the false picking rate. The designed C2f_DCNv2 module is used to replace the C2f module in the backbone network and the feature enhancement network, which improves the expression ability of the feature map, thereby greatly improving the accuracy and speed of green mandarin orange detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a green mandarin orange detection flow chart of the improved YOLOv8 network model of the present invention;
[0031] Figure 2 is a DCNv2 structural diagram of the present invention;
[0032] Figure 3 It is a structural diagram of the Deformable-LKA attention mechanism of the present invention;
[0033] Figure 4 It is a structural diagram of the C2f_DCNv2 module of the present invention;
[0034] Figure 5 It is a structural diagram of the improved YOLOv8 network architecture of the present invention. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] Example:
[0037] like Figure 1-Figure 5 As shown, taking the Wogan planting base in Shuitang Town, Xinping Yi and Dai Autonomous County, Yuxi City, Yunnan Province as an example, green Wogan detection is performed. The green Wogan detection method based on the improved YOLOv8 network model includes the following steps:
[0038] Step 1 is as follows:
[0039] Step 1.1: The shooting tool is a drone, which is equipped with an image sensor, an aperture of F2.8, and the resolution of the photos taken is 8K, and the resolution of the videos taken is 4K / 60P;
[0040] Step 1.2: The shooting location is the Wogan planting base in Shuitang Town, Xinping Yi and Dai Autonomous County, Yuxi City, Yunnan Province. The original data shot are all videos;
[0041] Step 1.3: There are two ways to shoot:
[0042] The first method was to move and shoot in four directions, front, back, left, and right, above the Wogan tree until the green Wogan fruits in the tree disappeared from the camera, ensuring that all the green Wogan fruits of a Wogan tree were photographed. This method selected 30 Wogan trees for sampling.
[0043] The second method is to scan all fruits from the top to the bottom on both the front and back sides of a Wogan fruit tree, which also ensures that all green Wogan fruits of a Wogan fruit tree are photographed. This method selects 10 Wogan fruit trees for sampling. Step 2 is as follows:
[0044] Step 2.1: Take one frame of each video at an interval of 30 to avoid overfitting of the model during training due to too many repetitions of the same picture. Finally, 3,500 green mandarin orange pictures were obtained.
[0045] Step 2.2: Use labelimg annotation tool to annotate the bounding boxes of 3500 green mandarin orange images;
[0046] First, create a folder named orange_data, and create two folders named images and labels under it to store 3,500 images and their corresponding labels.
[0047] Then, determine that the only target in the image is orange, and add a classes.txt file to the labels folder to store the orange category;
[0048] Finally, after importing 3,500 pictures, change the save format to YOLO, and create rectangular boxes to mark the oranges in the pictures.
[0049] Step 3 is as follows:
[0050] The labeled Wogan data set is divided into training set, validation set and test set in a ratio of 5:1:1, of which 2500 images are used as training set, 500 images are used as validation set and 500 images are used as test set.
[0051] Step 4 is as follows:
[0052] Step 4.1: Improved YOLOv8 network model, replace Conv in the backbone network and feature enhancement network with DCNv2; DCNv2 is an improvement on the deformable convolution DCN. Compared with DCN, DCNv2 introduces learnable deformation parameters while increasing the learnable offset. This not only has a stronger receptive field adjustment capability, but also enables the convolution kernel to adaptively adjust its position on the input feature map. Due to the large number and density of Wogan, there are problems of overlap and occlusion, resulting in complex changes in the shape and size of the detected target at different angles. The introduction of DCNv2 can better capture the tiny details of the target and adapt to the shape of the target, reducing the missed detection rate and effectively improving the detection effect.
[0053] Step 4.2: Improved YOLOv8 network model, replace Conv in CBS convolution structure in bottleneck with DCNv2, build DBS structure, thus realize the construction of Bottleneck_Dcnv2 structure; among them, bottleneck is an important component of C2f, and its internal structure uses CBS convolution structure. Among them, CBS convolution structure consists of a standard convolution (Conv) layer, BN layer and Silu layer. Standard convolution has considerable limitations in the green mandarin scene with dense targets, which makes C2f's feature learning inaccurate. The implementation of Bottleneck_Dcnv2 improves the model's ability to extract target features.
[0054] Step 4.3: Improved YOLOv8 network model, introduce Deformable-LKA attention mechanism in Bottleneck_Dcnv2; Deformable-LKA attention mechanism includes: standard 2D convolution (Conv2D), deformable convolution with offset (Deformable Convolution, Deform-DW Conv2D), calculation of offset field (Offsets Field), activation function GELU. Since green mandarin oranges are generally small and similar in color to their leaves and surrounding green plants, the traditional YOLOv8 algorithm will have the problem of misdetection when detecting them. The introduction of the Deformable-LKA attention mechanism has greatly reduced the mispick rate.
[0055] Step 4.4: Improved YOLOv8 network model, replace the stacked ordinary Bottleneck in the C2f structure with Bottleneck_Dcnv2, thus completing the construction of the C2f_DCNv2 module. And use the C2f_DCNv2 module to replace the C2f module in the backbone network and feature enhancement network; among them, the C2f_DCNv2 module first uses 1×1 convolution to change the number of channels for the input features, and then uses the Split operation to replace the 1×1 convolution to split the features. At the same time, the network receptive field is increased by stacking multiple DCNv2 modules, and more jump connections are used. In this way, the network has a richer structure with gradient flow while reducing the number of parameters, which can extract richer multi-scale features and improve the expression ability of the feature map.
[0056] Step 4.6: Input the images in the training set into the improved YOLOv8 network model for training to obtain a trained improved YOLOv8 network model.
[0057] Step 5 is as follows:
[0058] Step 5.1: Feature extraction. The preprocessed green mandarin orange image is input into the improved YOLOv8 network model, and the network will extract features from the green mandarin orange image. In the backbone layer, the green mandarin orange image is convoluted and pooled layer by layer to extract the key features of the green mandarin orange image.
[0059] Step 5.2: Feature fusion. The green mandarin orange image features extracted from different layers are received by the neck layer. The neck layer is responsible for the aggregation and processing of features. The received green mandarin orange image feature information is fused in the neck layer, and the learned features are output to the head layer.
[0060] Step 5.3: Bounding box prediction. The head layer maps the learned features to the prediction task, and the network model predicts the bounding boxes of green mandarin oranges based on the extracted features. These bounding boxes mark the location and size of the green mandarin oranges that may exist in the green mandarin orange image;
[0061] Step 5.4: Non-maximum suppression. Since the network model may predict multiple overlapping bounding boxes, a non-maximum suppression (NMS) operation is required. NMS removes those bounding boxes that have a high degree of overlap with the highest-scoring bounding box, thereby ensuring that each green mandarin orange is accurately marked by only one bounding box;
[0062] Step 5.5: Classification and confidence score. For each predicted bounding box, the network model will also classify the target and give a confidence score. Since there is only one category of green mandarin oranges, the predicted bounding box only shows the category of green mandarin oranges. The confidence score indicates the confidence of the network model in the prediction result;
[0063] Step 5.6: Output the results. The detection results after the above steps will be output, including the category of each detected target, the location and size of the bounding box, and the confidence score.
[0064] In order to verify the effect of the present invention, the inventors conducted comparative experiments and ablation experiments respectively, and the data sets used in the experiments were all self-built green mandarin orange data sets.
[0065] Experimental example:
[0066] Experimental environment of the present invention:
[0067] (1) The software and hardware environment required for the operation of this system is shown in Table 1.
[0068] Table 1 Hardware and software environment
[0069]
[0070] The training environment required for the operation of this system is shown in Table 2.
[0071] Table 2 Training environment
[0072]
[0073] Comparative experiment of the present invention:
[0074] Table 3 Comparative experimental results
[0075]
[0076] By comparing the experimental results, it can be seen that the improved algorithm of the present invention has been improved in terms of precision, recall and mAP0.5 compared with YOLOv3, YOLOv3-tiny, YOLOv5 and YOLOv6 algorithms, and has better performance in terms of parameter quantity, model size and computational complexity.
[0077] Ablation experiment of the present invention:
[0078] Table 4 Module ablation experiment results
[0079]
[0080] Note: √ means adding modules; - means not adding modules; bold means optimal data values
[0081] From the ablation experiment results, it can be seen that in the ablation experiment of designing deformable convolution DCNv2 to replace standard convolution Conv with YOLOv8n as the baseline, replacing C2f with C2f_DCNv2, and Deformable_LKA attention mechanism, after replacing the standard convolution Conv with DCNv2, mAP@0.5 increased by 0.1 percentage points and Recall increased by 0.5 percentage points; after replacing C2f of the backbone network with C2f_DCNv2, although Recall did not increase, mAP@0.5 increased by 0.7 percentage points; after adding Deformable_LKA attention mechanism in C2f, mAP@0.5 increased by 0 .6 percentage points, and Recall increased by 1.5 percentage points; replacing the standard convolution Conv with deformable convolution DCNv2 and replacing C2f with C2f-DCNv2, its mAP@0.5 increased by 0.8 percentage points, and Recall increased by 1.0 percentage points; designing C2f_DCNv2 and adding Deformable_LKA attention mechanism, its mAP@0.5 increased by 1.3 percentage points, and Recall increased by 1.4 percentage points. On this basis, replacing the standard convolution Conv with deformable convolution DCNv2, mAP@0.5 increased by 1.4 percentage points, and Recall increased by 2.2 percentage points. This fully verifies the effectiveness of the improvement of the present invention.
[0082] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A green mandarin orange detection method based on an improved YOLOv8 network model, characterized in that: Step 1: Obtain self-built green Wogan data and use drones to fly to the Wogan planting base to shoot videos of immature green Wogan; Step 2: Data processing: capture green mandarin orange images from the green mandarin orange video at 30 frames per second, and use labelimg to annotate the obtained green mandarin orange images with rectangular frames; Step 3: Dataset division: divide the labeled Wogan dataset into training set, validation set and test set; Step 4: Train the improved YOLOv8 network model; Step 5: Use the trained improved YOLOv8 network model to detect the green mandarin oranges and obtain the green mandarin orange detection results; The Step 4 is specifically as follows: Step 4.1: In the improved YOLOv8 network model, replace Conv in the backbone network and feature enhancement network with DCNv2; Among them, the DCNv2 is an improvement on the deformable convolution DCN; Step 4.2: The improved YOLOv8 network model replaces the Conv in the CBS convolution structure in the bottleneck with DCNv2 to build the DBS structure to achieve the construction of the Bottleneck_Dcnv2 structure; Among them, the bottleneck is an important component of C2f, and the CBS convolution structure is used internally. The CBS convolution structure consists of a layer of standard convolution Conv, a BN layer, and a Silu layer; Step 4.3: The improved YOLOv8 network model introduces the Deformable-LKA attention mechanism in Bottleneck_Dcnv2; The Bottleneck_Dcnv2 structure includes DBS, DBS and Deformable-LKA connected in sequence, and the DBS structure consists of Dcnv2, BN layer and Silu layer; The Deformable-LKA attention mechanism includes: standard 2D convolution, deformable convolution with offset, calculation of offset field and activation function GELU; Step 4.4: In the improved YOLOv8 network model, the stacked ordinary Bottleneck in the C2f structure is replaced with Bottleneck_Dcnv2, thereby completing the construction of the C2f_DCNv2 module, and the C2f_DCNv2 module is used to replace the C2f module in the backbone network and the feature enhancement network; The C2f_DCNv2 module first uses 1×1 convolution to change the number of channels for the input features, and then uses the Split operation to replace the 1×1 convolution to split the features. At the same time, it increases the network receptive field by stacking multiple DCNv2 modules and uses more skip connections. Step 4.5: Input the images in the training set into the improved YOLOv8 network model for training to obtain a trained improved YOLOv8 network model.
2. A green mandarin orange detection method based on an improved YOLOv8 network model according to claim 1, characterized in that: The Step 1 is specifically as follows: Step 1.1: The shooting tool is a drone, which is equipped with an image sensor, an aperture of F2.8, and the resolution of the photos taken is 8K, and the resolution of the videos taken is 4K / 60P; Step 1.2: The original data captured are all videos; Step 1.3: There are two ways to shoot: The first method is to move and shoot in four directions, front, back, left, and right, above the Wogan tree until the green Wogan fruits in the tree disappear in the camera lens, ensuring that all the green Wogan fruits of the tree are captured; The second method is to scan all the fruits from top to bottom on both the front and back sides of a Wogan fruit tree, which also ensures that all the green Wogan fruits of a Wogan fruit tree are photographed.
3. A green mandarin orange detection method based on an improved YOLOv8 network model according to claim 1, characterized in that: The Step 2 is specifically as follows: Step 2.1: Take one frame from each video at an interval of 30 frames to avoid overfitting during model training caused by too many repetitions of the same picture, and finally obtain 3500 green mandarin orange pictures; Step 2.2: Use the labelimg annotation tool to annotate 3500 green mandarin orange pictures with rectangular frames: First, create a folder named orange_data, and create two folders, image and labels, in the folder to store 3500 images and their corresponding labels. Then, determine that the only target in the image is orange, and add a classes.txt file to the labels folder to store the orange category; Finally, after importing 3,500 pictures, change the save format to YOLO, and create rectangular boxes to mark the oranges in the pictures.
4. A green mandarin orange detection method based on an improved YOLOv8 network model according to claim 1, characterized in that: The Step 3 is specifically as follows: The labeled Wogan dataset is divided into training set, validation set and test set in a ratio of 5:1:
1.
5. A green mandarin orange detection method based on an improved YOLOv8 network model according to claim 1, characterized in that: The Step 5 is specifically as follows: Input the green mandarin orange images in the divided test set into the trained improved YOLOv8 network model for detection; Step 5.1: The input green mandarin orange image is used by the network model to extract green mandarin orange features; Step 5.2: The network model integrates the extracted green mandarin orange features; Step 5.3: Predict the bounding box of the green mandarin orange; Step 5.4: Perform non-maximum suppression on all bounding boxes to ensure that each green orange is accurately marked by only one bounding box; Step 5.5: Give the category and confidence score to the marked bounding box; Step 5.6: Finally, output the detection results of green mandarin oranges.
