A method for persimmon fruit detection based on an improved YOLOv5 model
By improving the YOLOv5 model and utilizing a bidirectional pyramid network and an incentive attention mechanism, the problem of persimmon fruit detection in complex orchard environments was solved, achieving more efficient persimmon fruit identification and harvesting.
Patent Information
- Application Number
- CN202410352123.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-26
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-03-26
AI Technical Summary
Existing machine learning methods struggle to effectively detect dense, shading, and overlapping persimmon fruits in complex orchard environments, resulting in low efficiency for harvesting robots.
An improved YOLOv5 model is adopted, which enhances feature extraction and bounding box regression through a bidirectional pyramid network, squeezing and incentive attention mechanisms, and EIoU loss function, thereby improving detection performance.
This improved the accuracy and speed of persimmon fruit detection and enhanced the detection performance of the harvesting robot.
Smart Images

Figure CN118196757B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of fruit detection methods, and relates to a method for detecting persimmon fruit based on an improved YOLOv5 model. Background Technology
[0002] The use of robots for fruit harvesting in modern agriculture has become a new agricultural trend. Using robots for fruit harvesting offers several advantages: reduced manpower, lower labor costs, and increased labor productivity. Robotic mobile devices typically employ object detection technology to assist robots in harvesting fruit in complex environments. Factors such as fluctuating lighting, complex backgrounds, dense fruit distribution, overlapping persimmons, branches and leaves, camera angles, and distances all affect object detection in orchards. Traditional machine learning methods are unsuitable for detecting persimmons in complex and changing environments, while deep learning methods are still not ideal for detecting densely packed, overlapping persimmons in complex orchard environments, with significant differences in persimmon detection results among different deep learning methods. Therefore, researching a persimmon detection method with high accuracy and fast speed is crucial for improving the harvesting efficiency of harvesting robots. Summary of the Invention
[0003] The purpose of this invention is to provide a persimmon fruit detection method based on an improved YOLOv5 model. The existing YOLOv5 model is improved by using a bidirectional pyramid network, squeezing and excitation attention mechanisms, and EIoU loss function. This can more effectively achieve multi-scale feature fusion, improve the model's feature extraction capabilities, and improve detection performance.
[0004] The technical solution adopted in this invention is a persimmon fruit detection method based on an improved YOLOv5 model, which is implemented according to the following steps:
[0005] Step 1: Acquire persimmon images, perform image preprocessing, and then divide the preprocessed images into validation set, test set, and training set;
[0006] Step 2: Improve the YOLOv5 model to obtain the improved YOLOv5 model;
[0007] Step 3: Train the improved YOLOv5 model using images from the training set to obtain the optimal weights;
[0008] Step 4: Apply the optimal weights to the test set to obtain the model evaluation index, and use the evaluated and improved YOLOv5 model to perform persimmon fruit detection.
[0009] The invention is further characterized in that:
[0010] Step 1 is as follows:
[0011] Step 1.1: Collect persimmon images of different sizes, colors, shapes, poses, brightness, shadows, and occlusion levels as the raw dataset;
[0012] Step 1.2: The persimmon images in the original dataset are sequentially subjected to data enhancement processes including flipping, adding noise, motion blurring, and random brightness adjustment;
[0013] Step 1.3 involves manually labeling the persimmon images in the original dataset after enhancement in Step 1.2 using the LabelImg software, marking the category on each persimmon image. Each persimmon image must contain at least one category.
[0014] Step 1.4: Use a Python script to convert the XML file obtained in Step 1.3 into a TXT file, and then divide it into a validation set, a test set, and a training set.
[0015] In step 1, the ratio of the training set, validation set, and test set is 8:1:1.
[0016] Step 2 involves improving the YOLOv5 model by replacing the concat layer of the neck network with a bidirectional pyramid network module, adding a squeezing and attention-inducing mechanism to the C3 module in the neck network, and using EIoU as the model's loss function.
[0017] Step 3 specifically involves:
[0018] Step 3.1: Use the persimmon images in the training set as input to the improved YOLOv5 model, and use a convolutional module with 64 channels and a kernel size of 6×6 to extract features, perform feature dimensionality reduction, and normalize the feature map.
[0019] Step 3.2, the feature maps obtained in step 3.1 are then processed by a CBS module with 128 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0020] Step 3.3: The feature map after feature dimensionality reduction in step 3.2 is then processed by the C3 module with 128 channels and BottleNeck repeated 3 times to enhance feature representation capabilities;
[0021] Step 3.4: The feature map processed in step 3.3 is then processed by a CBS module with 256 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0022] Step 3.5: The feature map processed in step 3.4 is then processed by the C3 module with 256 channels and BottleNeck repeated 6 times to enhance its feature representation capability.
[0023] Step 3.6: The feature map processed in step 3.5 is then processed by a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0024] Step 3.7: The feature map processed in step 3.6 is then processed through the C3 module with 512 channels and BottleNeck repeated 9 times to enhance its feature representation capability;
[0025] Step 3.8: The feature map processed in step 3.7 is then processed by a CBS module with 1024 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0026] Step 3.9: The feature map processed in step 3.8 then passes through the C3 module, which has 1024 channels and repeats BottleNeck 3 times.
[0027] Step 3.10: The feature map processed in step 3.9 is then subjected to spatial pyramid pooling by a pooling layer.
[0028] Step 3.11: The feature map processed in step 3.10 is then processed again through a CBS module with 512 channels and a kernel size of 1×1 to extract image features;
[0029] 3.12, Apply the nearest neighbor upsampling operator to the feature map obtained in step 3.11 to perform an upsampling operation, and obtain a feature map with the same dimension as the output feature map in step 3.7;
[0030] Step 3.13: The features output from Step 3.7 and Step 3.12 are spliced and fused using a bidirectional pyramid network module according to the one-dimensional channel dimension to obtain the fused features;
[0031] Step 3.14: The fused features obtained in step 3.13 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times.
[0032] Step 3.15: Input the output of step 3.14 into a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0033] Step 3.16: Apply the nearest neighbor upsampling operator to the output of step 3.15 to perform an upsampling operation to obtain features of the same dimension as the output features of step 3.5;
[0034] Step 3.17: Concatenate and fuse the outputs of Step 3.16 and Step 3.5 according to the one-dimensional channel dimension to obtain the fused feature;
[0035] Step 3.18: The fused features obtained in step 3.17 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 256 channels and BottleNeck is repeated 3 times.
[0036] Step 3.19: Use the output of step 3.18 as input to access CBS with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0037] Step 3.20: Input the outputs of Step 3.19 and Step 3.15 back into the bidirectional pyramid network module and splice and fuse them according to the one-dimensional channel dimension to obtain the fused features;
[0038] Step 3.21: Input the output of step 3.20 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times.
[0039] Step 3.22: The output of step 3.21 is then input into a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0040] Step 3.23: The outputs of steps 3.21 and 3.11 are used as inputs to the bidirectional pyramid network module and spliced and fused according to the one-dimensional channel dimension to obtain the fused features;
[0041] Step 3.24: Input the output of step 3.23 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 1024 channels and BottleNeck is repeated 3 times.
[0042] Step 3.25: After passing the feature maps output from Steps 3.18, 3.21, and 3.25 through a convolutional layer, input them into the prediction network of the YOLOv5 model to predict the category.
[0043] Step 3.26: Following the process in steps 3.1-3.25, the improved YOLOv5 model is trained using persimmon images from the training set. During training, the EIoU loss function is used as the loss function for the improved YOLOv5 model. By setting the maximum number of iterations, batch size, initial learning rate, weight decay coefficient, optimizer, and data path of the improved YOLOv5 model, the trained improved YOLOv5 model is obtained.
[0044] The C3 module with the added squeezing and attention incentive mechanism is specifically as follows: the squeezing and attention incentive mechanism is added to the convolutional layer with a kernel size of 1×1 and a stride of 1 in the C3 module, and then the processed result is concatenated with the Bottleneck module in the C3 module in the one-dimensional channel dimension.
[0045] The maximum number of iterations is 150, the batch size is 24, the initial learning rate is 0.01, the optimizer uses stochastic gradient descent (SGD), and the weight decay coefficient is 0.0005.
[0046] The validation set is used to test the performance of the improved YOLOv5 model.
[0047] The beneficial effects of this invention are:
[0048] This invention employs a bidirectional pyramid network (BiFPN) as the neck network of the existing YOLOv5 model, which can more effectively achieve multi-scale feature fusion. Furthermore, by integrating the squeeze and stimulated attention (SE) mechanism into the C3 module, the model's feature extraction capability is further enhanced. Finally, EIoU is selected as the loss function, resulting in more accurate bounding box regression and object detection, thus improving convergence speed. Results show that the improved YOLOv5 model network exhibits excellent performance in precision, recall, and mAP, representing a significant improvement over the existing YOLOv5 model. It can be observed that the improved YOLOv5 model performs significantly better in terms of object detection box accuracy, confidence, and detection performance. Attached Figure Description
[0049] Figure 1 This is a diagram of the network structure of YOLOv5 in the existing technology;
[0050] Figure 2 This is a network structure diagram of BiFPN in the existing technology;
[0051] Figure 3 This is a network structure diagram of SE in the existing technology;
[0052] Figure 4 This is a network structure diagram of the C3 module incorporating the squeezing and attention-stimulating mechanisms of this invention;
[0053] Figure 5 This is a diagram of the improved YOLOv5 network structure of this invention;
[0054] Figure 6 This is a diagram of the CIoU loss function structure;
[0055] Figure 7 This is a diagram of the EIoU loss function structure;
[0056] Figure 8a This is a comparison chart of mAP from ablation experiments;
[0057] Figure 8b This is a comparison chart of the accuracy of ablation experiments.
[0058] Figure 8c This is a comparison chart of recall rates in ablation experiments;
[0059] Figure 9 This is a graph showing the detection results of persimmons at different ripeness levels using the existing YOLOv5 algorithm and the improved YOLOv5 algorithm of this invention in Embodiment 3 of the present invention. Detailed Implementation
[0060] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0061] Example 1
[0062] This invention relates to a method for detecting persimmon fruit based on an improved YOLOv5 model, which is implemented according to the following steps:
[0063] Step 1: Acquire persimmon images, perform image preprocessing, and then divide the preprocessed images into validation, test, and training sets; specifically:
[0064] Step 1.1: Collect persimmon images of different sizes, colors, shapes, poses, brightness, shadows, and occlusion levels as the raw dataset;
[0065] Step 1.2: The persimmon images in the original dataset are sequentially subjected to data enhancement processes including flipping, adding noise, motion blurring, and random brightness adjustment;
[0066] Step 1.3 involves manually labeling the persimmon images in the original dataset after enhancement in Step 1.2 using the LabelImg software, marking the category on each persimmon image. Each persimmon image must contain at least one category.
[0067] Step 1.4: Use a Python script to convert the XML file obtained in Step 1.3 into a TXT file, and then divide it into a validation set, a test set, and a training set. The ratio of the training set, validation set, and test set is 8:1:1.
[0068] Step 2, improve the YOLOv5 model, such as... Figure 5 As shown, specifically: the concat layer of the YOLOv5 model's neck network is replaced with a Bidirectional Pyramid Network (BiFPN) module; a squeezing and excitation attention mechanism (SE attention mechanism) is added to the C3 module in the YOLOv5 model's neck network; and EIoU is used as the model's loss function to obtain an improved YOLOv5 model; where, as Figure 4 As shown, the C3 module with the added squeezing and attention incentive mechanism is specifically as follows: the squeezing and attention incentive mechanism is added to the convolutional layer with a kernel size of 1×1 and a stride of 1 in the C3 module, and then the processed result is concatenated with the Bottleneck module in the C3 module in the one-dimensional channel dimension.
[0069] Step 3: Train the improved YOLOv5 model using images from the training set to obtain the optimal weights; specifically:
[0070] Step 3.1: Use the persimmon images in the training set as input to the improved YOLOv5 model, and use a convolutional module with 64 channels and a kernel size of 6×6 to extract features, perform feature dimensionality reduction, and normalize the feature map.
[0071] Step 3.2, the feature maps obtained in step 3.1 are then processed by a CBS module with 128 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0072] Step 3.3: The feature map after feature dimensionality reduction in step 3.2 is then processed by the C3 module with 128 channels and BottleNeck repeated 3 times to enhance feature representation capabilities;
[0073] Step 3.4: The feature map processed in step 3.3 is then processed by a CBS module with 256 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0074] Step 3.5: The feature map processed in step 3.4 is then processed by the C3 module with 256 channels and BottleNeck repeated 6 times to enhance its feature representation capability.
[0075] Step 3.6: The feature map processed in step 3.5 is then processed by a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0076] Step 3.7: The feature map processed in step 3.6 is then processed through the C3 module with 512 channels and BottleNeck repeated 9 times to enhance its feature representation capability;
[0077] Step 3.8: The feature map processed in step 3.7 is then processed by a CBS module with 1024 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction.
[0078] Step 3.9: The feature map processed in step 3.8 then passes through the C3 module, which has 1024 channels and repeats BottleNeck 3 times.
[0079] Step 3.10: The feature map processed in step 3.9 is then subjected to spatial pyramid pooling by a pooling layer.
[0080] Step 3.11: The feature map processed in step 3.10 is then processed again through a CBS module with 512 channels and a kernel size of 1×1 to extract image features;
[0081] 3.12, Apply the nearest neighbor upsampling operator to the feature map obtained in step 3.11 to perform an upsampling operation, and obtain a feature map with the same dimension as the output feature map in step 3.7;
[0082] Step 3.13: The features output from Step 3.7 and Step 3.12 are spliced and fused using a bidirectional pyramid network module according to the one-dimensional channel dimension to obtain the fused features;
[0083] Step 3.14: The fused features obtained in step 3.13 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times.
[0084] Step 3.15: Input the output of step 3.14 into a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0085] Step 3.16: Apply the nearest neighbor upsampling operator to the output of step 3.15 to perform an upsampling operation to obtain features of the same dimension as the output features of step 3.5;
[0086] Step 3.17: Concatenate and fuse the outputs of Step 3.16 and Step 3.5 according to the one-dimensional channel dimension to obtain the fused feature;
[0087] Step 3.18: The fused features obtained in step 3.17 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 256 channels and BottleNeck is repeated 3 times.
[0088] Step 3.19: Use the output of step 3.18 as input to access CBS with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0089] Step 3.20: Input the outputs of Step 3.19 and Step 3.15 back into the bidirectional pyramid network module and splice and fuse them according to the one-dimensional channel dimension to obtain the fused features;
[0090] Step 3.21: Input the output of step 3.20 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times.
[0091] Step 3.22: The output of step 3.21 is then input into a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction;
[0092] Step 3.23: The outputs of steps 3.21 and 3.11 are used as inputs to the bidirectional pyramid network module and spliced and fused according to the one-dimensional channel dimension to obtain the fused features;
[0093] Step 3.24: Input the output of step 3.23 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 1024 channels and BottleNeck is repeated 3 times.
[0094] Step 3.25: After passing the feature maps output from Steps 3.18, 3.21, and 3.25 through a convolutional layer, input them into the prediction network of the YOLOv5 model to predict the category.
[0095] Step 3.26: Following the process in steps 3.1-3.25, the improved YOLOv5 model is trained using persimmon images from the training set. During training, the EIoU loss function is used as the loss function for the improved YOLOv5 model. By setting the maximum number of iterations, batch size, initial learning rate, weight decay coefficient, optimizer, and data path of the improved YOLOv5 model, the trained improved YOLOv5 model is obtained.
[0096] Step 4: Apply the optimal weights to the test set to obtain the model evaluation index, and use the evaluated and improved YOLOv5 model to perform persimmon fruit detection.
[0097] Example 2
[0098] Based on Example 1, the maximum number of iterations is 150, the batch size is 24, the initial learning rate is 0.01, the optimizer uses stochastic gradient descent (SGD), and the weight decay coefficient is 0.0005.
[0099] Example 3
[0100] To verify the effectiveness of this invention, this embodiment collected persimmon images under different conditions from various image websites, totaling 653 images of persimmons with different sizes, colors, shapes, poses, brightness, shadows, and occlusion levels. To facilitate training, the images were compressed into uniform pixels and saved in JPG format. To prevent overfitting during training and improve the robustness and generalization of the experimental dataset, data augmentation (horizontal flip, vertical flip, Gaussian noise, motion blur, random brightness) was used to expand the 653 images to 4571 images.
[0101] Persimmon growth can be divided into two stages: immature (Gpersimmon) and mature (Persimmon). Persimmon image data was manually labeled using LabelImg software. This software allows for the selection of persimmon objects within each image and the labeling of their classification information, such as maturity level. The labeled dataset requires that each image contain at least one labeled persimmon object to ensure the completeness and accuracy of the dataset. To ensure the effectiveness of the labeling, fruits obscuring more than two-thirds of their surface are not labeled, and an Extensible Markup Language (XML) file containing category and coordinate information is generated after labeling. The labeled dataset is divided into three groups: training set, validation set, and test set, with a ratio of 8:1:1 between each group.
[0102] The existing YOLOv5 model has four different network structures, such as... Figure 1 As shown, these are YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5n, respectively. These four different network structures have different feature extraction modules, convolutional kernels, model sizes, and model parameters. They consist of four parts: the input network, the backbone network, the neck network, and the prediction network.
[0103] The input network includes Mosaic data augmentation, adaptive anchor box computation, and adaptive image scaling. The backbone mainly consists of a CBS module (convolutional, BN, SiLU activation function), SPPF (Spatial Pyramid Fast Pooling Layer), and C3 (Centralized Convolutional) module. In the neck network, YOLOv5 employs a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) structure to fuse image features and transmit them to the detection end. The Feature Pyramid Network (FPN) is a multi-scale feature fusion method that introduces feature maps of different scales and fuses these features at different levels, thereby achieving multi-scale target detection capabilities. Specifically, FPN generates multi-scale feature maps by using upsampling and downsampling operations in the lower-level network, and then fuses these feature maps through lateral connections, allowing higher-level features to contain fine-grained information from lower-level features, improving the accuracy of target detection. The Path Aggregation Network (PAN) is a top-down feature propagation method that introduces a feature propagation module to propagate higher-level features to lower-level features, thus achieving cross-level feature fusion. Specifically, PAN performs upsampling and downsampling operations on low-level and high-level features, transferring semantic information from high-level features to low-level features. This allows low-level features to acquire richer semantic information, improving the model's ability to detect small targets. By combining the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN), YOLOv5 fully utilizes feature information from different levels in the image, achieving multi-scale feature fusion. The fused features are then transmitted to the detection end, enabling accurate detection of targets of different scales and sizes. This structural design allows YOLOv5 to achieve high detection accuracy and robustness in various complex scenarios. The output uses a loss function (CIOU) to measure the overlap between the predicted and ground truth boxes. Simultaneously, to eliminate candidate boxes with high overlap, the YOLOv5 model employs non-maximum suppression.
[0104] To balance the application of the model in agricultural practice with the requirements of real-time performance, this paper selects YOLOv5 as the benchmark model. However, the benchmark model still has difficulties in detecting persimmons in complex orchard environments with dense distribution, diverse postures, leaf occlusion, and fruit overlap, especially performing poorly on persimmon datasets with high diversity.
[0105] This invention improves the existing YOLOv5 model by replacing the concat layer of the neck network of the YOLOv5 model with a bidirectional pyramid network module. This approach can more effectively achieve multi-scale feature fusion. Squeezing and excitation attention mechanisms are added to the C3 module in the neck network of the YOLOv5 model to better enhance the model's feature extraction capabilities. EIoU is used as the loss function of the model to obtain more accurate bounding box regression and object detection, thereby improving the convergence speed and obtaining an improved YOLOv5 model.
[0106] The training process according to step 3 in Example 1 is as follows:
[0107] Images of persimmon fruits from the training set are used as input. A convolutional module with 64 channels and a 6×6 kernel is used to extract features, perform dimensionality reduction, and normalize the feature map. Then, a CBS module is used to extract features and perform dimensionality reduction to obtain the feature map. Next, a C3 module with 128 channels and BottleNeck repeated 3 times is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency. Finally, a C3 module with 256 channels and BottleNeck repeated 6 times is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency. Finally, a CBS module is used to extract features and perform dimensionality reduction to obtain the feature map. The input feature map is obtained; then, the C3 module with 512 channels and BottleNeck repeated 9 times is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency; then, the CBS module is used to extract features and perform feature dimensionality reduction to obtain a feature map; then, the C3 module with 1024 channels and BottleNeck repeated 3 times is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency; the input feature map is pooled through the SPPF pooling layer to capture feature information at different scales, thereby improving object detection performance; then, CBS is used to extract features to obtain a feature map; then, the nearest neighbor upsampling operator is used to upsample the input feature map to obtain features of the same dimension as in step 3.7;
[0108] Then, the BiFPN module concatenates and fuses the features from steps 3.7 and 3.12 along a one-dimensional channel dimension to improve detection accuracy and recall. Next, the C3 module (512 channels, BottleNeck repeated 3 times, with SE attention mechanism) is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency. The CBS module is then used to extract features, obtaining a feature map. The nearest neighbor upsampling operator is then used to upsample the input feature map, resulting in features of the same dimension as in step 3.5. The BiFPN module concatenates and fuses the features from steps 3.5 and 3.16 along a one-dimensional channel dimension to improve detection accuracy and recall. Finally, the C3 module (256 channels, BottleNeck repeated 3 times, with SE attention mechanism) is used to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency. Performance; then the CBS module is used to extract features and perform feature dimensionality reduction to obtain a feature map; then the BiFPN module is used to concatenate and fuse the features from steps 3.15 and 3.19 according to a one-dimensional channel dimension to improve detection accuracy and recall; then the C3 module with 512 channels, BottleNeck repeated 3 times, and SE attention mechanism is used to extract rich features and enhance feature representation ability, while maintaining computational efficiency and improving model performance; then the CBS module is used to extract features and perform feature dimensionality reduction to obtain a feature map; then the BiFPN module is used to concatenate and fuse the features from steps 3.11 and 3.22 according to a one-dimensional channel dimension to improve detection accuracy and recall; then the C3 module with 1024 channels, BottleNeck repeated 3 times, and SE attention mechanism is used to extract rich features and enhance feature representation ability, while maintaining computational efficiency and improving model performance.
[0109] The output features from steps 3.18, 3.21, and 3.24 are passed through convolutional modules and then fed into the prediction network of the YOLOv5 model to predict the type of persimmon fruit. The CIoU loss function in the YOLOv5 network is changed to the EIoU loss function. The parameters in the YOLOv5 related files are adjusted: the image input size is 640 pixels × 640 pixels, the maximum number of iterations is 150, the batch size is 24, the initial learning rate is 0.01, the optimizer uses stochastic gradient descent (SGD), the weight decay coefficient is 0.0005, and the data path is modified. The model results are analyzed to assess the feature extraction capability of persimmon fruit.
[0110] In the original YOLOv5 network, FPN uses top-down channel fusion features, while PAN uses bottom-up channel fusion features. To achieve efficient bidirectional cross-scale connections and weighted feature fusion, BiFPN employs a novel approach. It introduces learnable weights, achieving feature fusion by learning the importance of different input features. This allows for dynamic adjustment of feature weights based on task requirements. BiFPN also employs multi-scale feature fusion strategies, both top-down and bottom-up. Propagating from higher-level features to lower-level features helps extract richer semantic information, while propagating from lower-level features to higher-level features provides more low-level detail. This bidirectional feature propagation effectively fuses features at different scales, improving model performance. Therefore, based on bidirectional paths, BiFPN uses a multi-level feature fusion mechanism to fuse features at different levels, preserving multi-scale semantic information and enhancing the accuracy and robustness of object detection. BiFPN has a simple network structure, introducing bidirectional paths and feature fusion without introducing additional components or modules. Therefore, it doesn't introduce a large number of parameters, ensuring efficient inference speed. The structure of BiFPN is as follows: Figure 2 As shown.
[0111] In YOLOv5, C3 refers to the CSP module of Darknet-53. The C3 module is a crucial component of YOLOv5, used for image feature extraction. Its main function is to extract rich features and enhance feature representation, improving model performance while maintaining computational efficiency. The SE attention mechanism learns attention weights for each channel, enabling the model to selectively focus on the most important and relevant features within each channel. This allows the model to more accurately capture highly discriminative features when handling complex tasks, thus improving performance and expressive power. Adaptive learning of channel weights through the SE attention mechanism makes feature selection and representation more efficient and accurate, effectively improving generalization and robustness. The SE module structure diagram is shown below. Figure 3 As shown in the diagram, the SE attention mechanism is integrated into the C3 module to form the C3SE module. The structure diagram of the C3SE module is shown below. Figure 4 As shown, C3SE can adaptively annotate the feature information and fine-grained features of target objects, further improving recognition accuracy.
[0112] Finally, EIoU was chosen as the model loss function. While the CIoU loss function considers the three central elements of a loss function, it can sometimes hinder the model from effectively optimizing similarity. The structure of the CIoU loss function is as follows: Figure 6As shown in the figure. Since the EIoU loss function can directly minimize the true difference between the height and width of the predicted bounding box and the ground truth bounding box, resulting in more accurate localization results and faster convergence speed, this paper chooses EIoU as the loss function. The structure of the EIoU loss function is as follows: Figure 7 As shown, the calculation formula is:
[0113]
[0114] Where h c and w c It is the height and width of the smallest bounding rectangle containing the predicted bounding box and the ground truth bounding box, ρ is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, and b is the center of the predicted bounding box. gt It is the center point of the true bounding box, w gt It is the width of the prediction box, h gt h is the height of the predicted bounding box, h is the height of the ground truth bounding box, and w is the width of the ground truth bounding box.
[0115] Therefore, compared with the existing YOLOv5 detection algorithm, the improved YOLOv5 model of this invention, named: E-YOLOv5-B-C3SE network model, has greatly improved the accuracy of persimmon detection in complex orchard environments. This progress is due to the use of BiFPN module and attention mechanism in this model, which replaces the loss function.
[0116] The model evaluation metrics used in this invention through the E-YOLOv5-B-C3SE network model are as follows: Precision (P), Recall (R), and Mean Accuracy (mAP). Precision (P) refers to the proportion of correctly detected persimmons out of all detected persimmons. Recall (R) refers to the proportion of samples correctly detected as persimmons out of all real persimmon samples. The combination of precision and recall is Mean Accuracy (AP). Mean Accuracy is the average of the average accuracies of n classes and is used to measure the overall model performance. The formulas for calculating P, R, and mAP are:
[0117]
[0118]
[0119] AP=∫0 1 P(R)dR
[0120]
[0121] In this context, TP (TruePositive) refers to the true positive instance, which is the number of times the model detects a correct persimmon as a correct persimmon; FP (FalsePositive) refers to the false positive instance, which is the number of times the model detects an incorrect persimmon as a correct persimmon; and FN (FalseNegative) refers to the false negative instance, which is the number of correct persimmons that are detected as incorrect persimmons. n is the number of categories.
[0122] Figure 8a This is a comparison chart of mAP from ablation experiments. Figure 8b This is a comparison chart of the accuracy of ablation experiments. Figure 8c This is a comparison chart of recall rates in ablation experiments. From... Figure 8a , Figure 8b and Figure 8c The YOLOv5 model achieved a precision of 92.9%, a recall of 89.8%, and an average precision of 94.8% with mAP_0.5. The E-YOLOv5-B-C3SE model achieved a precision of 95.8%, a recall of 92.2%, and an average precision of 96.8% with mAP_0.5. Compared to the original YOLOv5 network model, the E-YOLOv5-B-C3SE model improved precision, recall, and average precision by 2.9%, 2.4%, and 2%, respectively. Figures 8a-8c The results show the performance of the three improved methods on the model evaluation index, and all three improved methods outperform the original YOLOv5 model.
[0123] like Figure 9 As shown, the E-YOLOv5-B-C3SE algorithm model was used to detect persimmons. A representative detection result for each persimmon category was selected and briefly analyzed. To facilitate rapid analysis of the model's detection results, immature and ripe persimmons were labeled separately. The detection results demonstrate that the proposed E-YOLOv5-B-C3SE detection model can detect each persimmon category with high confidence even in complex backgrounds. It can also detect small target fruits and persimmons in multiple different locations. This model effectively improves the detection performance of the original YOLOv5 model.
[0124] The comprehensive analysis results show that the improved E-YOLOv5-B-C3SE model has significant advantages in persimmon fruit detection. The model effectively improves the detection performance of the original model, has a better target detection capability, and also improves the feature extraction capability of the original model for the target to be detected.
Claims
1. A method for persimmon fruit detection based on an improved YOLOv5 model, characterized in that, The specific steps are as follows: Step 1: Acquire persimmon images, perform image preprocessing, and then divide the preprocessed images into validation set, test set, and training set; Step 2: Improve the YOLOv5 model to obtain the improved YOLOv5 model; Specifically, the improvements to the YOLOv5 model are as follows: the concat layer of the neck network of the YOLOv5 model is replaced with a bidirectional pyramid network module, a squeezing and incentive attention mechanism is added to the C3 module in the neck network of the YOLOv5 model, and EIoU is used as the loss function of the model. Specifically, the C3 module with the added squeezing and attention incentive mechanism is as follows: the squeezing and attention incentive mechanism is added to the convolutional layer with a kernel size of 1×1 and a stride of 1 in the C3 module, and then the processed result is concatenated with the Bottleneck module in the C3 module in the one-dimensional channel dimension. Step 3: Train the improved YOLOv5 model using images from the training set to obtain the optimal weights; Step 4: Apply the optimal weights to the test set to obtain the model evaluation index, and use the evaluated and improved YOLOv5 model to perform the persimmon fruit detection method.
2. The method for persimmon fruit detection based on the improved YOLOv5 model according to claim 1, characterized in that, Step 1 specifically involves: Step 1.1: Collect persimmon images of different sizes, colors, shapes, poses, brightness, shadows, and occlusion levels as the raw dataset; Step 1.2: The persimmon images in the original dataset are sequentially subjected to data enhancement processes including flipping, adding noise, motion blurring, and random brightness adjustment; Step 1.3 involves manually labeling the persimmon images in the original dataset after enhancement in Step 1.2 using the LabelImg software, marking the category on each persimmon image. Each persimmon image must contain at least one category. Step 1.4: Use a Python script to convert the XML file obtained in Step 1.3 into a TXT file, and then divide it into a validation set, a test set, and a training set.
3. The method for persimmon fruit detection based on the improved YOLOv5 model according to claim 2, characterized in that, The ratio of the training set, validation set, and test set in step 1 is 8:1:
1.
4. The method for persimmon fruit detection based on the improved YOLOv5 model according to claim 1, characterized in that, Step 3 specifically involves: Step 3.1: Use the persimmon images in the training set as input to the improved YOLOv5 model, and use a convolutional module with 64 channels and a kernel size of 6×6 to extract features, perform feature dimensionality reduction, and normalize the feature map. Step 3.2, the feature maps obtained in step 3.1 are then processed by a CBS module with 128 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction; Step 3.3: The feature map after feature dimensionality reduction in step 3.2 is then processed by the C3 module with 128 channels and BottleNeck repeated 3 times to enhance feature representation capabilities; Step 3.4: The feature map processed in step 3.3 is then processed by a CBS module with 256 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction. Step 3.5: The feature map processed in step 3.4 is then processed through the C3 module with 256 channels and BottleNeck repeated 6 times to enhance its feature representation capability. Step 3.6: The feature map processed in step 3.5 is then processed by a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction. Step 3.7: The feature map processed in step 3.6 is then processed through the C3 module with 512 channels and BottleNeck repeated 9 times to enhance its feature representation capability; Step 3.8: The feature map processed in step 3.7 is then processed by a CBS module with 1024 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction. Step 3.9: The feature map processed in step 3.8 then passes through the C3 module, which has 1024 channels and repeats BottleNeck 3 times. Step 3.10: The feature map processed in step 3.9 is then subjected to spatial pyramid pooling by a pooling layer. Step 3.11: The feature map processed in step 3.10 is then processed again through a CBS module with 512 channels and a kernel size of 1×1 to extract image features; 3.12, Apply the nearest neighbor upsampling operator to the feature map obtained in step 3.11 to perform an upsampling operation, and obtain a feature map with the same dimension as the output feature map in step 3.7; Step 3.13: The features output from Step 3.7 and Step 3.12 are spliced and fused using a bidirectional pyramid network module according to the one-dimensional channel dimension to obtain the fused features; Step 3.14: The fused features obtained in step 3.13 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times. Step 3.15: Input the output of step 3.14 into a CBS module with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction; Step 3.16: Apply the nearest neighbor upsampling operator to the output of step 3.15 to perform an upsampling operation to obtain features of the same dimension as the output features of step 3.5; Step 3.17: Concatenate and fuse the outputs of Step 3.16 and Step 3.5 according to the one-dimensional channel dimension to obtain the fused feature; Step 3.18: The fused features obtained in step 3.17 are further enhanced by adding a squeezing and attention-stimulating mechanism to the C3 module to enhance feature representation. The C3 module with the squeezing and attention-stimulating mechanism has 256 channels and BottleNeck is repeated 3 times. Step 3.19: Use the output of step 3.18 as input to access CBS with 512 channels and a kernel size of 3×3 to extract image features and perform feature dimensionality reduction; Step 3.20: Input the outputs of Step 3.19 and Step 3.15 back into the bidirectional pyramid network module and splice and fuse them according to the one-dimensional channel dimension to obtain the fused features; Step 3.21: Input the output of step 3.20 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 512 channels and BottleNeck is repeated 3 times. Step 3.22: The output from step 3.21 is then input with 512 channels and a convolution kernel size of 3. The CBS module in module 3 extracts image features and performs feature dimensionality reduction; Step 3.23: The outputs of steps 3.21 and 3.11 are used as inputs to the bidirectional pyramid network module and spliced and fused according to the one-dimensional channel dimension to obtain the fused features; Step 3.24: Input the output of step 3.23 into the C3 module with the added squeezing and attention-stimulating mechanism. The C3 module with the added squeezing and attention-stimulating mechanism has 1024 channels and BottleNeck is repeated 3 times. Step 3.25: After passing the feature maps output from Steps 3.18, 3.21, and 3.25 through a convolutional layer, input them into the prediction network of the YOLOv5 model to predict the category. Step 3.26: Following the process in steps 3.1-3.25, train the improved YOLOv5 model using persimmon images from the training set. During training, use the EIoU loss function as the loss function for the improved YOLOv5 model. This is achieved by setting the maximum number of iterations and the batch size for the improved YOLOv5 model. The improved YOLOv5 model is obtained by setting the size, initial learning rate, weight decay coefficient, optimizer, and data path.
5. The method for persimmon fruit detection based on the improved YOLOv5 model according to claim 4, characterized in that, The maximum number of iterations is 150, batch The size is 24, the initial learning rate is 0.01, the optimizer uses stochastic gradient descent (SGD), and the weight decay coefficient is 0.0005.
6. The method for persimmon fruit detection based on the improved YOLOv5 model according to claim 1, characterized in that, The validation set is used to assess the performance of the improved YOLOv5 model after testing.
Citation Information
Patent Citations
Spinning cake defect detection method based on improved YOLOv5 model
CN115705637A
Wind turbine surface defect detection method
CN115719337A
