Improved strawberry detection method based on YOLOv8 model

By improving the YOLOv8 model and introducing the C2f_Faster, EfficientViT, and Dynamic Head modules, the accuracy and efficiency issues of strawberry detection were resolved, enabling efficient strawberry harvesting and improving the collection efficiency of agricultural automation.

CN118918429APending Publication Date: 2024-11-08CHANGSHA UNIVERSITY OF SCIENCE AND TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410555914.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-05-07
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing strawberry collection and testing technologies suffer from low accuracy and efficiency, especially when strawberry growing environments are complex and factors such as light and shade affect the accuracy of determining key quality characteristics such as ripeness and size.

Method used

Based on the YOLOv8 model, we improved it by introducing the C2f_Faster module, the EfficientViT module, and the Dynamic Head module to optimize the model's computational efficiency and detection accuracy. We then used LabelImg to annotate strawberry images, conducted training and ablation experiments, and deployed the model on a strawberry harvesting robot.

Benefits of technology

It improves the accuracy and efficiency of strawberry detection, enables efficient and accurate strawberry picking in complex environments, reduces damage to surrounding plants, and optimizes picking efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918429A_ABST
    Figure CN118918429A_ABST
Patent Text Reader

Abstract

The invention discloses an improved strawberry detection method based on a YOLOv8 model, and the method comprises the steps: 1, collecting an original image of a strawberry, and carrying out the marking of the strawberry through LabelImg; step 2, the YOLOv8 model is improved, a C2fFaster module is provided, and an OfficientViT module and a Dynamic Head module are integrated; 3, training the model to obtain an efficient training model, and obtaining a detection effect picture; 4, performing an ablation experiment; and step 5, realizing deployment and visualizing the actual application effect of the method. According to the method provided by the invention, a C2fFaster module is innovatively provided on a YOL Ov8 model, and an OfficitViT module and a Dynamic Head module are added. And an OfficientViT module is added, so that the detection efficiency of the model is enhanced. A C2fFaster module is provided, and Pconv is introduced into the C2f module, so that the calculation speed of the module is increased, and the calculation efficiency is improved. A Dynamic Head module is added, and the adaptability of feature fusion, a spatial feature attention mechanism and support for multiple tasks are introduced, so that the adaptability of the model is enhanced. The strawberry detection performance of the model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision processing, and in particular to a strawberry detection method based on an improved YOLOv8 model. Background Art

[0002] As a popular fruit, the quality of strawberries has an important impact on consumer satisfaction. However, the traditional method of picking strawberries mainly relies on manual labor, which is not only inefficient, but also difficult to accurately judge the key quality characteristics of strawberries, such as maturity, color, and size, resulting in uneven quality of picked strawberries.

[0003] In addition, as consumers' requirements for food safety and quality continue to increase, higher requirements are also put forward for strawberry collection and detection. Although the existing strawberry collection and detection technology has made some progress, there are still many problems.

[0004] On the one hand, due to the complexity of the strawberry growth environment (such as soil moisture, temperature, light, etc.), the surface characteristics of strawberries (such as color, shape, size, etc.) will change greatly, which brings great challenges to the collection and detection of strawberries. On the other hand, existing strawberry collection and detection technologies often rely on a single sensor or image processing technology, which is easily affected by factors such as light, shadow, and occlusion, resulting in low accuracy of detection results and low detection efficiency.

[0005] Therefore, there is an urgent need for an efficient detection method for strawberry characteristics to improve the accuracy and speed of strawberry collection and detection.

[0006] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the invention and should not be regarded as an admission or suggestion in any form that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a strawberry detection method based on an improved YOLOv8 model, so as to solve the problems of difficult strawberry positioning, easy false alarms and low detection efficiency caused by the characteristics of strawberry picking.

[0008] To achieve the above object, the present invention provides a strawberry picking detection method based on an improved YOLOv8 model, comprising:

[0009] Step 1: Collect the original strawberry picture and use LabelImg to label the strawberry;

[0010] Step 2: Improve the YOLOv8 model, propose the C2f_Faster module, and add the EfficientViT module and Dynamic Head module;

[0011] Step 3: Train the model to obtain an efficient training model and obtain the detection effect picture;

[0012] Step 4: Conduct ablation experiment.

[0013] Step 5: Implement deployment and visualize the actual application effect of the method.

[0014] Preferably, in the above technical solution, the strawberry pictures obtained in step 1 have different strawberry varieties, different growth stages and representative environmental conditions, including changes in light, background and strawberry density.

[0015] Preferably, in the above technical solution, in step 1, the strawberries are labeled using the LabelImg tool, and each labeling frame is strictly adjusted to ensure that the strawberries can be completely and accurately selected.

[0016] Preferably, in the above technical solution, step 2 of improving the YOLOv8 model includes:

[0017] (1) Integrate the EfficientViT module into the backbone network of YOLOV8. The EfficientViT module introduces a unique building block that seamlessly integrates two main components: a lightweight multi-scale linear attention (MSLA) module and a feed-forward network (FFN) accompanied by a deep convolution (DWConv). The process of this module can be expressed as:

[0018] Q, K, V = LP (Input)

[0019] Multi-Scale Tokens=Aggregate(Nearby Tokens)

[0020] Among them, nearby tokens can be represented by ReLU-based linear attention of Q, K, V tokens, and LP is the linear projection layer.

[0021] Output=Linear(Multi-Scale Tokens)

[0022] (2) The C2f module in the neck structure of the original YOLOv8 model is replaced by the proposed C2f_Faster module. The FLOPs of a PConv in this module is:

[0023]

[0024] It is only 1 / 16 of the traditional convolution. In addition, its memory access is:

[0025]

[0026] It is only 1 / 4 of the traditional convolution.

[0027] (3) Add the Dynamic Head module before the convolution operation in the head prediction part of the original YOLOv8 model.

[0028] 1) The scale-aware attention calculation formula is:

[0029]

[0030] Among them, F is the feature tensor.

[0031] 2) The calculation formula of its spatial perception attention is:

[0032]

[0033] Among them, K represents the number of sparse sampling points, and p k +Δp k is the shift position of the self-learning spatial offset.

[0034] 3) The calculation formula of its task-aware attention is:

[0035] π C (F)·F=max(α1(F)·F c +β1(F),α2(F)·F c +β2(F)

[0036] Among them, F c is the feature slice of the cth channel.

[0037] Preferably, in the above technical solution, step 3 trains the training set of the strawberry data set to obtain a trained efficient detection model of the improved method.

[0038] Preferably, in the above technical solution, step 3 inputs the original strawberry picture into the trained model, obtains the test result of the model, and outputs the predicted picture with the target prediction box and confidence.

[0039] Preferably, in the above technical solution, step 4 of performing an ablation experiment on the strawberry dataset includes performing combined training on each module to obtain the map value and GFLOPs of each model, and comparing the specific results.

[0040] Preferably, in the above technical solution, step 5 deploys the method on a strawberry robot, visualizes the detection results of the strawberry picking robot, and applies the strawberry picking robot to actual agricultural production.

[0041] (1) The present invention is an improved strawberry picking detection method based on the YOLOv8 model. The C2f_Faster module is innovatively proposed on the original YOLOv8 model, and the EfficientViT module and the Dynamic Head module are added. Although the detection effect of the model is slightly reduced by adding the EfficientViT module, the GFLOPs of the model is significantly reduced, which enhances the detection efficiency of the model. The C2f_Faster module is proposed, and Pconv is introduced into the C2f module to enhance the calculation speed of the module, thereby enhancing the calculation efficiency. The Dynamic Head module is added to introduce the adaptability of feature fusion, the spatial feature attention mechanism and the support for multi-tasks, thereby enhancing the adaptability of the model. Improve the performance and efficiency of the model for strawberry detection.

[0042] (2) The present invention is an improved strawberry detection method based on the YOLOv8 model. When this method is deployed on a strawberry harvesting robot and applied to agricultural automation in the real world, the specific effect exceeds the theoretical practical application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is an overall flow chart of the strawberry detection method improved based on the YOLOv8 model according to the present invention;

[0044] Figure 2 This is the improved YOLOv8 model framework diagram;

[0045] Figure 3 This is the added EfficientViT module structure diagram;

[0046] Figure 4 It is the Multi-Scale Linear Attention in the EfficientViT module

[0047] Structural diagram;

[0048] Figure 5 It is the proposed C2f_Faster module structure diagram;

[0049] Figure 6 This is the added Dynamic Head module structure diagram;

[0050] Figure 7 It is the detection effect diagram of the improved method model;

[0051] Figure 8 This is the structure diagram of the strawberry picking robot;

[0052] Fig. 9 It is an improvement method deployment and actual application effect diagram. DETAILED DESCRIPTION

[0053] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0054] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0055] like Figures 1 to 9 As shown, a strawberry detection method based on an improved YOLOv8 model according to a specific embodiment of the present invention comprises the following steps:

[0056] Step 1: Collect the original strawberry picture and use LabelImg to label the strawberry

[0057] (1) Considering the challenges that arise when collecting robots are used in strawberry detection tasks, our strawberry images include different strawberry varieties, different growth stages, and representative environmental conditions, including changes in lighting, background, and strawberry density. These perfectly expose the problems faced in strawberry detection tasks, such as different strawberry varieties, different maturity, different growth environments, leaf occlusion, and different light intensities.

[0058] (2) For the labeling of the original strawberry images, we invited three professionals in the strawberry planting industry to use LabelImg to label the original strawberry images. Under the operation of the professionals, the labeling boxes were strictly adjusted to accurately and completely select the strawberries.

[0059] Step 2: Improve the YOLOv8 model, propose the C2f_Faster module and add the EfficientViT module and Dynamic Head module

[0060] (1) The EfficientViT module is integrated into the backbone network of YOLOV8, enabling multi-scale linear attention for high-resolution dense prediction. This integration enables the model to capture the complex details in the strawberry image, making it more robust to changes in size, shape, and maturity. The EfficientViT module introduces a unique building block that seamlessly integrates two main components: a lightweight multi-scale linear attention (MSLA) module and a feed-forward network (FFN) accompanied by deep convolution (DWConv). The process of this module can be expressed as:

[0061] Q, K, V = LP (Input)

[0062] Multi-Scale Tokens=Aggregate(Nearby Tokens)

[0063] Among them, nearby tokens can be represented by ReLU-based linear attention of Q, K, V tokens, and LP is the linear projection layer.

[0064] Output=Linear(Multi-Scale Tokens)

[0065] This multi-scale approach ensures that the attention mechanism can focus on both broader patterns and finer details simultaneously. Furthermore, to improve efficiency and performance, a ReLU-based global attention is applied to these multi-scale tokens. The resulting outputs are then concatenated and passed through a final linear projection layer, ensuring the fusion of features across scales. This innovative attention method ensures that both broad and detailed images of depth are processed efficiently. An effective and efficient process.

[0066] (2) The C2f module in the neck structure of the original YOLOv8 model is changed to the proposed C2f_Faster module. This is used to improve computational efficiency. The Fasternet module is seamlessly integrated into the YOLOV8 architecture of the C2f module to construct the proposed C2f_Faster module. This module optimizes the number of floating-point operations per second (FLOPS) to speed up the execution of the neural network while maintaining detection accuracy. In the Fasternet module, a new partial convolution (PConv) is proposed that can extract spatial features more efficiently by reducing redundant calculations and memory accesses at the same time. The FLOPs of a PConv in this module are:

[0067]

[0068] It is only 1 / 16 of the traditional convolution. In addition, its memory access is:

[0069]

[0070] It is only 1 / 4 of the traditional convolution.

[0071] (4) Add the Dynamic Head module before the convolution operation in the head prediction part of the original YOLOv8 model. Combine the object detection head with the attention mechanism. This enhancement improves the model's ability to focus on relevant features, improves detection accuracy, and reduces computational overhead.

[0072] 1) The scale-aware attention calculation formula is:

[0073]

[0074] Among them, F is the feature tensor.

[0075] 2) The calculation formula of its spatial perception attention is:

[0076]

[0077] Among them, K represents the number of sparse sampling points, and p k +Δp k is the shift position of the self-learning spatial offset.

[0078] 3) The calculation formula of its task-aware attention is:

[0079] π C (F)·F=max(α1(F)·F c +β1(F),α2(F)·F c +β2(F)

[0080] Among them, F c is the feature slice of the cth channel.

[0081] Step 3: Train the model to obtain an efficient training model and obtain the detection effect picture

[0082] During the training process, the training set of the strawberry dataset is used to train the model, and the validation set is used for validation during the training process to obtain an efficient YOLOv8 detection model, which enables the model to maintain a certain accuracy and superior speed during the strawberry detection task.

[0083] Step 4: Conduct ablation experiment

[0084] Ablation experiments are conducted on this method to verify the effectiveness of each module and analyze the reasons why the module produces an effect. By comparing the ablation of each module, we can verify the effectiveness of each module. The result is that after adding the EfficientViT module, the map value of the model decreases slightly, but the GFLOPs of the model is significantly reduced, that is, the calculation amount of the model is significantly reduced, thereby significantly improving the detection efficiency of the model. After adding the C2f_Faster module, the map value of the model increases, and the GFLOPs of the model decreases significantly, thereby verifying the high efficiency of the module. After adding the Dynamic Head module, although the GFLOPs of the model increases slightly, the map value of the model is significantly improved, that is, the detection effect of the model is significantly improved, thereby improving the effectiveness of the model.

[0085] Step 5: Implement deployment and visualize the actual application effect of the method.

[0086] (1) The improved method was deployed on a strawberry harvesting robot to test the effectiveness of the method in actual agricultural production applications. We seamlessly integrated the method with the SOTA robotic arm to form an overall vision-based harvesting system. The visual part of the harvesting system uses the K210 vision module from Yabo Intelligent Company, and the robotic arm uses the TeeArm six-axis robotic arm. Combined with the precision of the robotic arm, a collaborative platform was created that can autonomously identify and harvest strawberries.

[0087] (2) When locating strawberries, we placed the vision module parallel to the ground. We conducted a total of 15 experiments in a laboratory environment, showing consistent and accurate detection for the robotic arm in strawberries at different crop densities. The vision-guided approach ensured selective picking, minimized damage to surrounding plants, and optimized harvesting efficiency. This planned deployment model is consistent with the vision of sustainable and resource-efficient agriculture.

[0088] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A strawberry detection method based on an improved YOLOv8 model, characterized in that: include: Step 1: Collect the original strawberry picture and use LabelImg to label the strawberry; Step 2: Improve the YOLOv8 model, propose the C2f_Faster module, and add the EfficientViT module and Dynamic Head module; Step 3: Train the model to obtain an efficient training model and obtain the detection effect picture; Step 4: Conduct ablation experiment; Step 5: Implement deployment and visualize the actual application effect of the method.

2. The strawberry detection method based on the improved YOLOv8 model according to claim 1, characterized in that, The strawberry images obtained in step 1 have different strawberry varieties, different growth stages, and representative environmental conditions, including changes in light, background, and strawberry density, etc.

3. The strawberry detection method based on the YOLOv8 model improvement according to claim 1, characterized in that, Step 1: Use the LabelImg tool to label the strawberries. Each labeling box is strictly adjusted to ensure that the strawberries can be fully selected.

4. The strawberry detection method based on the YOLOv8 model improvement according to claim 1, characterized in that, Step 2 improves the YOLOv8 model by: (1) Integrate the EfficientViT module into the backbone network of YOLOV8. The EfficientViT module introduces a unique building block that seamlessly integrates two main components: a lightweight multi-scale linear attention (MSLA) module and a feed-forward network (FFN) accompanied by a deep convolution (DWConv). The process of this module can be expressed as: Q, K, V = LP (Input) Multi-Scale Tokens=Aggregate(Nearby Tokens) Among them, nearby tokens can be represented by ReLU-based linear attention of Q, K, V tokens, and LP is the linear projection layer. Output=Linear(Multi-Scale Tokens) (2) The C2f module in the neck structure of the original YOLOv8 model is replaced by the proposed C2f_Faster module. The FLOPs of a PConv in this module is: It is only 1 / 16 of the traditional convolution. In addition, its memory access is: It is only 1 / 4 of the traditional convolution. (3) Add the Dynamic Head module before the convolution operation in the head prediction part of the original YOLOv8 model. 1) The scale-aware attention calculation formula is: Among them, F is the feature tensor. 2) The calculation formula of its spatial perception attention is: Among them, K represents the number of sparse sampling points, and p k +Δp k is the shift position of the self-learning spatial offset. 3) The calculation formula of its task-aware attention is: p C (F)·F=max(α1(F)·F c +β1(F),α2(F)·F c +β2(F)) Among them, F c is the feature slice of the cth channel.

5. The strawberry detection method based on the YOLOv8 model improvement according to claim 1, characterized in that, Step 3 trains the training set of the strawberry data set to obtain an efficient detection model after training of the improved method.

6. The strawberry detection method based on the improved YOLOv8 model according to claim 1, characterized in that, Step 3: Input the original strawberry image into the trained model, obtain the test result of the model, and output the predicted image with the target prediction box and confidence.

7. The strawberry detection method based on the improved YOLOv8 model according to claim 1, characterized in that, Step 4 performs an ablation experiment on the Strawberry dataset, including combining and training each module, obtaining the map value and GFLOPs of each model, and comparing the specific results.

8. The strawberry detection method based on the improved YOLOv8 model according to claim 1, characterized in that, Step 5 deploys this method on a strawberry robot, visualizes the detection results of the strawberry picking robot, and applies the strawberry picking robot to actual agricultural production.