Kiwi fruit small target detection method in complex environment based on improved YOLOv8
By improving the YOLOv8 model, combining the EMBC module, CARAFE upsampling operator and Dyhead probe, the accuracy and speed of kiwi fruit small target detection in complex environments are solved, and efficient kiwi fruit positioning and automatic picking support is achieved.
Patent Information
- Application Number
- CN202411783475.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-03-28
AI Technical Summary
In complex environments, it is difficult for the prior art to quickly and accurately identify small kiwi fruit targets, limiting the efficient and automated picking capabilities of picking robots.
By improving the YOLOv8 model, replacing the C2f module with the EMBC module, introducing the CARAFE upsampling operator and Dyhead probe head, the model's characterization ability and detection accuracy are enhanced.
It significantly improves the detection ability of small targets, can quickly and accurately identify kiwis in complex environments, and provides reliable support for kiwi positioning and automatic picking.
Smart Images
Figure CN119723562A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target recognition technology, and more specifically to a kiwi fruit small target detection method in a complex environment based on improved YOLOv8. Background Art
[0002] Kiwifruit is rich in nutrition, unique in flavor and has high economic benefits. In recent years, China's kiwifruit industry has developed rapidly, with a total kiwifruit output of 23,900 tons and a planting area of 67,000 hectares, both of which are among the highest in the world. At present, kiwifruit picking mainly relies on manual labor, which is labor-intensive, and the picking cost accounts for more than 25% of the production cost. In order to overcome the increasingly serious labor shortage, improve picking efficiency, and extend shelf life, the development of an accurate and efficient picking robot has become a research hotspot. In the working process of the picking robot, the accuracy and speed of target fruit recognition and positioning are the key factors that determine the picking efficiency. Therefore, how to provide a kiwifruit small target detection method in a complex environment based on an improved YOLOv8 is a problem that technicians in this field urgently need to solve. Summary of the invention
[0003] In view of this, the present invention provides a kiwi fruit small target detection method in a complex environment based on improved YOLOv8, which can run efficiently on mobile devices with limited computing resources, and can quickly and accurately identify target fruits in a complex environment, providing support for realizing automated picking.
[0004] In order to achieve the above object, the present invention provides the following technical solutions:
[0005] A kiwi fruit small target detection method in a complex environment based on improved YOLOv8, comprising the following steps:
[0006] S1, collecting unharvested kiwifruit images by on-site photography and online search;
[0007] S2, expand and annotate the collected images to obtain a kiwi fruit dataset;
[0008] S3, improve the YOLOv8 model by replacing the C2f module with the EMBC module, introducing the CARAFE upsampling operator and the Dyhead detector head;
[0009] S4, improve the YOLOv8 model based on the kiwi fruit dataset training to obtain the optimal weight;
[0010] S5. Detect kiwifruit in complex environments through the trained improved YOLOv8 model.
[0011] Optionally, S2 specifically includes: expanding the collected images by rotating, partially enlarging, and adding noise to improve the generalization ability of the improved YOLOv8 model, and using the LabelImg annotation tool to annotate the expanded images.
[0012] Optionally, S3 replaces the C2f module in the head and backbone network of the YOLOv8 model with the EMBC module; the EMBC module expands the channels of the input feature map through a 1×1 convolution layer, and then extracts features through a depthwise separable convolution unit, and explicitly models the interdependence between feature channels through the ESE unit, enhances useful features and suppresses unimportant features, improves the representation ability of the model, and then compresses the number of channels through a 1×1 convolution layer. Finally, the residual connection unit is used to alleviate the gradient vanishing problem in the deep network and accelerate the training process of the improved YOLOv8 model.
[0013] Optionally, the CARAFE upsampling operator in S3 includes a Kernel prediction module and a content-aware reorganization module; wherein the Kernel prediction module compresses the feature channels of the input feature map from C to C through 1×1 convolution. m , using a kernel size of K encoder The convolution layer generates a reorganized convolution kernel based on the input features, and finally uses the Softmax function to perform up ×K up The content-aware reconstruction module dynamically utilizes the generated reorganized convolution kernel W l’ Reorganize input features in local regions.
[0014] Optionally, the Dyhead detector works as follows: given a feature vector Where L represents the number of layers of the feature pyramid, S = H × W, S represents the spatial dimension of the feature map, C represents the number of channels of the feature map, and the attention calculation formula is as follows:
[0015]
[0016] Where π(·) is an attention function, which includes three sequentially executed attention mechanisms, each focusing on only one aspect:
[0017]
[0018] In the formula, π C ,π S ,π L Corresponding to the attention functions applied to C, S, and L dimensions respectively.
[0019] Optional, scale-aware attention function π L Specifically:
[0020]
[0021] In the formula, represents the average pooling process, σ(·) represents the sigmoid function, and f(·) represents the linear function approximated by the 1×1 convolutional layer.
[0022] Optional, spatially aware attention function π S Specifically:
[0023]
[0024] Where K is the number of sparse sampling locations, Δp k is the learnable spatial offset, Δm k is at position p k The importance scalar of self-learning.
[0025] Optionally, task-aware attention function π C Specifically:
[0026]
[0027] In the formula, F c is the feature slice of the cth channel, α 1 , β 1 , α 2 , β 2 is a hyperfunction used to learn the control activation threshold.
[0028] It can be seen from the above technical solution that compared with the prior art, the present invention provides a method for detecting small targets of kiwifruit in a complex environment based on improved YOLOv8, which has the following beneficial effects: the present invention improves YOLOv8, and the improved model significantly improves the detection capability of small targets, and can provide reliable support for subsequent kiwifruit positioning and automatic picking. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0030] Figure 1 It is a flow chart of the kiwi fruit small target detection method in a complex environment of the present invention;
[0031] Figure 2This is a schematic diagram of the improved YOLOv8 model structure of the present invention;
[0032] Figure 3 It is a schematic diagram of the EMBC module structure of the present invention;
[0033] Figure 4 It is a schematic diagram of the ESE unit structure of the present invention;
[0034] Figure 5 Schematic diagram of the structure of the CARAFE upsampling operator of the present invention;
[0035] Figure 6 It is a schematic diagram of the structure of the Dyhead detection head of the present invention. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0037] The embodiment of the present invention discloses a method for detecting small kiwi fruit targets in a complex environment based on improved YOLOv8. Figure 1 As shown, the following steps are included:
[0038] S1, collecting unharvested kiwifruit images by on-site photography and online search;
[0039] S2, expand and annotate the collected images to obtain a kiwi fruit dataset;
[0040] S3, improve the YOLOv8 model by replacing the C2f module with the EMBC module, introducing the CARAFE upsampling operator and the Dyhead detector head. The improved YOLOv8 model is as follows Figure 2 As shown;
[0041] S4, improve the YOLOv8 model based on the kiwi fruit dataset training to obtain the optimal weight;
[0042] S5. Detect kiwifruit in complex environments through the trained improved YOLOv8 model.
[0043] Furthermore, S2 specifically includes: expanding the collected images by rotating, partially enlarging, and adding noise to improve the generalization ability of the improved YOLOv8 model, and using the LabelImg annotation tool to annotate the expanded images.
[0044] Furthermore, S3 replaces the C2f module in the head and backbone network of the YOLOv8 model with the EMBC module; Figure 3 As shown in the figure, the EMBC module expands the channels of the input feature map through a 1×1 convolution layer, then extracts features through a depthwise separable convolution unit, explicitly models the interdependence between feature channels through the ESE unit, enhances useful features and suppresses unimportant features, improves the representation ability of the model, and then compresses the number of channels through a 1×1 convolution layer. Finally, the residual connection unit is used to alleviate the gradient vanishing problem in the deep network and accelerate the training process of the improved YOLOv8 model.
[0045] As an effective attention mechanism, the SE module originally contains two fully connected layers, which play the role of reducing and expanding the dimension of the feature channel respectively. However, while the dimensionality reduction operation reduces the computational complexity, it also inevitably leads to a certain degree of information channel loss. In order to further improve the effect of feature extraction, in the embodiment of the present invention, the ESE unit is introduced, such as Figure 4 As shown in Figure 1, the core idea is to simplify the traditional two fully connected layers into one, thereby better maintaining the integrity of the information between channels without explicit dimensionality reduction.
[0046] Further, such as Figure 5 As shown in Figure 2, the CARAFE upsampling operator in S3 includes a kernel prediction module and a content-aware reorganization module. The kernel prediction module compresses the feature channels of the input feature map from C to C through 1×1 convolution. m , thereby reducing the number of channels and reducing computational cost, using a kernel size of K encoder The convolution layer generates a reorganized convolution kernel based on the input features to enhance the use of context information. Finally, the Softmax function is used to up ×K up The content-aware reconstruction module dynamically utilizes the generated reorganized convolution kernel W l’ The input features are reorganized in local areas. This method is implemented through a weighted sum operation, where each pixel makes different contributions in the reorganization process according to the weights assigned to its neighboring pixels. The goal is to enhance the information of related points in the local area, thereby improving the semantic richness and quality of the upsampled feature map.
[0047] The nearest neighbor interpolation method only interpolates based on the values of adjacent pixels, ignoring the spatial relationship and contextual information between pixels. This causes the edges of kiwifruit to become blurred and lose details in areas where the light changes dramatically or is covered by shadows. For small targets and occluded kiwifruit, due to the limited number of pixels they occupy, detailed information is relatively scarce. In the process of nearest neighbor interpolation, edge and texture information is more likely to be lost, resulting in a decrease in detection accuracy. In order to overcome these defects, in an embodiment of the present invention, a CARAFE upsampling operator is introduced. The CARAFE upsampling operator dynamically generates upsampling convolution kernels based on the distribution information of the features, and strengthens the extraction of contextual semantic information of the features.
[0048] Furthermore, the working process of the Dyhead detector is as follows: given a feature vector Where L represents the number of layers of the feature pyramid, S = H × W, S represents the spatial dimension of the feature map, and C represents the number of channels of the feature map. The structure of the Dyhead detector is as follows: Figure 6 As shown, the attention calculation formula is as follows:
[0049]
[0050] Where π(·) is an attention function, which includes three sequentially executed attention mechanisms, each focusing on only one aspect:
[0051]
[0052] In the formula, π C ,π S ,π L The attention functions are applied to the C, S, and L dimensions respectively. The Dyhead detector can realize scale perception, spatial perception, and task perception in a unified manner, effectively capture the characteristic information of kiwifruit in the image, and dynamically adjust the attention allocation, so that the model can more accurately identify and locate kiwifruit in complex scenes.
[0053] Furthermore, the scale-aware attention function π L Specifically:
[0054]
[0055] In the formula, represents the average pooling process, σ(·) represents the sigmoid function, and f(·) represents the linear function approximated by the 1×1 convolutional layer.
[0056] In this embodiment of the present invention, the scale-aware attention function π LFirst, average pooling is applied to reduce the spatial dimension of the feature map, retaining key global information, which helps the network capture features at different scales; next, a 1×1 convolutional layer is used to perform cross-channel linear transformation on the pooled feature map to learn the dependencies between different scales and possibly reduce the number of channels; finally, a hard Sigmoid function is applied to normalize the features and generate attention weights for each scale, which are then used to recalibrate the corresponding scale features in the original feature map, thereby achieving dynamic fusion and emphasis across scales. Optionally, a ReLU activation function is introduced after the linear transformation to add nonlinear effects.
[0057] Furthermore, the spatial perception attention function π S Specifically:
[0058]
[0059] Where K is the number of sparse sampling locations, Δp k is the learnable spatial offset, Δm k is at position p k The importance scalar of self-learning.
[0060] In this embodiment of the present invention, the spatial perception attention function π S The working process of can be divided into two main steps: sparse sampling and cross-layer feature fusion using deformable convolution. First, deformable convolution is used to sparsely sample key locations on the feature map. Deformable convolution allows the convolution kernel to move freely on the input feature map, allowing it to cover areas that are more likely to contain useful information. This step is achieved through the self-learned spatial offset Δp k This is achieved by enabling the network to focus on discriminative areas. After sparse sampling, the spatially aware attention function π S Aggregate features from different scales (or levels) at the same spatial position. This process is achieved by weighted summation. The importance scalar Δm k It is also self-learning, reflecting the importance of features of different scales at corresponding spatial positions.
[0061] Furthermore, the task-aware attention function π C Specifically:
[0062]
[0063] In the formula, F c is the feature slice of the cth channel, α 1 , β 1 , α 2 , β 2 is a hyperfunction used to learn the control activation threshold.
[0064] In this embodiment of the present invention, the task-aware attention function π C First, global pooling is performed in the L×S dimension to reduce the spatial dimension and extract global features. Then the first fully connected layer takes the output of global pooling as input to learn the linear combination between features. The second fully connected layer further processes these combined features and enhances the expressiveness of the model by introducing the nonlinear activation function ReLU; then, a normalization layer is applied to ensure data stability; finally, a shifted Sigmoid function is applied to normalize the output to the range [-1,1].
[0065] In the embodiment of the present invention, the method of the present invention is experimented, and the specific experimental environment is: the hardware environment of the server platform is that the graphics card is NVIDIA GeForce RTX 4090D, the CPU is AMD EPYC 9754, and the memory is 60GB. The software environment is Windows 10, Python 3.8, PyTorch 2.0.0, CUDA 11.8. The experimental results are shown in Table 1:
[0066] Table 1:
[0067]
[0068] It can be seen that compared with the existing YOLOv8n model, the kiwi fruit small target detection method of the embodiment of the present application has higher recall rate (Recall), precision rate (Precision), average precision when the IoU threshold is 0.5 (mAP0.5), and average precision when the IoU threshold ranges from 0.5 to 0.95 (mAP0.5: 0.95).
[0069] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0070] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A kiwi fruit small target detection method in a complex environment based on improved YOLOv8, characterized in that: The following steps are involved: S1, collecting unharvested kiwifruit images by on-site photography and online search; S2, expand and annotate the collected images to obtain a kiwi fruit dataset; S3, improve the YOLOv8 model by replacing the C2f module with the EMBC module, introducing the CARAFE upsampling operator and the Dyhead detector head; S4, improve the YOLOv8 model based on the kiwi fruit dataset training to obtain the optimal weight; S5. Detect kiwifruit in complex environments through the trained improved YOLOv8 model.
2. According to the method for detecting small kiwi fruit targets in complex environments based on improved YOLOv8 in claim 1, it is characterized in that: S2 specifically includes: expanding the collected images by rotating, partially enlarging, and adding noise to improve the generalization ability of the improved YOLOv8 model, and using the LabelImg annotation tool to annotate the expanded images.
3. According to the method for detecting small kiwi fruit targets in complex environments based on improved YOLOv8 in claim 1, it is characterized in that: S3 replaces the C2f module in the head and backbone network of the YOLOv8 model with the EMBC module; the EMBC module expands the channels of the input feature map through a 1×1 convolution layer, and then extracts features through a depthwise separable convolution unit. The ESE unit explicitly models the interdependence between feature channels, enhances useful features and suppresses unimportant features, improves the representation ability of the model, and then compresses the number of channels through a 1×1 convolution layer. Finally, the residual connection unit is used to alleviate the gradient vanishing problem in the deep network and accelerate the training process of the improved YOLOv8 model.
4. According to a method for detecting small kiwi fruit targets in a complex environment based on improved YOLOv8 according to claim 1, it is characterized in that: The CARAFE upsampling operator in S3 includes a kernel prediction module and a content-aware reorganization module. The kernel prediction module compresses the feature channels of the input feature map from C to C through 1×1 convolution. m , using a kernel size of K encoder The convolution layer generates a reorganized convolution kernel based on the input features, and finally uses the Softmax function to perform up ×K up The content-aware reconstruction module dynamically utilizes the generated reorganized convolution kernel W l’ Reorganize input features in local regions.
5. According to a method for detecting small kiwi fruit targets in a complex environment based on improved YOLOv8 according to claim 1, it is characterized in that: The working process of the Dyhead detector is as follows: given a feature vector Where L represents the number of layers of the feature pyramid, S = H × W, S represents the spatial dimension of the feature map, C represents the number of channels of the feature map, and the attention calculation formula is as follows: Where π(·) is an attention function, which includes three sequentially executed attention mechanisms, each focusing on only one aspect: In the formula, π C ,π S ,π L Corresponding to the attention functions applied to C, S, and L dimensions respectively.
6. According to claim 5, a kiwi fruit small target detection method in a complex environment based on improved YOLOv8 is characterized in that: Scale-aware attention function π L Specifically: In the formula, represents the average pooling process, σ(·) represents the sigmoid function, and f(·) represents the linear function approximated by the 1×1 convolutional layer.
7. According to claim 5, a kiwi fruit small target detection method in a complex environment based on improved YOLOv8 is characterized in that: Spatial perception attention function π S Specifically: Where K is the number of sparse sampling locations, Δp k is the learnable spatial offset, Δm k is at position p k The importance scalar of self-learning.
8. According to claim 5, a kiwi fruit small target detection method in a complex environment based on improved YOLOv8 is characterized in that: Task-aware attention function π C Specifically: In the formula, F c is the feature slice of the cth channel, α 1 , β 1 , α 2 , β 2 is a hyperfunction used to learn the control activation threshold.