Watermelon and watermelon stem identification method based on improved YOLOv8n-obb

Abstract

The invention discloses a watermelon and watermelon stem identification method based on improved YOLOv8n-obb. The watermelon and watermelon stem identification method comprises the following steps: S1, acquiring watermelon images at different angles under different weather and illumination conditions in a watermelon greenhouse and a watermelon field by using a camera, and making a data set; s2, performing improvement by taking YOLOv8n-obb as a basic model to obtain an improved model; s3, training the data set in the S1 by using the improved model to obtain a watermelon and watermelon stem identification model, and outputting a weight file in a. Pt format; and S4, loading the weight file generated in the step S3, and detecting the watermelon and watermelon stem identification model by taking the test set in the step S1 as an input image to obtain a detection result of the watermelon and the watermelon stem. The method improves the recognition effect when shielding exists, and improves the precision.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology in the agricultural direction, and specifically to a method for identifying watermelons and their stems based on improved YOLOv8n-obb. Background Art

[0002] In the watermelon production process, watermelon picking is one of the important links, which requires a lot of manual participation. However, due to labor shortage and aging, as well as low mechanization and repetitive labor, picking is time-consuming and laborious. The use of picking robots can effectively alleviate many of the above problems.

[0003] For watermelon picking robots, the key information that needs visual positioning is the position of the watermelon and the position of the handle that needs to be cut. The accuracy of watermelon and handle detection and positioning directly affects the work quality of the watermelon picking robot, so the recognition quality needs to be improved. Watermelon recognition requires both the identification of large target watermelons and small target handles. However, small target recognition has problems such as feature loss, missed detection, and high false detection rate. At the same time, when recognizing watermelons, there are problems such as the fruit and the background color are similar, and there are leaf occlusions. In recent years, with the development of deep learning in the field of computer vision, deep learning algorithms based on convolutional neural networks have achieved accuracy far exceeding traditional recognition algorithms on major open source image data sets. Therefore, in order to improve recognition efficiency, deep learning detection algorithms are chosen to identify watermelons.

[0004] Deep learning algorithms can be divided into single-stage algorithms and two-stage algorithms in the field of target detection. Compared with the two-stage algorithm, the one-stage algorithm directly extracts the category and location information of the target from the image, and does not need to go through the two-stage algorithm to generate candidate boxes. Therefore, it has the advantages of less calculation and faster detection speed, and is easier to meet the real-time requirements in actual application scenarios. In the prior art, YOLOv8 is a member of the single-stage algorithm YOLO series, with excellent performance. Although it can identify watermelons, there is still room for improvement. Summary of the invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides a method for recognizing watermelons and their stems based on improved YOLOv8n-obb, which improves the recognition effect when there is occlusion and improves the accuracy.

[0006] The technical solution adopted by the present invention to solve its technical problem is:

[0007] A method for identifying watermelons and their stems based on improved YOLOv8n-obb, comprising the following steps:

[0008] S1. Use cameras to collect watermelon images from different angles under different weather and lighting conditions in watermelon greenhouses and watermelon fields and create a dataset;

[0009] S2. Improve YOLOv8n-obb as the basic model to obtain the improved model. The process is as follows:

[0010] S21. Improve the C2f module of the Backbone part of the YOLOv8n-obb model, and design the C2f_RFAConv module in combination with the receptive field attention convolution RFAConv to replace the C2f module.

[0011] S22. Improve the Head part of the YOLOv8n-obb model and add a small target detection layer to obtain a feature detection layer with n×n pixels.

[0012] S23. Improve the Neck part of the YOLOv8n-obb model and add the SEAM multi-head attention network module;

[0013] S3, using the improved model to train the data set in S1, obtain a watermelon and stem recognition model and output a weight file in .pt format;

[0014] S4. Load the weight file generated in S3 and use the test set in S1 as the input image to detect the watermelon and stem recognition model to obtain the detection results of the watermelon and stem.

[0015] Furthermore, in S1, the process of making the watermelon dataset is as follows: cropping the captured picture; manually annotating the watermelon and its stem with a rotating frame using the roLabelImg annotation tool, and including the stem when annotating the stem; after the annotation is completed, performing data enhancement by flipping, translating, rotating, and adding noise to the picture to obtain the dataset; and randomly dividing the dataset in a ratio of 7:2:1 to obtain a training set, a validation set, and a test set.

[0016] Furthermore, in S21, the C2f-RFAConv module is designed in combination with the receptive field attention convolution RFAConv, and the process is as follows:

[0017] The receptive field attention convolution RFAConv module is introduced. This module uses average pooling AvgPool to aggregate the global information of each receptive field feature, uses grouped convolution of size 1×1 for information interaction, and finally uses the normalized exponential function Softmax to emphasize the importance of each feature in the receptive field feature. The calculation formula of RFA is as follows:

[0018] F = softmax(g 1×1 (AvgPool(X)))×ReLU(Norm(g n×n (X)))

[0019] Among them, gn×n Represents a grouped convolution of size n×n, n represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and ReLU is the activation function;

[0020] The RFAConv module is used to replace the second convolution of the Bottleneck part in the original C2f module to obtain the C2f_RFAConv module.

[0021] Furthermore, in S22, n is taken as 160, and a small target detection layer is added to obtain a feature detection layer with pixels of 160×160. The small target detection layer is amplified before the detection head structure of the original YOLOv8n-obb model, and the output feature map size of this layer is 160×160 pixels.

[0022] In the S23, a SEAM multi-head attention network module is added, and the process is as follows: the SEAM module uses a channel and spatial hybrid module CSMM to learn the importance of different channels. The CSMM module used has three different sizes, and its step size is determined by the patch size, which is 6, 7 and 8 respectively. Then the three modules output the results and perform average pooling AvgPool, and perform channel expansion Channel exp operation, and finally multiply to provide enhanced feature representation; CSMM uses different patches for multi-scale feature processing, and uses deep separable convolution to learn the correlation between spatial dimensions and channels, and then performs point-by-point convolution to combine the outputs of deep separable convolution; the SEAM module is added to all connected positions of the Neck part and the Head part of the YOLOv8n-obb model.

[0023] In S4, the detection results of the watermelon and the stem include: the location, category name and confidence of the watermelon and the stem.

[0024] The technical concept of the present invention is: using YOLOv8n-obb as the basic network model, and replacing the C2f module of the Backbone part of the YOLOv8n-obb model by designing the C2f_RFAConv module in combination with the receptive field attention convolution RFAConv, thereby strengthening the spatial feature extraction capability of the model at multiple scales. At the same time, by adding a small target detection layer, the model's recognition capability for the melon handle is improved. By introducing the SEAM multi-head attention network module, the model's detailed processing capability for spatial dimensions and channels is enhanced, the model's detection effect when occluded is improved, and the applicability of the model is enhanced. Through the above improvements, the model's recognition accuracy for watermelons and melon handles is effectively improved, especially the model's detection capability when occluded is improved, and the applicability of the model is enhanced.

[0025] The beneficial effects of the present invention are mainly manifested in: improving the recognition effect when there is occlusion and improving the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of the process for implementing the detection method of the present invention;

[0027] Figure 2 This is an overall schematic diagram of the improved network structure of the present invention;

[0028] Figure 3 It is a structural schematic diagram of the RFAConv module of the present invention;

[0029] Figure 4 It is a schematic diagram of the structure of the C2f_RFAConv module of the present invention;

[0030] Figure 5 It is a structural schematic diagram of the SEAM module of the present invention;

[0031] Figure 6 It is a schematic diagram of the structure of the CSMM module of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings.

[0033] Reference Figure 1 to Figure 6 , a watermelon and its stem recognition method based on improved YOLOv8n-obb, such as Figure 1 As shown, the following steps are included:

[0034] S1. Use cameras to collect watermelon images from different angles under different weather and lighting conditions in watermelon greenhouses and watermelon fields and create a dataset. The process is as follows:

[0035] S11, cropping the captured image. The original image has various sizes, including 1980×1080, 1280×720, etc. The portion to be detected is cropped, and the cropped size is unified to 640×640 pixels;

[0036] S12, using the roLabelImg annotation tool to manually perform rotating frame annotation on the watermelon and its stem, and the stem needs to be included when annotating the stem;

[0037] S13. After the annotation is completed, the data is enhanced by flipping, translating, rotating and adding noise to the image to obtain the data set;

[0038] S14, randomly divide the data set into a ratio of 7:2:1 to obtain a training set, a validation set, and a test set;

[0039] S2. Improve YOLOv8n-obb as the basic model to obtain an improved model with the following structure: Figure 2As shown, the process is as follows:

[0040] S21. Improve the C2f module of the Backbone part of the YOLOv8n-obb model, and design the C2f_RFAConv module in combination with the receptive field attention convolution RFAConv to replace the C2f module.

[0041] Specifically, the structure of the receptive field attention convolution RFAConv module is as follows Figure 3 As shown in the figure, this module uses average pooling AvgPool to aggregate the global information of each receptive field feature, uses grouped convolution of size 1×1 for information interaction, and finally uses the normalized exponential function Softmax to emphasize the importance of each feature in the receptive field feature. In general, the calculation formula of RFA is as follows:

[0042] F=Sooftmax(g 1×1 (AvgPool(X)))×ReLU(Norm(g n×n (X)))

[0043] Among them, g n×n Represents a grouped convolution of size n×n, n represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and ReLU is the activation function;

[0044] The RFAConv module is used to replace the second convolution of the Bottleneck part in the original C2f module to obtain the RFAConv_Bottleneck module. Then the RFAConv_Bottleneck module is used to replace all the Bottleneck modules in C2f to obtain the C2f_RFAConv module. The specific structure is as follows: Figure 4 As shown;

[0045] S22. Improve the Head part of the YOLOv8n-obb model and add a small target detection layer to obtain a feature detection layer with a pixel size of 160×160.

[0046] Specifically, a small target detection layer is added before the detection head structure of the original YOLOv8n-obb model. The output feature map size of this layer is 160×160 pixels. At the same time, the original detection head structure is retained. The final model has 4 layers of detection heads, which process feature maps with pixel sizes of 160×160, 80×80, 40×40, and 20×20 respectively.

[0047] S23. Improve the Neck part of the YOLOv8n-obb model and add the SEAM multi-head attention network module;

[0048] Specifically, the structure of the SEAM module is as follows: Figure 5 As shown in Figure 1, the SEAM module uses a channel and spatial hybrid module CSMM to learn the importance of different channels. The CSMM module used has three different sizes, and its step size is determined by the patch size, which is 6, 7 and 8 respectively. Then the three modules output the results and perform average pooling AvgPool, and then perform channel expansion Channel exp operation, and finally multiply to provide enhanced feature representation; the structure of CSMM is shown in Figure 1 Figure 6 As shown in , it uses patches of different sizes for multi-scale feature processing, uses deep separable convolution to learn the correlation between spatial dimensions and channels, and then combines the outputs of deep separable convolution through point-by-point (1x1) convolution; the SEAM module is added to all the locations where the Neck part of the YOLOv8n-obb model is connected to the Head part, as shown in Figure 2 As shown, there are 4 positions in total;

[0049] S3, using the improved model to train the training set in S1, obtain a watermelon and stem recognition model and output a weight file in .pt format;

[0050] S4. Load the weight file generated in S3 and use the test set in S1 as the input image to detect the watermelon and stem recognition model to obtain the detection results of the watermelon and stem, which include the location, category name, and confidence level of the watermelon and stem.

[0051] The solution of this embodiment uses YOLOv8n-obb as the basic network model, and replaces the C2f module of the Backbone part of the YOLOv8n-obb model by designing the C2f_RFAConv module in combination with the receptive field attention convolution RFAConv, thereby strengthening the spatial feature extraction capability of the model at multiple scales; at the same time, by adding a small target detection layer, the model's recognition capability of the melon handle is improved, and by introducing the SEAM multi-head attention network module, the model's detailed processing capability of spatial dimensions and channels is enhanced, the model's detection effect when occluded is improved, and the model's applicability is enhanced. The solution of this embodiment effectively improves the model's recognition accuracy of watermelons and melon handles, especially improves the model's detection capability when occluded, and enhances the model's applicability.

[0052] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A watermelon and its stem recognition method based on improved YOLOv8n-obb, It is characterized in that The method comprises the following steps: S1. Use cameras to collect watermelon images from different angles under different weather and lighting conditions in watermelon greenhouses and watermelon fields and create a dataset; S2. Improve YOLOv8n-obb as the basic model to obtain the improved model. The process is as follows: S21. Improve the C2f module of the Backbone part of the YOLOv8n-obb model, and design the C2f_RFAConv module in combination with the receptive field attention convolution RFAConv to replace the C2f module. S22. Improve the Head part of the YOLOv8n-obb model and add a small target detection layer to obtain a feature detection layer with n×n pixels. S23. Improve the Neck part of the YOLOv8n-obb model and add the SEAM multi-head attention network module; S3, using the improved model to train the data set in S1, obtain a watermelon and stem recognition model and output a weight file in .pt format; S4. Load the weight file generated in S3 and use the test set in S1 as the input image to detect the watermelon and stem recognition model to obtain the detection results of the watermelon and stem.

2. A method for identifying watermelons and their stems based on improved YOLOv8n-obb as claimed in claim 1, It is characterized in that In S1, the process of making a watermelon dataset is as follows: cropping the captured image; manually annotating the watermelon and its stem with a rotating frame using the roLabelImg annotation tool, and including the stem when annotating the stem; after the annotation is completed, performing data enhancement by image flipping, translation, rotation, and adding noise to obtain a dataset; and randomly dividing the dataset in a ratio of 7:2:1 to obtain a training set, a validation set, and a test set.

3. A method for identifying watermelons and their stems based on improved YOLOv8n-obb as claimed in claim 1 or 2, It is characterized in that In S21, the C2f-RFAConv module is designed in combination with the receptive field attention convolution RFAConv, and the process is as follows: The receptive field attention convolution RFAConv module is introduced. This module uses average pooling AvgPool to aggregate the global information of each receptive field feature, uses grouped convolution of size 1×1 for information interaction, and finally uses the normalized exponential function Softmax to emphasize the importance of each feature in the receptive field feature. The calculation formula of RFA is as follows: F=Softmax(g 1×1 (AvgPool(X)))×ReLU(Norm(gn ×n (X))) Among them, g n×n Represents a grouped convolution of size n×n, n represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and ReLU is the activation function; The RFAConv module is used to replace the second convolution of the Bottleneck part in the original C2f module to obtain the C2f_RFAConv module.

4. A method for identifying watermelons and their stems based on improved YOLOv8n-obb as claimed in claim 1 or 2, It is characterized in that In S22, n is 160, and a small target detection layer is added to obtain a feature detection layer with a pixel size of 160×160. The small target detection layer is amplified before the detection head structure of the original YOLOv8n-obb model, and the output feature map size of this layer is 160×160 pixels.

5. A method for identifying watermelons and their stems based on improved YOLOv8n-obb as claimed in claim 1 or 2, It is characterized in that In the S23, a SEAM multi-head attention network module is added, and the process is as follows: the SEAM module uses a channel and spatial hybrid module CSMM to learn the importance of different channels. The CSMM module used has three different sizes, and its step size is determined by the patch size, which is 6, 7 and 8 respectively. Then the three modules output the results and perform average pooling AvgPool, and perform channel expansion Channel exp operation, and finally multiply to provide enhanced feature representation; CSMM uses different patches for multi-scale feature processing, and uses deep separable convolution to learn the correlation between spatial dimensions and channels, and then performs point-by-point convolution to combine the outputs of deep separable convolution; the SEAM module is added to all connected positions of the Neck part and the Head part of the YOLOv8n-obb model.

6. A method for identifying watermelons and their stems based on improved YOLOv8n-obb as claimed in claim 1 or 2, It is characterized in that In S4, the detection results of the watermelon and the stem include: the location, category name and confidence of the watermelon and the stem.

Images