Peanut segmentation method based on improved YOLOv8n-seg
By improving the YOLOv8n-seg model, replacing the module and introducing feature fusion and attention mechanisms, the problem of insufficient peanut segmentation accuracy and adaptability is solved, and more efficient peanut feature extraction and precise segmentation is achieved, which is suitable for mobile deployment.
Patent Information
- Application Number
- CN202410963688.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-18
AI Technical Summary
Traditional methods have low accuracy in peanut segmentation and are not adaptable to different scenarios, making it difficult to effectively improve peanut feature extraction ability and segmentation accuracy.
Improve the YOLOv8n-seg model, and by replacing some modules with C2f_DCNv3 and DCNv3, introducing scale sequence feature fusion SSFF and triple feature encoder TFE module, as well as an efficient multi-scale attention mechanism EMA, build the YOLOv8n-seg-DSE model to enhance feature extraction capabilities .
It improves the accuracy of peanut segmentation and model adaptability, can segment peanuts more accurately, and reduces the model size, making it easier to deploy on the mobile side.
Smart Images

Figure CN118918327A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image segmentation, and in particular to a peanut segmentation method based on improved YOLOv8n-seg. Background Art
[0002] Peanuts, also known as longevity nuts, are important edible oil resources and high-quality plant protein resources with high application value. my country occupies an important position in the world's peanut production, with the highest total output, the highest average per-acre yield, the lowest sales price and the largest export volume.
[0003] For the peanut segmentation problem, the traditional method uses the watershed algorithm to segment the stuck peanuts, but the segmentation accuracy of the watershed algorithm is not high, and the adaptability to segmentation in different scenarios is not good enough. With the development of deep learning, detection models represented by the YOLO series have developed rapidly. Among them, the YOLOv8-seg model takes into account both target detection and instance segmentation. It can effectively complete the detection of peanut defects while segmenting, which can reduce a lot of time and the cost of manual detection of peanut defects. However, for peanut segmentation, how to improve the model's ability to extract peanut features and improve the accuracy of peanut segmentation is an urgent problem to be solved. Summary of the invention
[0004] In view of the needs of the application field and the shortcomings of the background technology, the purpose of the present invention is to provide a peanut segmentation method based on improved YOLOv8n-seg.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is:
[0006] Step 1: Collect peanut images, perform data augmentation and annotation on the peanut images to generate a peanut dataset;
[0007] Step 2: Divide the peanut dataset into training set and validation set according to the set ratio;
[0008] Step 3: Build an improved YOLOv8n-seg model, replace some C2f modules and Conv modules in the backbone network of the YOLOv8n-seg model with C2f_DCNv3 modules and DCNv3 modules respectively; use the scale sequence feature fusion SSFF module and the triple feature encoder TFE module in the neck network; add an efficient multi-scale attention mechanism EMA module to the backbone network to obtain the improved YOLOv8n-seg-DSE model;
[0009] Step 4: Use the training set and validation set to train and validate the YOLOv8n-seg-DSE model to obtain a trained peanut segmentation model;
[0010] Step 5: Get the peanut image to be segmented, input the trained peanut segmentation model to obtain the segmentation detection result;
[0011] The process of constructing the improved YOLOv8n-seg model in step 3 is as follows:
[0012] The 2nd, 3rd and 4th C2f modules in the backbone network of the YOLOv8n-seg model are replaced by C2f_DCNv3 modules, and the 3rd, 4th and 5th Conv modules are replaced by deformable convolution DCNv3 modules to improve the model's ability to extract features from deformable objects; a scale sequence feature fusion SSFF module and two triple feature encoder TFE modules are used in the neck network of the YOLOv8n-seg model to reconstruct the feature fusion layer to achieve the fusion of feature information of different feature maps; an efficient multi-scale attention mechanism EMA module is added in front of the SPPF module in the backbone network to enhance the model's feature extraction capability, and finally the improved YOLOv8n-seg-DSE model is obtained; the 0th to 10th layers in the backbone network of the YOLOv8n-seg-DSE model are connected to Conv, Conv, C2f, DCNv3, C2f DCNv3, DCNv3, C2f DCNv3, DCNv3, C2f DCNv3, EMA, SPPF; the 11th to 26th layers in the neck network are Conv, Conv, TFE, C2f, Conv, Conv, TFE, C2f, Conv, Concat, C2f, Conv, Conact, C2f, SSFF, Add;The output of the 10th layer SPPF is connected to the input of the 11th layer Conv, the output of the 4th layer C2f_DCNv3 is connected to the input of the 12th layer Conv, the outputs of the 12th layer Conv, the 6th layer C2f_DCNv3, and the 11th layer Conv are connected to the input of the 13th layer TFE, the output of the 13th layer TFE is connected to the input of the 14th layer C2f, the output of the 14th layer C2f is connected to the input of the 15th layer Conv, the output of the 2nd layer C2f is connected to the input of the 16th layer Conv, the outputs of the 16th layer Conv, the 4th layer C2f_DCNv3, and the 15th layer Conv are connected to the input of the 17th layer TFE, the output of the 17th layer TFE is connected to the input of the 18th layer C2f, the output of the 18th layer C2f is connected to the input of the 19th layer Conv, and the 19th layer The outputs of the Conv layer and the 15th Conv layer are connected to the input of the 20th Concat layer, the output of the 20th Concat layer is connected to the input of the 21st C2f layer, the output of the 21st C2f layer is connected to the input of the 22nd Conv layer, the outputs of the 22nd Conv layer and the 11th Conv layer are connected to the input of the 23rd Concat layer, the output of the 23rd Concat layer is connected to the input of the 24th C2f layer, the outputs of the 4th C2f_DCNv3 layer, the 6th C2f_DCNv3 layer and the 8th C2f_DCNv3 layer are connected to the input of the 25th SSFF layer, the outputs of the 18th C2f layer and the 25th SSFF layer are connected to the input of the 26th Add layer, and the 26th Add layer, the 21st C2f layer and the 24th C2f layer are connected to a segmentation head Segment respectively. ;
[0013] Furthermore, in the step 1, the specific method of performing data enhancement on the peanut image is image flipping, image rotation and image translation.
[0014] Furthermore, in step 2, the peanut dataset is divided into a training set and a validation set in a ratio of 8:2.
[0015] Furthermore, in step 3, the DCNv3 module deforms the convolution kernel during the convolution process so that it can adapt to the changes in local features at different positions. By introducing learnable deformation parameters, the shape and sampling position of the convolution kernel can be adaptively adjusted according to the local features in the peanut feature map, so that the model can better extract peanut features.
[0016] Furthermore, in the step 3, the scale sequence feature fusion SSFF module inputs features of three different scales: large, medium and small. First, the number of channels of the medium-scale and small-scale features is adjusted to 256 using 1×1 convolution, and the size of the medium-scale and small-scale features is adjusted to the same as the large-size feature size using the nearest neighbor interpolation method; secondly, the dimension expansion method is used to convert each feature map from a three-dimensional tensor consisting of height, width and channel to a four-dimensional tensor consisting of depth, height, width and channel, and the four-dimensional tensor is spliced in the depth dimension; finally, 3D convolution, 3D batch normalization and activation function are used to complete the scale sequence feature extraction.
[0017] Furthermore, in the step 3, the triple feature encoder TFE module uses three feature maps of different scales as input. First, the convolution module is used to adjust the number of channels of the large-size features to the same as the medium-size features, and a hybrid structure of maximum pooling and average pooling is used for down-sampling, which helps to maintain the diversity of high-resolution features and images; secondly, for small-size features, the convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for up-sampling, which helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets; finally, the large, medium, and small-size feature maps of the same size are convolved once and spliced in the channel dimension.
[0018] Furthermore, in the step 3, the efficient multi-scale attention mechanism EMA module adopts three parallel paths to extract attention channel weights for grouping feature maps. The first two are 1×1 branches, and the third is a 3×3 branch. In the 1×1 branch, two 1D global average pooling operations are used to encode the channels along two spatial directions; for the 3×3 branch, a stacked 3×3 kernel is used to capture multi-scale feature representation. The three branches integrate the advantages of channel attention and spatial attention, while capturing global channel dependencies and local spatial features, and obtaining more comprehensive feature information between channel and spatial dimensions.
[0019] Compared with the background technology, the gain effect of the present invention is:
[0020] 1. The improved YOLOv8n-seg-DSE model enhances the peanut feature extraction capability, improves the accuracy of peanut segmentation, and can segment peanuts more accurately.
[0021] 2. Compared with YOLOv8n-seg, the improved YOLOv8n-seg-DSE model is smaller and easier to deploy on mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is a flow chart of the present invention;
[0023] Figure 2 This is an example image of peanut outline annotation;
[0024] Figure 3 This is the YOLOv8n-seg network structure diagram;
[0025] Figure 4 This is the YOLOv8n-seg-DSE network structure diagram;
[0026] Figure 5 This is the principle diagram of deformable convolution;
[0027] Figure 6 It is the structural diagram of the scale sequence feature fusion SSFF module;
[0028] Figure 7 It is the module structure diagram of triple feature encoder TFE;
[0029] Figure 8 This is the module structure diagram of the efficient multi-scale attention mechanism EMA;
[0030] Fig. 9 This is an example of a peanut segmentation result image. DETAILED DESCRIPTION
[0031] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods (examples), but they are not intended to limit the present invention.
[0032] The main steps of the present invention are as follows Figure 1 As shown, specifically:
[0033] Step 1: Collect peanut images, perform data augmentation and annotation on the peanut images to generate a peanut dataset;
[0034] In this embodiment, the specific method of data enhancement for the peanut image is image flipping, image rotation and image translation. Figure 2 As shown in the figure, the peanut dataset is annotated using the image annotation tool labelme. The number of images before data augmentation is 600, and the number of images after data augmentation is 2770.
[0035] Step 2: Divide the peanut dataset into training set and validation set according to the set ratio;
[0036] In this embodiment, the peanut dataset is divided into a training set and a validation set in a ratio of 8:2.
[0037] Step 3: Build an improved YOLOv8n-seg model, such as Figure 3As shown in the figure, some C2f modules and Conv modules in the backbone network of the YOLOv8n-seg model are replaced with C2f_DCNv3 modules and DCNv3 modules respectively; the scale sequence feature fusion SSFF module and the triple feature encoder TFE module are used in the neck network; the efficient multi-scale attention mechanism EMA module is added to the backbone network to obtain the improved YOLOv8n-seg-DSE model, which is as follows:
[0038] like Figure 4As shown in the figure, the 2nd, 3rd and 4th C2f modules in the backbone network of the YOLOv8n-seg model are replaced by C2f_DCNv3 modules, and the 3rd, 4th and 5th Conv modules are replaced by deformable convolution DCNv3 modules to improve the model's ability to extract features from deformable objects; a scale sequence feature fusion SSFF module and two triple feature encoder TFE modules are used in the neck network of the YOLOv8n-seg model to reconstruct the feature fusion layer to achieve the fusion of feature information of different feature maps; an efficient multi-scale attention mechanism EMA module is added in front of the SPPF module in the backbone network to enhance the model's feature extraction capability. Finally, the improved YOLOv8n-seg-DSE model is obtained; the 0th to 10th layers in the backbone network of the YOLOv8n-seg-DSE model are connected respectively Conv, Conv, C2f, DCNv3, C2f_DCNv3, DCNv3, C2f_DCNv3, DCNv3, C2f_DCNv3, EMA, and SPPF; the 11th to 26th layers in the neck network are Conv, Conv, TFE, C2f, Conv, Conv, TFE, C2f, Conv, Concat, C2f, Conv, Conact, C2f, SSFF, and Add;The output of the 10th layer SPPF is connected to the input of the 11th layer Conv, the output of the 4th layer C2f_DCNv3 is connected to the input of the 12th layer Conv, the outputs of the 12th layer Conv, the 6th layer C2f_DCNv3, and the 11th layer Conv are connected to the input of the 13th layer TFE, the output of the 13th layer TFE is connected to the input of the 14th layer C2f, the output of the 14th layer C2f is connected to the input of the 15th layer Conv, the output of the 2nd layer C2f is connected to the input of the 16th layer Conv, the outputs of the 16th layer Conv, the 4th layer C2f_DCNv3, and the 15th layer Conv are connected to the input of the 17th layer TFE, the output of the 17th layer TFE is connected to the input of the 18th layer C2f, the output of the 18th layer C2f is connected to the input of the 19th layer Conv, and the 19th layer The outputs of the Conv layer and the 15th Conv layer are connected to the input of the 20th Concat layer, the output of the 20th Concat layer is connected to the input of the 21st C2f layer, the output of the 21st C2f layer is connected to the input of the 22nd Conv layer, the outputs of the 22nd Conv layer and the 11th Conv layer are connected to the input of the 23rd Concat layer, the output of the 23rd Concat layer is connected to the input of the 24th C2f layer, the outputs of the 4th C2f_DCNv3 layer, the 6th C2f_DCNv3 layer and the 8th C2f_DCNv3 layer are connected to the input of the 25th SSFF layer, the outputs of the 18th C2f layer and the 25th SSFF layer are connected to the input of the 26th Add layer, and the 26th Add layer, the 21st C2f layer and the 24th C2f layer are connected to a segmentation head Segment respectively. ;
[0039] like Figure 5 As shown in the figure, the DCNv3 module deforms the convolution kernel during the convolution process so that it can adapt to the changes in local features at different positions. By introducing learnable deformation parameters, the shape and sampling position of the convolution kernel can be adaptively adjusted according to the local features in the peanut feature map, so that the model can better extract peanut features.
[0040] like Figure 6 As shown in the figure, the input of the scale sequence feature fusion SSFF module is features of three different scales: large, medium and small. First, the number of channels of the medium-scale and small-scale features is adjusted to 256 using 1×1 convolution, and the size of the medium-scale and small-scale features is adjusted to the same as the large-size feature size using the nearest neighbor interpolation method; secondly, the dimension expansion method is used to convert each feature map from a three-dimensional tensor consisting of height, width and channel to a four-dimensional tensor consisting of depth, height, width and channel, and the four-dimensional tensor is spliced in the depth dimension; finally, 3D convolution, 3D batch normalization and activation function are used to complete the scale sequence feature extraction.
[0041] like Figure 7 As shown in the figure, the triple feature encoder TFE module uses feature maps of three different scales as input. First, the convolution module is used to adjust the number of channels of the large-size features to the same as the medium-size features, and the hybrid structure of maximum pooling and average pooling is used for down-sampling, which helps to maintain the diversity of high-resolution features and images; secondly, for small-size features, the convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for up-sampling, which helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets; finally, the feature maps of large, medium, and small sizes with the same size are convolved once and spliced in the channel dimension.
[0042] like Figure 8 As shown in the figure, the efficient multi-scale attention mechanism EMA module adopts three parallel paths to extract attention channel weights for grouping feature maps. The first two are 1×1 branches, and the third is a 3×3 branch. In the 1×1 branch, two 1D global average pooling operations are used to encode the channels along two spatial directions; for the 3×3 branch, a stacked 3×3 kernel is used to capture multi-scale feature representations. The three branches integrate the advantages of channel attention and spatial attention, while capturing global channel dependencies and local spatial features, and obtaining more comprehensive feature information between channel and spatial dimensions.
[0043] Step 4: Use the training set and validation set to train and validate the YOLOv8n-seg-DSE model to obtain a trained peanut segmentation model;
[0044] In this embodiment, an experimental comparison is performed on the models before and after improvement under the same data set, with 100 epochs, as shown in Table 1, where YOLOv8n-seg is the unimproved basic model, and YOLOv8n-seg-DSE is an improved model using C2f_DCNv3, DCNv3, SSFF, TFE and EMA modules.
[0045] Table 1 Experimental results of the model before and after improvement
[0046]
[0047] As can be seen from the table above, compared with YOLOv8n-seg, the precision, recall, mAP50 and mAP50-95 of the improved model increased by 1%, 1.1%, 1.1% and 1.1% respectively. The model size is also reduced, and it can segment peanuts more accurately.
[0048] Step 5: Get the peanut image to be segmented, input the trained peanut segmentation model to obtain the segmentation detection result, namely the peanut category and peanut mask.
[0049] In this embodiment, there are two types of peanuts, namely, non-defective peanuts and defective NG peanuts. Fig. 9 As shown, the peanuts in the peanut segmentation result image example are all defect-free, and a mask for each peanut is obtained.
[0050] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A peanut segmentation method based on improved YOLOv8n-seg, characterized by: The following steps are involved: Step 1: Collect peanut images, perform data augmentation and annotation on the peanut images to generate a peanut dataset; Step 2: Divide the peanut dataset into training set and validation set according to the set ratio; Step 3: Build an improved YOLOv8n-seg model, replace some C2f modules and Conv modules in the backbone network of the YOLOv8n-seg model with C2f_DCNv3 modules and DCNv3 modules respectively; use the scale sequence feature fusion SSFF module and the triple feature encoder TFE module in the neck network; add an efficient multi-scale attention mechanism EMA module to the backbone network to obtain the improved YOLOv8n-seg-DSE model; Step 4: Use the training set and validation set to train and validate the YOLOv8n-seg-DSE model to obtain a trained peanut segmentation model; Step 5: Get the peanut image to be segmented, input the trained peanut segmentation model to obtain the segmentation detection result; The process of constructing the improved YOLOv8n-seg model in step 3 is as follows: The 2nd, 3rd and 4th C2f modules in the backbone network of the YOLOv8n-seg model are replaced by C2f_DCNv3 modules, and the 3rd, 4th and 5th Conv modules are replaced by deformable convolution DCNv3 modules to improve the model's ability to extract features from deformable objects; a scale sequence feature fusion SSFF module and two triple feature encoder TFE modules are used in the neck network of the YOLOv8n-seg model to reconstruct the feature fusion layer to achieve the fusion of feature information of different feature maps; an efficient multi-scale attention mechanism EMA module is added in front of the SPPF module in the backbone network to enhance the model's feature extraction capability. The improved YOLOv8n-seg-DSE model is obtained; the 0th to 10th layers in the backbone network of the YOLOv8n-seg-DSE model are connected respectively Conv, Conv, C2f, DCNv3, C2f_DCNv3, DCNv3, C2f_DCNv3, DCNv3, C2f_DCNv3, EMA, and SPPF; the 11th to 26th layers in the neck network are Conv, Conv, TFE, C2f, Conv, Conv, TFE, C2f, Conv, Concat, C2f, Conv, Conact, C2f, SSFF, and Add;The output of the 10th layer SPPF is connected to the input of the 11th layer Conv, the output of the 4th layer C2f_DCNv3 is connected to the input of the 12th layer Conv, the outputs of the 12th layer Conv, the 6th layer C2f_DCNv3, and the 11th layer Conv are connected to the input of the 13th layer TFE, the output of the 13th layer TFE is connected to the input of the 14th layer C2f, the output of the 14th layer C2f is connected to the input of the 15th layer Conv, the output of the 2nd layer C2f is connected to the input of the 16th layer Conv, the outputs of the 16th layer Conv, the 4th layer C2f_DCNv3, and the 15th layer Conv are connected to the input of the 17th layer TFE, the output of the 17th layer TFE is connected to the input of the 18th layer C2f, the output of the 18th layer C2f is connected to the input of the 19th layer Conv, and the 19th layer The outputs of the Conv layer and the 15th Conv layer are connected to the input of the 20th Concat layer, the output of the 20th Concat layer is connected to the input of the 21st C2f layer, the output of the 21st C2f layer is connected to the input of the 22nd Conv layer, the outputs of the 22nd Conv layer and the 11th Conv layer are connected to the input of the 23rd Concat layer, the output of the 23rd Concat layer is connected to the input of the 24th C2f layer, the outputs of the 4th C2f_DCNv3 layer, the 6th C2f_DCNv3 layer and the 8th C2f_DCNv3 layer are connected to the input of the 25th SSFF layer, the outputs of the 18th C2f layer and the 25th SSFF layer are connected to the input of the 26th Add layer, and the 26th Add layer, the 21st C2f layer and the 24th C2f layer are connected to a segmentation head Segment respectively. ; 2. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In the step 1, the specific method of performing data enhancement on the peanut image is image flipping, image rotation and image translation.
3. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In step 2, the peanut dataset is divided into a training set and a validation set in a ratio of 8:
2.
4. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In step 3, the DCNv3 module deforms the convolution kernel during the convolution process so that it can adapt to the changes in local features at different positions. By introducing learnable deformation parameters, the shape and sampling position of the convolution kernel can be adaptively adjusted according to the local features in the peanut feature map, so that the model can better extract peanut features.
5. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In the step 3, the scale sequence feature fusion SSFF module inputs features of three different scales: large, medium and small. First, the number of channels of the medium-scale and small-scale features is adjusted to 256 using 1×1 convolution, and the size of the medium-scale and small-scale features is adjusted to the same as the large-size feature size using the nearest neighbor interpolation method; secondly, the dimension expansion method is used to convert each feature map from a three-dimensional tensor consisting of height, width and channel to a four-dimensional tensor consisting of depth, height, width and channel, and the four-dimensional tensor is spliced in the depth dimension; finally, 3D convolution, 3D batch normalization and activation function are used to complete the scale sequence feature extraction.
6. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In the step 3, the triple feature encoder TFE module uses three feature maps of different scales as input. First, the convolution module is used to adjust the number of channels of the large-size features to the same as the medium-size features, and the hybrid structure of maximum pooling and average pooling is used for down-sampling, which helps to maintain the diversity of high-resolution features and images; secondly, for small-size features, the convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for up-sampling, which helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets; finally, the feature maps of large, medium and small sizes with the same size are convolved once and spliced in the channel dimension.
7. The peanut segmentation method based on improved YOLOv8n-seg according to claim 1, characterized in that: In step 3, the efficient multi-scale attention mechanism EMA module uses three parallel paths to extract attention channel weights for grouping feature maps, the first two are 1×1 branches, and the third is a 3×3 branch. Two 1D global average pooling operations are used in the 1×1 branch to encode channels along two spatial directions; For the 3×3 branch, a stacked 3×3 kernel is used to capture multi-scale feature representations. The three branches integrate the advantages of channel attention and spatial attention, while capturing global channel dependencies and local spatial features, and obtaining more comprehensive feature information between channel and spatial dimensions.
Citation Information
Patent Citations
Deep learning-based tea bud detection method and model building method thereof
CN116580305A
Deep learning-based throwing tool automatic positioning and tracking system and method
CN117789098A
Lightweight target detection network design method for indoor semantic SLAM system
CN118097365A
Fan blade defect detection method and system based on improved SSD model
WO2023173598A1
Cited By
Seed germination segmentation method and seed germination rate and germination rate identification method
CN119339087A
Seed germination segmentation method, seed germination rate and germination rate identification method
CN119339087B
Field crop phenotype intelligent acquisition method and robot
CN119478050A
A method and robot for intelligently acquiring field crop phenotypes
CN119478050B
Wild rice salt tolerance intelligent evaluation method and system based on deep learning
CN120198779A
