Crushed soybean particle instance segmentation method based on YOLOv8

Abstract

The invention discloses a broken soybean particle instance segmentation method based on YOLOv8, a new Particle Seg-YOLOv8 model is constructed, and the model fuses a DBB module in a detection head so as to enhance effective information interaction between feature channels and accurately extract detail information of broken soybean particles. In an NECK structure, a feature fusion module is improved based on ASF-YOLO, and a Sobel edge detection operator is integrated, so that the sensitivity and the recognition capability of broken edges are enhanced, and the edge features of broken particles are captured more clearly. In addition, the RFA convolutional layer is used for replacing part of traditional convolutional layers, so that the responsiveness of the model to spatial features is enhanced, and the detection capacity of small targets and broken particles is improved. According to the improved model, on the basis of ensuring high calculation efficiency, the segmentation precision and robustness of the broken soybeans are remarkably improved, and a more accurate solution is provided for breakage rate detection and quality control in the agricultural field.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and agricultural quality inspection, and particularly relates to a method for instance segmentation of broken soybean particles based on YOLOv8. Background Art

[0002] In agriculture and granaries, the quality inspection of broken soybeans is an important link to ensure product quality and safety. Traditional manual inspection methods cannot meet the requirements of efficient and precise quality inspection due to low efficiency and susceptibility to human factors. With the rapid development of computer vision and deep learning technologies, image-based automated inspection methods have gradually become the mainstream. However, due to the broken state and size differences of soybean particles, traditional object detection and instance segmentation methods have obvious deficiencies in capturing small targets and details, resulting in low segmentation accuracy and robustness.

[0003] Currently, in the soybean segmentation task, deep convolutional neural networks (CNNs) are often used for object detection. However, these methods generally rely on traditional convolutional layers and feature fusion modules and cannot fully mine the detailed features of broken particles. Although the existing YOLO series models perform excellently in object detection, in the instance segmentation task, especially in complex backgrounds, they face problems such as low segmentation accuracy and insufficient extraction of edge information. Therefore, improving the instance segmentation accuracy of broken soybeans and enhancing the model's ability to capture small targets and complex details have become urgent technical problems to be solved in the current agricultural quality inspection field. Summary of the Invention

[0004] Aiming at the above-mentioned existing technical deficiencies, the purpose of the present invention is to provide a method for instance segmentation of broken soybean particles based on YOLOv8, which can solve the problems of low detection accuracy of small targets (low algorithm accuracy), large interference from complex backgrounds, high missed detection rate, parameter sharing limitations, and insensitivity to edge information in the prior art. This method can improve segmentation accuracy and robustness, enhance the detection and segmentation ability of broken soybean particles in complex backgrounds, and significantly improve the accuracy of instance segmentation of broken soybeans in the case of severe stacking, providing a more accurate solution for researchers to detect the breakage rate and quality control of broken particles in the agricultural field.

[0005] To solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The present invention provides a method for instance segmentation of broken soybean particles based on YOLOv8, specifically including the following steps:.

[0007] Step 1, data preprocessing and enhancement: Obtain the image dataset of broken soybean particles, and perform data enhancement through rotation, flipping, adding Gaussian noise, and brightness adjustment to generate a training set and a validation set labeled in the YOLO format;

[0008] Step 2, Improvement of the backbone network: Replace the standard convolution modules of the P2, P3, P4, and P5 layers in the backbone feature extraction network of YOLOv8 with RFAConv (Receptive-Field Attention Convolution) convolution modules. The RFAConv convolution module dynamically adjusts the convolution kernel parameters through an attention mechanism, enhancing the ability to extract local features of broken particles; RFAConv enhances the feature extraction ability of the convolution layer, adapts to the detailed features in the small target detection scenario, and can improve the detection accuracy and robustness.

[0009] Step 3, Optimization of multi-scale feature fusion: Replace the Neck structure of the original YOLOv8 with an improved ASF-YOLO framework. The improved ASF-YOLO framework has an advantage in detecting complex backgrounds and small targets by combining global context information and edge information. The improvements include:

[0010] Integrate a global context information enhancement module in the channel attention module, extract global features through global average pooling and generate adaptive channel weights, enhancing the model's ability to model the global characteristics of the target;

[0011] Embed the Sobel edge detection operator in the "Add" module of feature fusion, calculate the horizontal gradient edge_x and vertical gradient edge_y respectively, and fuse the gradient magnitude M to strengthen the edge features and improve the network's ability to detect small targets;

[0012] Step 4, Optimization of the detection head structure: Introduce a multi-branch DBB (Diverse BranchBlock) module into the YOLOv8 detection head. The DBB module is incorporated for the classification and regression prediction of small targets. Diverse features are extracted through multi-branch convolution and pooling operations, and the outputs of each branch are fused through adaptive weights after normalization; improving the network's performance in different detection tasks.

[0013] Step 5, Implementation of instance segmentation: Use the Segment_DBB module to perform segmentation prediction on the multi-scale feature maps of the P3, P4, and P5 layers, fuse the multi-scale features to generate accurate instance segmentation results, and finally improve the segmentation accuracy and target recognition rate to complete the accurate segmentation of small targets in the image;

[0014] Step 6, Model training and validation: Use the training set to train the improved Particle Seg-YOLOv8 model, and evaluate the segmentation accuracy and robustness through the validation set.

[0015] Preferably, in step two, the specific implementation of the RFAConv convolution module is as follows: in the P2, P3, P4, and P5 layers of the YOLOv8 backbone network, the original convolution kernel parameters are multiplied by the spatial attention weights in position to generate spatially adaptive convolution kernel parameters, solving the problem of parameter sharing. The spatial attention weights are calculated from the spatial position information of the feature map.

[0016] Preferably, in step three, horizontal convolutional kernels and vertical convolutional kernels are respectively used to convolve the input feature map to obtain the gradient images edge_x and edge_y in the horizontal and vertical directions, and the gradient magnitude is calculated, where ∈ is a stable constant, used to avoid numerical instability during the square sum and square root process. The edge magnitude M is fused into the original feature map to further strengthen the edge information of the object, which is especially suitable for cases with complex shapes or blurred boundaries in object segmentation and detection, thereby improving the detection accuracy and segmentation performance of the model.

[0017] Preferably, in step four, in the four convolutional blocks of the detection head, a multi-branch structure including 1x1 convolution, 3x3 convolution, and average pooling is introduced. After the outputs of each branch are normalized by the BN layer, they are weighted and fused through the adaptive weight formula: for weighted fusion

[0018] where, W i represents the weight of branch i, μ is the learning rate, and L is the loss function. The adaptive combination process ensures finding the optimal feature fusion strategy among multiple branches, and the weight update is dynamically adjusted based on the contribution of each branch to the loss function.

[0019] Preferably, in step five, the segmentation prediction method of the Segment_DBB module is as follows: the feature maps of the P3, P4, and P5 layers are respectively upsampled to the same resolution, and the multi-scale features are fused through concatenation operation, and the final segmentation mask is generated by combining the classification probability and bounding box information output by the DBB module.

[0020] Preferably, in step one, the specific parameters of data augmentation include: applying ±15% brightness adjustment, Gaussian noise σ = 0.05, and random horizontal / vertical flipping to the original image to expand the diversity of the dataset.

[0021] Preferably, the environment configuration for model training in step six is as follows: using the PyTorch 1.11.0 framework, Python3.8 programming environment, CUDA 11.3 acceleration library, and completing the training on a Windows10 system equipped with an RTX 3090 graphics card.

[0022] Preferably, the performance evaluation metrics of the method include: Intersection over Union (IoU), missed detection rate, and false detection rate, and the segmentation accuracy improvement of the improved model in the stacking and adhesion scenarios is verified by comparing with the original YOLOv8 model.

[0023] Preferably, the implementation manner of the global context information enhancement module is as follows: perform global average pooling on the input feature map to generate a global feature vector, generate channel weights through a fully connected layer, and multiply them with the original feature map channel by channel.

[0024] Preferably, the calculation method of the attention weight of the RFAConv convolution module is as follows: extract the spatial saliency information of the feature map through a spatial attention mechanism, generate an attention map with the same size as the input feature map, and use it to dynamically adjust the convolution kernel parameters.

[0025] The beneficial effects of the present invention are as follows:

[0026] 1. The parameter sharing of traditional convolutional layers leads to insufficient feature diversity and cannot adapt to the detail differences of broken particles. In this method, receptive field attention convolution is introduced in the backbone network (P2 - P5 layers). The convolution kernel weights at different spatial positions are dynamically adjusted through the attention mechanism to strengthen the feature extraction in key regions. The convolution kernel parameters of each receptive field are generated by multiplying the general convolution kernel with the attention weight, solving the problem of parameter sharing. It improves the ability to capture details of small targets and broken particles, improves the detection accuracy, and enhances the robustness of the model to complex scenarios (such as stacked particles).

[0027] 2. Traditional Neck modules ignore global context and edge information during feature fusion, resulting in blurred boundary segmentation. This method uses the ASF - YOLO framework to replace the original Neck module, and realizes multi - scale fusion through cross - scale feature splicing, serial scaling, and Add feature superposition. The Sobel edge detection operator is integrated in the Add module to calculate the horizontal / vertical gradient magnitude M to strengthen the edge features. This method can greatly improve the edge sensitivity and significantly improve the boundary segmentation accuracy of broken particles; the target recognition ability in complex backgrounds is improved through the global context information enhancement module (such as channel attention mechanism).

[0028] 3. Traditional detection heads have a single feature expression ability and are difficult to adapt to the classification and regression of multi - scale targets in complex scenarios. This method introduces the DBB module in the detection head, and extracts diverse features through a combination of multi - branch convolution + pooling (such as 1x1 convolution, 3x3 convolution, average pooling). The adaptive weight fusion formula dynamically optimizes the branch feature combination. It improves feature diversity and greatly reduces the classification and regression errors of the detection head for small targets. Through BN and Pad layer normalization processing, the training stability is enhanced.

[0029] 4. The traditional segmentation module fails to adequately fuse multi-scale features, resulting in incomplete segmentation of the target area. This method performs multi-scale fusion on the features of layers P3 - P5, combines the output of the DBB branch, and generates a high-resolution segmentation mask. By fusing the gradient magnitude and performing adaptive upsampling, the prediction accuracy of the edge region is enhanced. This significantly improves the segmentation intersection over union (IoU), especially for the segmentation of adhesive particles. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0031] Figure 1 is a flowchart of a method for instance segmentation of broken soybean particles based on YOLOv8 provided by the present invention;

[0032] Figure 2 is a schematic diagram of the original YOLOV8 model structure provided by the present invention;

[0033] Figure 3 is a schematic diagram of the Particle Seg - YOLO model structure provided by the present invention;

[0034] Figure 4 is a schematic diagram of the data annotation method in the embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of the RFA mechanism provided by the present invention;

[0036] Figure 6 is a schematic diagram of the structure of ASF - YOLO provided by the present invention;

[0037] Figure 7 is a schematic diagram of the segmentation results of the YOLOv8 model and the Particle Seg - YOLO model provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0039] As Figures 1 - 6As shown in the figure, this embodiment provides a method for instance segmentation of broken soybean particles based on YOLOv8, including the following steps:

[0040] Step 1: Obtain pictures of broken soybeans.

[0041] Step 2: Use the labelme tool to manually annotate the obtained images and divide them into a training set and a validation set.

[0042] Use the labelme annotation tool to select the YOLO format to manually annotate the images after manual screening: The specific method of data annotation is as Figure 3 shown. Manually annotate each soybean instance in the image. Note that when annotating, the label box should coincide with the target area as much as possible. In this way, annotate all the soybean instances that appear, and the label name is huangdou; save both the label name and the position in a.txt file. After annotation, randomly divide the dataset into a training set and a validation set according to a ratio of 4:1.

[0043] Step 3: Build a Particle Seg-YOLOv8 model.

[0044] 1) Replace the convolution module of the backbone feature extraction network with the RFAConv convolution module:

[0045] Replace the convolution modules in the P2, P3, P4, and P5 layers with the RFAConv convolution module, effectively solving the problem of parameter sharing. The attention mechanism is introduced, and each unique convolution kernel parameter K in the receptive field i is obtained by multiplying the general convolution kernel parameter K by the corresponding attention weight A i This convolution allows the network to assign different importance to features at different spatial positions during the feature extraction process, enhancing the ability of the YOLOv8 model to capture key features.

[0046] 2) Use the improved ASF-YOLO framework to replace the original Neck layer:

[0047] Add a global context information enhancement module to the feature fusion structure of ASF-YOLO to increase the network's ability to capture global information. Specifically, add global context operations to the channel attention module. By adaptively adjusting the channel weights of the features, the model can extract global context information, thereby modeling the global characteristics of the target and improving robustness. The global context enhancement module keeps the original number of channel parameters unchanged during the calculation process;

[0048] In the "Add" module of the feature fusion structure of ASF-YOLO, a Sobel edge detection operation is added to enhance the model's sensitivity to object boundaries. The Sobel operator extracts edge features by calculating the gradients of the image in the horizontal and vertical directions. Specifically, the Sobel operator uses two 3x3 convolutional kernels to calculate the gradients in the horizontal and vertical directions respectively:

[0049] and

[0050] where G x is used to calculate the gradient in the horizontal direction, and G y is used to calculate the gradient in the vertical direction. By convolving these two convolutional kernels with the input feature map respectively, the gradient images edge_x and edge_y in the horizontal and vertical directions are obtained. Then, the gradient magnitude M of each pixel is calculated as the edge feature:

[0051]

[0052] where ∈ is a sufficiently small positive number to avoid numerical instability during the square sum and square root process. The edge magnitude M is fused into the original feature map to further strengthen the edge information of the object, which is particularly suitable for cases with complex shapes or blurred boundaries in object segmentation and detection, thereby improving the detection accuracy and segmentation performance of the model;

[0053] 1) Improve the original detection head of YOLOv8 into a Segment-DBB detection head:

[0054] In the detection head, the Diverse Branch Block (DBB) module is used. The DBB module processes the input feature map through a multi-branch convolutional structure, and uses branches composed of various convolutional kernels and pooling operations to further enhance the diversity of feature representation. The output of this module is the weighted fusion of the features of each branch, and its calculation formula is:

[0055]

[0056] where f i represents different convolutional or pooling operations, and N represents the number of branches. By introducing different convolutional and pooling operations, the DBB module can extract richer features;

[0057] Use BN and Pad layers to standardize and smooth the branch outputs: Each DBB branch is standardized through the BatchNormalization (BN) layer after convolution and smoothed through the BNAndPadLayer. This processing aims to enhance feature stability and reduce training fluctuations;

[0058] Add the combination of 1x1 convolution and 3x3 convolution, as well as the average pooling branch in the DBB module. This combination method is smoothed and normalized by the BNAndPadLayer, further enhancing the expression ability of multi-branch features. At the same time, in the DBB module, adaptive combination is performed on different branch features. Through the learning of weight parameters, automatic weighting of features is achieved to ensure the extraction of the best combination among different branch features. Its adaptive weight update formula is as follows:

[0059]

[0060] where, W i represents the weight of branch i, μ is the learning rate (usually taken as 10 -6 or 10 -8 , to ensure the stable convergence of the model), and L is the loss function. The adaptive combination process ensures the finding of the optimal feature fusion strategy among multiple branches.

[0061] The Segment-DBB module integrates the native prediction head of YOLO, takes the output of the DBB module as the input layer for object detection, and predicts the bounding box and classification probability. Through the processing and fusion of multi-branch features by the DBB module, the Segment-DBB module effectively improves the performance of object detection without significantly increasing the computational amount.

[0062] Step 4, train the Particle Seg-YOLOv8 model.

[0063] Input the training set in the labeled broken soybean image dataset into the Particle Seg-YOLOv8 model for training to obtain the best.pt weight file, and then obtain the trained Particle Seg-YOLOv8 model. After the training is completed, use the validation set for validation.

[0064] Step 5, use the trained Particle Seg-YOLOv8 model to identify the broken soybean images in the sampling shovel.

[0065] Detect using the trained YOLOv8 model and Particle Seg-YOLOv8 model respectively on the same test dataset, and the comparison results are as Figure 7 shown. From Figure 7It can be seen that, compared with the YOLOv8 model, the Particle Seg-YOLOv8 model has stronger broken soybean segmentation ability, performs better in distinguishing adhesion and stacking situations, has more thorough multi-scale feature fusion, stronger ability to segment edge regions, and lower omission and misdetection rates. Even in a complex un-preprocessed sampling environment, it has good segmentation effects, has good robustness and generalization ability, and can meet the requirements of unmanned inspection of the qualified rate of soybean crushing in the oil pressing workshop.

[0066] The framework used in this embodiment is Pytorch1.11.0, the Python version is 3.8, the CUDA version is 11.3, the training system is Winsows10, and the graphics card model used for training is RTX 3090 24GB.

[0067] Based on the YOLOv8 model, the present invention enhances the extraction of detailed features by introducing the RFAConv convolution module to solve the problem of information loss in small target detection; adopts the feature fusion mechanism of ASF-YOLO, combines the Zoom_cat, ScalSeq, and Add modules to perform feature fusion at multiple scales, and enhances the edge perception ability; incorporates the DBB module into the detection head to improve the detection ability of small targets; and realizes instance segmentation through the Segment_DBB module, further improving the segmentation accuracy.

[0068] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A method for segmenting broken soybean particles based on YOLOv8, characterized in that: The following steps are involved: Step 1: Data preprocessing and enhancement: Obtain a dataset of broken soybean particle images, perform data enhancement by rotating, flipping, adding Gaussian noise, and adjusting brightness, and generate training and validation sets annotated in YOLO format; Step 2: Improve the backbone network: replace the standard convolution modules of the P2, P3, P4, and P5 layers in the backbone feature extraction network of YOLOv8 with RFAConv convolution modules. The RFAConv convolution module dynamically adjusts the convolution kernel parameters through the attention mechanism to enhance the ability to extract local features of broken particles. Step 3: Multi-scale feature fusion optimization: The improved ASF-YOLO framework is used to replace the original YOLOv8 Neck structure. The improvements include: The global context information enhancement module is integrated into the channel attention module to extract global features and generate adaptive channel weights through global average pooling, thereby enhancing the model's ability to model the global characteristics of the target. The Sobel edge detection operator is embedded in the "Add" module of feature fusion to calculate the horizontal gradient edge_x and the vertical gradient edge_y respectively, and the gradient amplitude M is fused to strengthen the edge features and improve the network's ability to detect small targets; Step 4: Optimize the detection head structure: Introduce a multi-branch Diverse Branch Block module, referred to as the DBB module, into the YOLOv8 detection head. Diversified features are extracted through multi-branch convolution and pooling operations. The outputs of each branch are standardized and fused through adaptive weights. Step 5: Instance segmentation implementation: Use the Segment_DBB module to perform segmentation prediction on the multi-scale feature maps of the P3, P4, and P5 layers, and fuse the multi-scale features to generate accurate instance segmentation results; Step 6: Model training and verification: Use the training set to train the improved Particle Seg-YOLOv8 model, and use the verification set to evaluate the segmentation accuracy and robustness.

2. The method according to claim 1, characterized in that In step 2, the specific implementation method of the RFAConv convolution module is as follows: in the P2, P3, P4, and P5 layers of the YOLOv8 backbone network, the original convolution kernel parameters are multiplied by the spatial attention weights by position to generate spatially adaptive convolution kernel parameters to solve the parameter sharing problem. The spatial attention weights are calculated through the spatial position information of the feature map.

3. The method according to any one of claims 1 to 2, characterized in that ,In step three, horizontal convolution kernel is used And the vertical convolution kernel Convolve the input feature map to obtain the horizontal and vertical gradient images edge_x and edge_y, and calculate the gradient amplitude Where ∈ is a stability constant.

4. The method according to any one of claims 1 to 3, characterized in that: In step 4, a multi-branch structure including 1x1 convolution, 3x3 convolution and average pooling is introduced into the four convolution blocks of the detection head. After the output of each branch is standardized by the BN layer, the adaptive weight formula is used: Weighted Fusion Among them, W i represents the weight of branch i, μ is the learning rate, L is the loss function, the adaptive combination process ensures that the optimal feature fusion strategy is found in multiple branches, and the weight update is dynamically adjusted based on the contribution of each branch to the loss function.

5. The method according to any one of claims 1 to 4, characterized in that: In step 5, the segmentation prediction method of the Segment_DBB module is as follows: upsample the feature maps of the P3, P4, and P5 layers to the same resolution, fuse the multi-scale features through splicing operations, and combine the classification probability and bounding box information output by the DBB module to generate the final segmentation mask.

6. The method according to any one of claims 1 to 5, characterized in that: In step 1, the specific parameters of data enhancement include: applying ±15% brightness adjustment, Gaussian noise σ = 0.05, and random horizontal / vertical flipping to the original image to expand the diversity of the data set.

7. The method according to any one of claims 1 to 6, characterized in that: The environment configuration for model training in step six is: using the PyTorch 1.11.0 framework, Python 3.8 programming environment, CUDA 11.3 acceleration library, and completing the training on a Windows 10 system equipped with an RTX3090 graphics card.

8. The method according to any one of claims 1 to 7, characterized in that: The performance evaluation indicators of the method include: intersection over union (IoU), missed detection rate, and false detection rate. The improved model is compared with the original YOLOv8 model to verify the improved segmentation accuracy in stacking and adhesion scenarios.

9. The method according to any one of claims 1 to 8, characterized in that: The global context information enhancement module is implemented by performing global average pooling on the input feature map to generate a global feature vector, generating channel weights through a fully connected layer and multiplying the weights with the original feature map channel by channel.

10. The method according to any one of claims 1 to 9, characterized in that: The attention weight calculation method of the RFAConv convolution module is as follows: the spatial saliency information of the feature map is extracted through the spatial attention mechanism, and an attention map with the same size as the input feature map is generated for dynamically adjusting the convolution kernel parameters.

Images