Machine part identification and segmentation method based on YOLOv8
By introducing GAM, SAC and BiFPN structures into YOLO v8, and using Focaler-IoU to improve the bounding box regression loss, the problem of YOLO's poor performance in small object detection and instance segmentation tasks is solved, achieving higher accuracy and robustness.
Patent Information
- Application Number
- CN202411339460.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-01-21
AI Technical Summary
YOLO performs poorly in small object detection and instance segmentation tasks, especially when dealing with small or densely arranged parts, and is prone to missed or missed inspections, and is not very accurate in object segmentation of complex shapes.
On the basis of YOLO v8, GAM, SAC and BiFPN structures are introduced, and Focaler-IoU is used to improve the bounding box regression loss to improve feature extraction capabilities and bounding box regression accuracy.
It has significantly improved the model's mechanical part instance segmentation and recognition effects in actual industrial applications, improved indicators such as precision, recall, and mAP, and enhanced the performance and robustness of the model.
Smart Images
Figure CN119339076A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine parts target detection and segmentation, and particularly relates to a YOLOv8m-GAM-BiFPN segmentation model which integrates a YOLO v8m-seg model with a GAM, SAC, and BiFPN structure and adopts Focaler-IoU to improve the bounding box regression loss. Background Art
[0002] With the rapid development of artificial intelligence technology, research in the field of computer vision has gradually deepened into many practical application fields. Among them, instance segmentation and recognition technology of machine parts has received widespread attention as an important part in the fields of intelligent manufacturing and automated production. In actual industrial applications, there are many types of machine parts and a large number of morphological differences. Traditional image processing methods are difficult to meet the needs of efficient and accurate recognition. Deep learning has better recognition ability, calculation speed and detection accuracy than traditional algorithms. Therefore, with the help of deep learning technology, especially the integrated application of target detection and instance segmentation, it has become an important way to solve this problem. First, YOLO's performance in small object detection is relatively weak. Due to its network structure and feature extraction mechanism, YOLO often misses or misdetects when dealing with small or densely arranged parts. This is particularly prominent in industrial environments because many parts may overlap in images and are small in size. Secondly, YOLO's performance in instance segmentation tasks is limited. Although YOLO can realize bounding box detection of objects, its accuracy in accurately segmenting object edges, especially when the object shape is complex, is not as good as some models designed specifically for instance segmentation. This may lead to difficulties in subsequent processing steps for application scenarios that require high-precision segmentation. In order to improve the instance segmentation and recognition of mechanical parts in actual industrial applications, this paper adopts the YOLO v8 model to implement the part recognition and segmentation tasks. Based on YOLO v8, a targeted and efficient network for instance segmentation and recognition of mechanical parts in complex scenarios is designed. Summary of the invention
[0003] The purpose of the present invention is to improve the deficiencies in the above-mentioned background technology and improve the instance segmentation and recognition effect of mechanical parts in actual industrial applications. Therefore, a YOLO v8 model is specially designed to realize the recognition and segmentation tasks of parts. On the basis of YOLO v8, a targeted and efficient network suitable for instance segmentation and recognition of mechanical parts in complex scenes is designed. In the feature extraction backbone, switchable hole convolution and global attention mechanism are introduced to improve the feature extraction capability; in the feature fusion layer, a more complex feature fusion structure is added to replace the original FPN-PAN structure to better fuse feature information; for the bounding box regression loss, the Focaler-IoU loss is introduced to improve the deficiencies of CIoU, thereby improving the performance and stability of the model. In order to achieve the above technical objectives, the technical solution of the present invention is:
[0004] A YOLO v8m-GAM-BiFPN segmentation model that combines the YOLO v8m-seg model with the GAM, SAC, and BiFPN structures, and uses Focaler-IoU to improve the bounding box regression loss.
[0005] In the structural optimization of the YOLO v8 algorithm, the feature extraction layer is a crucial link. In order to further improve the performance and accuracy of the model, the present invention introduces the GAM module and integrates it into the feature extraction part of the backbone. GAM is a global attention mechanism that can effectively capture the channel correlation and spatial correlation between the input feature maps, thereby enhancing the expressiveness and semantic information of the features. The module includes two parts: channel attention and spatial attention, which are implemented by linear mapping and convolution operations respectively. Channel attention compresses and expands the channel dimension of the input feature map through linear mapping, and then activates it through the ReLU activation function to obtain the channel attention weight. This step of operation can make the model pay more attention to the important information between different channels, which helps to improve the expressiveness and semantic information of the features. Spatial attention uses convolution operations to perform spatial convolution on the channel dimension, and then obtains the spatial attention weight through batch normalization and Sigmoid activation function. This step of operation can make the model pay more attention to the spatial structure and local details of the feature map, which helps to improve the positioning accuracy and detection performance of the model for the target.
[0006] To further improve the feature extraction capability of the YOLO v8 model, the present invention improves the C2f module by converting the 3×3 convolutional layer into SAC. Two global context modules are embedded before and after the SAC component. These modules are designed to understand the image content more comprehensively and provide more extensive context information to the SAC component, thereby enhancing the model's ability to perceive features.
[0007] In the implementation, the present invention utilizes the characteristics of dilated convolution and captures features of different scales by adjusting different dilation rates. A smaller dilation rate can capture local detail information, while a larger dilation rate can capture broader context information. In this way, the model can flexibly select the appropriate dilation rate according to the characteristics of the input data and task requirements, thereby better understanding the image content and improving the effect of feature extraction.
[0008] The present invention introduces the Global Attention Module (GAM) module and switchable dilated convolution (SAC) when optimizing the feature extraction layer of the YOLO v8 algorithm. GAM enhances feature expression capabilities through channel and spatial attention mechanisms, while SAC is applied in the C2f module to capture features of different scales by dynamically adjusting the dilation rate. These improvements effectively improve the model's ability to perceive image content and the effect of feature extraction.
[0009] The present invention adopts the BiFPN structure as the feature fusion layer. The BiFPN structure fuses multi-scale information through two fusion paths from top to bottom and from bottom to top, and adds two lateral connection paths on the basis of the original FPN-PAN structure, further enhancing the semantic information and improving the detection performance. In addition, the fusion of the backbone features effectively solves the problem of image information loss due to resolution changes, further improving the detection performance of small targets. Through these improvements, the Neck layer plays a more critical role in the YOLO v8 network, enabling the model to show better performance and robustness in target detection tasks.
[0010] The present invention calculates the difference values of width and height respectively to replace the aspect ratio based on CIoU. The EIoU loss function consists of three parts: overlap loss, center distance loss, and width and height loss. Among them, overlap loss and center distance loss are similar to CIoU, while width and height loss directly minimizes the difference between the width and height of the real box and the anchor box to more comprehensively reflect the similarity between the target box and the anchor box, thereby improving the performance and robustness of the model. The EIoU loss function is as follows:
[0011]
[0012] Among them, c w and c h They represent the width and height of the minimum detection box covering the real box and the anchor box respectively.
[0013] Since CIoU does not consider the impact of difficult and easy sample distribution on bounding box regression, the Focaler-IoU proposed based on the existing bounding box regression method is used to focus on different regression samples through linear interval mapping. The formula is as follows:
[0014]
[0015] d and u are the lower and upper thresholds of IoU, respectively, [d,u]∈[0,1]. By adjusting the values of d and u, the loss function can pay more attention to the samples with medium degree. Applying Focaler-IoU loss to EIoU loss, we get L Focaler-EIoU :
[0016] L Focaler-EIoU =L EIoU +IoU-IoU Focaler
[0017] The present invention also discloses a YOLO v8m-GAM-BiFPN segmentation model, which integrates the YOLO v8m-seg model with the GAM, SAC, and BiFPN structures, and adopts the Focaler-IoU method to improve the bounding box regression loss, including the following steps:
[0018] S1: A batch of car wheel-related parts photographed from a cooperative factory are used as a dataset, including single parts and framed parts. The improved YOLO v8m-GAM-BiFPN segmentation model is trained: the seven types of parts datasets are used for training to obtain the trained model weights;
[0019] S2: Load the trained model and input the test set to obtain the performance indicators of the model in the object detection and instance segmentation tasks, including precision, recall, mAP50 and mAP50-95;
[0020] S3: The four performance indicators of precision, recall, mAP50 and mAP50-95 can roughly show the effect of object detection and instance segmentation.
[0021] Beneficial effects: Compared with the prior art, the present invention integrates the YOLO v8m-seg model with the GAM, SAC, and .BiFPN structures, and adopts the Focaler-IoU method to improve the bounding box regression loss. Compared with the traditional YOLO v8, the present invention can significantly improve the precision, recall, and mAP indicators. These results show that the proposed improved method has a significant effect in improving the performance and robustness of the model, and can perform high-precision recognition and segmentation of machine parts. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is the YOLO v8 technical roadmap in the present invention;
[0023] Figure 2 It is the GAM structure diagram;
[0024] Figure 3 It is a schematic diagram of the SAC structure;
[0025] Figure 4 This is a schematic diagram of the optimization of the feature extraction layer that integrates GAM and SAC;
[0026] Figure 5 This is a schematic diagram of the Neck layer structure of BiFPN. DETAILED DESCRIPTION
[0027] The content of the present invention is further described below in conjunction with the accompanying drawings.
[0028] like Figure 1 The YOLO v8 technical roadmap in the present invention is shown, which includes the following steps:
[0029] S1: Preliminary data preprocessing: The present invention uses data enhancement methods such as mirroring, adding noise, adjusting brightness, and translating images to expand the data set. After data enhancement, the parts of each category are counted again, and the ratio of the training set to the test set of the entire data set is 3:1, as shown in Table 1;
[0030] Table 1: Statistical information of the number of various parts
[0031] Component Training set Test Set class1 1434 386 class2 1579 395 class3 1141 461 class4 1395 465 class5 2069 759 class6 3004 748 class7 3713 1049
[0032] S2: Training of object detection and instance segmentation: Utilizing the characteristics of dilated convolution, different dilation rates are adjusted to capture features of different scales. A smaller dilation rate can capture local detail information, while a larger dilation rate can capture broader contextual information. In this way, the model can flexibly select an appropriate dilation rate based on the characteristics of the input data and task requirements, so as to better understand the image content and improve the effect of feature extraction. The BiFPN structure fuses multi-scale information through two fusion paths, top-down and bottom-up, and adds two lateral connection paths based on the original FPN-PAN structure to further enhance semantic information. In addition, the fusion of backbone features effectively solves the problem of image information loss due to resolution changes, and further improves the detection performance of small objects. After training, the object detection and instance segmentation weights can be obtained;
[0033] S3: Input the test set images into the trained classification network. After testing, we can get four performance indicators: precision, recall, mAP50 and mAP50-95, from which we can understand the effectiveness of object detection and instance segmentation.
[0034] like Figure 3The figure shows a schematic diagram of the SAC structure. Two global context modules are embedded before and after the SAC component. These modules are designed to understand the image content more comprehensively and provide the SAC component with more extensive context information, thereby enhancing the model's ability to perceive features. SAC uses an additional convolutional layer through a switch function mechanism to dynamically generate a weight value to control the contribution of two different convolution paths. This weight value is dynamically calculated based on the input feature map. The model adaptively chooses to use a standard convolution path or an adjustable hole convolution path based on the characteristics of the input data.
[0035] Figure 4 The figure shows the optimization diagram of the feature extraction layer that combines GAM and SAC. GAM enhances the feature expression ability through channel and spatial attention mechanisms, while SAC is applied in the C2f module to capture features of different scales by dynamically adjusting the hole rate. These improvements effectively improve the model's perception of image content and the effect of feature extraction.
[0036] Figure 5 The figure shows the schematic diagram of the Neck layer structure of BiFPN. The BiFPN structure fuses multi-scale information through two fusion paths, top-down and bottom-up, and adds two lateral connection paths on the basis of the original FPN-PAN structure, further enhancing the semantic information and improving the detection performance. In addition, the fusion of backbone features effectively solves the problem of image information loss due to resolution change, further improving the detection performance of small targets. Through these improvements, the Neck layer plays a more critical role in the YOLO v8 network, enabling the model to show better performance and robustness in target detection tasks.
[0037] This implementation case proposes a YOLO v8m-GAM-BiFPN segmentation model by integrating the YOLO v8m-seg model with the GAM, SAC, and BiFPN structures, and using Focaler-IoU to improve the bounding box regression loss. The experimental results show that the improved model has significant improvements in precision, recall, and mAP. The experimental results show that the proposed improvement method has a significant effect in improving model performance and robustness.
[0038] Table 2 shows the performance comparison between the improved YOLO v8m-GAM-BiFPN and YOLO v8m-seg, including precision, recall, mAP50 and mAP50-95:
[0039] Table 2 Performance comparison between improved YOLO v8m-GAM-BiFPN and YOLO v8m-seg
[0040] Model Precision(B) Recall(B) mAP50(B) mAP50-95(B) YOLOv8m-seg 93.3％ 90.4％ 93.2％ 85.9％ YOLOv8m-GAM-BiFPN 94.9％ 91.0％ 94.1％ 87.2％ Model Precision(M) Recall(M) mAP50(M) mAP50-95(M) YOLOv8m-seg 94.3％ 89.0％ 93.0％ 76.2％ YOLOv8m-GAM-BiFPN 94.5％ 90.7％ 93.7％ 77.2％
[0041] In summary, the target detection and instance segmentation of the present invention are higher in accuracy than those of YOLO v8, and the fusion of backbone features effectively solves the problem of image information loss due to resolution change, further improves the detection performance of small targets, and is more conducive to the subsequent analysis of machine parts detection and segmentation.
Claims
1. A machine parts recognition and segmentation model based on YOLOv8, characterized by: The model combines the YOLOv8m-seg model with the GAM (global attention module), SAC (switchable atrous convolution) and BiFPN (dual-path feature pyramid network) structures, and improves the bounding box regression loss through Focaler-IoU, achieving efficient instance segmentation and recognition of machine parts in complex scenes.
2. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The GAM module enhances the expressiveness of features through channel attention and spatial attention mechanisms, and can effectively capture the channel correlation and spatial correlation between input feature maps, thereby improving recognition accuracy.
3. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The SAC module dynamically adjusts the dilation rate of the dilated convolution in the feature extraction layer and combines the global context module to capture features of different scales, thereby enhancing the model's perception and feature extraction capabilities of image content.
4. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The BiFPN structure adopts two multi-scale fusion paths from top to bottom and from bottom to top, and adds a lateral connection path, which can further enhance semantic information and improve small target detection performance.
5. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The Focaler-IoU loss function focuses on medium-difficulty samples in the bounding box regression task through linear interval mapping, which improves the robustness of the model in complex scenes and the accuracy of bounding box positioning.
6. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The model introduces an improved bounding box regression loss EIoU to calculate the width and height difference between the detection box and the anchor box respectively to improve the performance and stability of the model.
7. The machine part recognition and segmentation model based on YOLOv8 according to claim 1, characterized in that: The dataset is expanded through data augmentation techniques, including mirroring, adding noise, adjusting brightness, and translating images, to improve the model's ability to recognize machine parts of various shapes.