Drainage pipeline defect detection method based on improved YOLOv8s

Abstract

The invention discloses a drainage pipeline defect detection method based on improved YOLOv8s, and the method comprises the steps: obtaining a drainage pipeline defect data set, carrying out the data enhancement of the drainage pipeline defect data set, and dividing the drainage pipeline defect data set into a training set, a verification set and a test set; according to the method, YOLOv8s is used as a network basic model, a CBAM attention module is introduced into a backbone network, and feature extraction of pipeline defects is enhanced; an FPN-PAN structure of a neck network is replaced by a BiFPN structure, so that the network feature fusion capability is improved; a new Wise-IoU loss function is adopted to improve the detection precision of the model; and finally, a Conv module in a part of the network is replaced by a GhostConv module, and a C2f module is improved into a C2f-RepGhost module, so that the weight of the network is reduced. Conveying the training set to a drainage pipeline defect detection model for training; and conveying a to-be-detected drainage pipeline defect picture to the trained drainage pipeline defect detection model to obtain a final detection result. Compared with an original model, the detection precision is improved, and support is provided for defect detection of the drainage pipeline.

Description

Technical Field

[0001] The present invention belongs to the technical field of drainage pipe defect detection, and in particular relates to a drainage pipe defect detection method based on improved YOLOv8s. Background Art

[0002] With the continuous development of science and technology, deep learning networks are increasingly used in the field of target detection. Traditional pipeline defect detection methods mainly include manual well detection and the use of periscope measuring equipment to detect drainage pipes, but they all have problems such as low detection accuracy, easy missed detection, and false detection. Therefore, it is particularly important to use efficient, intelligent, and automated technologies to reduce manpower and material resources and improve the efficiency of drainage pipe defect detection. At present, there are many algorithms in the field of target detection. YOLOv8s is one of them. It includes the use of the backbone network to extract features from the input image and the feature fusion of the neck network, and finally completes the classification of the target image through the detection network.

[0003] In the task of defect detection of drainage pipes, since a large number of drainage pipes are used to discharge industrial wastewater and domestic sewage, various defects often occur inside the pipes. In addition, changes in various environmental factors will lead to uncertainty in the position and shape of the target object. Internal defects such as misalignment may be blocked by obstacles, resulting in inaccurate identification. There may be many defects in the same image. In addition, the defects in the drainage pipes vary greatly in scale and are not fixed in position, which will bring certain difficulties to detection. Summary of the invention

[0004] In view of the above problems, the present invention provides a model based on improved YOLOv8s. First, a CBAM (Convolutional Block Attention Module) attention module is introduced after each C2f module of the backbone network to enhance the network's ability to extract occluded targets; the BiFPN (Bidirectional Feature Pyramid Network) structure is used in the neck network to replace the original FPN-PAN (Feature Pyramid Network-Path Aggregation Network) structure, and cross-scale connections and additional connections are added between input and output nodes at the same level, so that the defects with a large number of types and large scale changes in drainage pipe defects have better detection effects; a new loss function Wise-IoU (Wise Intersection over Union) is used to speed up the convergence of the model; finally, the GhostConv module and the C2f-RepGhost module are used to lightweight the network so that it does not occupy a large amount of computing resources. Specifically, the following steps are included:

[0005] 1. A drainage pipe defect detection method based on improved YOLOv8s, characterized by comprising the following steps:

[0006] Step 1. Collect drainage pipe defect data images, perform data enhancement on the collected images, and then divide them into training sets, verification sets, and test sets;

[0007] Step 2. Build a drainage pipe defect detection model based on the improved YOLOv8s. After the drainage pipe defect image is processed by the convolution module and C2f residual module of the backbone network to extract features, it is processed by the CBAM attention module to increase the weight of the area of ​​interest in the image.

[0008] Step 3. Send the feature map extracted by the backbone network to the neck network, and introduce the BiFPN structure in the neck network to replace the FPN-PAN structure. It uses a bidirectional connection operation to integrate information from different levels of the pyramid, which is conducive to capturing more scale features;

[0009] Step 4. Use the loss function Wise-IoU instead of the CIoU (Complete Intersection over Union) loss function to optimize the model performance, use the outlier degree as an indicator to evaluate the anchor box, and adopt a reasonable gradient gain allocation strategy to effectively reduce the harmful gradients generated by low-quality examples;

[0010] Step 5. Use the GhostConv convolution module and C2f-RepGhost module to improve the model in a lightweight way, perform low-cost convolution operation on the input feature map to obtain feature information, and then concatenate the generated features with the convolution results of the standard channel to obtain the output features, effectively reducing the number of model parameters;

[0011] Step 6. Train and adjust the hyperparameters of the improved YOLOv8s model. After determining the optimal parameters, use the test set to evaluate the detection performance of the model.

[0012] Preferably, in step 1, a real water pipe defect data image is obtained, and the image is annotated using LabelImg software, and then the acquired data set is enhanced. The image enhancement uses random rotation, adding noise, adjusting brightness and other means to enrich the number of defect images, totaling 10,000 images. The enhanced data is divided into training set, validation set and test set in a ratio of 8:1:1.

[0013] Preferably, in step 2, a CBAM attention module is introduced into the backbone network of the YOLOv8s network structure to improve the performance of the detection model by enhancing the feature extraction capability of the region of interest.

[0014] Preferably, in step 3, the FPN-PAN structure is replaced with a weighted bidirectional feature pyramid BiFPN structure in the neck network. The weighted bidirectional feature pyramid BiFPN structure can learn the weights of different input features in the network, set different weights for different feature layers, reduce the connection of unnecessary layers and increase some skip layer connections, so as to integrate more features without increasing the computational cost.

[0015] Preferably, in step 4, the detection head network uses a Wise-IoU loss function to replace the original CIoU loss function, and the Wise-IoU loss function formula is as follows:

[0016]

[0017]

[0018]

[0019] Where b i represents the predicted bounding box coordinates of the i-th target, g i represents the true bounding box coordinates of the i-th target; IoU(b i ,g i ) represents the degree of overlap between the i-th predicted bounding box and the true bounding box; W g and H g The size of the smallest enclosing box; The superscript * in the text indicates W g and H g is separated from the computational graph; x and y are the coordinates of the center point of the predicted bounding box; x gt and gt is the coordinate of the center point of the real box; β and r are the outlier degree and gradient gain respectively; δ is the penalty hyperparameter; L IoU is the running mean of the index with momentum; is the monotonic focusing coefficient.

[0020] Preferably, in step 5, the model is lightweighted using a GhostConv convolution module and a C2f-RepGhost module. In order to reduce the amount of network calculation, the GhostConv module first generates a feature map with a smaller channel through traditional convolution, and the newly acquired feature map further reduces the amount of calculation through cheap operation to generate a new feature map, and finally splices the front and back two sets of feature maps as the output result; the C2f-RepGhost module is essentially an improvement on the Bottleneck in the C2f module, wherein the RepGhost module omits the 1×1 convolution in the first step of the original Ghost module, replaces the Concat operation with the Add operation, further reduces the amount of calculation, and moves the ReLU after the Add operation so that the module satisfies the structural reparameterization rule. Finally, the BN operation is added in the identity mapping branch to make the training process nonlinear.

[0021] The present invention has the following beneficial effects:

[0022] The present invention introduces the CBAM attention module into the backbone network of the original YOLOv8s model to enhance the feature extraction capability of the backbone network and effectively improve the problem of low recognition accuracy caused by missed detection and occlusion.

[0023] The present invention uses a weighted bidirectional feature pyramid BiFPN structure to replace the FPN-PAN structure. The BiFPN structure effectively utilizes high-level and low-level feature information for fusion, thereby making the prediction box of the model more accurate and also performing well in the case of multiple targets.

[0024] The present invention adopts the Wise-IoU loss function to make the model converge faster and improve the accuracy of defect identification.

[0025] The present invention uses a lightweight GhostConv module and a C2f-RepGhost module. The lightweight module not only reduces the number of model parameters but also improves the detection speed of the model and is more friendly to the deployment and application of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a drainage pipe defect detection method based on improved YOLOv8s in the present invention;

[0027] Figure 2 It is the overall framework diagram of the detection model of the present invention;

[0028] Figure 3 This is the structure diagram of the convolution module CBS of the present invention;

[0029] Figure 4 This is a structural diagram of the bottleneck module Bottleneck of the present invention;

[0030] Figure 5 This is a structural diagram of the residual module C2f of the present invention;

[0031] Figure 6 This is the structural diagram of the attention mechanism module CBAM of the present invention;

[0032] Figure 7 This is the structural diagram of the feature fusion module BiFPN of the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0034] The present invention provides a drainage pipe defect detection method based on improved YOLOv8s, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0035] Step 1. Collect drainage pipe defect data images, perform data enhancement on the collected images, and then divide them into training sets, verification sets, and test sets;

[0036] The step 1 specifically includes:

[0037] The collection of drainage pipe defect data images is completed in rainwater pipes and sewage pipes by the amphibious exploration mobile robot of Hefei Harbin Institute of Technology Zhiling Intelligent Technology Co., Ltd. The robot is used to walk in the pipe to shoot pipeline defect images. Common types of defects in drainage pipes include scaling, misalignment, deposition, leakage and obstacles. LabelImg software is used to annotate and classify the collected defect images, and finally the acquired data set is enhanced. Image enhancement uses random rotation, adding noise and adjusting brightness to enrich the number of defect images, totaling 10,000 images. The enhanced data is divided into training set, validation set and test set in a ratio of 8:1:1.

[0038] Step 2. Introduce the CBAM attention module into the backbone network of the YOLOv8s network for feature extraction;

[0039] The step 2 specifically includes:

[0040] The CBAM attention module is introduced into the backbone network of the YOLOv8s network structure. The backbone network of YOLOv8s is used as the feature extraction part. The CBAM attention module is added after each C2f module before feature fusion. The CBAM attention module is performed in two steps. In the channel attention processing part, firstly, a global maximum pooling downsampling and a global average pooling downsampling operation are performed on the input feature map respectively. The feature map is changed from the original H×W×C to two 1×1×C feature maps. After the MLP change, two 1×1×C feature maps are output. The feature maps are added and processed by the sigmoid activation function to obtain the Channel Attention. For the spatial attention processing part, firstly, the global maximum pooling downsampling and the global average pooling downsampling are performed on the channel dimension to obtain the feature map whose size is H×W×1. The two feature maps are spliced ​​in the channel dimension, and the spliced ​​result is subjected to the convolution operation to reduce the dimension. Finally, the output result is obtained by the sigmoid activation function. After the dual processing of space and channel, the model can more effectively understand the target features.

[0041] Step 3. Use the weighted bidirectional feature pyramid BiFPN structure to replace the FPN-PAN structure for multi-scale feature fusion;

[0042] The step 3 specifically includes:

[0043] The weighted bidirectional feature pyramid BiFPN structure is used to replace the original FPN-PAN structure in the neck network of YOLOv8s. The weighted bidirectional feature pyramid BiFPN structure can learn the weights of different input features in the network, set different weights for different feature layers, reduce the connection of unnecessary layers and add some skip layer connections, so as to integrate more features without increasing the computational cost, and still have good performance when the target scale changes greatly.

[0044] Step 4. Use the loss function Wise-IoU instead of CIoU loss function to optimize model performance;

[0045] The step 4 specifically includes:

[0046] In the detection head network, the Wise-IoU loss function is used to replace the original CIoU loss function. The Wise-IoU loss function formula is as follows:

[0047]

[0048]

[0049]

[0050] Where b irepresents the predicted bounding box coordinates of the i-th target, g i represents the true bounding box coordinates of the i-th target; IoU(b i ,g i ) represents the degree of overlap between the i-th predicted bounding box and the true bounding box; W g and H g The size of the smallest enclosing box; The superscript * in the text indicates W g and H g is separated from the computational graph; x and y are the coordinates of the center point of the predicted bounding box; x gt and gt is the coordinate of the center point of the real box; β and r are the outlier degree and gradient gain respectively; δ is the penalty hyperparameter; L IoU is the running mean of the index with momentum; is the monotonic focusing coefficient.

[0051] Due to the influence of the lighting intensity of the defect shooting, the shooting angle selection and the camera performance, there will be some low-quality target box samples in the acquired data set. When the model generates high-quality anchor boxes for low-quality examples, it will produce a large IoU loss. To solve this problem, the Wise-IoU loss function constructs an attention-based border loss based on the distance metric, and adds a focusing mechanism by constructing a gradient gain calculation method, which weakens the penalty of the geometric metric and makes the initial defect detection model have better generalization performance. The dynamic non-monotonic focusing mechanism uses outliers instead of IoU to evaluate the quality of anchor boxes and provides a wise gradient gain allocation strategy. This strategy reduces the competitiveness of high-quality anchor boxes while reducing the harmful gradients generated by low-quality examples.

[0052] Step 5. Use the GhostConv module and C2f-RepGhost module to improve the network in a lightweight way;

[0053] The step 5 specifically includes:

[0054] In the YOLOv8s network, the Conv module is used to perform convolution operations. The present invention replaces the traditional convolution module Conv with the GhostConv module and improves the residual module C2f into the C2f-RepGhost module. The GhostConv module is performed in two steps, namely conventional convolution and Ghost generation, wherein the conventional convolution operation convolves the input feature map to obtain the output. GhostNet first uses ordinary convolution, batch normalization and an activation function ReLU, and generates some inherent feature maps after compressing the number of channels of the input image, and obtains more feature maps through cheap operations to increase features. The cheap operation is a combination of Depthwise Convolution, batch normalization and an activation function ReLU, and then different feature maps are spliced, and the feature map finally obtained is the output. The cheap operation is a combination of depthwise separable convolution, batch normalization and an activation function ReLU. The C2f-RepGhost module is essentially an improvement on the Bottleneck in the C2f module, where the RepGhost module omits the 1×1 convolution in the first step of the Ghost module, replaces the concatenation operation with the Add operation, further reduces the amount of calculation, moves the ReLU after the Add operation, so that the module meets the structural reparameterization rule, and finally adds the BN operation in the identity mapping branch to bring nonlinearity to the training process, thereby making the network lightweight and reducing the number of model parameters and the amount of calculation.

[0055] Step 6. Train the improved YOLOv8s model and adjust the hyperparameters. After determining the optimal parameters, use the test set to test the model performance. The main evaluation indicators include mean average precision (mAP), precision, recall, computational effort (GFLOPs), parameter quantity (Params), and model size.

[0056] The step 6 specifically includes:

[0057] Before training the improved YOLOv8s model, the hyperparameter selected cosine learning rate decay, iterated training for 200 rounds, input the training set into the improved initial defect detection model for training, and obtained the best weight file, and then evaluated the prediction results of the improved YOLOv8s lightweight network model. Specifically, mAP, Precision, Recall, GFLOPs, Params and model size are mainly used as evaluation criteria for the improved network model. Among them, mAP represents the average recognition accuracy of the ratio of the intersection and union of the predicted box and the true box at a certain threshold. mAP comprehensively analyzes the precision and recall rates to more comprehensively evaluate the model performance. In addition, the number of model parameters, the amount of calculation and the model size all have an impact on the model in real-time detection.

[0058] The evaluation formula of the model is as follows:

[0059]

[0060]

[0061]

[0062]

[0063] In the formula, TP means that the prediction is positive and the real sample is positive; FP means that the prediction is positive and the real sample is negative; FN means that the prediction result is negative and the real sample is positive; Precision means the number of correctly predicted positive samples / the number of all predicted positive samples; Recall means the number of correctly predicted positive samples / the total number of all positive samples; AP means the area enclosed by the PR curve and the coordinate axis; mAP means the average recognition accuracy of all categories.

[0064] Furthermore, in order to verify that each improved part of the present invention contributes to improving the model performance, the original YOLOv8 model is used as the benchmark model, and an ablation experiment is performed to evaluate the performance of the model using evaluation indicators such as mAP@0.5, Precision, and Recall. The ablation experiment results are shown in Table 1.

[0065] Table 1 Comparison table of three optimization methods of YOLOv8s

[0066]

[0067] From the results in Table 1, we can see that the improved YOLOv8s network model has the highest average accuracy, among which the mAP@0.5 value is 86.9%, which is 3.3% higher than that of the original YOLOv8s network model. The parameters, calculation amount and model size of the improved model are increased by 3.7%, 1.4% and 3.6% respectively compared with the original YOLOv8s network model. The improved YOLOv8s model has improved in both precision and recall, but in order to reduce the requirements for deploying the model, the model still needs to be further lightweight.

[0068] Furthermore, the improved model is defined as YOLOv8s-CBW. The YOLOv8s network model is lightweighted and improved by the model lightweight improvement method of the present application. The results of the detection of the drainage pipe data set by the improved YOLOv8s lightweight network model YOLOv8s-Light and the YOLOv8s network model are shown in Table 2:

[0069] Table 2 Comparison of test results before and after model lightweighting

[0070]

[0071] It can be seen from Table 2 that compared with the original YOLOv8s network model, the improved YOLOv8s lightweight network model has a parameter amount reduced by 13.7%, mAP@0.5 increased by 2.6%, the calculation amount reduced by 14.6%, and the model size is 88.4% of the original model. Compared with the YOLOv8s-CBW model, the improved YOLOv8s lightweight network model has a parameter amount reduced by 16.7%, and mAP@0.5 has decreased slightly, but it can still be maintained above 86%. The calculation amount has decreased by 15.8%, the model size is 85.4% of the YOLOv8s-CBW model, and the precision and recall rates have decreased slightly. This shows that the improved YOLOv8s lightweight network model can reduce the calculation amount and model parameter amount while basically maintaining the recognition accuracy, which can meet the requirements of real-time detection. At the same time, it has stronger deployment capabilities and can provide a better foundation for drainage pipe defect detection.

[0072] Finally, it should be noted that the specific implementation scheme described above further illustrates the purpose, technical scheme and beneficial effects of the present invention. The above embodiments are only used to illustrate the technical scheme of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art should understand that any modification or equivalent substitution of the technical scheme of the present invention is included in the scope of protection of the present invention.

Claims

1. A drainage pipe defect detection method based on improved YOLOv8s, characterized in that: The following steps are involved: Step 1. Collect drainage pipe defect data images, perform data enhancement on the collected images, and then divide them into training sets, verification sets, and test sets; Step 2. Build a drainage pipe defect detection model based on improved YOLOv8s. After the drainage pipe defect image is processed by the convolution module and C2f residual module of the backbone network to extract features, it is processed by the CBAM attention module to increase the weight of the area of ​​interest in the image. In step 2, a CBAM attention module is introduced after the residual module C2f of the backbone network. This module performs dual processing on the spatial part and the channel part of the feature map to enhance the feature extraction capability of the model. Step 3. Send the feature map extracted by the backbone network to the neck network, and introduce the BiFPN structure in the neck network to replace the FPN-PAN structure. It uses a bidirectional connection operation to integrate information from different levels of the pyramid, which is conducive to capturing more scale features; In the step 3, a BiFPN structure is introduced into the neck network to replace the FPN-PAN structure. The BiFPN structure uses a skip connection to remove nodes in the FPN-PAN structure that contribute less to feature fusion. When the input and output nodes are at the same level, an additional connection operation is added to improve the feature fusion capability. Step 4. Use the loss function Wise-IoU instead of CIoU loss function to optimize model performance, use the outlier degree as an indicator to evaluate the anchor box, and adopt a reasonable gradient gain allocation strategy to effectively reduce the harmful gradients generated by low-quality examples; In step 4, the loss function Wise-IoU is used to replace the CIoU loss function. Its distance metric constructs an attention-based border loss, and an additional focusing mechanism is added by constructing a calculation method for gradient gain, thereby weakening the penalty of geometric metrics and effectively reducing the impact of low-quality images on the model. Step 5. Use the GhostConv convolution module and C2f-RepGhost module to improve the model in a lightweight way, perform low-cost convolution operation on the input feature map to obtain feature information, and then concatenate the generated features with the convolution results of the standard channel to obtain the output features, effectively reducing the number of model parameters; In step 5, the Conv module of the model is replaced by the GhostConv module, and the C2f module is improved to the C2f-RepGhost module. The GhostConv module uses the cheap operation and the standard convolution result as the final output result; in the C2f-RepGhost module, the RepGhost module replaces the Concat operation in the Ghost module with the Add operation to further reduce the amount of calculation, moves the ReLU to after the Add operation, and finally adds the BN operation in the identity mapping branch, so that the training process brings nonlinearity; Step 6. Train and adjust the hyperparameters of the improved YOLOv8s model. After determining the optimal parameters, use the test set to evaluate the detection performance of the model.

Images