A sewer pipe defect identification method based on improved YOLOv10

Abstract

The application discloses a kind of drainage pipeline defect identification method based on improved YOLOv10, belong to defect detection field and image recognition field.The method includes: collecting drainage pipeline defect image, carries out pre-processing operation to the data set collected, obtains the data set as training model;YOLOv10 model's main network and head network are improved, and the improved YOLOv10 model is obtained;Using the data set obtained as training model, the improved YOLOv10 model is trained and verified, obtains the optimal YOLOv10 model of drainage pipeline defect, completes drainage pipeline defect identification.The application provides more smooth category transition basis, reduces information loss and improves the accuracy of feature extraction, optimizes the processing capacity of model to small object and low resolution image.For the drainage pipeline defect image collected, there are still problems such as uneven image illumination and low resolution of picture after frame processing, improve the detection efficiency and detection precision of model.

Description

Technical Field

[0001] This invention belongs to the fields of defect detection and image recognition, and specifically relates to a method for identifying defects in drainage pipes based on an improved YOLOv10. Background Technology

[0002] In recent years, with the rapid development of computer vision and image processing technologies, automated defect identification technology has made significant progress in the application of drainage pipeline systems, demonstrating its potential in the field of drainage pipeline defect detection. Currently, computer vision and image processing technologies in the field of drainage pipeline defect detection include deep learning models such as Convolutional Neural Networks (CNN), U-Net, YOLO series algorithms, Transformer-based network architectures, and the YOLO (You Only Look Once) series of algorithms.

[0003] Although deep learning technology has made some progress in the identification of defects in drainage pipes, several challenges remain in practical applications. First, defects in drainage pipes exhibit diverse forms, including cracks, deformations, blockages, and corrosion, and these defects can be highly similar. This is especially true when image quality is poor, lighting is uneven, or viewing angles change significantly, making it difficult for deep learning models to accurately identify them. Second, the complex internal environment of drainage pipes, with sewage, silt, and water flow often creating cluttered backgrounds, reduces the distinction between the background and defects, making the model susceptible to background interference during training. Third, while existing YOLO algorithms offer high speed and accuracy, their performance remains unsatisfactory in noisy environments or when defects are highly similar to normal pipe structures, particularly in the detection and identification of small defects, where false negatives or missed detections are common. Therefore, for the identification of defects in drainage pipes, complex lighting conditions (low light / uneven lighting) lead to low contrast and blurred texture in defect areas, making it easy for traditional models to miss or misdetect defects. Due to the fixed receptive field of conventional convolution, small-scale defect features are easily lost in multi-scale defect morphology (cracks / corrosion / sediment, etc.). The low imaging resolution of monitoring equipment results in the loss of defect edges and texture details, which limits the classification accuracy. How to achieve robust perception and efficient and accurate identification of multi-scale defects in complex scenarios has become a core problem that urgently needs to be solved. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for identifying defects in drainage pipes based on an improved YOLOv10.

[0005] To achieve the above technology, the specific steps include:

[0006] S1. Collect images of defects in drainage pipes, perform preprocessing operations on the collected dataset, and obtain the dataset to be used as a training model.

[0007] The steps involved in performing preprocessing operations include:

[0008] S1.1 The obtained CCTV video of the internal inspection of the drainage pipe is processed into frames, each frame is made into an image, and the global image of the defect is selected as the sample data; whereby the global image is defined as the image taken by the camera directly in front of the inside of the pipe.

[0009] In this embodiment, the selected method is: manual experience screening;

[0010] S1.2. Classify the acquired sample data into defect types; the defect type classification includes: surface defects, structural defects, and functional defects.

[0011] Surface defects include: surface cracks and surface damage;

[0012] Structural defects include: inter-pipe displacement, inter-pipe fracture, inter-pipe deformation, and inter-pipe leakage;

[0013] The functional defects are: blockage deposits inside the pipe, tree roots, and obstructions.

[0014] S1.3. Use image enhancement algorithms to perform enhancement operations on the sample data after the defect type classification to obtain the image-enhanced dataset;

[0015] Methods to enhance operation include:

[0016] S1.3.1. Contrast-limited adaptive histogram equalization (CLAHE) and homomorphic filtering (HF) are used to process images with uneven illumination and low contrast. The results of the two methods are then weighted and fused according to a preset ratio, as shown in the following expression:

[0017]

[0018] In the formula, and The images are the results of homomorphic filtering (HF) and contrast-limited adaptive histogram equalization (CLAHE), respectively. and These are the fusion weighting coefficients for homomorphic filtering (HF) and contrast-limited adaptive histogram equalization (CLAHE), respectively. The resulting image after fusion;

[0019] S1.3.2. Perform non-subsampled contourlet transform on the fused image to remove noise. The expression is as follows:

[0020]

[0021] In the formula, This represents the input image, i.e., the fused result image; and These represent a low-pass filter and a high-pass filter, respectively. Indicates the number of directions. An index representing the number of directions; Indicates in each direction Apply a high-pass filter The result;

[0022] S1.3.3. Using the standard deviation of the local region as the noise standard deviation, an improved Bayes-Shrink threshold is used to remove noise from the high-frequency coefficients. The improved Bayes-Shrink threshold formula is as follows:

[0023]

[0024] In the formula, The standard deviation of noise; The number of coefficients;

[0025] S1.3.4. A nonlinear mapping function is used to enhance details, and an inverse NSCT transform is performed to obtain the enhanced image dataset. The detail enhancement formula is as follows:

[0026]

[0027] In the formula, The noise threshold is obtained at the l-th decomposition scale and the d-th decomposition direction. This represents the maximum absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient. The absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient; The direction coefficient for the d-th decomposition scale is given by the l-th decomposition scale. For enhancement coefficient; For coefficient factors; Enhance the results for details;

[0028] S1.4 Divide the enhanced dataset into training set, validation set and test set according to a preset ratio;

[0029] S1.5 Use the annotation tool to mark defects in the partitioned and enhanced dataset, and convert it into a YOLO format file.

[0030] S2. Improve the backbone network and head network of the YOLOv10 model to obtain the improved YOLOv10 model.

[0031] The improvement involves adding a multi-scale dilated attention mechanism (MSDA) to the head layer of the network. MSDA obtains the query, key, and value of the feature map through linear projection. Then, the channels of the feature map are divided into a predetermined number of heads, and sliding window dilated attention (SWDA) is performed in each head with a different dilation rate. The MSDA formula is as follows:

[0032]

[0033]

[0034] In the formula, Indicates the first Output features of each head; Indicates the first The query vector of the head; Indicates the first The key vector of each head; No. A vector of values ​​for each head; Indicates the first i The rate of expansion of the size;

[0035] The backbone and head networks in the YOLOv10 model are improved by the following steps:

[0036] S2.1. A multi-scale dilated attention mechanism (MSDA) is introduced between the standard convolutional layers (Conv layers) and the efficient object detection network (ELAN) in the YOLOv10 model backbone. The steps are as follows:

[0037] S2.1.1 Perform a linear transformation on the input feature map to create a query vector, a key vector, and a value vector; In this invention, the input feature map is the drainage pipe defect image dataset obtained in S1;

[0038] S2.1.2. Divide the channels of the input feature map into a preset number of heads (groups), and use a different dilation rate for each head to perform self-attention operation; generate different receptive fields through different dilation rates to form different scale feature maps;

[0039] S2.1.3. Perform a dot product operation on the query vector and the key vector to obtain the attention score, where the attention score formula is:

[0040]

[0041] In the formula, Indicates attention score; Represents the query vector; Represents a key vector; superscript Indicates transpose;

[0042] S2.1.4. Normalize the attention scores using the Softmax function to obtain the attention weights. Multiply the value vector of each scale feature map by the corresponding attention weight to obtain the weighted value vector. The formula for the weighted value vector is as follows:

[0043]

[0044] In the formula, Represents a value vector;

[0045] S2.1.5. Concatenate all the scale-weighted value vectors to obtain the final multi-scale attention feature representation, which serves as the output of the attention layer;

[0046] S2.2 Replace the standard convolutional layers in the head and backbone network structures with spatial-to-depth convolutional (SPD-Conv) to obtain richer feature information;

[0047] The improved network structure is as follows:

[0048] The image is input into the backbone network and passes through the following steps:

[0049] A standard convolutional layer is used to form the P1 feature layer;

[0050] The SPD-Conv layer processes the P1 feature and outputs 128 channels to form the P2 feature layer.

[0051] The C2f module (a composite module containing multiple convolutions and residual connections) processes P2 features;

[0052] The SPD-Conv layer processes the output of C2f, with 256 output channels, forming the P3 feature layer;

[0053] The C2f module processes the P3 feature;

[0054] The SPD-Conv layer processes the output of C2f, with 512 output channels, forming the P4 feature layer;

[0055] The C2f module processes the P4 feature;

[0056] The SPD-Conv layer processes the output of C2f, with 1024 output channels, forming the P5 feature layer;

[0057] The C2fCIB module processes P5 features;

[0058] The SPPF module (fixed-size spatial pyramid pooling) processes the output of C2fCIB to enhance feature representation;

[0059] The PSA (Position-Sensitive Attention) module processes the output of SPPF to enhance sensitivity to information at different locations.

[0060] The output of the backbone network is stored in the head network:

[0061] The upsampling layer upsamples the P5 features to match the P4 features;

[0062] The concatenation layer (Concat) concatenates the upsampled P5 features with the P4 features;

[0063] The C2f module processes the spliced ​​features;

[0064] The upsampling layer upsamples the processed features to match the P3 features;

[0065] The Concat layer concatenates the upsampled features with the P3 features.

[0066] The C2f module processes the spliced ​​features;

[0067] The SPD-Conv layer processes the output of C2f;

[0068] The Concat layer concatenates the output of SPD-Conv with the previously processed P4 features;

[0069] The C2f module processes the spliced ​​features;

[0070] The SPD-Conv layer processes the output of C2f to match the size of the P5 feature;

[0071] The Concat layer concatenates the output of SPD-Conv with the previously processed P5 features.

[0072] S3. Using the obtained dataset as the training model, train and validate the improved YOLOv10 model to obtain the optimal YOLOv10 model for drainage pipe defects. The steps are as follows:

[0073] S3.1 Use the pre-trained weights YOLOv10n on the open-source dataset as the initial weights of the optimized YOLOv10 model;

[0074] In this embodiment, the open-source dataset is the COCO dataset; the training set used to train the YOLOv10 model is a dataset of images of defects in drainage pipes after image enhancement. The training set is used to train the improved YOLOv10 network model. Based on the idea of ​​transfer learning, the weights yolov10n pre-trained on the open-source COCO dataset are used as the initial weights of the optimized YOLOv10 model to avoid large fluctuations in the loss value that may occur when the network is trained from scratch, thereby improving the convergence performance of the network.

[0075] S3.2. Divide the images into batches (set batch_size=8), adjust the size of each image to 640x640, 3 channels (RGB), and obtain a floating-point tensor of (batch_size, 3, 640, 640);

[0076] S3.3, Backbone is responsible for extracting multi-level features, and it downsamples step by step through convolution and cross-stage partial networks (CSP module), as follows:

[0077] S3.3.1, Spatial-to-Depth Convolution (SPD-Conv) and C2f modules constitute shallow feature extraction, capturing low-level edge and texture features of the input drainage pipe image and outputting shallow results;

[0078] S3.3.2, Spatial-to-Depth Convolution SPD-Conv and C2f modules constitute intermediate feature extraction, extracting shallow intermediate features, including parts and object contours, and outputting intermediate results;

[0079] S3.3.3, repeated downsampling and C2f modules constitute deep feature extraction, further increasing the number of channels, capturing high-level semantic features, and outputting high-level results;

[0080] S3.4 The neck network enhances detection capabilities through multi-scale feature fusion, combining top-down and bottom-up approaches, including the following steps:

[0081] S3.4.1 Top-down upsampling: Deep features are upsampled and concatenated with shallow features through an upsampling layer, and then fused through convolution to generate a fused multi-scale feature map;

[0082] S3.4.2 Bottom-up secondary fusion: The fused features are downsampled again and concatenated with the features of the lower layer to enhance cross-scale information and obtain an optimized multi-level feature map;

[0083] S3.5 The head network achieves high-precision prediction through multi-level feature fusion and spatial information enhancement;

[0084] S3.5.3, Multi-scale prediction output;

[0085] S3.6. Use the training set obtained as the training model dataset to train the optimized YOLOv10 network model.

[0086] S3.7. Supervised learning is performed on the labeled defect information using the obtained validation set in the training model dataset. The validation effect feedback is provided to the training set during network training, and parameter adjustments and overfitting judgments are made in a timely manner to update the YOLOv10 model. The initial weights are automatically adjusted during network training based on the YOLOv10 training dataset to generate the weights after training.

[0087] S3.8. After the network training is completed, the weights are loaded into the improved YOLOv10 model. The prediction module is used to compare and analyze the prediction effect of the test set obtained as the training model dataset. After determining the optimal model, the optimal YOLOv10 identification model for drainage pipe defects in this embodiment is obtained.

[0088] S3.9. Use the obtained optimal YOLOv10 identification model for drainage pipe defects to identify the defect images of drainage pipes, and mark the location and type of defects in the drainage pipes in the form of rectangular prediction boxes.

[0089] Beneficial effects of the present invention

[0090] In processing images of defects in drainage pipes, this invention introduces a method of limiting contrast adaptive histogram equalization and homomorphic filtering followed by data fusion, providing a smoother basis for subsequent feature extraction and classification of different defect features.

[0091] This invention replaces the convolutions in the backbone of the YOLOv10 model with spatial-to-depth convolutions (SPD-Conv), which optimizes the model's ability to process small objects and low-resolution images by reducing information loss and improving the accuracy of feature extraction.

[0092] This invention addresses the problems of uneven illumination and low resolution of images after frame-segmentation in the collected images of drainage pipe defects. It introduces a multi-scale dilated attention mechanism into the YOLOv10 model, which enhances the receptive field of the feature map without increasing the computational load, improves the model's ability to capture local details, and improves the model's detection efficiency and accuracy. Attached Figure Description

[0093] Figure 1 This is a flowchart of the steps of the present invention;

[0094] Figure 2 This invention provides a partial topological diagram of a drainage pipe structure.

[0095] Figure 3The images provided by this invention are sample pictures of drainage pipe defects, wherein part a is a first schematic diagram of pipe functional defects; part b is a schematic diagram of pipe surface defects; part c is a second schematic diagram of pipe functional defects; and part d is a schematic diagram of pipe structural defects.

[0096] Figure 4 A flowchart for image enhancement of drainage pipes provided by the present invention;

[0097] Figure 5 The images provided by this invention are enhanced images of drainage pipe defects, wherein part a is a first schematic diagram of enhanced pipe functional defects; part b is a schematic diagram of enhanced pipe surface defects; part c is a second schematic diagram of enhanced pipe functional defects; and part d is a schematic diagram of enhanced pipe structural defects.

[0098] Figure 6 An improved YOLOv10 network structure diagram provided for this invention;

[0099] Figure 7 This invention provides the working principle of the Multi-Scale Hollow Attention (MSDA) introduced for YOLOv10;

[0100] Figure 8 The image provided by this invention shows the results of defect recognition of drainage pipes. Part a is a schematic diagram of pipe surface cracks and inter-pipe fractures; part b is a schematic diagram of pipe surface cracks, inter-pipe displacement, and the presence of roots; part c is a schematic diagram of pipe inter-pipe seepage and surface damage; and part d is a schematic diagram of pipe displacement and sediment deposition inside the pipe. Detailed Implementation

[0101] The present invention will be further described in detail below with reference to specific embodiments.

[0102] like Figure 1 As shown, a method for identifying defects in drainage pipes based on an improved YOLOv10 includes the following steps:

[0103] S1. Collect images of defects in drainage pipes, perform preprocessing operations on the collected dataset, and obtain the dataset to be used as a training model.

[0104] In this embodiment, the acquisition of drainage pipe defect images involves: acquiring images of pipe defects in actual drainage pipes. The acquisition location is a university in Chenggong District, Kunming City, Yunnan Province. A CCTV pipe inspection robot KT-965 is used to enter the drainage pipes and capture images of pipe defects. The inspection pipe diameter range is 300-3000mm, and the inspection length range is 0-120m. Based on the high-incidence locations of pipe defects, key acquisition points are selected. Joints are easily affected by soil movement or groundwater pressure, resulting in displacement or leakage. Elbows and tees, due to changes in water flow direction, are subjected to greater fluid impact forces, which not only easily cause wear but also easily accumulate impurities and deposits, leading to blockages. In areas with unstable foundations or frequent heavy vehicle traffic, ground subsidence may occur, causing pipe displacement or breakage. Pipes that have been in use for a long time may become brittle due to material aging, making them prone to breakage and corrosion, cracks, and other problems. If there are trees above the pipes, their roots may penetrate the pipe cracks and gradually widen these cracks, eventually leading to pipe rupture. Based on changes in ground humidity, spring and autumn may be the peak seasons for defects. Defect images will be collected on March 15, March 16, July 10, and July 11, 2024, with the images collected each day from 12:00 PM to 5:00 PM.

[0105] The data acquisition project specifically includes:

[0106] Based on the locations where pipe defects are likely to occur, pipe defects in the drainage system of the old dormitory area were selected as the data collection targets.

[0107] The drainage mains in the old dormitory area have numerous bends and tees, and the ground above the pipes is covered with vegetation and trees. This area also has a higher probability of material aging, making pipe defects easier to detect. A topological diagram of the drainage system is attached. Figure 2 As shown, the burial depth of the main pipeline is 2.5~5.0, and the pipe diameter is DN60~200. Among them, the pipe diameter of DN60~100 is made of PVC (polyvinyl chloride) pipe, and the pipe diameter of DN100~200 is made of HDPE double wall corrugated pipe.

[0108] In this case, the drainage main pipe was in operation for a long time, with a high water level and a lot of silt accumulation in most sections. Therefore, it was necessary to block water and clean the silt from the pipe before the inspection.

[0109] The pipeline was sealed by measures such as airbags and wall construction, then the water was pumped out and dewatered. The pipeline was then cleaned manually or by high-pressure water guns. Solid debris was removed from the pipeline manually and bagged, while sludge was sucked out by a sewage truck and transported to a designated treatment point. After the pipeline was cleared, it was flushed with high-pressure water guns to ensure that the CCTV inspection robot could pass through smoothly and fully expose the pipeline defects.

[0110] After the dredging and cleaning are completed, place the inspection robot into the drain pipe. After the robot's meter counter is zeroed, allow it to crawl along the pipe. The camera's movement trajectory should remain on the central axis of the pipe, with a deviation not exceeding 10% of the pipe diameter. Adjust the camera position promptly and appropriately to obtain the best image. When the robot is in a pipe with a diameter greater than 200mm, its travel speed should not exceed 0.15m / s; the defect location should be the distance displayed on the instrument plus the robot's own length (approximately 80cm).

[0111] The steps involved in performing preprocessing operations include:

[0112] S1.1 The obtained CCTV video of the internal inspection of the drainage pipe is processed into frames, each frame is made into an image, and the global image of the defect is selected as the sample data; whereby the global image is defined as the image taken by the camera directly in front of the inside of the pipe.

[0113] In this embodiment, the selected method is: manual experience screening;

[0114] S1.2. Classify the acquired sample data into defect types; the defect type classification includes: surface defects, structural defects, and functional defects, such as... Figure 3 As shown in parts a, b, c, and d;

[0115] Surface defects include: surface cracks and surface damage;

[0116] Structural defects include: inter-pipe displacement, inter-pipe fracture, inter-pipe deformation, and inter-pipe leakage;

[0117] The functional defects are: blockage deposits inside the pipe, tree roots, and obstructions.

[0118] S1.3, such as Figure 4 As shown, an image enhancement algorithm is used to perform enhancement operations on the sample data after the defect type is classified, resulting in an image-enhanced dataset;

[0119] Methods to enhance operation include:

[0120] S1.3.1. Contrast-limited adaptive histogram equalization (CLAHE) and homomorphic filtering (HF) are used to process images with uneven illumination and low contrast. The results of the two methods are then weighted and fused according to a preset ratio, as shown in the following expression:

[0121]

[0122] In the formula, and The images are the results of homomorphic filtering (HF) and contrast-limited adaptive histogram equalization (CLAHE), respectively. and These are the fusion weighting coefficients for homomorphic filtering (HF) and contrast-limited adaptive histogram equalization (CLAHE), respectively. The resulting image after fusion;

[0123] In this embodiment, and The preset ratios are set to 0.4 and 0.6 to balance overall contrast and local detail;

[0124] S1.3.2. Perform non-subsampled contourlet transform on the fused image to remove noise. The expression is as follows:

[0125]

[0126] In the formula, This represents the input image, i.e., the fused result image; and These represent a low-pass filter and a high-pass filter, respectively. Indicates the number of directions. An index representing the number of directions; Indicates in each direction Apply a high-pass filter The result;

[0127] S1.3.3. Using the standard deviation of the local region as the noise standard deviation, an improved Bayes-Shrink threshold is used to remove noise from the high-frequency coefficients. The improved Bayes-Shrink threshold formula is as follows:

[0128]

[0129] In the formula, The standard deviation of noise; The number of coefficients;

[0130] S1.3.4. A nonlinear mapping function is used to enhance details, and an inverse NSCT transform is performed to obtain the enhanced image dataset. The detail enhancement formula is as follows:

[0131]

[0132] In the formula, The noise threshold is obtained at the l-th decomposition scale and the d-th decomposition direction. This represents the maximum absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient. The absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient; The direction coefficient for the d-th decomposition scale is given by the l-th decomposition scale. For enhancement coefficient ( ); For coefficient factors ( ); Enhance the results for details;

[0133] S1.4 Divide the enhanced dataset into training set, validation set and test set according to a preset ratio;

[0134] In this embodiment, the preset ratio is 7:2:1;

[0135] S1.5. Use annotation tools to mark defects in the partitioned enhanced dataset and convert it into a YOLO format file;

[0136] The method for labeling defects in the segmented, enhanced dataset and converting it to YOLO format is as follows: Based on the defect categories and their locations on the pipes in the enhanced drainage pipe images, the LabelMe annotation tool is used to mark the defect regions on the images. Using the image segmentation data format of the COCO dataset as a standard, a corresponding format (.json) is generated for each image. A Python script is written to convert each image's corresponding format (.json) into a YOLO format file (.txt). Based on the YOLO-annotated training set, validation set, and unannotated test set, a dataset for training the model is generated, such as... Figure 5 Parts a, b, c, and d are shown in the diagram.

[0137] S2, such as Figure 6 As shown, the backbone network and head network of the YOLOv10 model are improved to obtain the improved YOLOv10 model.

[0138] The improvement method is as follows: Figure 7 As shown, a multi-scale dilated attention mechanism (MSDA) is added to the head network (head layer), which obtains the query, key, and value of the feature map through linear projection. Then, the channels of the feature map are divided into a predetermined number of heads, and a sliding window dilated attention (SWDA) operation is performed in different heads with different dilation rates. The MSDA formula is as follows:

[0139]

[0140]

[0141] In the formula, Indicates the first Output features of each head; Indicates the first The query vector of the head; Indicates the first The key vector of each head; No. A vector of values ​​for each head; Indicates the first i The rate of expansion of the size; in this embodiment, Take 3;

[0142] The backbone and head networks in the YOLOv10 model are improved by the following steps:

[0143] S2.1. A multi-scale dilated attention mechanism (MSDA) is introduced between the standard convolutional layers (Conv layers) and the efficient object detection network (ELAN) in the YOLOv10 model backbone. The steps are as follows:

[0144] S2.1.1 Perform a linear transformation on the input feature map to create a query vector, a key vector, and a value vector; In this invention, the input feature map is the drainage pipe defect image dataset obtained in S1;

[0145] S2.1.2. Divide the channels of the input feature map into a preset number of heads (groups), and use a different dilation rate for each head to perform self-attention operation; generate different receptive fields through different dilation rates to form different scale feature maps;

[0146] In this embodiment, the preset number of heads is set to 3, and the expansion rate is [1, 2, 3];

[0147] S2.1.3. Perform a dot product operation on the query vector and the key vector to obtain the attention score, where the attention score formula is:

[0148]

[0149] In the formula, Indicates attention score; Represents the query vector; Represents a key vector; superscript Indicates transpose;

[0150] S2.1.4. Normalize the attention scores using the Softmax function to obtain the attention weights. Multiply the value vector of each scale feature map by the corresponding attention weight to obtain the weighted value vector. The formula for the weighted value vector is as follows:

[0151]

[0152] In the formula, Represents a value vector;

[0153] S2.1.5. Concatenate all the scale-weighted value vectors to obtain the final multi-scale attention feature representation, which serves as the output of the attention layer;

[0154] The formula for concatenating value vectors is:

[0155]

[0156] In the formula, This represents the final multi-scale attention features; This represents the weighted vector in the first head; This represents the weighted vector in the second head; This represents the weighted vector in the third head;

[0157] S2.2 Replace the standard convolutional layers in the head and backbone network structures with spatial-to-depth convolutional (SPD-Conv) to obtain richer feature information;

[0158] The improved network structure is as follows:

[0159] The image is input into the backbone network, and then passes through:

[0160] A standard convolutional layer with 64 output channels, a kernel size of 3, and a stride of 2 forms the P1 feature layer.

[0161] The SPD-Conv layer processes the P1 feature and outputs 128 channels to form the P2 feature layer.

[0162] The C2f module (a composite module containing multiple convolutions and residual connections) processes P2 features;

[0163] The SPD-Conv layer processes the output of C2f, with 256 output channels, forming the P3 feature layer;

[0164] The C2f module processes the P3 feature;

[0165] The SPD-Conv layer processes the output of C2f, with 512 output channels, forming the P4 feature layer;

[0166] The C2f module processes the P4 feature;

[0167] The SPD-Conv layer processes the output of C2f, with 1024 output channels, forming the P5 feature layer;

[0168] The C2fCIB module processes P5 features;

[0169] The SPPF module (fixed-size spatial pyramid pooling) processes the output of C2fCIB to enhance feature representation;

[0170] The PSA (Position-Sensitive Attention) module processes the output of SPPF to enhance sensitivity to information at different locations.

[0171] The output of the backbone network is stored in the head network:

[0172] The upsampling layer upsamples the P5 features to match the P4 features;

[0173] The Concat layer concatenates the upsampled P5 features with the P4 features;

[0174] The C2f module processes the spliced ​​features;

[0175] The upsampling layer upsamples the processed features to match the P3 features;

[0176] The Concat layer concatenates the upsampled features with the P3 features.

[0177] The C2f module processes the spliced ​​features;

[0178] The SPD-Conv layer processes the output of C2f;

[0179] The Concat layer concatenates the output of SPD-Conv with the previously processed P4 features;

[0180] The C2f module processes the spliced ​​features;

[0181] The SPD-Conv layer processes the output of C2f to match the size of the P5 feature;

[0182] The Concat layer concatenates the output of SPD-Conv with the previously processed P5 features.

[0183] S3. Using the obtained dataset as the training model, train and validate the improved YOLOv10 model to obtain the optimal YOLOv10 model for drainage pipe defects. The steps are as follows:

[0184] S3.1 Use the pre-trained weights YOLOv10n on the open-source dataset as the initial weights of the optimized YOLOv10 model;

[0185] In this embodiment, the open-source dataset is the COCO dataset; the training set used to train the YOLOv10 model is a dataset of images of defects in drainage pipes after image enhancement. The training set is used to train the improved YOLOv10 network model. Based on the idea of ​​transfer learning, the weights yolov10n pre-trained on the open-source COCO dataset are used as the initial weights of the optimized YOLOv10 model to avoid large fluctuations in the loss value that may occur when the network is trained from scratch, thereby improving the convergence performance of the network.

[0186] S3.2. Divide the images into batches (set batch_size=8), adjust the size of each image to 640x640, 3 channels (RGB), and obtain a floating-point tensor of (batch_size, 3, 640, 640);

[0187] S3.3 The backbone network is responsible for extracting multi-level features, and it downsamples step by step through convolution and cross-stage partial networks (CSP module), as follows:

[0188] S3.3.1, Spatial-to-Depth Convolution SPD-Conv(3x3, stride=2) and C2f module (including residual blocks and cross-stage connections) constitute shallow feature extraction, capturing low-level edge and texture features of the input drainage pipe image, and output (batch_size, 64, 320, 320);

[0189] S3.3.2, Spatial-to-depth convolution SPD-Conv(3x3, stride=2) and C2f module (deeper residual structure) constitute mid-level feature extraction, extracting mid-level features, including parts and object contours, and output (batch_size, 128, 160, 160);

[0190] S3.3.3, repeated downsampling and C2f modules constitute deep feature extraction, further increasing the number of channels, capturing high-level semantic features (such as overall defects), and outputting (batch_size, 1024, 20, 20);

[0191] S3.4 The neck network enhances detection capabilities through multi-scale feature fusion, combining top-down and bottom-up approaches, including the following steps:

[0192] S3.4.1 Top-down upsampling: The deep features (1024x20x20) are upsampled and concatenated with the shallow features (512x40x40) through the upsampling layer, and then fused by convolution to generate a fused multi-scale feature map (e.g., 512x40x40).

[0193] S3.4.2 Bottom-up secondary fusion: The fused features are downsampled again and further concatenated with the features at a lower level to enhance cross-scale information and obtain optimized multi-level feature maps (P3: 256x80x80, P4: 512x40x40, P5: 1024x20x20).

[0194] S3.5 The head network achieves high-precision prediction through multi-level feature fusion and spatial information enhancement. The specific process is as follows:

[0195] S3.5.1 Top-down feature fusion: Upsample P5 features (1024×20×20 → 1024×40×40), concatenate them with P4 (512×40×40), and compress them to 512×40×40 through the C2f module, fusing deep semantics and mid-level details; the second upsampled result (512×40×40 → 512×80×80), concatenate it with P3 (256×80×80), and compress it to 256×80×80 through the C2f module, enhancing the expression of small target features;

[0196] S3.5.2 Bottom-up Spatial Enhancement: SPD-Conv spatial compression (80×80 → 40×40) is performed on the P3 level feature (256×80×80), expanding the channels to 1024×40×40. After concatenation with the P4 feature, it is compressed to 512×40×40 by C2f, preserving detailed information. SPD-Conv downsampling (40×40 → 20×20) is performed on the P4 level feature (512×40×40), expanding the channels to 1024×20×20. After concatenation with the original P5, it is fused to 1024×20×20 by the C2fCIB module, enhancing the global context.

[0197] S3.5.3 Multi-scale prediction output: P3 (256×80×80) outputs high-resolution detection results after 1x1 convolution, focusing on small targets (such as tiny defects); P4 (512×40×40) outputs medium-resolution detection results, covering medium-sized targets (such as tree branches, obstacles, etc.); P5 (1024×20×20) outputs low-resolution detection results, capturing large targets (such as blockage deposits).

[0198] S3.6. Use the training set obtained as the training model dataset to train the optimized YOLOv10 network model.

[0199] S3.7. Supervised learning is performed on the labeled defect information using the obtained validation set in the training model dataset. The validation effect feedback is provided to the training set during network training, and parameter adjustments and overfitting judgments are made in a timely manner to update the YOLOv10 model. The initial weights are automatically adjusted during network training based on the YOLOv10 training dataset to generate the weights after training.

[0200] S3.8. After the network training is completed, the weights are loaded into the improved YOLOv10 model. The prediction module is used to compare and analyze the prediction effect of the test set obtained as the training model dataset. After determining the optimal model, the optimal YOLOv10 identification model for drainage pipe defects in this embodiment is obtained.

[0201] In this embodiment, comparative analysis is used to evaluate the recognition effect. The evaluation content includes comparative analysis of the recognition accuracy of drainage pipe defects, recognition time and model real-time performance.

[0202] In this embodiment, the experimental environment and hardware platform used for network model training and validation specifically include: the operating system is WIN11, based on the PyTorch deep learning framework, the CPU is a 14-core, 20-thread 12th Gen Intel(R) Core(TM) i9-12900H, and the GPU is a GTX3060; the training epochs are 90 and 300 respectively for network performance comparison and analysis.

[0203] S3.9. Using the obtained optimal YOLOv10 defect recognition model for drainage pipes, defect images of drainage pipes are identified, and the location and type of defects in the drainage pipes are marked in the form of rectangular prediction boxes. The results are as follows: Figure 8 As shown in parts a, b, c, and d, the detection results of drainage pipe defects are all above 0.8, except for surface damage in part c, which shows a value of 0.79. This indicates that the detection results of the model of the present invention are accurate.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for identifying defects in drainage pipes based on an improved YOLOv10, characterized in that, Includes the following steps: S1. Collect images of defects in drainage pipes, perform preprocessing operations on the collected dataset, and obtain the dataset to be used as a training model. The steps include: S1.1 The obtained CCTV video of the internal inspection of the drainage pipe is processed into frames, each frame is made into an image, and the global image of the defect is selected as the sample data; whereby the global image is defined as the image taken by the camera directly in front of the inside of the pipe. S1.

2. Classify the obtained sample data into defect types; S1.

3. Use image enhancement algorithms to perform enhancement operations on the sample data after the defect type classification to obtain the image-enhanced dataset; S1.4 Divide the enhanced dataset into training set, validation set and test set according to a preset ratio; S1.

5. Use annotation tools to mark defects in the partitioned enhanced dataset and convert it into a YOLO format file; The steps of performing enhancement operations on the sample data after defect type classification using image enhancement algorithms to obtain the image-enhanced dataset include: S1.3.

1. Contrast-limited adaptive histogram equalization and homomorphic filtering are used to process images with uneven illumination and low contrast. The results of the two methods are then weighted and fused according to a preset ratio, as shown in the following expression: ； In the formula, and The images are the results of homomorphic filtering and contrast-limited adaptive histogram equalization, respectively. and These are the fusion weighting coefficients for homomorphic filtering and contrast-limited adaptive histogram equalization, respectively. The resulting image after fusion; S1.3.

2. Perform non-subsampled contourlet transform on the fused image to remove noise. The expression is as follows: ； In the formula, This represents the input image, i.e., the fused result image; and These represent a low-pass filter and a high-pass filter, respectively. Indicates the number of directions. An index representing the number of directions; Indicates in each direction Apply a high-pass filter The result; S1.3.

3. Using the standard deviation of the local region as the noise standard deviation, an improved Bayes-Shrink threshold is used to remove noise from the high-frequency coefficients. The improved Bayes-Shrink threshold formula is as follows: ； In the formula, The standard deviation of noise; The number of coefficients; S1.3.

4. A nonlinear mapping function is used to enhance details, and an inverse NSCT transform is performed to obtain the enhanced image dataset. The detail enhancement formula is as follows: ； In the formula, The noise threshold is obtained at the l-th decomposition scale and the d-th decomposition direction. This represents the maximum absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient. The absolute value of the decomposition direction coefficient at the l-th decomposition scale and the d-th decomposition direction coefficient; The direction coefficient for the d-th decomposition scale is given by the l-th decomposition scale. For enhancement coefficient; For coefficient factors; Enhance the results for details; S2. Improve the backbone network and head network of the YOLOv10 model to obtain the improved YOLOv10 model. The improvement is as follows: a multi-scale dilated attention mechanism is added to the head network to obtain the query, key, and value of the feature map through linear projection; the channels of the feature map are divided into a preset number of heads, and a sliding window dilation attention operation is performed in different heads with different dilation rates; spatial-to-depth convolution is used to replace the standard convolutional layers in the head network and the backbone network structure to obtain richer feature information. S3. Using the obtained dataset as the training model, train and validate the improved YOLOv10 model to obtain the optimal YOLOv10 model for drainage pipe defects and complete the identification of drainage pipe defects.

2. The drainage pipe defect identification method based on improved YOLOv10 according to claim 1, characterized in that: In the process of classifying the acquired sample data into defect types, the defect types include surface defects, structural defects, and functional defects. Surface defects include: surface cracks and surface damage; Structural defects include: inter-pipe displacement, inter-pipe fracture, inter-pipe deformation, and inter-pipe leakage; The functional defects are: blockage deposits inside the pipe, tree roots, and obstacles.

3. The drainage pipe defect identification method based on improved YOLOv10 according to claim 1, characterized in that: The steps for improving the backbone and head networks of the YOLOv10 model to obtain the improved YOLOv10 model include: S2.

1. Introduce a multi-scale dilated attention mechanism between the standard convolutional layers and the efficient object detection network in the backbone network of the YOLOv10 model; S2.

2. Replace the standard convolutional layers in the head network and backbone network structure with spatial-to-depth convolutions to obtain richer feature information.

4. The drainage pipe defect identification method based on improved YOLOv10 according to claim 3, characterized in that: The steps of introducing a multi-scale dilated attention mechanism between the standard convolutional layers and the efficient object detection network in the YOLOv10 model backbone include: S2.1.1 Perform a linear transformation on the input feature map to create a query vector, key vector, and value vector; the input feature map is the drainage pipe defect image dataset obtained in S1. S2.1.

2. The input feature map channels are segmented into a preset number of heads, and each head is used with a different dilation rate to perform self-attention operation; different receptive fields are generated by different dilation rates, forming different scale feature maps; S2.1.3 Perform a dot product operation on the query vector and the key vector to obtain the attention score; S2.1.

4. Use the Softmax function to normalize the attention score to obtain the attention weights. Multiply the value vector of each scale feature map by the corresponding attention weight to obtain the weighted value vector. S2.1.

5. Concatenate the value vectors after weighting all scales to obtain the final multi-scale attention feature representation.

Images