Drainage pipe network defect detection method based on data optimization and multi-model fusion

Abstract

The invention relates to the technical field of drainage pipe network defect detection and computer vision, in particular to a drainage pipe network defect detection method based on data optimization and multi-model fusion, and the method comprises the steps: collecting a video in a pipeline through a detection robot, and carrying out the frame extraction to obtain an original image; carrying out the quality optimization of the image through employing a lens fouling restoration algorithm, an image defogging algorithm and an illumination enhancement algorithm; positioning a defect area by using the target detection model, and performing pixel-level segmentation through the U-Net semantic segmentation model to obtain a defect contour; visual features and morphological features are extracted, feature fusion is carried out in combination with an attention mechanism, and finally a classifier is input to realize accurate identification of defect types. Targeted solutions are provided for multiple common imaging defects in the drainage pipe network, the method can adapt to complex and changeable actual detection environments, good generalization ability and practical value are achieved, and reliable technical support can be provided for intelligent operation and maintenance of the urban drainage pipe network.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of sewer network defect detection and computer vision technology, in particular to a sewer network defect detection method based on data optimization and multi-model fusion. BACKGROUND

[0002] Sewer network is an important infrastructure of city. Periodic, efficient and accurate defect detection and evaluation of sewer network is the top priority of city management and safe operation. However, traditional sewer network detection methods (such as manual visual inspection, video detection based on closed-circuit television, etc.) have strong subjectivity, low efficiency, high labor cost and difficulty in quantifying complex defects, which cannot meet the needs of modern smart city for rapid and objective analysis of massive pipe network data. Deep learning technology provides a new technical path for breaking through the bottleneck of traditional detection due to its excellent performance in image recognition, segmentation and classification.

[0003] The performance of deep learning model is highly dependent on the quality of the data it learns. In the complex scene of sewer network detection, the original collected video images often have problems such as uneven illumination, turbid water, low contrast, diverse defect morphology and lack of key defect samples, which seriously restrict the actual upper limit of model performance. Therefore, systematic optimization of original data is the core and foundation for the implementation of the technical solution of the present application. SUMMARY

[0004] The purpose of the present application is to provide a sewer network defect detection method based on data optimization and multi-model fusion to solve the problems raised in the background technology.

[0005] In order to solve the above technical problems, the present application provides the following technical solution: a sewer network defect detection method based on data optimization and multi-model fusion, comprising: S1, detecting the internal camera of the sewer network by a detection robot, collecting internal video data of the sewer network, extracting video frames at fixed frame intervals, obtaining an original image sequence, and performing size normalization processing on the image; S2, performing image quality optimization processing on the normalized original image sequence by using lens contamination repair, image dehazing and illumination enhancement methods; This invention uses image enhancement and restoration algorithms to process the original image, generating a quality-optimized image. During this process, Pix2Pix is ​​used to synthesize various forms of lens smudge effects, and a U-Net-structured image inpainting model is trained to learn how to recover a clean image from images with simulated smudges. This model is applied to actual video frames to remove water droplets or mud spots from the lens. Then, a guided filtering-based local transmittance optimization algorithm is used to recover a clear, fog-free image, resolving the blurring caused by water vapor and dust inside pipes. Finally, a contrast-limited adaptive histogram equalization (CLAHE) algorithm is employed to locally and adaptively adjust the image's brightness and contrast, addressing uneven lighting issues.

[0006] S3. Input the optimized image into the pre-trained target detection model to locate the defect region, and perform pixel-level segmentation on the located defect region to obtain a pixel-level defect contour binary mask. S4. Visual and morphological features are extracted from the located defect area, and the features are fused through an attention mechanism. The results are then input into a classifier to output the defect type and confidence level.

[0007] Furthermore, the method for lens damage repair in the image quality optimization process of the normalized original image sequence in step S2 includes the following steps: S211. Use the Pix2Pix network to synthesize simulated soiled images and construct a dataset of clean and soiled image pairs. Each element in the clean and soiled image pair dataset corresponds to a set of simulated soiled images of the same pipe in the drainage network and a set of simulated clean images of the corresponding pipe. S212. Using simulated smudged images of each data pair in the clean and smudged image pair dataset as input, and simulated clean images of the corresponding data pairs in the clean and smudged image pair dataset as targets, train the U-Net structure image inpainting model using L1 loss and perceptual loss until it can effectively learn the ability to remove occlusions and restore background texture. S213. Based on the trained image restoration model, obtain the lens damage restoration image corresponding to each image in the normalized original image sequence.

[0008] Furthermore, in the image quality optimization process of the normalized original image sequence in S2, the image dehazing adopts a local transmittance optimization algorithm based on guided filtering. The specific implementation steps are as follows: S221. Model the image after the contamination repair based on the atmospheric scattering model, and estimate the initial transmittance using the dark channel prior. S222. Use guided filtering to perform edge-preserving optimization on the transmittance map to obtain refined transmittance. S223. Combine the estimated global atmospheric light value and recover the clear, fog-free image through inverse operation.

[0009] The local transmittance optimization dehazing method based on guided filtering in this invention utilizes an atmospheric scattering model. Modeling is performed on the blurred image of the pipeline, where, This represents the degraded, blurred image of the pipeline, i.e., the observation value; A clear image of the pipe when there is no scattering; Let A represent the initial transmittance; let A represent the estimated global atmospheric light value. First, the dark channel prior is used. (in, This represents the value of the dark channel, which is approximately equal to 0, and c belongs to {R, G, B}. This means that when c∈{R, G, B}, the corresponding pixel y takes the minimum value in the RGB three channels; (Represents the set of pixels in the image to be processed) Estimate the initial transmittance Where ω is the dehazing intensity factor (ω=0.95~0.98 is recommended for pipeline scenes), used to retain a small amount of scattering (to avoid overestimating transmittance and causing image distortion), balancing the deblurring effect with realism. Subsequently, guided filtering is used to optimize the transmittance map by edge preservation, resulting in a refined transmittance. ;in, This represents the average local slope. G represents the average local intercept; G represents the guide map, i.e., the grayscale image of the degraded blurred pipeline image; finally, it is combined with the estimated global atmospheric light value. Through inverse operation It restores a clear, fog-free image, effectively resolving the blurring issue caused by moisture and dust inside pipes.

[0010] Furthermore, in the image quality optimization process of the normalized original image sequence in S2, the illumination enhancement adopts a contrast-limited adaptive histogram equalization algorithm, and the specific implementation steps are as follows: S231. Convert the dehazed image from the RGB color space to the LAB color space; S232. Apply the CLAHE algorithm to the brightness channel L to obtain the updated brightness channel L value, while the remaining channel values ​​remain unchanged. S233. Convert the updated LAB color space corresponding to the image after dehazing back to the RGB color space to obtain the final image quality optimization result for each image in the normalized original image sequence.

[0011] Furthermore, in step S3, YOLOv8 is used as the target detection model. It is initialized using weights pre-trained on the COCO dataset and fine-tuned on the drainage network defect dataset to obtain a pre-trained target detection model. The drainage network defect dataset includes drainage network image data labeled with one or more defects such as cracks, deposits, and tree root intrusion. During the defect region localization process of the optimized image, the pre-trained target detection model outputs the corresponding defect region bounding boxes and their confidence scores in the optimized image based on the input optimized image; and the set of regions within the defect region bounding boxes with confidence scores greater than a preset value is taken as the defect region localization result for the corresponding optimized image.

[0012] Furthermore, in the process of performing pixel-level segmentation on the located defect region to obtain a pixel-level defect contour binary mask, the lightweight U-Net semantic segmentation model is used. The image region corresponding to each element in the defect region localization result of the optimized image is used as the input of the lightweight U-Net semantic segmentation model, and the corresponding pixel-level defect contour binary mask is output.

[0013] Furthermore, S4 performs visual feature extraction on the located defect region using a multi-scale convolutional neural network, and the multi-scale convolutional neural network employs a three-layer convolutional structure, including: The first convolutional layer uses a 7×7 convolutional kernel to extract global features; The second convolutional layer uses a 5×5 convolutional kernel to extract medium-scale features; The third convolutional layer uses a 3×3 convolutional kernel to extract detailed features; The multi-scale convolutional neural network ultimately outputs a 256-dimensional visual feature vector.

[0014] Furthermore, during the morphological feature extraction of the located defect region in step S4, based on the pixel-level defect contour binary mask generated in step S3, the defect region area, defect contour perimeter, defect bounding box aspect ratio, defect region circularity, and the first 7 Zernike moments are calculated to form an 11-dimensional morphological feature vector.

[0015] In the 11-dimensional morphological feature vector of this invention, the area of ​​the defect region represents the total number of pixels within the defect region, reflecting the size of the defect; the perimeter of the defect outline represents the total number of pixels in the defect outline, used to describe boundary complexity; the aspect ratio of the defect bounding box is the ratio of the width to the height of the defect bounding box, characterizing the shape tendency; the roundness of the defect region is calculated by 4π × defect region area / defect outline perimeter², used to quantify the degree to which the defect is close to a circle; the calculation of the first 7 Zernike moments (Z00, Z11, Z20, Z22, Z31, Z33, Z40) is used to provide rotation-invariant shape descriptions.

[0016] Furthermore, the specific implementation steps of feature fusion via the attention mechanism in S4 include: Projecting 256-dimensional visual feature vectors onto 128 dimensions; Project the 11-dimensional morphological feature vectors to 32 dimensions; The dimensionality-reduced visual feature vector projection and the dimensionality-reduced morphological feature vector projection are concatenated to form a 160-dimensional joint feature, which is then input into the attention feature fusion network to calculate the weights of the visual and morphological features respectively; the visual and morphological features are then weighted and fused.

[0017] Furthermore, in the process of inputting the classifier into S4 and outputting the defect type and confidence level, the classifier is a lightweight neural network, including: a batch normalization layer, a 64-dimensional hidden layer, a ReLU activation function, Dropout regularization, and a Softmax output layer; the input of the classifier is a 160-dimensional joint feature obtained by weighted fusion of visual features and morphological features; the number of nodes in the output layer of the classifier corresponds to the number of defect categories, and the output result is the probability distribution of each defect category; During training, the cross-entropy loss function and Adam optimizer are used to jointly train the multi-scale convolutional neural network, the attention feature fusion network, and the classifier end-to-end, and finally output the specific classification results of the defects and their corresponding confidence scores.

[0018] Compared with the prior art, the beneficial effects achieved by the present invention are: (1) By introducing a data quality optimization module, and comprehensively using a variety of algorithms such as image restoration, defogging and illumination enhancement, the complex environmental interference such as uneven illumination inside the pipe, water vapor blurring and lens stains is effectively overcome, the clarity and recognizability of the image are improved, and high-quality input data is provided for the subsequent defect identification model, which fundamentally improves the detection accuracy and robustness of the model. (2) This invention uses YOLOv8 for rapid defect region localization and combines it with lightweight U-Net to achieve pixel-level fine segmentation, which not only ensures the detection speed but also achieves accurate extraction of defect contours. It is suitable for scenarios in actual engineering where both efficiency and accuracy are required. (3) This invention fully utilizes the complementary information of defects in visual appearance and geometric structure by deeply fusing visual features and morphological features and combining attention mechanism for weighted fusion, effectively improving the model’s ability to distinguish complex and diverse defect types, especially in cases where samples are scarce or morphology is varied. (4) This invention proposes targeted solutions for a variety of common imaging defects in drainage pipe networks. It can adapt to complex and ever-changing actual detection environments, has good generalization ability and practical value, and can provide reliable technical support for the intelligent operation and maintenance of urban drainage pipe networks. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to the present invention. Figure 2 This is a schematic diagram of an image quality optimization processing algorithm for a drainage pipe network defect detection method based on data optimization and multi-model fusion according to the present invention. Figure 3 This is a structural block diagram of step S4 in the drainage pipe network defect detection method based on data optimization and multi-model fusion of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-3 The present invention provides a technical solution: such as Figure 1 As shown, this embodiment provides a method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion, including: S1. An inspection robot performs internal camera inspection on the drainage pipe network, collects video data inside the drainage pipe network, and extracts video frames at fixed frame intervals to obtain the original image sequence. The images are then normalized in size. In this embodiment, a video processing tool is used to extract frames from the video at a fixed interval of 1 frame per second to obtain the original image sequence. All images are then scaled to a uniform size of 128x128 pixels.

[0022] S2. Image quality optimization is performed on the normalized original image sequence using lens damage repair, image dehazing, and illumination enhancement methods. like Figure 2 The diagram shown illustrates an image quality optimization processing algorithm. The method for lens damage repair in S2, which optimizes the image quality of the normalized original image sequence, includes the following steps: S211. Use the Pix2Pix network to synthesize simulated soiled images and construct a dataset of clean and soiled image pairs. Each element in the clean and soiled image pair dataset corresponds to a set of simulated soiled images of the same pipe in the drainage network and a set of simulated clean images of the corresponding pipe. S212. Using simulated smudged images of each data pair in the clean and smudged image pair dataset as input, and simulated clean images of the corresponding data pairs in the clean and smudged image pair dataset as targets, train the U-Net structure image inpainting model using L1 loss and perceptual loss; until it can effectively learn the ability to remove occlusions and restore background texture; the training parameters corresponding to training the image inpainting model in this embodiment include: the optimizer is Adam, the initial learning rate is 1e-4, the batch size is 8, and the training period is 100.

[0023] S213. Based on the trained image restoration model, obtain the lens damage restoration image corresponding to each image in the normalized original image sequence.

[0024] In step S2, the image quality optimization process for the normalized original image sequence employs a guided filtering-based local transmittance optimization algorithm for image dehazing. The specific implementation steps are as follows: S221. Model the image after the contamination repair based on the atmospheric scattering model, and estimate the initial transmittance using the dark channel prior. S222. Use guided filtering to perform edge-preserving optimization on the transmittance map to obtain refined transmittance. S223. Combine the estimated global atmospheric light value and recover the clear, fog-free image through inverse operation.

[0025] In step S2, the image quality optimization process for the normalized original image sequence employs a contrast-limited adaptive histogram equalization algorithm for illumination enhancement. The specific implementation steps are as follows: S231. Convert the dehazed image from the RGB color space to the LAB color space; S232. Apply the CLAHE algorithm to the brightness channel L to obtain the updated brightness channel L value, while the remaining channel values ​​remain unchanged. S233. Convert the updated LAB color space corresponding to the image after dehazing back to the RGB color space to obtain the final image quality optimization result for each image in the normalized original image sequence.

[0026] S3. Input the optimized image into the pre-trained target detection model to locate the defect region, and perform pixel-level segmentation on the located defect region to obtain a pixel-level defect contour binary mask. Step S3 uses YOLOv8 as the target detection model, initializes it with weights pre-trained on the COCO dataset, and fine-tunes it on the drainage network defect dataset to obtain the pre-trained target detection model; the drainage network defect dataset includes drainage network image data annotated with one or more defects such as cracks, deposits, and tree root intrusion. During the defect region localization process of the optimized image, the pre-trained target detection model outputs the corresponding defect region bounding boxes and their confidence scores in the optimized image based on the input optimized image; and the set of regions within the defect region bounding boxes with confidence scores greater than a preset value is taken as the defect region localization result for the corresponding optimized image.

[0027] In the process of performing pixel-level segmentation on the located defect region in S3 to obtain a pixel-level defect contour binary mask, a lightweight U-Net semantic segmentation model is used. The image region corresponding to each element in the defect region localization result of the optimized image is used as the input of the lightweight U-Net semantic segmentation model, and the corresponding pixel-level defect contour binary mask is output.

[0028] The specific structure of the lightweight U-Net semantic segmentation model in this embodiment is as follows: Since the image size is uniformly scaled to 128x128 pixels, therefore, in the encoder, Input: 128x128x3; Block 1: Two 3x3 convolutions (32 channels) → ReLU → Max pooling → Output 64x64x32; Block 2: Two 3x3 convolutions (64 channels) → ReLU → Max pooling → Output 32x32x64; Block 3: Two 3x3 convolutions (128 channels) → ReLU → Max pooling → Output 16x16x128; Block4: Two 3x3 convolutions (256 channels) → ReLU → Max pooling → Output 8x8x256; In the decoder, Block5: Transposed convolution (256→128, stride 2) → concatenated with the output of Block3 (to obtain 256 channels) → two 3x3 convolutions (128 channels) → output 16x16x128; Block6: Transposed convolution (128→64, stride 2) → concatenated with the output of Block2 (to obtain 128 channels) → two 3x3 convolutions (64 channels) → output 32x32x64; Block7: Transposed convolution (64→32, stride 2) → concatenated with the output of Block1 (to get 64 channels) → two 3x3 convolutions (32 channels) → output 64x64x32; Block8: Transposed convolution (32→32, stride 2) → concatenated with Block1 output (upsampled to 128x128x32) (to obtain 64 channels) → two 3x3 convolutions (32 channels) → output 128x128x32; In the output layer, 1x1 convolution (32→1) → sigmoid → 128x128x1.

[0029] S4. Visual and morphological features are extracted from the located defect area, and the features are fused through an attention mechanism. The results are then input into a classifier to output the defect type and confidence level.

[0030] like Figure 3 As shown, S4 extracts visual features from the located defect area using a multi-scale convolutional neural network, and the multi-scale convolutional neural network employs a three-layer convolutional structure, including: The first convolutional layer uses a 7×7 convolutional kernel to extract global features; The second convolutional layer uses a 5×5 convolutional kernel to extract medium-scale features; The third convolutional layer uses a 3×3 convolutional kernel to extract detailed features; The multi-scale convolutional neural network ultimately outputs a 256-dimensional visual feature vector.

[0031] In the process of extracting morphological features of the located defect region in step S4, based on the pixel-level defect contour binary mask generated in step S3, the defect region area, defect contour perimeter, defect boundary box aspect ratio, defect region circularity, and the first 7 Zernike moments are calculated to form an 11-dimensional morphological feature vector.

[0032] The specific implementation steps of feature fusion via the attention mechanism in S4 include: Projecting 256-dimensional visual feature vectors onto 128 dimensions; Project the 11-dimensional morphological feature vectors to 32 dimensions; The dimensionality-reduced visual feature vector projection and the dimensionality-reduced morphological feature vector projection are concatenated to form a 160-dimensional joint feature, which is then input into the attention feature fusion network to calculate the weights of the visual and morphological features respectively; the visual and morphological features are then weighted and fused.

[0033] In the process of the S4 input classifier outputting defect type and confidence, the classifier is a lightweight neural network, including: batch normalization layer, 64-dimensional hidden layer, ReLU activation function, Dropout regularization and Softmax output layer; the input of the classifier is a 160-dimensional joint feature obtained by weighted fusion of visual features and morphological features; the number of nodes in the output layer of the classifier corresponds to the number of defect categories, and the output result is the probability distribution of each defect category; During training, the cross-entropy loss function and Adam optimizer are used to jointly train the multi-scale convolutional neural network, the attention feature fusion network, and the classifier end-to-end, and finally output the specific classification results of the defects and their corresponding confidence scores.

[0034] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0035] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion, characterized in that, include: S1. The internal video inspection of the drainage pipe network is carried out by the inspection robot, the video data inside the drainage pipe network is collected, and the video frames are extracted at fixed frame intervals to obtain the original image sequence. The image size is normalized. S2. Image quality optimization is performed on the normalized original image sequence using lens damage repair, image dehazing, and illumination enhancement methods. S3. Input the optimized image into the pre-trained target detection model to locate the defect region, and perform pixel-level segmentation on the located defect region to obtain a pixel-level defect contour binary mask. S4. Visual and morphological features are extracted from the located defect area, and the features are fused through an attention mechanism. The results are then input into a classifier to output the defect type and confidence level.

2. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: The method for lens damage repair in the image quality optimization process of the normalized original image sequence, as described in S2, includes the following steps: S211. Use the Pix2Pix network to synthesize simulated soiled images and construct a dataset of clean and soiled image pairs. Each element in the clean and soiled image pair dataset corresponds to a set of simulated soiled images of the same pipe in the drainage network and a set of simulated clean images of the corresponding pipe. S212. Using the simulated dirty image of each data pair in the clean and dirty image pair dataset as input, and the simulated clean image of the corresponding data pair in the clean and dirty image pair dataset as target, train the U-Net structure image inpainting model using L1 loss and perceptual loss. S213. Based on the trained image restoration model, obtain the lens damage restoration image corresponding to each image in the normalized original image sequence.

3. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 2, characterized in that: In step S2, the image quality optimization process for the normalized original image sequence employs a guided filtering-based local transmittance optimization algorithm for image dehazing. The specific implementation steps are as follows: S221. Model the image after the contamination repair based on the atmospheric scattering model, and estimate the initial transmittance using the dark channel prior. S222. Use guided filtering to perform edge-preserving optimization on the transmittance map to obtain refined transmittance. S223. Combine the estimated global atmospheric light value and recover the clear, fog-free image through inverse operation.

4. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: In step S2, the image quality optimization process for the normalized original image sequence employs a contrast-limited adaptive histogram equalization algorithm for illumination enhancement. The specific implementation steps are as follows: S231. Convert the dehazed image from the RGB color space to the LAB color space; S232. Apply the CLAHE algorithm to the brightness channel L to obtain the updated brightness channel L value, while the remaining channel values ​​remain unchanged. S233. Convert the updated LAB color space corresponding to the image after dehazing back to the RGB color space to obtain the final image quality optimization result for each image in the normalized original image sequence.

5. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: Step S3 uses YOLOv8 as the target detection model, initializes it with weights pre-trained on the COCO dataset, and fine-tunes it on the drainage network defect dataset to obtain the pre-trained target detection model; the drainage network defect dataset includes drainage network image data annotated with one or more defects such as cracks, deposits, and tree root intrusion. During the defect region localization process of the optimized image, the pre-trained target detection model outputs the corresponding defect region bounding boxes and their confidence scores in the optimized image based on the input optimized image; and the set of regions within the defect region bounding boxes with confidence scores greater than a preset value is taken as the defect region localization result for the corresponding optimized image.

6. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: In the process of performing pixel-level segmentation on the located defect region in S3 to obtain a pixel-level defect contour binary mask, a lightweight U-Net semantic segmentation model is used. The image region corresponding to each element in the defect region localization result of the optimized image is used as the input of the lightweight U-Net semantic segmentation model, and the corresponding pixel-level defect contour binary mask is output.

7. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: The S4 method extracts visual features from the located defect region using a multi-scale convolutional neural network, which employs a three-layer convolutional structure, including: The first convolutional layer uses a 7×7 convolutional kernel to extract global features; The second convolutional layer uses a 5×5 convolutional kernel to extract medium-scale features; The third convolutional layer uses a 3×3 convolutional kernel to extract detailed features; The multi-scale convolutional neural network ultimately outputs a 256-dimensional visual feature vector.

8. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: In the process of extracting morphological features of the located defect region in step S4, based on the pixel-level defect contour binary mask generated in step S3, the defect region area, defect contour perimeter, defect boundary box aspect ratio, defect region circularity, and the first 7 Zernike moments are calculated to form an 11-dimensional morphological feature vector.

9. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: The specific implementation steps of feature fusion via the attention mechanism in S4 include: Projecting 256-dimensional visual feature vectors onto 128 dimensions; Project the 11-dimensional morphological feature vectors to 32 dimensions; The dimensionality-reduced visual feature vector projection and the dimensionality-reduced morphological feature vector projection are concatenated to form a 160-dimensional joint feature, which is then input into the attention feature fusion network to calculate the weights of the visual and morphological features respectively; the visual and morphological features are then weighted and fused.

10. The method for detecting defects in drainage pipe networks based on data optimization and multi-model fusion according to claim 1, characterized in that: In the process of the S4 input classifier outputting defect type and confidence, the classifier is a lightweight neural network, including: batch normalization layer, 64-dimensional hidden layer, ReLU activation function, Dropout regularization and Softmax output layer; the input of the classifier is a 160-dimensional joint feature obtained by weighted fusion of visual features and morphological features; the number of nodes in the output layer of the classifier corresponds to the number of defect categories, and the output result is the probability distribution of each defect category; During training, the cross-entropy loss function and Adam optimizer are used to jointly train the multi-scale convolutional neural network, the attention feature fusion network, and the classifier end-to-end, and finally output the specific classification results of the defects and their corresponding confidence scores.

Images