A method and system for identifying areas of functional defects in a sewer pipe

Abstract

The present application relates to the technical field of image recognition, and in particular to a sewer pipe functional defect area identification method and system, the method provided by the present application mainly includes obtaining an initial image inside the current pipe, and pre-processing the initial image to obtain a target image; a plurality of historical images of the front of the pipe and the characteristics of the blockage are obtained, a semantic segmentation model based on mobileNet v2 is constructed, the target image is identified based on the semantic segmentation model, the semantic segmentation result of the pipe diameter area and the fault area in the output image is output, and the blockage rate and the blockage area are calculated. Through the above method, detailed optical parameters, including the sight distance, are no longer required when calculating the area, only the pipe diameter is required, and the traditional method is usually complicated to calculate, the present method is simple, convenient and fast, secondly, the present method can quantitatively calculate the area by only using the pipe diameter, which is helpful for objectively and quantitatively evaluating the actual situation of the defect.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to a method and system for identifying the area of ​​functional defects in drainage pipes. Background Technology

[0002] As urban drainage systems age, problems such as siltation, collapse, and tree root intrusion often occur inside the pipes, severely affecting drainage capacity and threatening urban operational safety. While the widely used CCTV (closed-circuit television) inspection method can visually capture images of the pipe interior, obstacle identification and assessment largely rely on manual interpretation, making quantitative analysis difficult, especially lacking the ability to automatically calculate indicators such as obstacle area and volume.

[0003] During the operation of drainage pipes, due to the combined influence of various factors, they are prone to sediment accumulation, tree root intrusion, and blockage by debris, which greatly affects drainage performance, leading to functional defects and severely impacting normal use. Traditional detection methods are inefficient, costly, and their assessment of blockage severity is influenced by human experience, only indicating whether a blockage exists, exhibiting significant subjectivity and making them unsuitable for precise maintenance. Similarly, traditional image recognition algorithms can only determine whether a blockage exists, but cannot quantify the specific degree of blockage (the area of ​​the blockage).

[0004] Traditional image area recognition and calculation techniques typically require high image quality. However, drainage pipe inspection usually employs CCTV inspection methods, resulting in low-quality image data that is pure image data without reference coordinates. Traditional 3D measurement methods often rely on hardware devices such as lasers, structured lights, or depth cameras, but these are costly to deploy in complex and narrow drainage environments and have poor adaptability. Therefore, there is an urgent need for an intelligent identification method for the functional defect area of ​​sewers based on pure image data. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for identifying the area of ​​functional defects in drainage pipes, in order to solve the above-mentioned problems in the prior art.

[0006] This invention is achieved through the following technical solution:

[0007] In a first aspect, the present invention provides a method for identifying the area of ​​functional defects in drainage pipes, comprising:

[0008] Obtain an initial image of the current pipeline interior, and preprocess the initial image to obtain the target image;

[0009] Acquire several historical images of pipes with frontal features and blockages, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into test and training sets;

[0010] Construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, until the semantic segmentation model’s recognition accuracy of the target region meets the preset threshold, and output the semantic segmentation model that meets the preset threshold.

[0011] The target image is identified based on the semantic segmentation model, and the semantic segmentation results of the pipe diameter area and the fault area in the image are output.

[0012] Based on the semantic segmentation results, the pixel count of the pipe cross section and the pixel count of the fault area are calculated respectively, and the blockage rate is calculated by the pixel count of the pipe cross section and the pixel count of the fault area.

[0013] Obtain the pipe diameter, calculate the pipe cross-sectional area based on the pipe diameter, calculate the blockage area based on the pipe cross-sectional area and the blockage rate, and output the blockage area as the result.

[0014] Preferably, the preprocessing of the initial image to obtain the target image includes:

[0015] The initial image is cleaned by removing blurry and abnormally exposed frames, and retaining frontal, distortion-free images to obtain the target image.

[0016] Preferably, the annotation of the historical images based on labelme for different congestion categories includes;

[0017] First, the pipe cross-section is labeled, and then the blockage features of the blockage area are labeled based on the historical images of the pipe cross-section that have already been labeled.

[0018] After completing the annotation of the blockage features, delete the annotation of the pipe cross-section.

[0019] Preferably, the semantic segmentation model-based identification of the target image and the output semantic segmentation results of the pipe diameter region and the fault region in the image include establishing a label mapping. The label mapping is a mapping function model from global labels of the pipe diameter region and the blockage region annotated with labelme in historical images to index pointers.

[0020] In the formula, For a set of tags, As background, For sedimentation, For obstacles, For the pipe cross-section, and , This represents a mapping relationship.

[0021] Preferably, it also includes generating corresponding digital labels based on the recognition results through a mapping function model, and changing the color of different digital labels through a color palette;

[0022] After changing the color using the color palette, a mask is generated. The semantic segmentation result output by the DeepLab V3 + intelligent monitoring model is converted into a binary mask, where faulty areas are marked as 1 and non-faulty areas are marked as 0. For each pixel (x,y), its index value I(x,y) is defined.

[0023] The semantic segmentation results are converted into PNG images and used to generate an indexed image of the matrix.

[0024] Preferably, the step of calculating the pixel count of the pipe cross-section and the pixel count of the fault region based on the semantic segmentation result includes:

[0025]

[0026] In the formula, For pixel counting, Let be the RGB value of the i-th pixel. These are RGB value sequences, For indicator functions, This represents the total number of pixels in the image.

[0027] Preferably, the calculation of the blockage rate using pixel counts of the pipe cross-section and pixel counts of the fault area includes:

[0028]

[0029] In the formula, For congestion rate, Count the number of pixels in the faulty area. Count the number of pixels for the pipe cross-section. This is an indicator function for the fault region. This is an indicator function for the pipe cross-section. Let be the pixel value of the i-th pixel in the image of the fault region. Let i be the pixel value of the i-th pixel in the pipe cross-section image. These are the RGB value sequences of the fault area image.

[0030] Preferably, obtaining the pipe diameter and calculating the pipe cross-sectional area based on the pipe diameter includes:

[0031]

[0032] In the formula, The cross-sectional area of ​​the pipe. This refers to the pipe diameter.

[0033] Preferably, obtaining the blockage area based on the pipe cross-sectional area and the blockage rate includes:

[0034]

[0035] In the formula, The area of ​​the blockage.

[0036] Secondly, the present invention also provides a drainage pipe functional defect area identification system for performing the above-described drainage pipe functional defect area identification method, comprising:

[0037] The preprocessing module is configured to acquire an initial image of the current pipe interior, preprocess the initial image to obtain a target image, acquire several historical images of the front of the pipe with blockage features, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into a test set and a training set.

[0038] The recognition module is configured to construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, and output the semantic segmentation model that meets the preset threshold in terms of the recognition accuracy of the target region; and recognize the target image based on the semantic segmentation model, and output the semantic segmentation results of the pipe diameter region and the fault region in the image.

[0039] The output module is configured to calculate the pixel count of the pipe cross section and the pixel count of the fault area based on the semantic segmentation results, calculate the blockage rate using the pixel count of the pipe cross section and the pixel count of the fault area, obtain the pipe diameter, obtain the pipe cross section area based on the pipe diameter, obtain the blockage area based on the pipe cross section area and the blockage rate, and output the blockage area as the result.

[0040] The technical solution of the present invention has at least the following advantages and beneficial effects:

[0041] The method provided by this invention mainly includes: acquiring an initial image of the current pipeline interior; preprocessing the initial image to obtain a target image; acquiring several historical images of the pipeline front with blockage features; constructing a semantic segmentation model based on MobileNet v2; recognizing the target image based on the semantic segmentation model; outputting the semantic segmentation results of the pipe diameter region and the fault region in the image; and calculating the blockage rate and the area of ​​the blockage. This method eliminates the need for detailed optical parameters, including line-of-sight distance, when calculating the area; only the pipe diameter is required. Furthermore, traditional methods are often computationally cumbersome, while this method is simple, convenient, and fast. Moreover, this method can quantitatively calculate the area using only the pipe diameter, which helps to objectively and quantitatively evaluate the actual situation of the defect. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of the process of the present invention;

[0044] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0046] The independently described modules or sub-modules may or may not be physically separated; they may be implemented in software or hardware, and some modules or sub-modules may be implemented in software, with the processor calling the software to implement the function of these modules or sub-modules, while other modules or sub-modules may be implemented in hardware, such as through hardware circuits. Furthermore, some or all of the modules can be selected to achieve the purpose of this application's solution according to actual needs.

[0047] Please refer to Figures 1-2 The present invention provides a method for identifying the area of ​​functional defects in drainage pipes, comprising:

[0048] S1: Image acquisition and preprocessing: Acquire an initial image of the current pipeline interior, and preprocess the initial image to obtain the target image;

[0049] The process involves collecting images of the inside of sewer pipes using closed-circuit television (CCTV) systems or pipe inspection robots. These images cover different pipe diameters and various blockage conditions (such as silt deposits, mineral deposits, tree roots, and industrial waste). The collected images are then cleaned (by removing blurry or abnormally exposed frames and retaining undistorted frontal images). Clear images of the front of the pipe are selected to provide visual information for subsequent analysis.

[0050] S2: Optimize the parameters of the Deeplabv+ model training set, obtain several historical images of pipes with blockage features, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into test and training sets; construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, until the semantic segmentation model's recognition accuracy of the target region meets the preset threshold, and output the semantic segmentation model that meets the preset threshold;

[0051] Specifically, we obtained a publicly available sewer pipe dataset, cleaned the data, and selected 1000 images of the front of the pipes that showed signs of blockage. We used LabelMe to label the deposition / obstacle categories and pipe cross-sections (prioritizing labeling pipe cross-sections, then labeling blockage areas within already labeled pipe cross-sections, and finally deleting the pipe cross-section labels to ensure they are stored separately from the blockage labels). We randomly selected 30% of the data as the test set and the remaining 70% as the training set to train a DeepLab V3+ framework. We built a semantic segmentation model based on MobileNetv2, fusing features of sewer pipe deposition and obstruction states (random flipping + color temperature perturbation enhancement) as input. We optimized the feature fusion method to extract and analyze sewer pipe features, learning the image features of sewer pipe blockage states. During training, we continuously optimized the feature fusion parameters until the model's accuracy in identifying fault areas met a preset threshold (validation set satisfies mIoU ≥ 0.9).

[0052] The DeepLab V3+ framework includes an encoder-decoder design: a backbone network (such as Xception or ResNet) acts as the encoder to extract features, and the decoder gradually restores resolution through upsampling, combined with skip connections to preserve shallow details; enhanced ASPP: captures multi-scale features through dilated convolutions (dilated convolutions) to cover different receptive fields and solve the problem of target size differences; depthwise separable convolutions are applied in the decoder to reduce computation and improve model efficiency; and dilated convolution optimization: balances feature resolution and computational cost, avoiding information loss caused by traditional downsampling.

[0053] Semantic segmentation models, through their deep feature extraction capabilities and residual structures, form the core foundation of semantic segmentation models (such as Faster R-CNN and RetinaNet). When combined with modules such as FPN and RPN, they can efficiently handle multi-scale object detection and instance segmentation tasks. Among them, hierarchical feature extraction: low-level (such as Res2) preserves spatial details (edges, textures); high-level (such as Res5) extracts semantic information (object category).

[0054] Secondly, labelme is used for data annotation in computer vision tasks such as object detection, semantic segmentation, and instance segmentation. It supports polygon, rectangle, circle, line, and point annotations and generates JSON format annotation files, suitable for deep learning model training.

[0055] S3: Intelligent obstacle recognition and acquisition of pipe diameter data, recognition of target image based on semantic segmentation model, and output of semantic segmentation results of pipe diameter area and fault area in image;

[0056] The image to be identified is used as input data to identify the location and shape of pipe defects in the image, and the semantic segmentation results of the pipe diameter area and fault area in the image are output. The pipe cross-sectional area is obtained by using the pipe diameter of the pipe to be identified in the image to be identified in advance.

[0057] S4: Calculate the obstacle blockage rate. Based on the semantic segmentation results, calculate the pixel count of the pipe cross section and the pixel count of the fault area respectively, and calculate the blockage rate by the pixel count of the pipe cross section and the pixel count of the fault area.

[0058] The calculation of the congestion rate includes label mapping, color palette, mask generation, image generation, pixel count of deposition / obstacle region, and congestion rate.

[0059] S5: Calculate the area of ​​the obstacle, obtain the pipe diameter, obtain the pipe cross-sectional area based on the pipe diameter, obtain the blockage area based on the pipe cross-sectional area and the blockage rate, and output the blockage area as the result.

[0060] The method provided by this invention mainly includes: acquiring an initial image of the current pipeline interior; preprocessing the initial image to obtain a target image; acquiring several historical images of the pipeline front with blockage features; constructing a semantic segmentation model based on MobileNet v2; recognizing the target image based on the semantic segmentation model; outputting the semantic segmentation results of the pipe diameter region and the fault region in the image; and calculating the blockage rate and the area of ​​the blockage. This method eliminates the need for detailed optical parameters, including line-of-sight distance, when calculating the area; only the pipe diameter is required. Furthermore, traditional methods are often computationally cumbersome, while this method is simple, convenient, and fast. Moreover, this method can quantitatively calculate the area using only the pipe diameter, which helps to objectively and quantitatively evaluate the actual situation of the defect.

[0061] In one exemplary embodiment of the present invention, the semantic segmentation model-based identification of the target image and the output semantic segmentation results of the pipe diameter region and the fault region in the image include establishing a label mapping. The label mapping is a mapping function model from global labels of the pipe diameter region and the blockage region annotated using labelme on historical images to an index pointer.

[0062] In the formula, For a set of tags, As background, For sedimentation, For obstacles, For the pipe cross-section, and , This represents a mapping relationship.

[0063] An exemplary embodiment of the present invention further includes generating corresponding digital labels based on the recognition results through a mapping function model, and changing the color of different digital labels through a color palette;

[0064] The identified results can be mapped to labels using a mapping function model. Colors are changed using a color palette for different labels to easily distinguish between the background, pipe cross-sections, and blockage areas (deposits and obstacles). The color palette is a fixed sequence of RGB values, which can be represented as:

[0065]

[0066] in: (Each color is a 3D vector, such as...) =(0,0,0) represents the corresponding background.

[0067] CJ for =(160, 82, 45), ZW is =(255, 255, 0), Circle is =(255, 255,255) has a background fill of (0,0,0).

[0068] After changing the color using the color palette, a mask is generated. The semantic segmentation result output by the DeepLab V3+ intelligent monitoring model is converted into a binary mask, where faulty areas are marked as 1 and non-faulty areas are marked as 0. For each pixel (x,y), its index value I(x,y) is defined, which is determined by the polygon region.

[0069]

[0070] in, Let m be the label of the k-th polygon, and m() be the index value.

[0071] The semantic segmentation results are converted into PNG images and used to generate an indexed image of the matrix.

[0072] After generating pixels, to facilitate a more intuitive view of blockages or pipe cross-sections, it is necessary to generate an image from the above processing results. This step converts the semantic segmentation results into a PNG image, generating an indexed image. For a matrix:

[0073]

[0074] Where H×W are the image dimensions, representing the height and width respectively, and the RGB image after palette mapping satisfies:

[0075]

[0076] In the formula, The processed pixel values, This is a pixel transformation.

[0077] In one exemplary embodiment of the present invention, the step of calculating the pixel count of the pipe cross-section and the pixel count of the fault region based on the semantic segmentation result includes:

[0078]

[0079] In the formula, For pixel counting, Let be the RGB value of the i-th pixel. These are RGB value sequences, This is an indicator function; it returns 1 if true and 0 otherwise. This represents the total number of pixels in the image.

[0080] Specifically, the calculation of the blockage rate by counting pixels in the pipe cross-section and the pixel count in the fault area includes:

[0081]

[0082] In the formula, For congestion rate, Count the number of pixels in the faulty area. Count the number of pixels for the pipe cross-section. This is an indicator function for the fault region. This is an indicator function for the pipe cross-section. Let be the pixel value of the i-th pixel in the image of the fault region. Let i be the pixel value of the i-th pixel in the pipe cross-section image. These are the RGB value sequences of the fault area image.

[0083] In one exemplary embodiment of the present invention, obtaining the pipe diameter and calculating the pipe cross-sectional area based on the pipe diameter includes:

[0084]

[0085] In the formula, The cross-sectional area of ​​the pipe. This refers to the pipe diameter.

[0086] Specifically, obtaining the blockage area based on the pipe cross-sectional area and the blockage rate includes:

[0087]

[0088] In the formula, The area of ​​the blockage.

[0089] Secondly, the present invention also provides a drainage pipe functional defect area identification system for performing the above-described drainage pipe functional defect area identification method, comprising:

[0090] The preprocessing module is configured to acquire an initial image of the current pipe interior, preprocess the initial image to obtain a target image, acquire several historical images of the front of the pipe with blockage features, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into a test set and a training set.

[0091] The recognition module is configured to construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, and output the semantic segmentation model that meets the preset threshold in terms of the recognition accuracy of the target region; and recognize the target image based on the semantic segmentation model, and output the semantic segmentation results of the pipe diameter region and the fault region in the image.

[0092] The output module is configured to calculate the pixel count of the pipe cross section and the pixel count of the fault area based on the semantic segmentation results, calculate the blockage rate using the pixel count of the pipe cross section and the pixel count of the fault area, obtain the pipe diameter, obtain the pipe cross section area based on the pipe diameter, obtain the blockage area based on the pipe cross section area and the blockage rate, and output the blockage area as the result.

[0093] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer software product, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying the area of ​​functional defects in drainage pipes, characterized in that, include: Obtain an initial image of the current pipeline interior, and preprocess the initial image to obtain the target image; Acquire several historical images of pipes with frontal features and blockages, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into test and training sets; Construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, until the semantic segmentation model’s recognition accuracy of the target region meets the preset threshold, and output the semantic segmentation model that meets the preset threshold. The target image is identified based on the semantic segmentation model, and the semantic segmentation results of the pipe diameter area and the fault area in the image are output. Based on the semantic segmentation results, the pixel count of the pipe cross section and the pixel count of the fault area are calculated respectively, and the blockage rate is calculated by the pixel count of the pipe cross section and the pixel count of the fault area. Obtain the pipe diameter, calculate the pipe cross-sectional area based on the pipe diameter, calculate the blockage area based on the pipe cross-sectional area and the blockage rate, and output the blockage area as the result; The pixel counts of the pipe cross section and the pixel counts of the fault region are calculated based on the semantic segmentation results, including: In the formula, For pixel counting, Let be the RGB value of the i-th pixel. These are RGB value sequences, For indicator functions, This represents the total number of pixels in the image. The calculation of the blockage rate using pixel counts of the pipe cross-section and pixel counts of the fault area includes: In the formula, For congestion rate, Count the number of pixels in the faulty area. Count the number of pixels across the pipe cross-section. This is an indicator function for the fault region. This is an indicator function for the pipe cross-section. Let be the pixel value of the i-th pixel in the image of the fault region. Let i be the pixel value of the i-th pixel in the pipe cross-section image. These are the RGB value sequences of the fault area image; The process of obtaining the pipe diameter and calculating the pipe cross-sectional area based on the pipe diameter includes: In the formula, The cross-sectional area of ​​the pipe. The diameter of the pipe; The method of obtaining the blockage area based on the pipe cross-sectional area and the blockage rate includes: In the formula, The area of ​​the blockage.

2. The method for identifying the area of ​​functional defects in drainage pipes according to claim 1, characterized in that, The preprocessing of the initial image to obtain the target image includes: The initial image is cleaned by removing blurry and abnormally exposed frames, and retaining frontal, distortion-free images to obtain the target image.

3. The method for identifying the area of ​​functional defects in drainage pipes according to claim 1, characterized in that, The annotation of the historical images based on labelme for different congestion categories includes; First, the pipe cross-section is labeled, and then the blockage features of the blockage area are labeled based on the historical images of the pipe cross-section that have already been labeled. After completing the annotation of the blockage features, delete the annotation of the pipe cross-section.

4. The method for identifying the area of ​​functional defects in drainage pipes according to claim 1, characterized in that, The semantic segmentation model is used to identify the target image, and the semantic segmentation results of the pipe diameter region and fault region in the output image include establishing a label mapping. The label mapping is a mapping function model from global labels of the pipe diameter region and blockage region annotated with labelme in historical images to index pointers. In the formula, For a set of tags, As background, For sedimentation, For obstacles, For the pipe cross-section, and , This represents a mapping relationship.

5. The method for identifying the area of ​​functional defects in drainage pipes according to claim 4, characterized in that, It also includes generating corresponding digital labels based on the recognition results using a mapping function model, and changing the color of different digital labels using a color palette. After changing the color using the color palette, a mask is generated. The semantic segmentation result output by the DeepLab V3 + intelligent monitoring model is converted into a binary mask, where faulty areas are marked as 1 and non-faulty areas are marked as 0. For each pixel (x,y), its index value I(x,y) is defined. The semantic segmentation results are converted into PNG images and used to generate an indexed image of the matrix.

6. A system for identifying the area of ​​functional defects in drainage pipes, characterized in that, A method for identifying the area of ​​functional defects in a drainage pipe as described in any one of claims 1-5, comprising: The preprocessing module is configured to acquire an initial image of the current pipe interior, preprocess the initial image to obtain a target image, acquire several historical images of the front of the pipe with blockage features, label the historical images with different blockage categories based on labelme, and divide the labeled historical images into a test set and a training set. The recognition module is configured to construct a semantic segmentation model based on MobileNet v2, train the semantic segmentation model based on the training set, set a preset threshold, and output the semantic segmentation model that meets the preset threshold in terms of the recognition accuracy of the target region; and recognize the target image based on the semantic segmentation model, and output the semantic segmentation results of the pipe diameter region and the fault region in the image. The output module is configured to calculate the pixel count of the pipe cross section and the pixel count of the fault area based on the semantic segmentation results, calculate the blockage rate using the pixel count of the pipe cross section and the pixel count of the fault area, obtain the pipe diameter, obtain the pipe cross section area based on the pipe diameter, obtain the blockage area based on the pipe cross section area and the blockage rate, and output the blockage area as the result.

Images