A pipeline crack length estimation method based on video images, medium and equipment

Abstract

The present application relates to the technical field of image processing, in particular to a pipeline crack length estimation method based on video images, medium and equipment, comprising the following steps: A, extracting images with cracks from the obtained pipeline video and making data sets; B, establishing a segmentation model to segment the images in the obtained data set to obtain crack images containing only cracks; C, marking the crack images to obtain the center coordinate position of the crack images; D, based on the crack images, obtaining the coordinate positions of all pixel points on the crack, and calculating according to the obtained center coordinate position and the coordinate positions of all pixel points on the crack to obtain the length of the crack on the pipeline, which can overcome the obstacle problem of the shooting angle to the crack length quantification, thereby realizing the effect that the crack length in the pipeline inner wall can be measured only by the pipeline detection image, and significantly improving the crack length quantification precision, providing a reference basis for further repair of the drainage pipeline.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for estimating the length of pipe cracks based on video images. Background Technology

[0002] Urban drainage pipes are a vital component of a city's normal operation, responsible not only for transporting wastewater from residents and industrial production but also for collecting and discharging rainwater. Due to the age of many drainage pipe networks, complex underground environments, and improper operation and maintenance, pipe ruptures and other defects are becoming increasingly prominent. These defects significantly impact the drainage capacity of the network, exacerbating the risks of sewage overflows, water pollution, and urban flooding, ultimately leading to economic losses and even loss of life and property for residents.

[0003] CCTV (Closed-Circuit Television) is a commonly used method for detecting defects in drainage pipes. However, with a large amount of CCTV data, inspectors need to spend a lot of time and energy to find defects and assess their severity. Moreover, the current standards and definitions are quite broad, and due to differences in inspector experience, the same defect may be classified into different levels. Therefore, to reduce the workload of inspectors, ensure the objectivity of defect identification, and improve the standardization of defect diagnosis, various computer vision algorithms are gradually being applied.

[0004] Computer vision technology is a comprehensive discipline encompassing computer science, image processing, and pattern recognition. It is used to simulate human visual functions, observe the state of objects, and extract key information. Deep learning algorithms based on computer vision technology are beginning to be widely applied to CCTV data processing for drainage pipelines, such as object detection and semantic segmentation. Object detection technology was first applied to pipeline defect detection, for example, the YOLO (You Only Look Once) series of algorithms and R-CNN (Region Convolutional Neural Networks).

[0005] In CCTV images, it can use anchor frames to mark the specific location and type of pipeline defects, significantly reducing the workload of inspection personnel. However, target detection technology using anchor frames cannot mark the specific morphology of pipeline defects.

[0006] Semantic segmentation technology has begun to be applied in pipeline defect detection, such as the U-Net and DeepLab algorithms. Semantic segmentation can segment defects in CCTV footage to obtain their specific morphology, thereby further assisting inspectors in assessing the severity of pipeline defects. However, this method still cannot fully automate pipeline defect identification; the process still requires manual defect assessment by inspectors. Furthermore, due to the complex internal environment of drainage pipelines and poor shooting conditions, the application of computer vision technology in pipeline inspection still faces challenges. Summary of the Invention

[0007] To address the aforementioned problems in existing technologies, this invention provides a method for quantifying the length of pipe cracks based on video images. This method overcomes the obstacle of shooting angle to crack length quantification, thereby enabling the measurement of the length of cracks in the inner wall of pipes using only pipe detection images, providing a reference for further repair of drainage pipes.

[0008] In a first aspect, embodiments of this application provide a method for quantifying the length of pipe cracks based on video images, including the following steps:

[0009] A. Extract images of pipes with cracks from the acquired pipe videos and create a dataset;

[0010] B. Establish a segmentation model to segment the images in the acquired dataset to obtain crack images containing cracks;

[0011] C, Mark the crack image to obtain the center coordinates of the crack image;

[0012] D. Based on the crack image, obtain the coordinate positions of all pixels on the crack. Calculate the length of the crack on the pipe based on the obtained center coordinate position and the coordinate positions of all pixels on the crack.

[0013] In one alternative of the first aspect, step C specifically involves judging the crack image to determine whether there is a pipe joint in the crack image. If there is a pipe joint, the center coordinate position of the pipe joint is marked; if there is no pipe joint, the center coordinate position of the crack image is marked.

[0014] In another alternative to the first aspect, before performing step D, a preprocessing step is performed to remove pixels in the crack image that do not need to be counted.

[0015] In another alternative to the first aspect, before performing step D, preprocessing is performed, specifically, the pixels of the crack image are traversed, and if there are at least three non-zero pixels at the positions of the left and right sides, the bottom side, and the left and right sides of the bottom side of a pixel, or if the pixels on the left and right sides of a pixel are all zero, then the value of the pixel is set to zero.

[0016] In another alternative to the first aspect, in step D, the formula for calculating the crack length is:

[0017] L=α∑L i

[0018] Where L represents the crack length, L i α represents the actual length corresponding to pixel i, and α represents the correction coefficient. Since the camera shooting angle and image resolution affect the quantization accuracy of the crack, the more tilted the shooting angle and the lower the image resolution, the lower the quantization accuracy of the crack. Therefore, a correction coefficient needs to be added for adjustment.

[0019] In another alternative to the first aspect, in step D, the formula for calculating the crack length is specifically as follows:

[0020]

[0021] Among them, C i C represents the calculated pipe circumference. s This indicates the actual circumference of the pipeline.

[0022] In another alternative to the first aspect, the formulas for calculating the calculated pipe circumference and the actual pipe circumference are as follows:

[0023]

[0024] Among them, R s x represents the actual pipe radius. c The x-coordinate representing the center position, y c The ordinate representing the center coordinate position, x i The x-coordinate of pixel i is represented by the y-coordinate. i This represents the ordinate of pixel i.

[0025] In another alternative to the first aspect, in step B, the segmentation model established includes an encoder and a decoder connected in sequence, the encoder including a feature extraction layer for feature extraction and a pooling layer for capturing features at different scales connected in sequence.

[0026] Secondly, embodiments of this application provide a computer storage medium storing a computer program, which includes program instructions. When executed by a processor, the program instructions can implement a method for estimating the length of a pipe crack based on a video image, provided by the first aspect or any implementation of the first aspect of this application.

[0027] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform a method for estimating the length of a pipe crack based on a video image provided by the first aspect or any implementation thereof of the embodiments of this application.

[0028] The beneficial technical effects of this invention include:

[0029] 1. This invention proposes a method for estimating the length of pipe cracks based on video images, which breaks through the bottleneck of computer vision technology in the identification of defects in municipal pipelines. By using pipeline inspection images and combining the DeepLab V3+ semantic segmentation model and crack length estimation method, the method realizes the function of quantifying the length of cracks in the inner wall of the pipeline using only pipeline inspection images.

[0030] 2. The DeepLab V3+ semantic segmentation model was used, which has higher accuracy in crack segmentation compared to the FCN model and can significantly improve the accuracy of crack length quantization. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of a pipe crack length estimation method based on video images according to the present invention;

[0033] Figure 2 This is a network structure diagram of the semantic segmentation model in this invention;

[0034] Figure 3 This is a schematic diagram of the preprocessing of crack images in Embodiment 1 of the present invention;

[0035] Figure 4 This is a schematic diagram of calculating the pipe circumference based on pixel i in a crack image in Embodiment 1 of the present invention;

[0036] Figure 5The graph shows the performance of three different segmentation models in Embodiment 1 of the present invention. Detailed Implementation

[0037] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0038] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0039] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0040] Example 1:

[0041] Reference Figure 1 A method for estimating the length of a pipe crack based on video images, comprising the following steps:

[0042] Step A: Obtain the dataset, which comes from CCTV videos. Extract images of damaged pipes from the CCTV videos and create the dataset.

[0043] Step B: Establish a segmentation model to segment the images in the acquired dataset to obtain crack images containing only cracks.

[0044] Reference Figure 2Furthermore, the segmentation model is a DeepLab V3+ semantic segmentation model with MobileNet V2 as the backbone network. The segmentation model consists of an encoder and a decoder connected in sequence. The encoder includes a feature extraction layer and a pooling layer connected in sequence. The feature extraction layer is MobileNet V2, used to extract low-level features, while the pooling layer is the ASPP model, used to extract high-level features. Images from the dataset are input into the segmentation model, and through processing by the sequentially connected decoder and encoder, the output is a crack image containing only the cracks.

[0045] Specifically, the feature extraction layer of MobileNet V2 includes two sequentially connected residual structures. Both the first and second residual structures include three convolutional blocks. In the first residual structure, convolutional block 1 has a 1x1 kernel with a stride of 1, convolutional block 2 has a 3x3 kernel with strides of 1 and 3x3, and convolutional block 3 has a 1x1 kernel with a stride of 1. In the second residual structure, convolutional block 1 has a 1x1 kernel with a stride of 1, convolutional block 2 has a 3x3 kernel with strides of 1 and 3x3, and convolutional block 3 has a 1x1 kernel with a stride of 1. Specifically, convolutional blocks 1 and 2 in both the first and second residual structures include sequentially connected convolutional layers, batch normalization layers, and activation function layers. Convolutional block 3 in both the first and second residual structures includes sequentially connected convolutional layers and batch normalization layers. The output of the first residual structure and the input image of the feature extraction layer are concatenated and then used as the input of the second residual structure.

[0046] The segmentation model incorporates the ASPP Model structure, which includes convolutional layers with multiple kernels of different sizes connected in sequence and max pooling layers. This can further enhance the model's feature extraction capabilities, with the aim of expanding the receptive field so that the segmentation model can understand broader contextual information and address the problem of spatial information loss.

[0047] Specifically, the ASPP Model pooling layer includes five convolutional blocks of different sizes arranged in parallel, comprising a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence. The first convolutional block has a 1*1 kernel, a stride of 1, and a residual of 1. The second convolutional block has a 3*3 kernel, a stride of 1, and a residual of 12. The third convolutional block has a 3*3 kernel, a stride of 1, padding of 24, and a residual of 24. The fourth convolutional block has a 3*3 kernel, a stride of 1, padding of 36, and a residual of 36. The fifth convolutional block has a 3*3 kernel and a stride of 1. Specifically, the network structure of the fifth convolutional block consists of an adaptive average pooling layer, a convolutional layer, a batch normalization layer, an activation function layer, and a bilinear interpolation layer connected in sequence. The convolution results of the five convolutional blocks of different sizes are concatenated by channels and then passed through another convolutional block with a 1*1 kernel and a stride of 1. After regularization, high-level features are extracted.

[0048] The decoder's network structure consists of five convolutional blocks of different sizes connected in sequence, including convolutional layers, batch normalization layers, and activation function layers connected in sequence. The first convolutional block has a 1*1 kernel, a stride of 1, and padding of 0. The second convolutional block has a 3*3 kernel, a stride of 1, and padding of 1. The third convolutional block has a 3*3 kernel, a stride of 1, and padding of 1.

[0049] The low-level features extracted by the MobileNet V2 feature extraction layer are input into the decoder. After passing through convolution block one, they are concatenated with the high-level features extracted by the ASPP pooling layer model. Then, they pass through convolution block two and convolution block three in sequence, and then through the bilinear interpolation layer to obtain the output image, i.e., the crack image.

[0050] The established segmentation model was trained using 133 images of drainage pipes from CCTV surveillance videos. These images were flipped, translated, and cropped to obtain an expanded training dataset of 730 images with a resolution of 1920*1080. The dataset was then divided into training and validation sets at a 9:1 ratio. The training set contained 657 images, and the validation set contained 73 images. The validation set images were manually annotated to generate annotation files in JSON format, a lightweight data exchange format.

[0051] Cross-entropy loss is introduced as the loss function. During training, the learning rate is set to 0.0001 to stabilize the deep structure of the segmentation model. The batch size is set to 4, and 4 images are processed in each batch until the loss function converges to the preset value to obtain the trained segmentation model.

[0052] Step C: Mark the crack image using the Hough circle detection method to obtain the coordinates of the center of the pipe joint.

[0053] Further, step C specifically involves judging the crack image to determine whether there is a pipe joint in the crack image. If there is a pipe joint, the coordinate position of the center of the pipe joint is obtained. If there is no pipe joint, the coordinate position of the center of the crack image is marked.

[0054] Reference Figure 3 Before executing step D, preprocessing is performed to remove pixels that do not need to be counted in the crack image. Specifically, the pixels in the crack image are traversed. If there are two or more non-zero pixels on the side and below of the pixel, or if all the side pixels are zero, then the value of the pixel is set to zero.

[0055] Specifically, if pixel A is a non-zero pixel in the crack image, and there are at least three non-zero pixels on its left and right sides, bottom side, and bottom left and right sides (i.e., there are more than two non-zero pixels among pixels a1-a5), then the value of pixel A is set to 0. If the values ​​of pixels a1 and a5 are both zero, then pixel A is directly set to zero. The width of the annotation line is only one pixel.

[0056] Step D: Based on the crack image, obtain the coordinate positions of all pixels i on the crack, and calculate the length of the crack on the pipe according to the obtained center coordinate position and the coordinate positions of all pixels i on the crack.

[0057] Reference Figure 4 Let the coordinates of pixel i be (x, y, y). i ,y i The coordinates of the center O are (x, y). c ,y c Construct a circle as a pipe with the straight-line distance between point i and the center O as its radius. Calculate the circumference of this circle and compare it with the actual pipe circumference. The calculation formula is:

[0058]

[0059] Among them, C i x represents the calculated pipe circumference. c The x-coordinate of the center O is represented by y. c The ordinate of the center O is represented by x. i The x-coordinate of pixel i is represented by the y-coordinate. i C represents the ordinate of pixel i. s R represents the actual circumference of the pipe. s This indicates the actual pipe radius.

[0060] By combining the calculated pipe circumference and the actual pipe circumference, the actual length corresponding to each pixel i is calculated using the following formula:

[0061]

[0062] The actual length of the crack is obtained by calculating the actual length corresponding to each pixel i on the crack and summing the results. The calculation formula is as follows:

[0063]

[0064] Since the camera's shooting angle and image resolution affect the quantization accuracy of cracks, the more tilted the shooting angle and the lower the image resolution, the lower the crack quantization accuracy. Therefore, this invention incorporates a correction coefficient α as an adjustment to obtain the corrected actual length of the crack. The calculation formula is:

[0065]

[0066] This invention effectively combines the DeepLab V3+ semantic segmentation model and CCTV images of municipal drainage pipes, overcoming the obstacle of shooting angle to the quantification of crack length, thus realizing the function of measuring the length of cracks in the inner wall of pipes using only CCTV images.

[0067] In the verification experiment, 50 images were extracted from the real CCTV video database for verification. Among them, 25 images had a resolution of 480*260 and a correction factor α of 2.0, while the other 25 images had a resolution of 1920*1080 and a correction factor α of 1.0.

[0068] Reference Figure 5 A DeepLab V3+ network with Xception-65 as the backbone and an FCN-32s network were constructed, and together with the separation model constructed in this invention, they were used in a 12th generation... Core TM A comparative analysis was conducted on a host machine with an i7-12700KF (20 CPUs) and an NVIDIA GeForce RTX 3080 with 10GB of memory.

[0069] The calculation results using the method of this invention will be compared with the actual results from the field to verify the feasibility of the method. Since the length information of the cracks in these actual images is unknown, the crack lengths estimated using the method of this invention will be used to determine their damage level based on national and local standards for classifying crack levels in urban drainage pipes.

[0070] The calculation results of the method of the present invention are shown in the table below:

[0071] Expressing meaning

[0072]

[0073]

[0074] Experimental Results Analysis: Compared to the DeepLab V3+ model and FCN model using Xception-65 as the backbone network, the DeepLab V3+ model using MobileNet V2 as the backbone network exhibits higher accuracy. The better the performance of the semantic segmentation model, the higher the accuracy of crack length quantization. The crack quantization method of the DeepLab V3+ model using MobileNet V2 as the backbone network can correctly quantize 27 cracks in 50 cases, demonstrating high accuracy and precision.

[0075] The crack estimation method proposed in this invention works by scaling each pixel proportionally to obtain its actual length, thereby calculating the total crack length. Since the pixel size is small, the length, width, and diagonal dimensions of the pixel are considered to be consistent. Therefore, this method will be used to calculate inclined cracks, as well as horizontal and vertical cracks.

[0076] The method proposed in this invention is named DL-CLQ, DeepLab-Crack Length quantification method, which is a pipeline crack quantification method. The P-CQN method proposed in this invention has great potential for application in the quantification and identification of other defects and can be further developed and applied to other defect identification applications.

[0077] Example 2:

[0078] A computer-readable storage medium storing computer instructions for causing a computer to perform a method for estimating the length of a pipe crack based on video images, as proposed in Embodiment 1.

[0079] Example 3:

[0080] An electronic device includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform a method for estimating the length of a pipe crack based on video images, as proposed in Embodiment 1.

[0081] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for estimating the length of a pipe crack based on video images, characterized in that, Including the following steps: A. Extract images of pipes with cracks from the acquired pipe videos and create a dataset; B. Establish a segmentation model to segment the images in the acquired dataset to obtain crack images containing cracks; the segmentation model includes an encoder and a decoder connected in sequence, the encoder includes a feature extraction layer and a pooling layer connected in sequence, the feature extraction layer is MobileNet V2, used to extract low-level features, and the pooling layer is ASPPModel, used to extract high-level features; C. Analyze the crack image to determine if there is a pipe joint. If there is a pipe joint, obtain the coordinates of the center of the pipe joint. D. Based on the crack image, obtain the coordinate positions of all pixels on the crack. Calculate the length of the crack on the pipe based on the obtained center coordinate position and the coordinate positions of all pixels on the crack. In step D, the formula for calculating the crack length is: in, Indicates the crack length. Represents pixels The corresponding actual length, This represents a correction factor, which is used to adjust the impact of camera shooting angle and image resolution on the quantization accuracy of cracks; The formula for calculating crack length is as follows: in, This represents the calculated pipe circumference. Indicates the actual circumference of the pipe; The formulas for calculating the pipe circumference and the actual pipe circumference are as follows: in, Indicates the actual pipe radius. The x-coordinate representing the position of the center of the circle. The ordinate representing the position of the center of the circle. Represents pixels x-coordinate Represents pixels The ordinate.

2. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform a method for estimating the length of a pipe crack based on a video image as described in claim 1.

3. An electronic device, characterized in that, The device includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the pipe crack length estimation method based on video images as described in claim 1.

Images