Video-image-based pipeline crack length estimation method, and medium and device

Abstract

The present invention relates to the technical field of image processing, and in particular to a video-image-based pipeline crack length estimation method, and a medium and a device. The method comprises the steps of: A, extracting, from an acquired pipeline video, an image showing a crack, and making the image into a data set; B, establishing a segmentation model, and segmenting the image in the acquired data set, so as to obtain a crack image that only includes the crack; C, labeling the crack image to obtain a center coordinate position of the crack image; and D, on the basis of the crack image, acquiring the coordinate positions of all pixel points on the crack, and performing calculation on the basis of an obtained circle center coordinate position and the coordinate positions of all the pixel points on the crack, so as to obtain the length of the crack on a pipeline. The problem of a photographing angle of view hindering the quantization of the length of a crack can be overcome, thus achieving the effect of measuring the length of a crack on an inner wall of a pipeline simply by means of a pipeline detection image, and the quantization accuracy of the length of the crack can be significantly improved, thus providing a reference basis for further repairs of a drainage pipeline.

Description

A method, medium and device for estimating pipeline crack length based on video images Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for estimating the length of a pipeline crack based on video images. Background Art

[0002] Urban drainage pipelines are crucial for the normal operation of cities. They not only transport sewage and industrial wastewater, but also collect and drain urban rainfall. Due to the age of many drainage pipelines, complex underground environments, and improper operation and maintenance, pipeline ruptures are an increasingly common problem. These ruptures significantly impact the drainage capacity of the network, exacerbating the risks of sewage overflows, water pollution, and urban waterlogging, ultimately causing economic losses and even loss of life and property.

[0003] CCTV (Closed-Circuit Television) is a common method for detecting drainage pipe defects. However, with such a large amount of CCTV footage, inspectors need to spend considerable time and effort to identify defects and assess their severity. Furthermore, current standards define these defects broadly, and the same defect may be classified at different levels depending on the experience of the inspectors. Therefore, to reduce the workload of inspectors, ensure the objectivity of defect identification, and improve the standardization of defect diagnosis, various computer vision-based algorithms are gradually being applied.

[0004] Computer vision technology is a comprehensive discipline encompassing computer science, image processing, pattern recognition, and other fields. It is used to simulate human vision, observe the state of objects, and extract key information. Deep learning algorithms based on computer vision are beginning to be widely used in processing CCTV data from drainage pipes, such as object detection and semantic segmentation. Object detection technology was early applied to pipeline defect detection, such as the YOLO (You Only Look Once) series of algorithms and R-CNN (Region Convolutional Neural Networks).

[0005] In CCTV images, it can use anchor frames to mark the specific location and type of pipeline defects, significantly reducing the workload of inspectors. However, object detection technology using anchor frames cannot mark the specific form of pipeline defects.

[0006] Semantic segmentation technology, such as the U-Net and DeepLab algorithms, is beginning to be applied to pipeline defect detection. This technology can segment defects in CCTV footage, revealing their specific morphology, thereby helping inspectors assess the severity of pipeline defects. However, this method still cannot fully automate pipeline defect identification; manual assessment is still required. Furthermore, due to the complex internal environment of drainage pipes and poor imaging conditions, the application of computer vision technology to pipeline inspection remains problematic. Summary of the Invention

[0007] In response to the above-mentioned problems existing in the prior art, the present invention provides a method for quantifying pipeline crack length based on video images, which can overcome the problem of the quantification of crack length caused by the shooting angle, thereby achieving the effect of measuring the length of cracks on the inner wall of the pipeline only with the pipeline detection image, providing a reference basis for further repair of the drainage pipeline.

[0008] In a first aspect, an embodiment of the present application provides a method for quantifying pipeline crack length based on video images, comprising the steps of:

[0009] A, extract images of cracked pipes from the acquired pipeline videos and make them into a dataset;

[0010] B. Establish a segmentation model to segment the images in the acquired data set to obtain crack images containing cracks;

[0011] C, mark the crack image to obtain the center coordinate position of the crack image;

[0012] D. Based on the crack image, the coordinate positions of all pixels on the crack are obtained, and the length of the crack on the pipeline is calculated based on the obtained coordinate position of the center of the circle and the coordinate positions of all pixels on the crack.

[0013] In an optional solution of the first aspect, step C specifically comprises judging the crack image to determine whether there is a pipe joint in the crack image. If there is a pipe joint, marking the center coordinate position of the pipe joint; if there is no pipe joint, marking the center coordinate position of the crack image.

[0014] In another optional solution of the first aspect, before executing step D, preprocessing is further performed, wherein the preprocessing is to remove pixels in the crack image that do not need to be counted.

[0015] In another optional scheme of the first aspect, before executing step D, preprocessing is also performed. Specifically, the pixel points of the crack image are traversed. If there are at least three non-zero pixel points on the left and right sides, the bottom side, and the left and right sides of the bottom side of a pixel point, or if the pixel points on the left and right sides of a pixel point are all zero, the value of the pixel point is appended with zero.

[0016] In another optional solution of the first aspect, in step D, the calculation formula for the crack length is:

[0017]

[0018] in, represents the crack length, Represents pixel points The corresponding actual length, Represents the correction coefficient. Since the camera shooting angle and image resolution will affect the quantification accuracy of cracks, the more inclined the shooting angle and the lower the image resolution, the lower the crack quantification accuracy, so a correction coefficient needs to be added as an adjustment.

[0019] In another optional solution of the first aspect, in step D, the calculation formula for the crack length is specifically:

[0020]

[0021] in, represents the calculated pipe circumference, Indicates the actual pipe circumference.

[0022] In another optional solution of the first aspect, the calculated pipeline circumference and the actual pipeline circumference are calculated as follows:

[0023]

[0024] in, Indicates the actual pipe radius, The horizontal coordinate represents the center coordinate position. The vertical coordinate represents the center coordinate position, Represents pixel points The horizontal axis, Represents pixel points The vertical coordinate of .

[0025] In another optional solution of the first aspect, in step B, the established segmentation model includes an encoder and a decoder connected in sequence, and the encoder includes a feature extraction layer for feature extraction and a pooling layer for capturing features of different scales connected in sequence.

[0026] In a second aspect, an embodiment of the present application provides a computer storage medium storing a computer program, the computer program including program instructions. When the program instructions are executed by a processor, a method for estimating pipeline crack length based on video images provided in the first aspect of the embodiment of the present application or any one of the implementations of the first aspect can be implemented.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform a method for estimating pipeline crack length based on video images provided by the first aspect of the embodiment of the present application or any one of the implementations of the first aspect.

[0028] The beneficial technical effects of the present invention include:

[0029] 1. This paper proposes a pipeline crack length estimation method based on video images, breaking through the bottleneck of computer vision technology in municipal pipeline disease identification. By using pipeline exploration images, combined with the DeepLab V3+ semantic segmentation model and crack length estimation method, it can achieve the function of quantifying the length of pipeline inner wall cracks only using pipeline exploration images.

[0030] 2. The DeepLab V3+ semantic segmentation model is used. Compared with the FCN model, its crack segmentation accuracy is higher and can significantly improve the crack length quantification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] FIG1 is a flow chart of a method for estimating pipeline crack length based on video images according to the present invention;

[0033] FIG2 is a network structure diagram of the semantic segmentation model in the present invention;

[0034] FIG3 is a schematic diagram of preprocessing of a crack image in Example 1 of the present invention;

[0035] FIG4 is a diagram of a crack image based on pixel points in the first embodiment of the present invention. Calculate the pipeline circumference diagram;

[0036] FIG5 is a performance diagram of three different segmentation models in the first embodiment of the present invention. Implementation Method

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0038] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application. Different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, even though the embodiment may not be clearly described in the following text.

[0039] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the elements described without departing from the scope of the present application. Various examples may appropriately omit, replace, or add various processes or components. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted, or combined. In addition, features described in some examples may be combined in other examples. Example

[0040] 1 , a method for estimating pipeline crack length based on video images includes the following steps:

[0041] Step A: obtaining a data set, wherein the data set comes from CCTV videos, extracting images of pipelines with defects from the CCTV videos, and creating a data set.

[0042] Step B: Establish a segmentation model to segment the images in the acquired data set to obtain crack images containing only cracks.

[0043] Referring to Figure 2, the segmentation model uses a DeepLab V3+ semantic segmentation model with MobileNet V2 as the backbone. The segmentation model consists of an encoder and a decoder connected in sequence. The encoder includes a feature extraction layer and a pooling layer connected in sequence. The feature extraction layer uses MobileNet V2 to extract low-level features, while the pooling layer uses the ASPP model to extract high-level features. Images from the dataset are input to the segmentation model, which, through processing by the decoder and encoder, outputs crack images containing only cracks.

[0044] Specifically, the feature extraction layer MobileNet V2 includes two residual structures connected in sequence. The first residual structure and the second residual structure both include three convolution blocks. In the first residual structure, the convolution kernel of convolution block 1 is 1*1 with a stride of 1, the convolution kernel of convolution block 2 is 3*3 with a stride of 1, 3*3, and the convolution kernel of convolution block 3 is 1*1 with a stride of 1. In the second residual structure, the convolution kernel of convolution block 1 is 1*1 with a stride of 1, the convolution kernel of convolution block 2 is 3*3 with a stride of 1, 3*3, and the convolution kernel of convolution block 3 is 1*1 with a stride of 1. Specifically, convolution block 1 and convolution block 2 in the first residual structure and the second residual structure both include convolution layers, batch normalization layers, and activation function layers connected in sequence, and convolution block 3 in the first residual structure and the second residual structure both include convolution layers and batch normalization layers connected in sequence. The output of the first residual structure is concatenated with the input image of the feature extraction layer and then used as the input of the second residual structure.

[0045] The ASPP Model structure is added to the segmentation model, which includes multiple convolutional layers and maximum pooling layers with multiple convolution kernels of different sizes connected in sequence. This can further enhance the model's feature extraction capabilities. The purpose is to expand the receptive field, enabling the segmentation model to understand broader contextual information and address the problem of spatial information loss.

[0046] Specifically, the ASPP Model pooling layer includes five convolution blocks of different sizes arranged in parallel, including a convolution layer, a batch normalization layer, and an activation function layer connected in sequence. The convolution kernel of the first convolution block is 1*1, the step size is 1, and the residual is 1. The convolution kernel of the second convolution block is 3*3, the step size is 1, and the residual is 12. The convolution kernel of the third convolution block is 3*3, the step size is 1, the padding is 24, and the residual is 24. The convolution kernel of the fourth convolution block is 3*3, the step size is 1, the padding is 36, and the residual is 36. The convolution kernel of the fifth convolution block is 3*3, and the step size is 1. Specifically, the network structure of the fifth convolution block is an adaptive average pooling layer, a convolution layer, a batch normalization layer, an activation function layer, and a bilinear difference layer connected in sequence. The convolution results of the five convolution blocks of different sizes are channel-spliced ​​and passed through a convolution block again. The convolution kernel is 1*1 and the step size is 1. After regularization, high-level features are extracted.

[0047] The network structure of the decoder consists of five convolution blocks of different sizes connected in sequence, including a convolution layer, a batch normalization layer, and an activation function layer connected in sequence. The convolution kernel of convolution block one is 1*1, the stride is 1, and the padding is 0. The convolution kernel of convolution block two is 3*3, the stride is 1, and the padding is 1. The convolution kernel of convolution block three is 3*3, the stride is 1, and the padding is 1.

[0048] The low-level features extracted by the feature extraction layer MobileNet V2 are input to the decoder. After passing through convolution block 1, they are spliced ​​with the high-level features extracted by the pooling layer ASPP Model. They then pass through convolution blocks 2 and 3 in sequence, and finally pass through a bilinear interpolation layer to obtain the output image, i.e., the crack image.

[0049] To train the established segmentation model, we selected 133 drainage pipe images from CCTV inspection videos and performed operations such as flipping, translating, and cropping on the images. This yielded an expanded training dataset of 730 images with a resolution of 1920*1080. The dataset was split into a training set and a validation set at a ratio of 9:1, with 657 images in the training set and 73 images in the validation set. The validation set images were manually annotated to generate annotation files in JSON format, a lightweight data exchange format.

[0050] Cross entropy loss is introduced as the loss function. During the training process, the learning rate is set to 0.0001 to stabilize the deep structure of the segmentation model. The batch size is set to 4, and 4 images are processed in each batch until the loss function converges to the preset value to obtain a trained segmentation model.

[0051] Step C: marking the crack image using a Hough circle detection method to obtain the center coordinate position of the pipe joint.

[0052] Furthermore, step C specifically comprises judging the crack image to determine whether there is a pipe joint in the crack image. If there is a pipe joint, the center coordinate position of the pipe joint is obtained; if there is no pipe joint, the center coordinate position of the crack image is marked.

[0053] Referring to Figure 3, before executing step D, preprocessing is also performed. The preprocessing is to remove pixels that do not need to be included in the crack image. Specifically, the pixels of the crack image are traversed. If there are two or more non-zero pixels on the side and below the pixel point or the side pixels are all zero, the value of this pixel point is assigned zero.

[0054] Specifically, if pixel A is a non-zero pixel in the crack image and there are at least three non-zero pixels on its left and right sides, on the bottom side, and on the left and right sides of the bottom side, that is, if there are at least two non-zero pixels among pixels a1-a5, then the value of pixel A is assigned to 0. If the values ​​of pixels a1 and a5 are zero, then pixel A is directly assigned to zero. The width of the annotation line is only one pixel.

[0055] Step D: Based on the crack image, obtain all the pixels on the crack The coordinate position of the circle center and all the pixels on the crack are obtained. The coordinate position of the crack is used to calculate the length of the crack on the pipeline.

[0056] Referring to Figure 4, set pixel The coordinates are , the coordinates of the center O are Point Construct a circle as the pipeline with the straight-line distance from the center O as the radius, calculate the circumference of the circle, and compare it with the actual circumference of the pipeline. The calculation formula is:

[0057]

[0058] in, represents the calculated pipe circumference, represents the horizontal coordinate of the center O, represents the ordinate of the center O, Represents pixel points The horizontal axis, Represents pixel points The vertical coordinate, Indicates the actual pipe circumference, Indicates the actual pipe radius.

[0059] Calculate each pixel by calculating the pipe circumference and the actual pipe circumference The corresponding actual length is calculated as follows:

[0060]

[0061] By calculating each pixel on the crack The corresponding actual length is calculated and summed to obtain the actual length of the crack. The calculation formula is:

[0062]

[0063] Since the camera shooting angle and image resolution will affect the quantification accuracy of cracks, the more inclined the shooting angle and the lower the image resolution, the lower the crack quantification accuracy. Therefore, the present invention adds a correction factor As an adjustment, the actual length of the crack after correction is obtained. The calculation formula is:

[0064]

[0065] This paper effectively combines the DeepLab V3+ semantic segmentation model with CCTV images of municipal drainage pipes, overcoming the difficulty of quantifying crack length due to the shooting angle, thereby realizing the function of measuring the length of cracks on the inner wall of the pipe only with CCTV images.

[0066] Verification experiment, in the real CCTV video database, 50 images were intercepted for verification, of which 25 images had a resolution of 480*260 and the correction coefficient Set to 2.0, the resolution of the other 25 images is 1920*1080, the correction coefficient Set to 1.0.

[0067] 5 , a DeepLab V3+ network with Xception-65 as the backbone network and an FCN-32s network were constructed, and a comparative analysis was performed on a host equipped with a 12th Gen Intel® Core™ i7-12700KF (20 CPUs) and an NVIDIA GeForce RTX 3080 with 10G memory, together with the separation model constructed by the present invention.

[0068] The calculation results of the method of the present invention were compared with the actual results on site to verify the feasibility of the method of the present invention. Since the length information of the cracks in these real images is unknown, the length of the pipe cracks estimated by the method of the present invention is used to determine their damage level based on the national and local standards for the classification of crack levels in urban drainage pipes.

[0069] The calculation results of the method of the present invention are shown in the following table:

[0070] Express intention

[0071]

[0072] Experimental results show that compared to the DeepLab V3+ model and the FCN model using the Xception-65 backbone, the DeepLab V3+ model using the MobileNet V2 backbone achieves higher accuracy. The better the semantic segmentation model, the higher the accuracy of crack length quantization. The crack quantization method of the DeepLab V3+ model using the MobileNet V2 backbone correctly quantized 27 cracks out of 50 cases, demonstrating high accuracy and precision.

[0073] The crack estimation method proposed in this paper works by scaling each pixel to determine its actual length, thereby calculating the total crack length. Because pixels are small, their length, width, and diagonal dimensions are assumed to be consistent. Therefore, this method can be used to calculate both inclined cracks and horizontal and vertical cracks.

[0074] The proposed method is named DL-CLQ, short for DeepLab-Crack Length quantification, a pipeline crack quantification method. The proposed P-CQN method has great potential for quantitative identification of other defects and can be further developed and applied to other defect identification applications.

[0075] Example 2:

[0076] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute a pipeline crack length estimation method based on video images as proposed in the first embodiment.

[0077] Example 3:

[0078] An electronic device includes a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform a pipeline crack length estimation method based on video images as proposed in Example 1.

[0079] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection of the present invention.

Claims

1. A pipeline crack length estimation method based on video images, characterized in that: Includes steps: A, extract images of cracked pipes from the acquired pipeline videos and make them into a dataset; B, establish a segmentation model to segment the images in the acquired data set to obtain crack images containing cracks; C, mark the crack image to obtain the center coordinate position of the crack image; D. Based on the crack image, the coordinate positions of all pixels on the crack are obtained, and the length of the crack on the pipeline is obtained by calculating the coordinate position of the center of the circle and the coordinate positions of all pixels on the crack.

2. The method for estimating pipeline crack length based on video images according to claim 1, characterized in that: Step C specifically comprises judging the crack image to determine whether there is a pipe joint in the crack image. If there is a pipe joint, the center coordinate position of the pipe joint is marked. If there is no pipe joint, the center coordinate position of the crack image is marked.

3. The method for estimating pipeline crack length based on video images according to claim 1, characterized in that: Before executing step D, preprocessing is performed to remove abnormal pixels in the crack image.

4. A pipeline crack length estimation method based on video images according to any one of claim 3, characterized in that: Before executing step D, preprocessing is also performed, specifically, traversing the pixel points of the crack image. If there are at least three non-zero pixel points on the left and right sides, the bottom side, and the left and right sides of the bottom side of a pixel point, or if the pixel points on the left and right sides of a pixel point are all zero, the value of the pixel point is appended with zero.

5. The method for estimating pipeline crack length based on video images according to claim 1, characterized in that: In step D, the calculation formula for the crack length is: ,in, represents the crack length, Represents pixel The corresponding actual length, Indicates the correction factor.

6. The method for estimating pipeline crack length based on video images according to claim 5, characterized in that: In step D, the calculation formula for the crack length is specifically: ， in, represents the calculated pipe circumference, Indicates the actual pipe circumference.

7. The method for estimating pipeline crack length based on video images according to claim 6, characterized in that: The calculation formula for the calculated pipe circumference and the actual pipe circumference is: ， in, Indicates the actual pipe radius, The horizontal coordinate represents the center coordinate position. The ordinate represents the center coordinate position, Represents pixel The horizontal axis of Represents pixel The vertical coordinate of .

8. The method for estimating pipeline crack length based on video images according to claim 1, characterized in that: In step B, the segmentation model established includes an encoder and a decoder connected in sequence, and the encoder includes a feature extraction layer for feature extraction and a pooling layer for capturing features of different scales connected in sequence.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the pipeline crack length estimation method based on video images according to any one of claims 1 to 8.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory and the processor are in communication with each other, the memory stores computer instructions, and the processor executes the method for estimating the length of a pipeline crack based on video images as claimed in any one of claims 1 to 8 by executing the computer instructions.

Images