Underground drainage pipeline defect identification method based on double attention mechanism and receptive field expansion

Abstract

The application relates to an underground drainage pipeline defect identification method based on a double attention mechanism and a receptive field expansion, and the method comprises the following steps: S1, collecting image data and constructing an underground drainage pipeline defect data set; S2, building an SSD convolutional neural network and improving the SSD network in combination with underground drainage pipeline defect features; S3, performing data enhancement on the data set to expand the training data set, setting hyperparameters in the convolutional neural network, and training the model; and S4, reading test images by taking the trained improved SSD target detection model as a test network, and outputting the results of pipeline defect target identification. According to the technical scheme, a detection model based on deep learning is constructed for the specific scene of the underground drainage pipeline, the detection efficiency is high, the detection precision is greatly improved, the detection speed meets the real-time demand, and abundant and convenient basic data can be provided for the maintenance and care of the underground drainage pipeline.

Description

Technical Field

[0001] This invention relates to the field of underground drainage pipeline defect detection technology, and in particular to a method for identifying underground drainage pipeline defects based on a dual attention mechanism and receptive field amplification. Background Technology

[0002] Underground drainage pipes are a vital infrastructure for ensuring the smooth operation of cities, playing a crucial role in the functioning of modern urban areas. As cities expand, underground drainage pipes become increasingly dense, and with increasing usage time, various types of damage gradually occur, necessitating regular inspections to prevent sudden pipe collapses or functional degradation. Faced with the dense network of underground drainage pipes and the heavy workload of inspection and maintenance, the traditional method involved manually controlling closed-circuit television (CCTV) equipment to enter the underground drainage pipes and film their interiors. The recorded video footage was then evaluated and analyzed frame by frame to determine the presence of pipe defects. This process required workers to watch the video data for extended periods, and the accuracy of manual identification decreased with prolonged viewing time, making it extremely labor-intensive. While CCTV pipe defect detection technology is widely popular, it still relies on manual identification and cannot achieve intelligent defect recognition. With the development of deep learning, it has shown great potential in target defect identification, but research on its application to urban underground drainage pipe defect identification remains very limited. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of the existing technology by providing a method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field amplification. Specifically designed for underground drainage pipes, this invention constructs a deep learning-based detection model, achieving high detection efficiency, significantly improved detection accuracy, and real-time detection speed. This method also provides rich and convenient basic data for the maintenance and upkeep of underground drainage pipes.

[0004] Technical Solution: To achieve the above objectives, this invention proposes a method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field amplification. This method includes the following steps:

[0005] S1. Collect image data and construct a dataset of defects in underground drainage pipes;

[0006] S2. Construct an SSD convolutional neural network and improve the SSD network by combining the defect characteristics of underground drainage pipes;

[0007] S3. Perform data augmentation on the dataset to expand the training dataset, and set the hyperparameters in the convolutional neural network to train the SSD convolutional neural network.

[0008] S4. The trained and improved SSD convolutional neural network is used as the test network to read the test image and output the result of the pipeline defect target identification.

[0009] Furthermore, in step S1, the pipeline defect image is obtained by recording video inside the underground drainage pipeline using closed-circuit television equipment, and extracting video frames with defects from the video data as defect image data.

[0010] Furthermore, in step S1, defects in the image data are classified and labeled. The labeling content includes defect category and location. The labeling is performed using the Python-based labelme tool, and the VOC format dataset is output as an underground drainage pipe defect dataset.

[0011] Furthermore, in step S2, the method for improving the SSD network is as follows: the SeNet attention mechanism module is improved and added to the conv2_1 layer of the original SSD network; the RFB receptive field expansion module is improved and added to the conv7 layer of the original SSD network.

[0012] Furthermore, the improved SeNet network refers to improvements made to the SeNet network in two aspects: First, it uses a dual activation operation based on fully connected layers and convolutional layers to further enhance the calculation of the importance of different feature layers; second, it adopts a residual connection method to directly pass the input feature layer of the SeNet network to the output feature layer, and adds it to the output layer after multiple layers of calculation. After adding the improved SeNet network to the conv2_1 layer of the SSD network structure, the specific process of the improved SeNet network is as follows:

[0013] Global average pooling is performed on the feature map X output by the conv2_1 layer of the SSD network structure to obtain the global descriptor S:

[0014] S = (AvgPool(X))

[0015] In the formula, AvgPool represents the average pooling layer;

[0016] The first activation function E1 is obtained by passing two fully connected layers and the sigmoid activation function, and the second activation function E2 is obtained by passing two convolutional layers with a kernel size of 1 and the sigmoid activation function.

[0017] E1=σF2(δF1(S))

[0018] E2=σC2(δC1(E1))

[0019] In the formula, F1 and F2 are two fully connected layers, C1 and C2 are two convolutional layers with a kernel of 1, δ represents the ReLU non-linear activation function, and σ represents the Sigmoid non-linear activation function. After two activation operations, the input feature map X is multiplied by the activation function E2 to obtain the weighted feature map X′:

[0020]

[0021] In the formula, This indicates element-wise multiplication between matrices;

[0022] In addition to the feature map X, a residual connection is formed to obtain the output feature X″ containing weight information:

[0023]

[0024] In the formula, This represents the addition of corresponding elements between matrices;

[0025] The output feature map X″ is input into the conv2_2 layer of the SSD network.

[0026] Further improvements to the RFB network are as follows: two convolutional layers with kernels of 3 are used to replace the convolutional layers with kernels of 5 in the original network. The input feature layer of the RFB network is directly passed to the output feature layer by using residual connections, and then added to the output layer after multiple layers of computation. This is then added to the conv7 layer of the original SSD network.

[0027] Furthermore, in step S3, the dataset is randomly divided into two parts: one part is the training set, which is used to train the SSD convolutional neural network after data augmentation; the other part is the test set, which is used to test the SSD convolutional neural network.

[0028] Furthermore, in step S3, data augmentation includes flipping the image, adjusting its grayscale value, and scaling it up or down to expand its training set.

[0029] Furthermore, in step S3, the hyperparameters include batch size, initial learning rate, termination learning rate, optimizer setting, and training epochs; the hyperparameters are set to the batch size as 16, the initial learning rate as 0.01, the termination learning rate as 0.0001, the optimizer as SGD, and the training epochs as 600.

[0030] Furthermore, in step S4, the model detection results include defect category and location information.

[0031] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0032] This invention provides a defect identification method for underground drainage pipes based on a dual attention mechanism and receptive field amplification. For the specific scenario of underground drainage pipes, a detection model based on deep learning is constructed, which has high detection efficiency, greatly improves detection accuracy, and meets the real-time requirements of detection speed. It can provide rich and convenient basic data for the maintenance and upkeep of underground drainage pipes. Attached image description:

[0033] Figure 1 This is a flowchart of the underground drainage pipe defect identification method provided by the present invention;

[0034] Figure 2 These are training sample data on defects in underground drainage pipes;

[0035] Figure 3 This is a diagram of the improved SeNet network structure;

[0036] Figure 4 This is a diagram of the improved RFB network structure;

[0037] Figure 5 This is the network structure diagram of the improved SSD algorithm. Detailed Implementation

[0038] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0039] like Figure 1 As shown, this invention proposes a method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion. The method includes the following steps:

[0040] S1. Collect image data and construct a dataset of defects in underground drainage pipes;

[0041] S2. Construct an SSD convolutional neural network and improve the SSD network by combining the defect characteristics of underground drainage pipes;

[0042] S3. Perform data augmentation on the dataset to expand the training dataset, and set the hyperparameters in the convolutional neural network to train the SSD convolutional neural network.

[0043] S4. The trained and improved SSD convolutional neural network is used as the test network to read the test image and output the result of the pipeline defect target identification.

[0044] Furthermore, in step S1, the pipeline defect image is obtained by recording video inside the underground drainage pipeline using closed-circuit television equipment, and extracting video frames with defects from the video data as defect image data.

[0045] Further, in step S1, defects in the image data are classified and labeled, with the labeling content including defect category and location. The labeling is performed using the Python-based LabelMe tool, and the VOC format dataset is output as an underground drainage pipe defect dataset. The defects in the image data are classified and labeled to create a multi-label classification dataset; the classification and labeling of defects in the image data, with the labeling content including defect category and location, is performed using the Python-based LabelMe tool, and the resulting data is finally used as an underground drainage pipe defect dataset. Specifically, the pipe defects include eight categories: deformation, cracks, tree roots, sediment, concealed joints, insertions, residual walls, and misalignments. Figure 2 As shown.

[0046] Furthermore, in step S2, the method for improving the SSD network is as follows: the SeNet attention mechanism module is improved and added to the conv2_1 layer of the original SSD network; the RFB receptive field expansion module is improved and added to the conv7 layer of the original SSD network. Figure 5 The improved SSD algorithm network structure diagram is shown.

[0047] Furthermore, such as Figure 3 As shown, the improved SeNet network refers to improvements made to the SeNet network in two aspects: First, it uses a dual activation operation based on fully connected layers and convolutional layers to further enhance the calculation of the importance of different feature layers; second, it adopts a residual connection method to directly pass the input feature layer of the SeNet network to the output feature layer, and adds it to the output layer after multiple layers of calculation. After adding the improved SeNet network to the conv2_1 layer of the SSD network structure, the specific process of the improved SeNet network is as follows:

[0048] Global average pooling is performed on the feature map X output by the conv2_1 layer of the SSD network structure to obtain the global descriptor S:

[0049] S = (AvgPool(X))

[0050] In the formula, AvgPool represents the average pooling layer;

[0051] The first activation function E1 is obtained by passing two fully connected layers and the sigmoid activation function, and the second activation function E2 is obtained by passing two convolutional layers with a kernel size of 1 and the sigmoid activation function.

[0052] E1=σF2(δF1(S))

[0053] E2=σC2(δC1(E1))

[0054] In the formula, F1 and F2 are two fully connected layers, C1 and C2 are two convolutional layers with a kernel of 1, δ represents the ReLU non-linear activation function, and σ represents the Sigmoid non-linear activation function. After two activation operations, the input feature map X is multiplied by the activation function E2 to obtain the weighted feature map X′:

[0055]

[0056] In the formula, This indicates element-wise multiplication between matrices;

[0057] In addition to the feature map X, a residual connection is formed to obtain the output feature X″ containing weight information:

[0058]

[0059] In the formula, This represents the addition of corresponding elements between matrices;

[0060] The output feature map X″ is input into the conv2_2 layer of the SSD network.

[0061] Furthermore, such as Figure 4 As shown, the improvement to the RFB network is as follows: two convolutional layers with kernels of 3 are used to replace the convolutional layers with kernels of 5 in the original network. The input feature layer of the RFB network is directly passed to the output feature layer by using residual connections, and then added to the output layer after multiple layers of computation. This is then added to the conv7 layer of the original SSD network.

[0062] Furthermore, in step S3, the dataset is randomly divided into two parts: one part is the training set, which is used to train the SSD convolutional neural network after data augmentation; the other part is the test set, which is used to test the SSD convolutional neural network.

[0063] Furthermore, in step S3, data augmentation includes flipping the image, adjusting its grayscale value, and scaling it up or down to expand its training set.

[0064] Furthermore, in step S3, the hyperparameters include batch size, initial learning rate, termination learning rate, optimizer setting, and training epochs; the hyperparameters are set to the batch size as 16, the initial learning rate as 0.01, the termination learning rate as 0.0001, the optimizer as SGD, and the training epochs as 600.

[0065] Furthermore, in step S4, the model detection results include defect category and location information.

[0066] For the final defect identification results of underground drainage pipes, the experiment used the mean average precision (mAP), a commonly used metric in target detection algorithms, as the evaluation index. mAP is obtained by summing the mean precision (AP) of all categories, dividing by the number of categories, and taking the average, as shown in the formula. The effectiveness of the model algorithm is measured by the mAP value; that is, the higher the mAP value, the better the model's detection performance.

[0067]

[0068] In the formula, k is the number of categories, AP j Let be the average precision of the j-th class.

[0069] It should be noted that the above description of the embodiments is only for the purpose of helping to understand the method and core idea of ​​this application. For those skilled in the art, several improvements and modifications can be made to this application without departing from the principle of this application, and these improvements and modifications are also within the protection scope of the claims of this application.

Claims

1. A method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion, characterized in that, The method includes the following steps: S1. Collect image data and construct a dataset of defects in underground drainage pipes; S2. Construct an SSD convolutional neural network and improve the SSD network by combining the defect characteristics of underground drainage pipes; S3. Perform data augmentation on the dataset to expand the training dataset, and set the hyperparameters in the convolutional neural network to train the SSD convolutional neural network. S4. The trained and improved SSD convolutional neural network is used as the test network to read the test image and output the result of the pipeline defect target identification. In step S2, the method for improving the SSD network is as follows: the SeNet attention mechanism module is improved and added to the conv2_1 layer of the original SSD network; the RFB receptive field expansion module is improved and added to the conv7 layer of the original SSD network. The improved SeNet network refers to improvements made to the SeNet network in two aspects: First, it uses a dual activation operation based on fully connected layers and convolutional layers to further enhance the calculation of the importance of different feature layers; second, it adopts a residual connection method to directly pass the input feature layer of the SeNet network to the output feature layer, and adds it to the output layer after multiple layers of calculation. After adding the improved SeNet network to the conv2_1 layer of the SSD network structure, the specific process of the improved SeNet network is as follows: Feature maps output from the conv2_1 layer of the SSD network structure Perform global average pooling to obtain the global descriptor S: ； In the formula, AvgPool represents the average pooling layer; The first activation function is obtained through two fully connected layers and the sigmoid activation function. The second activation function is obtained by using two convolutional layers with kernels of 1 and the sigmoid activation function. : ； ； In the formula, and These are two fully connected layers, and These are two convolutional layers with kernels of 1, and δ represents the ReLU nonlinear activation function. Let X represent the sigmoid nonlinear activation function; after two activation operations, the input feature map X is multiplied by the activation function E2 to obtain the weighted feature map. : ； In the formula, This indicates element-wise multiplication between matrices; In addition to feature maps This forms a residual connection, yielding output features containing weight information. : ； In the formula, This represents the addition of corresponding elements between matrices; Output feature map Input to the conv2_2 layer of the SSD network; The improvements to the RFB network are as follows: two convolutional layers with a kernel of 3 are used to replace the convolutional layer with a kernel of 5 in the original network. The input feature layer of the RFB network is directly passed to the output feature layer by using residual connections, and then added to the output layer after multiple layers of computation. This is then added to the conv7 layer of the original SSD network.

2. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion as described in claim 1, characterized in that, In step S1, the pipeline defect image is obtained by recording video inside the underground drainage pipe using closed-circuit television equipment, and extracting video frames with defects from the video data as defect image data.

3. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion as described in claim 1, characterized in that, In step S1, defects in the image data are classified and labeled. The labeling content includes defect category and location. The labeling is performed using the Python-based labelme tool, and the VOC format dataset is output as an underground drainage pipe defect dataset.

4. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion as described in claim 1, characterized in that, In step S3, the dataset is randomly divided into two parts: one part is the training set, which is used to train the SSD convolutional neural network after data augmentation; the other part is the test set, which is used to test the SSD convolutional neural network.

5. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field amplification as described in claim 1, characterized in that, In step S3, data augmentation includes flipping the image, adjusting its grayscale value, and scaling it up or down to expand its training set.

6. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion as described in claim 1, characterized in that, In step S3, the hyperparameters include batch size, initial learning rate, termination learning rate, optimizer setting, and training epochs; the hyperparameters are set to the batch size as 16, the initial learning rate as 0.01, the termination learning rate as 0.0001, the optimizer as SGD, and the training epochs as 600.

7. The method for identifying defects in underground drainage pipes based on a dual attention mechanism and receptive field expansion as described in claim 1, characterized in that, In step S4, the model detection results include defect category and location information.

Images