Tunnel water leakage detection method based on data enhancement and multi-constraint loss function

Abstract

The application provides a tunnel water leakage detection method based on data enhancement and a multi-constraint loss function, and the method comprises the following steps: S1, adopting an inlaid CutMix method to perform data enhancement on a tunnel image, and splicing and fusing to form a new sample with comprehensive characteristics; S2, taking a YOLOv7 network as a skeleton structure, and introducing an efficient channel attention module; S3, using a loss function of a multi-constraint geometric condition to improve the output result. The application improves the autonomous learning and expression ability of the key features of the water leakage and the accuracy of the geometric shape of the prediction frame, and has good robustness and generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a tunnel water leakage detection method based on data enhancement and a multi-constraint loss function. BACKGROUND

[0002] Metro is an important part of urban rail transit system. With the continuous expansion of the city, the mileage of metro shield tunnels increases year by year, and tunnel diseases also occur in large numbers. Water leakage is an important component of tunnel diseases, and the water leakage problem of metro shield tunnels becomes increasingly prominent as the operation time increases. Water leakage directly erodes the internal structure of the tunnel, damages cables and ancillary devices, and poses a great hidden danger to train safety. Therefore, efficient and accurate water leakage detection technology plays a crucial role in the safe operation of shield tunnels.

[0003] Tunnel image acquisition methods include manual photography or mobile scanning technology. After obtaining the images, the target recognition and detection method is used to extract the water leakage disease distribution. Traditional target recognition methods use manual feature design to recognize water leakage areas by designing geometric shapes, textures, and gray level semantic features, such as using gray stretch and edge detection, region growing algorithm, etc. In the case of no occlusion and good light, such methods can effectively detect, but the algorithm requires more prior knowledge and is difficult to solve the detection task under the condition of occlusion and weak light.

[0004] With the rapid development of deep learning in target detection, deep networks can more autonomously learn multi-semantic and high-level features. Target detection based on convolutional neural networks (CNN) can be divided into single-stage networks that directly locate and classify, and two-stage networks that first find target regions and then classify and locate. Single-stage networks include OverFeat, You Only Look Once (YOLO) series, Single Shot MultiBox Detector (SSD), etc., which have the advantage of fast detection speed; two-stage networks include SPPNet, Faster R-CNN, Mask R-CNN, etc., which are time-consuming but have high accuracy. Water leakage detection has large target size differences and high similarity to the background, and existing methods have low detection accuracy, especially for small size water leakage. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of existing methods and provide a tunnel water leakage detection method based on data enhancement and a multi-constraint loss function.

[0006] To this end, the present application adopts the following technical solutions:

[0007] A tunnel water leakage detection method based on data enhancement and multi-constraint loss function, comprising the following steps:

[0008] S1, adopting the mosaic CutMix method to perform data enhancement on the tunnel image, and splicing and fusing to form a new sample with comprehensive characteristics

[0009] S2, the new sample obtained in S1 is input into a YOLOv7 deep learning network ;

[0010] S3, a loss function of multi-constraint geometric conditions is designed at the output end of the YOLOv7 deep learning network, which is used to improve the prediction box result, specifically:

[0011] Taking IoU, center point distance and length-width ratio of the minimum bounding rectangle as the loss function constraint conditions of the water leakage target detection, the calculation formula is as follows:

[0012] L=L IoU +L p2p +L asp ;

[0013]

[0014]

[0015] Wherein, L represents the loss function of the improved YOLOv7, L IoU , L p2p , L asp respectively represent the IoU, center point distance and MBR length-width ratio between the prediction box and the real box, b, b gt respectively represent the center points of the prediction box and the real box, ρ represents the Euclidean distance between the two points, C represents the diagonal line of the minimum closure region containing the prediction box and the real box, C w , C h is the width and height of the MBR containing the two boxes, h, h gt respectively represent the height of the prediction box and the real box, w, w gt respectively represent the width of the prediction box and the real box.

[0016] Thus, the improved prediction box is obtained.

[0017] Preferably, S1 is specifically:

[0018] The CutMix-based data enhancement method is used to combine two training samples (x A , y A ) and (x B , y B ) to generate a new training sample x∈RW×H×C and y respectively represent the shield tunnel intensity image and the shield tunnel water leakage label, first cut the same size region B A and x B The boundary box coordinates of region B A and region B B , the boundary box coordinates of region B A and region B B (r x , r y , r w , r h ); remove region B A in x A , and fill it with region B B in x B to form a new sample The calculation formula is as follows:

[0019]

[0020] In the formula, M is in {0, 1} W×H , a binary mask representing the position of the same size region B in the two images, is an element-wise multiplication, λ is sampled from a uniform distribution (0, 1), W represents the width of the image, H represents the height of the image, and C represents the channel of the image.

[0021] Preferably, S2 is specifically:

[0022] YOLOv7 network as the skeleton structure, introduce efficient channel attention module ECA, efficient layer aggregation network ELAN, spatial pyramid pooling full connection spatial pyramid convolution SPPCSPC.

[0023] Preferably, the adaptive attention weight in the ECA module is calculated by the following formula:

[0024]

[0025] Where |t| odd Indicates the nearest odd number of t, and in this paper, γ and b are respectively set to 2 and 1, and c is the number of input channels.

[0026] The tunnel water leakage detection method based on data enhancement and multi-constraint loss function has the following advantages:

[0027] 1. The present application can completely identify the water leakage stripes hidden by other objects, clearly identify the water leakage and provide accurate position information, improve the accuracy of the prediction box, and improve the model prediction accuracy and robustness, and has good prediction results.

[0028] 2. The application improves the key feature learning ability of the target detection network and improves the detection accuracy of water leakage disease targets by using the CutMix method for data enhancement of tunnel images. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the tunnel water leakage detection method of the application;

[0030] Figure 2 is a CutMix-based shield tunnel image data enhancement schematic diagram in the application;

[0031] Figure 3 is an improved YOLOv7 network structure diagram in the application;

[0032] Figure 4 is a multi-constraint geometric condition schematic diagram in the application;

[0033] Figure 5 is a prediction box change diagram before and after adding a multi-constraint geometric condition loss function in the embodiment of the application;

[0034] Figure 6 is a part of the sample diagram based on CutMix data enhancement in the embodiment of the application;

[0035] Figure 7 is a detail diagram of shield tunnel image recognition using Faster R-CNN technology;

[0036] Figure 8 is a detail diagram of shield tunnel image recognition using SSD technology;

[0037] Figure 9 is a detail diagram of shield tunnel image recognition using YOLOv5 technology;

[0038] Figure 10 is a detail diagram of shield tunnel image recognition using YOLOv7 technology;

[0039] Figure 11 is a detail diagram of shield tunnel image recognition using the technology of the application;

[0040] Figure 12 is the recognition of large-area water seepage shield tunnel images using Faster R-CNN technology;

[0041] Figure 13 is the recognition of large-area water seepage shield tunnel images using SSD technology;

[0042] Figure 14 is the recognition of large-area water seepage shield tunnel images using YOLOv5 technology;

[0043] Figure 15 is the recognition of large-area water seepage shield tunnel images using YOLOv7 technology;

[0044] Figure 16 is the recognition of large-area water seepage shield tunnel images using the technology of the application;

[0045] Figure 17 is the recognition of insufficient light shield tunnel images using Faster R-CNN technology;

[0046] Figure 18 is the recognition of insufficient light shield tunnel images using SSD technology;

[0047] Figure 19 is the recognition of insufficient light shield tunnel images using YOLOv5 technology;

[0048] Figure 20 is the recognition of insufficient light shield tunnel images using YOLOv7 technology;

[0049] Figure 21 is the recognition of insufficient light shield tunnel images using the technology of the application;

[0050] Figure 22 is the recognition of good light shield tunnel images using Faster R-CNN technology;

[0051] Figure 23 is the recognition of good light shield tunnel images using SSD technology;

[0052] Figure 24 is the recognition of good light shield tunnel images using YOLOv5 technology;

[0053] Figure 25 is the recognition of good light shield tunnel images using YOLOv7 technology;

[0054] Figure 26 is the recognition of good light shield tunnel images using the technology of the application. DETAILED DESCRIPTION

[0055] The technical solutions of the application will be further described in detail below in combination with the drawings and examples.

[0056] As shown in Figure 1 , the tunnel water leakage detection method based on data enhancement and multi-constraint loss function of the application comprises:

[0057] S1, using the mosaic CutMix method to perform data enhancement on the tunnel image, splicing and fusing into a new sample with comprehensive features.

[0058] Referring to Figure 2 , let x∈R W×H×C and y respectively represent the shield tunnel intensity image and the shield tunnel water leakage label. Two training samples (x A , y A ) and (x B , y B ) are combined to generate a new training sample has:

[0059]

[0060] In the formula, M∈{0,1} W×H is a binary mask representing the position of region B in the two images, ⊙ represents element-wise multiplication, λ is sampled from a uniform distribution (0, 1), W represents the width of the image, H represents the height of the image, and C represents the channel of the image.

[0061] Through the calculation of formula (1), first, the same size of region B A and region B B on x A and x B are cut A The bounding box coordinates of region B B and region B x are (r y , r w , r h ). Remove region B A in x A , and fill it with region B B in x B to form a new sample

[0062] S2, taking YOLOv7 network as the skeleton structure, introduces an efficient channel attention module, specifically:

[0063] YOLOv7 is a single-stage classic target detection deep learning network, and the network structure mainly includes three parts of Backbone, Neck and Head. In view of the problems that the sizes of the seepage water are inconsistent and the pixel ratio is generally small, and the key feature learning ability is relatively weakened, the application introduces an efficient channel attention (ECA) mechanism and a loss function with multiple constraint geometric conditions, embeds the ECA into the backbone structure, improves the extraction and expression ability of the key features, and combines the loss function with multiple constraint geometric conditions into the Head structure, improves the single intersection over union loss (IOU Loss), and guides the parameter update in the backward propagation process. The network of the application uses the multiple constraint loss function in the training stage to constrain the center point position and size of the prediction box, so as to limit the range of the prediction result and improve the result precision of the prediction box. The network structure is as shown in Figure 3 The ELAN represents an efficient layer aggregation network (ELAN). The steps are as follows: inputting the seepage water training sample, first expanding the sample through the CutMix-based data enhancement method; and secondly inputting the expanded sample into the Backbone, Neck and Head to obtain the seepage water prediction result.

[0064] As shown in Figure 3 It can be seen that the improved YOLOv7 network is obtained after the continuous three convolution, normalization and activation processes, and then input into the ECA module, which can be divided into the following steps:

[0065] Firstly, the convolution layer is used to obtain the seepage water features to generate a feature map with the same number of channels; secondly, the adaptive attention weight is calculated according to formula (2) to express in the form of a scalar; then the attention weight is obtained through a single 1×1 convolution layer, which represents the relative importance of each channel in the feature vector; finally, the output result with the same channel as the input feature is obtained through the channel-by-channel feature fusion calculation.

[0066]

[0067] In the formula, |t| odd represents the nearest odd number of t, and in the embodiment, γ and b are respectively set to 2 and 1, and c is the number of input channels.

[0068] It is clear that the function ψ gives greater channels a long-range interaction, and vice versa.

[0069] S3, a loss function of multi-constrained geometric conditions is designed, specifically:

[0070] There is a large aspect ratio gap in the seepage water image, and the classic IoU loss function may have a low IoU calculation value in the training process, thereby affecting the effective update of the backward propagation parameters, causing the seepage water prediction result to have a certain degree of false detection and missed detection problem. In view of this problem, the present application designs a loss function of the distance between the multi-constrained prediction frame and the real frame and the size, and takes the IoU, the center point distance and the aspect ratio of the minimum bounding rectangle (MBR) as the loss function constraint condition of the seepage water target detection, the principle is as shown in Figure 4 , the change of the prediction frame before and after improvement is as shown in Figure 5 , the calculation method is as shown in formulas (3), (4) and (5).

[0071] L = L IoU + L p2p + L asp (3)

[0072]

[0073]

[0074] Wherein L represents the loss function of improved YOLOv7; L IoU , L p2p , L asp respectively represent the IoU, the center point distance and the MBR aspect ratio between the prediction frame and the real frame; b, b gt respectively represent the center points of the prediction frame and the real frame; ρ represents the Euclidean distance between two points; C represents the diagonal length of the minimum closed region containing the prediction and real frame; C w , C h is the width and height of the MBR composed of two frames; h, h gt respectively represent the height of the prediction frame and the real frame; w, w gt respectively represent the width of the prediction frame and the real frame.

[0075] Embodiment

[0076] To verify the technical effect of the YOLOv7 shield tunnel water leakage detection method based on CutMix data enhancement and multi-constraint loss function compared with existing methods, select a segment of subway MLS point cloud data as experimental data. The comparison methods are: two-stage Faster R-CNN, single-stage SSD, YOLOv5, and original YOLOv7. In order to quantitatively evaluate the accuracy of the algorithm, the artificially labeled water leakage samples are used as the true value, and the precision (AP) measurement index is used to evaluate the algorithm accuracy, as shown in formula (6):

[0077]

[0078] In the formula, FP represents the negative samples predicted as positive by the model, including missed detection and false detection, FN represents the positive samples predicted as negative by the model, TP represents the positive samples predicted as positive by the model, and TN represents the negative samples predicted as negative by the model. The comparative analysis data is shown in the following table:

[0079] Table 1 Comparison table of the present application and the comparison method

[0080]

[0081] As shown in Figure 6 , some of the samples contain information about the water leakage area, effectively increasing the diversity of the samples and improving the proportion of positive samples in the overall samples, which helps the convergence of the model network.

[0082] As shown in Figures 7 to 26 , Faster R-CNN algorithm has better detection effect than SSD algorithm in good or bad light, while SSD algorithm has missed detection. The reason is that Faster R-CNN is a two-stage target detection algorithm of region recommendation and target classification, which adopts pre-defined anchor frame shape and scale to detect targets. The calculation is time-consuming but the accuracy is high; in addition, the single-stage SSD algorithm has poor detection effect on small target due to its multi-scale pyramid feature extraction structure.

[0083] YOLOv7 has better detection effect than YOLOv5, and the missed detection is significantly less than YOLOv5. YOLOv7 contains an efficient aggregation network (ELAN), which controls the shortest and longest gradient path, so that the network can learn more rich features and has good noise suppression ability. YOLOv5 has relatively weak detection performance on small targets, because its network structure adopts multi-layer down-sampling operation, which may cause the gradual loss of small target feature information in network training, thereby reducing the accuracy and recall rate of part of the water leakage identification.

[0084] Compared with the two-stage Faster R-CNN and the single-stage original YOLOv7 algorithm, the algorithm in this paper can not only identify the water leakage stripes that are covered by the interference such as occlusion, but also clearly identify the water leakage and provide accurate location information, with good detection accuracy and prediction results. This is because the detection algorithm in this paper introduces ECA and the loss function of multiple constraint geometric conditions, which can better extract the key features of the water leakage pixel part, and in the backward propagation, the original IoU and the center point distance and MBR length-width constraint are used. As can be seen from the large area water leakage shield tunnel image and the detail image, the algorithm in this paper can effectively extract the multi-scale water leakage target feature information in the image, and realize the high-precision extraction of the prediction frame.

Claims

1. A tunnel water leakage detection method based on data enhancement and multi-constraint loss function, characterized in that, Comprise the following steps: S1, the mosaic CutMix method is used for data enhancement of the tunnel image, and new samples with comprehensive characteristics are spliced and fused S2, the new sample obtained from S1 adding a YOLOv7 deep learning network; S3, a loss function of multiple constraint geometric conditions is designed at the output end of the YOLOv7 deep learning network, which is used to improve the prediction box result, specifically: Taking IoU, center point distance and the length-width of the minimum outer rectangle as the loss function constraint conditions of the leakage water target detection, the calculation formula is as shown in the following formula: L = L IoU + L p2p + L asp ; wherein L represents a loss function for improving YOLOv7, L IoU , L p2p , L asp respectively represent the IoU between the predicted frame and the real frame, the center point distance and the MBR aspect ratio, b, b gt respectively represent the center points of the predicted frame and the real frame, ρ represents the Euclidean distance between two points, C represents the diagonal line of the minimum closed region containing the predicted frame and the real frame, C w , C h is the width and height of the MBR containing the two frames, h, h gt respectively represent the height of the predicted frame and the real frame, w, w gt respectively represent the width of the predicted frame and the real frame; Thus, the improved prediction box is obtained.

2. The method of claim 1, wherein, S1 is specifically: A data augmentation method based on CutMix is used to combine two training samples (x A , y A ) and (x B , y B ) to generate a new training sample and y respectively represent the strength image of the shield tunnel and the leakage water label of the shield tunnel, first cut the same size area B A and area B B on x A and x B , the boundary box coordinates of area B A and area B B are (r x , r y , r w , r h ); remove area B A in x A , and fill it with area B B in x B to form a new sample The calculation formula is as follows: where M ∈ {0, 1} W×H where B denotes the binary mask representing the location of the region of the same size in two images, denotes the element-wise multiplication, λ is sampled from a uniform distribution (0, 1), W denotes the width of the image, H denotes the height of the image, and C denotes the channel of the image.

3. The method of claim 2, wherein, S2 is specifically: Taking YOLOv7 network as the skeleton structure, introducing efficient channel attention module ECA, efficient layer aggregation network ELAN, spatial pyramid pooling full connection spatial pyramid convolution SPPCSPC.

4. The method of claim 3, wherein, The adaptive attention weight in the ECA module is calculated by the following formula; where |t| < 1 odd denotes the nearest odd integer to t, and here γ and b are set to 2 and 1, respectively, and c is the number of input channels.

Images