A deep learning detection method for water leakage based on feature input enhancement

Abstract

This invention discloses a deep learning method for detecting water leakage defects based on feature input enhancement. The method includes: directional filtering and suppression of water leakage images, using the directional filtering results as the first part of the image feature channel input to a subsequent deep neural network; segmentation and enhancement of water leakage region signals: tunnel inspection image data is segmented and preprocessed using the Otsu threshold segmentation method, followed by smoothing using an image spatial domain filter; multiplication of the feature input enhancement weight matrix after spatial domain filtering with the original image, and using the resulting result as the second part of the image feature channel input to a subsequent deep neural network; and establishment of a water leakage detection and recognition model. This method significantly improves the performance of water leakage detection and recognition.

Description

Technical Field

[0001] The present invention belongs to the field of subway tunnel lining surface defect detection and identification, and in particular relates to a deep learning detection method for water leakage defects based on feature input enhancement. Background Art

[0002] Detecting surface defects in subway tunnels, especially the detection and identification of water leakage, is crucial for ensuring safe subway operation. Current detection technologies for water leakage in subway tunnels primarily include traditional manual inspection methods and computer vision. Manual inspection involves inspectors marking defects during inspections. With the increasing workload of subway maintenance, this method has led to a sharp increase in maintenance costs. Furthermore, due to limited working hours, inspectors often miss or misidentify defects. Computer vision methods include traditional image processing and deep learning. Traditional image processing methods utilize threshold segmentation, edge detection, and morphological analysis to detect and identify water leakage on tunnel surfaces. While these traditional image processing methods offer low computational complexity and minimal hardware requirements, they struggle to overcome interference from low contrast, uneven illumination, and severe background noise in subway tunnel surface defects. In recent years, deep learning methods have demonstrated excellent generalization and robustness, leading to their widespread application in civil structure health monitoring. Chinese invention patent publication number CN109615653A discloses a method for detecting and identifying water leakage areas based on deep learning and field of view projection models. This method collects point cloud data of the area to be tested and detects and identifies water leakage defects through a neural network model. Chinese invention patent publication number CN111899288A discloses a method for detecting and identifying water leakage areas in tunnels based on the fusion of infrared and visible light images. This method uses deep learning technology to fuse the infrared data and visible light data collected from the area to be tested inside the tunnel to detect and identify water leakage areas inside the tunnel. Both of the above methods have achieved more accurate experimental results than traditional image processing methods by leveraging the powerful feature extraction and pattern classification capabilities of deep learning networks.

[0003] However, when applying deep learning technology to actual subway tunnel surface defect inspections, the following problems still need to be solved:

[0004] Although background interference such as seams and pipelines exists on the tunnel lining surface, and their grayscale feature distribution is similar to that of water leakage defects, making them difficult to distinguish, analysis of multi-camera image acquisition content in actual tunnel inspection systems shows that background interference such as seams and pipelines captured by cameras generally exhibits a relatively clear horizontal and vertical distribution pattern across the entire image area. However, as a naturally occurring phenomenon, the distribution of boundary features of water leakage defects exhibits significant randomness. However, existing deep learning-based water leakage detection methods generally ignore this distinction, using only the original image content as input in deep learning network training. However, according to relevant machine learning case studies, less noise in the training samples is more conducive to improving target recognition performance. Clearly, this single training sample input method lacks the exploitation of the prior distribution patterns of water leakage images in tunnel inspections, thereby limiting the ultimate detection and recognition performance of water leakage defects. Summary of the Invention

[0005] In order to solve the problems existing in the prior art, the present invention provides a deep learning detection method for water leakage defects based on feature input enhancement.

[0006] To this end, the present invention adopts the following technical solutions:

[0007] A deep learning detection method for water leakage based on feature input enhancement includes the following steps:

[0008] S1, directional filtering suppression of water leakage images: Clutter such as tunnel surface joints and pipelines presents obvious directional characteristics in the camera image field. To this end, a multi-directional and multi-scale Gabor filter is used to convolve with the water leakage image to be detected, as shown in the following formula:

[0009]

[0010] Where z = (x, y) is the pixel coordinate; is the convolution operator; u∈{0,1,2,3,4,5,6,7} is the 8 direction angles selected by the Gabor filter v∈{1,2,3,4,5} is the 5 different scales k selected by Gabor filter v =2 -(v+2)π / 2 ;O u,v (z) is the convolution result of the Gabor filter with the water leakage image using direction u and scale v;

[0011] Use the convolution result O u,v The amplitude response of (z) is the filtering output result, as shown in the following formula:

[0012]

[0013] The directional filtering result is used as the first part of the subsequent deep neural network input image feature channel;

[0014] S2, signal segmentation and enhancement processing of water leakage area: The tunnel inspection image data is segmented and pre-processed using the image signal Otsu threshold segmentation method. The true value image of the detection result of a certain image area of the tunnel after being processed by the Otsu threshold segmentation method is assumed to be matrix B, which is defined as follows:

[0015]

[0016] Then use the image spatial domain filter to smooth the matrix B and generate a continuously differentiable feature input enhancement weight matrix Q. The specific formula is as follows:

[0017]

[0018] In the above formula, (x, y) is the pixel coordinate; is the convolution operator; s is a normalized 5×5 size average filter, which is generated as follows: s=ones(5,5) / 25,

[0019] Multiply the feature input enhancement weight matrix Q after image spatial domain filtering with the original image, and use the result as the second part of the subsequent deep neural network input image feature channel;

[0020] S3, establish a water leakage disease detection and identification model: the original image, directional filtering feature map and image signal segmentation processing feature map are connected in parallel in the channel direction and used as the feature input of the subsequent deep neural network; based on the input sample data after feature enhancement, the VGG16+FCN fully convolutional network is used to establish a water leakage disease detection and identification model.

[0021] Because strong interference objects such as seams and pipelines on the tunnel lining surface appear almost identical to leaking water in image grayscale distribution, the characteristics of the background and leaking water cannot be effectively distinguished, resulting in missed detections or false alarms in traditional deep learning recognition models. Based on this, the present invention analyzed a large amount of subway tunnel water leakage image data and confirmed that clutter such as seams and pipelines exhibits obvious directional characteristics. A two-dimensional Gabor filter can be used to capture the horizontal and vertical feature amplitude information in the tunnel surface image. The water leakage defect filtering result is obtained by subtracting it from the original image and used as the input for the subsequent fully convolutional neural network.

[0022] Considering that there is a significant brightness difference between the water leakage defect and the subway tunnel lining segment itself, and that the pixel grayscale in the water leakage area shows a certain degree of clustering, the present invention proposes to use the Otsu threshold method to segment the water leakage defect. However, since the result of the Otsu threshold segmentation algorithm is a binary signal, directly applying it to deep learning network training will prevent the gradient error from backpropagating. Based on this, the present invention proposes to use an image spatial domain filter to further smooth the Otsu threshold segmentation result, generate a continuously differentiable weak true value map, and obtain the water leakage defect segmentation processing feature map by multiplying it with the original image, which is used as the input of the subsequent fully convolutional neural network.

[0023] At present, most of the water leakage detection methods based on deep learning directly process a single original input image; however, as a special disease, water leakage has its environmental interference characteristics and its own grayscale aggregation attribute characteristics ignored. The present invention proposes to use directional filtering feature maps and image signal segmentation processing feature maps as supplementary features of the original image input, and introduce them into the subsequent deep learning network training in the form of feature channel expansion, which greatly improves the detection and recognition performance of water leakage diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the flow chart of directional filtering of tunnel water leakage images based on two-dimensional Gabor filter;

[0025] Figure 2 This is the signal segmentation and enhancement processing diagram of the water leakage area;

[0026] Figure 3 This is the result map of Gabor filter directional feature suppression;

[0027] Figure 4 This is the result of signal segmentation and enhancement in the water leakage area;

[0028] Figure 5 This is a network framework diagram for water leakage detection and identification;

[0029] Figure 6 This is a comparison chart of the detection results of the method of the present invention and the existing method, where a is the original image, b is the manually marked disease image, c is the detection result chart when only the original image is input, and d is the detection result chart of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0031] The water leakage disease deep learning detection method based on feature input enhancement of the present invention comprises the following steps:

[0032] S1: Water leakage image directional filtering suppression:

[0033] We combined the actual tunnel inspection image acquisition system to analyze a large amount of subway tunnel water leakage image data from the perspective of disease imaging, and confirmed the obvious directional characteristics of clutter such as tunnel surface joints and pipelines in the camera image field. Figure 1 As shown in the figure, the present invention uses a two-dimensional Gabor filter to suppress the directional characteristics of clutter. The Gabor filter is the product of a sinusoidal plane wave and a Gaussian kernel function, and has good local feature extraction capabilities. In order to determine the optimal filter parameters for the lining surface image, a multi-directional and multi-scale Gabor filter is convolved with the leakage image I to be detected, as shown in the following formula:

[0034]

[0035] In formula (1), z = (x, y) is the pixel coordinate; is the convolution operator; u∈{0,1,2,3,4,5,6,7} is the 8 direction angles selected by the Gabor filter v∈{1,2,3,4,5} is the 5 different scales k selected by Gabor filter v =2 -(v+2)π / 2 ;O u,v (z) is the convolution result of the Gabor filter with direction u and scale v and the leakage water image I.

[0036] In order to more effectively calculate the directional filtering features, the convolution result O u,v The amplitude response of (z) is the filtering output result, as shown in the following formula:

[0037]

[0038] By comparing and analyzing the Gabor filter output responses at different scales and directions, we can preliminarily determine the optimal parameters of the Gabor filter when the output response to strong interference objects such as tunnel lining surface joints and pipelines is the largest. Figure 3 As shown in Figure 1, when the two-dimensional Gabor filter parameters (u∈{0,4}v=5) are selected, the directional features of interference such as tunnel surface joints and pipelines are extracted, and the directional filtering results of water leakage defects are obtained by subtracting them from the original image.

[0039] S2: Segmentation and enhancement of water leakage area signal:

[0040] Since the pixel grayscale in the water leakage area shows a certain aggregation, based on the pixel grayscale aggregation analysis of the tunnel water leakage area, the present invention uses the image signal Otsu threshold segmentation method to perform segmentation preprocessing on the tunnel inspection image data, such as Figure 2As shown. Considering that Otsu method is a commonly used method for image threshold segmentation, the specific details of this method are not repeated here. Figure 4 As shown in Figure 2, suppose that the true value image of the detection result of a certain image area I of the tunnel after being processed by the Otsu algorithm is a matrix B, which is defined as follows:

[0041]

[0042] As can be seen from formula (3), the Otsu detection result is a binary signal. Therefore, directly using the binary detection result as the feature input of the deep learning network will cause the error loss function of the entire network to be non-differentiable, and thus the gradient error cannot be back-propagated. To avoid this phenomenon, the present invention proposes to use an image spatial domain filter to smooth the matrix B and generate a continuously differentiable feature input enhancement weight matrix Q. The specific formula is as follows:

[0043]

[0044] In the above formula, (x, y) is the pixel coordinate; is the convolution operator; s is a normalized 5×5 averaging filter, which is generated as follows: s=ones(5,5) / 25.

[0045] The feature input after image spatial domain filtering is enhanced by the weight matrix Q, multiplied by the original image, and the result is used as the second part of the subsequent deep neural network input image feature channel.

[0046] S3: Establish a water leakage disease detection and identification model, such as Figure 5 As shown in the figure, this paper proposes to establish a water leakage detection and identification network based on VGG16+FCN. The entire network model has 6 groups of convolutional layers, and each group of convolutional layers is separated by a pooling layer. Figure 5 Conv, Average Pool, ReLu, and DeConv represent convolution, average pooling, activation, and deconvolution operations, respectively. Steps 1 and 2 yield the original image's directional filtering feature map and the image signal segmentation processing feature map. These two feature maps are then connected in channel-parallel fashion and used as feature inputs for the deep neural network. The entire water leakage detection and recognition network, fed by the combined input of the original image, the directional filtering feature map, and the water leakage area segmentation feature map, can better perceive and learn the salient features of water leakage areas, thereby improving the detection and recognition performance of water leakage areas. Figure 6 This is a specific embodiment of the present invention. Figure c shows the detection result when only the original image is input, which may result in missed detections. The dotted circle indicates the missed detection area. Compared with the result of "combining the original image, the directional filtering feature map, and the water seepage area segmentation feature map," the present invention can detect the missed detection area on the right.

Claims

1. A deep learning detection method for water leakage based on feature input enhancement, comprising the following steps: S1, directional filtering suppression of water leakage images: Tunnel surface joints and pipelines show obvious directional characteristics in the camera image field of view. Therefore, a multi-directional and multi-scale Gabor filter is used to convolve with the water leakage image to be detected, as shown in the following formula: Where z = (x, y) is the pixel coordinate; is the convolution operator; u∈{0,1,2,3,4,5,6,7} is the 8 direction angles selected by the Gabor filter v∈{1,2,3,4,5} are the five different scales selected by Gabor filter; u,v (z) is the convolution result of the Gabor filter with the water leakage image using direction u and scale v; Use the convolution result O u,v The amplitude response of (z) is the filtering output result, as shown in the following formula: Subtract the filtered output from the original image to obtain a directional filtered feature map, and use the directional filtered feature map as the first part of the subsequent deep neural network input image feature channel; S2, signal segmentation and enhancement processing of water leakage area: The tunnel inspection image data is segmented and pre-processed using the image signal Otsu threshold segmentation method. The true value image of the detection result of a certain image area of the tunnel after being processed by the Otsu threshold segmentation method is assumed to be matrix B, which is defined as follows: Then use the image spatial domain filter to smooth the matrix B and generate a continuously differentiable feature input enhancement weight matrix Q. The specific formula is as follows: In the above formula, (x, y) is the pixel coordinate; is the convolution operator; s is a normalized 5×5 size average filter, which is generated as follows: s=ones(5,5) / 25, Multiply the feature input enhancement weight matrix Q after image spatial domain filtering with the original image to obtain the image signal segmentation processing feature map, and use the image signal segmentation processing feature map as the second part of the subsequent deep neural network input image feature channel; S3, establish a water leakage disease detection and recognition model: connect the original image, directional filtering feature map and image signal segmentation processing feature map in the channel direction and use them as feature inputs for the subsequent deep neural network; Based on the input sample data after feature enhancement, the subsequent deep neural network uses the VGG16+FCN fully convolutional network to establish a water leakage disease detection and identification model.

Images