Electric power pipeline valve leakage detection method

Abstract

The invention discloses an electric power pipeline valve leakage detection method, and belongs to the field of computer vision. The method comprises the steps that firstly, pipeline valve leakage images are collected, and then a training set used for improving the yoolo (you only look only once) and a video frame data set used for tiny leakage identification are made; then training a training set by using the network provided in the invention, judging the training condition through the LOSS (loss) change of the whole network, and obtaining a pipeline valve leakage detection model after the training is completed; on the other hand, micro leakage is identified through processing of a continuous video frame sequence; and finally, detecting the leakage of the pipeline valve by combining the pipeline valve leakage detection model and a tiny leakage detection method. When the detection method is used for detecting an actual scene, the processing efficiency of 25fps can be achieved, and the detection method is suitable for being applied to the electric power industry.

Description

Technical Field

[0001] The invention relates to the field of electric power intelligent video analysis, and in particular to a method for detecting leakage of electric power pipeline valves based on camera video streams. Background Art

[0002] The existing pipeline valve leakage detection technologies mainly include the following 6 types: 1. Audio and sound detection method; 2. Correlation analysis leakage method; 3. Acoustic wave detection; 4. In-pipe detection method; 5. Water conservancy parameter detection method; 6. Manual detection method. The audio and sound detection method places a leak detector on the pipeline to check whether there is leakage; the correlation analysis method uses a correlation leak detector to check the pipeline leakage; the acoustic wave detection method uses a high-sensitivity acoustic wave converter and acoustic methods to amplify and filter the collected leakage noise to detect whether there is leakage; the in-pipe detection method places a detector in the pipe with a signal device to find the leakage based on the electrical signal of the leakage point noise; the water conservancy parameter detection method compares the parameters in the water with and without leakage, simulates and calculates through a certain method, and compares the calculated results with the measured results to determine whether there is leakage; the manual detection method is to observe the current operation of the pipeline manually to observe whether there is leakage.

[0003] Through analysis and comparison, several existing methods all have shortcomings. In actual operation, the audio-visual detection method, correlation analysis leakage method, acoustic wave detection, and in-pipe detection method not only require a large number of sensors to support, but also the actual production environment may not allow the placement of sensors in the pipe, and cannot be used for large-scale real-time monitoring. Manual detection requires workers to pay attention to the pipeline valve information at all times, and requires a large amount of manpower to complete this work. In the actual production environment, it cannot meet the needs of real-time special monitoring by dedicated personnel. Summary of the invention

[0004] This patent combines existing intelligent video technology means, adds a pipeline valve leakage detection method for specific scenarios, and proposes an effective pipeline valve detection method based on image data.

[0005] Since pipeline valve leakage has many types and forms, and the leakage liquid forms caused by the size of the rupture at the pipeline valve and the pressure in the pipe are also ever-changing, in order to better complete the pipeline valve leakage detection, this patent adopts a multi-method combination design detection method. The method proposed in this patent is mainly divided into two parts, one is a leakage detection model based on a neural network, and the other is a small leakage information detection method based on video timing information and leakage characteristics. Among them, the former has a better detection effect on pipeline valve leakage in complex scenes; the latter, based on the former, through the use of timing information, can better identify small leakage of pipeline valves in indoor scenes.

[0006] The neural network is divided into four main modules. CSPDarknet53 is used as the backbone network BackBone. It mainly draws on the idea of ​​CSPNet and adds CSP modules to the original Resblock to achieve richer gradient combinations and reduce the amount of calculation. FPN is used as the feature fusion module of Neck to enhance the detection performance of small targets. SPP is used as the additional module of Neck to increase the receptive field of the network. Yolo is used as the Head to connect the output of Neck to complete the leakage category prediction function.

[0007] The specific structure of the model is as follows:

[0008] The input image tensor passes through a CBM module, which contains a convolution layer (3×3 convolution kernel) to increase the dimension to 32, as well as BatchNorm normalization and Mish activation function to obtain a 32-dimensional tensor of the original size. The calculation method is shown in equation (1).

[0009] Mish = x*tanh(ln(1+e x )) (1)

[0010] This tensor passes through 5 CSP modules.

[0011] The first CSP module starts with a 3×3 convolution kernel and a 64-dimensional convolution layer. The output output1 of the convolution layer is copied for backup. The original output then passes through a 1×1 convolution kernel and a 64-dimensional convolution layer. The output output2 of the convolution layer is copied. Next, it passes through a 1×1 convolution kernel and a 32-dimensional convolution layer and a 3×3 convolution kernel and a 64-dimensional convolution layer. Then, this output is added to the output2 matrix and passed through a 1×1 convolution kernel and a 64-dimensional convolution layer. The output of this layer is concatenated with output1 and then passed through a 1×1 convolution kernel and an output dimension of 64 to obtain a 64-dimensional tensor with unchanged size.

[0012] The use of output2 is the realization of the residual structure, and the final output of the residual block is obtained by matrix addition of the result after convolution with the original input; and the use of output1 is based on the CSPNet idea (the original input is divided into two parts, and a direct copy method is adopted here, one part is convolved, and the other part is spliced ​​with the result after convolution to get the final output).

[0013] The outputs of the 2nd, 3rd, and 4th CSP modules are used as the input of the next part, namely the input of the Neck part SPP and FPN.

[0014] The FPN structure improves the detection effect of small targets. The specific operation is to upsample the output of the SPP structure twice in combination with the 4th and 5th CSP modules.

[0015] The first upsampling includes two parts of input, one is the output of SPP, and the other is the output of the fourth CSP module. The output of SPP passes through a 1×1 convolution kernel, a convolution layer with an output dimension of 512, a 3×3 convolution kernel, a convolution layer with an output dimension of 1024, a 1×1 convolution kernel, a convolution layer with an output dimension of 512, a 1×1 convolution kernel, and a convolution layer with an output dimension of 256. The original image size is then doubled; and the fourth CSP module passes through a 1×1 convolution kernel and a convolution layer with an output dimension of 256. The two output results are spliced ​​to obtain a tensor with an output dimension of 512 and doubled size.

[0016] The second upsampling is similar. The output of the first upsampling is convolved multiple times and doubled in size, and then concatenated with the output of the fifth CSP module. Finally, the tensor is input into yolo_head (the head network in the yolo model) for result prediction.

[0017] Finally, the three outputs obtained from the FPN structure are processed by the convolution of yolo_head. Here, the convolution layer uses BatchNorm (batch normalization) standardization and LeakyReLU (leaky rectified linear unit) activation function. The calculation method is shown in equation (2).

[0018] LeakyReLU(x)=max(0,x)+negative_slope·min(0,x) (2)

[0019] Finally, multiple logistic classifiers are used for prediction to complete the identification and classification of leakage in the corresponding image. The calculation method is shown in equation (3).

[0020]

[0021] For tiny leaks that are difficult to detect, we first select a video set with tiny leaks based on the captured video, then extract the video frames one by one to find the possible locations of leaking droplets in each frame; determine whether it is a leak or noise interference through the motion patterns of droplets in multiple consecutive frames to complete the identification of tiny leaks.

[0022] Perform frame difference operation on the image sequence, that is, perform subtraction operation on adjacent continuous frames, take appropriate threshold for binarization, and obtain candidate target images. The candidate target images are binarized sequences, which contain moving targets and noise.

[0023] Save the candidate target image results of the most recent N frames. Each time a new frame of image arrives, perform frame difference calculation with the current Nth frame, take the resulting image frame as the last frame, and remove the most recent and earliest candidate target image. Record the point with a grayscale value of 1 in the kth frame as the candidate target point xi, establish an m×m window Wi with each candidate target in the kth frame as the center, and calculate the contribution Qi of a single window of a single image. The calculation method is shown in equation (4).

[0024]

[0025] The contribution of each window position of N frames of images is accumulated to calculate the candidate target confidence p(Xi), and the calculation method is shown in equation (5).

[0026]

[0027] Due to the continuity of moving targets in time and space, there will be pixels with a grayscale value of 1 in multiple frames of images at the window position where the real leakage is located. Since the isolated noise is randomly distributed, the accumulated grayscale value in the window is small. Therefore, a threshold (T=N / 2) can be set. For areas exceeding this limit, it is considered that there is a target, that is, there may be a leakage point.

[0028] A coordinate system is established with the length and width of the image as the x and y axes respectively, and the center points of the windows that may have leakage are taken as points in the coordinate system. If all adjacent center points can be fitted into a univariate function, and the function is monotonically unique within the scope of the fitting function of these points, it means that there is a tiny leakage in the image sequence, that is, the leakage has been found. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be described by way of the accompanying drawings, in which:

[0030] Figure 1 It is a work flow chart of the present invention.

[0031] Figure 2 It is the leakage detection model training process.

[0032] Figure 3 This is a penetration test rendering.

[0033] Figure 4 are the initial image and the binarization result image.

[0034] Figure 5 It is the detection result of tiny leakage point. DETAILED DESCRIPTION

[0035] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.

[0036] Any feature disclosed in the present invention (including any additional claims, abstract and drawings), unless otherwise stated, may be replaced by other alternative features that are equivalent or have similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

[0037] The pipeline valve leakage detection based on yolo and small target recognition proposed in this patent is implemented through the following steps:

[0038] Step S1, collecting pipeline valve leakage video data.

[0039] Step S2, processing the collected video data, and making a training set for improving the YOLO network and a data set for micro-leakage detection.

[0040] Step S21, use deep learning annotation software to annotate the image and classify the leakage in the image into three categories: water mist, water droplets, and water columns.

[0041] Step S22, creating a video sequence for tiny target detection, extracting videos with tiny leaks, and saving them frame by frame as a tiny leak detection data set.

[0042] Step S3, training the leakage detection model. Use the training set used to improve the YOLO network in step S2 for training. The training process is as follows: Figure 2 shown.

[0043] Step S4, obtain a pipeline valve leakage detection model, use the detection model to predict the training set and test set, the detection effect is as follows Figure 3 shown.

[0044] Step S5, based on the data set used for micro-leakage detection in step S2, the method for identifying micro-leakage is implemented. The first two frames of the selected video frames are reconnaissance operated, and an appropriate threshold is used for binarization to obtain a candidate target image. The initial image and the binarization result are shown in FIG. Figure 4 shown.

[0045] Step S51, continue to perform frame difference calculation and binarization processing on adjacent frames, and obtain a frame sequence after performing this operation 100 times in a row, and take a plurality of the sequences for display.

[0046] Step S6, determine the target window according to equation (4) and equation (5) to obtain all target windows where possible micro-leakage points may appear. The detection results are as follows: Figure 5 shown.

[0047] A coordinate system is established with the length and width of the image as the x and y axes respectively, and the center point of the window where leakage may exist is fitted as the point in the coordinate system. The window in the figure can be fitted into an approximate straight line, that is, the function of the window set fitting satisfies monotonicity, indicating that it is the movement of a tiny leakage, so the tiny leakage is detected.

Claims

1. A method for detecting leakage of a power pipeline valve, mainly comprising the following steps: Collect pipeline rupture videos and image files in various scenarios to ensure that the video captures clear and stable pipeline leakage scenes; Use the collected video images to create the training set of the YOLO-based improved network in the patent and the video frame dataset required for the small target recognition method; The improved network based on Yolo is used for training to obtain a model with pipeline valve leakage detection and classification capabilities; Based on the video time sequence information and the characteristics of the leaking liquid in the image, a method for identifying tiny leaks in the image is proposed; The pipeline valve leakage detection method required in this patent is comprehensively completed based on the network training model improved by Yolo and the small leakage identification method.

2. According to the method described in claim 1, a training set based on the YOLO improved network in the patent is prepared. First, the training set is annotated. The annotation method is as follows: Firstly, pipeline leakage video picture frames in the video data are extracted; According to the characteristics of pipeline leakage, the leakage is classified, and the images are annotated using deep learning annotation software to create a training set.

3. According to the method described in claim 1, a training set based on the YOLO improved network in the patent is prepared. First, the training set is labeled and classified. This patent classifies it into three categories: water mist, water droplets, and water columns.

4. According to the method described in claim 1, the key lies in training the network of the pipeline leakage detection model, based on the neural network, inputting the image to be detected, detecting whether there is leakage and the leakage category, and the network structure includes: Using CSPDarknet53 (Cross Stage Partial Network Darknet53) as BackBone, it mainly draws on the idea of ​​CSPNet (Cross Stage Partial Network) and adds CSP (Cross Stage Partial) module to the original Resblock (residual block) to achieve richer gradient combinations and reduce the amount of calculation; Use SPP (Spatial Pyramid Pooling) as an additional module of Neck to increase the receptive field of the network; Use FPN (Feature Pyramid Network) as the feature fusion module of Neck to enhance the detection performance of small targets; Yolo acts as the Head (head network) to connect the output of Neck to complete the prediction function.

5. According to the method of claim 1 or claim 4, in order to reduce the amount of calculation and achieve richer gradient combinations, a method of adding a CSP module to the original Resblock.

6. According to the method of claim 1 or claim 4, in order to increase the receptive field of the network, SPP is used as a method of Neck additional module.

7. According to the method of claim 1 or claim 4, in order to enhance the detection performance for small targets, FPN is used as a feature fusion module of Neck.

8. According to the method of claim 1, the time series information is used to identify small leakages that are difficult to detect. The identification method is as follows: Firstly, based on the collected videos, a video set with slight leakage is selected, and video frames are extracted one by one; Extract the possible leakage droplet positions in each frame; The motion morphology of droplets in multiple consecutive frames is used to determine whether it is leakage or noise interference, thereby completing the identification of tiny leaks.

9. According to the method of claim 1 or claim 8, in order to enhance the detection of micro-leakage, a method for calculating the contribution of a single window of a single image is: Save the results of the candidate target images of the most recent N frames. Every time a new frame of image arrives, perform frame difference calculation with the current Nth frame, take the resulting image frame as the last frame, and remove the most recent and earliest candidate target image; record the point with a gray value of 1 in the kth frame as the candidate target point xi, establish an m×m window Wi with each candidate target in the kth frame as the center, and calculate the contribution Qi of a single window of a single image; The contribution of each window position of N frames of images is accumulated to calculate the candidate target confidence p(Xi).

10. The method according to claim 1 or claim 8, wherein the method comprises: calculating the presence of micro-leakage in an image sequence to enhance the detection of micro-leakage: A coordinate system is established with the length and width of the image as the x and y axes respectively, and the center points of the windows that may have leakage are taken as points in the coordinate system. If all adjacent center points can be fitted into a univariate function, and the function is monotonically unique within the scope of the fitting function of these points, it means that there is a tiny leakage in the image sequence, that is, the leakage has been found.

Images