Method for accurately detecting foreign matters in pipeline

Abstract

The invention relates to the technical field of detection of foreign matters in a pipeline, and discloses an accurate detection method of foreign matters in the pipeline, which comprises the following steps: S1, segmenting video signals in the pipeline frame by frame to form continuous image frame signals, manually marking the foreign matters in the pipeline, and making an image data set of the foreign matters in the pipeline; s2, inputting the image data set of the foreign matters in the pipeline into a plurality of convolutional neural networks based on a multi-scale split convolution kernel for classification, and classifying the image data set into pictures containing and not containing the specific foreign matters; s3, integrating and counting the number of foreign matter pictures of each convolutional neural network based on the multi-scale split convolution kernel so as to realize foreign matter detection; according to the method, the convolutional neural networks based on the multi-scale split convolution kernel are respectively set for different types of foreign matters in the pipeline to identify and detect the foreign matters, so that accurate detection of the foreign matters is realized, omission or misjudgment is not easy to occur, and the maintenance and operation cost is remarkably reduced in an occasion requiring frequent detection.

Description

Technical Field

[0001] The present invention relates to the technical field of foreign object detection in pipelines, and specifically to a precise foreign object detection method in pipelines. Background Technique

[0002] In industrial and municipal infrastructure, pipeline systems play a crucial role in transporting resources such as water, oil, and natural gas. The integrity and functional efficiency of these pipelines are essential for ensuring the smooth transportation of resources. However, foreign objects may accumulate inside the pipelines for various reasons, such as stones, branches, plastic products, etc. These foreign objects not only hinder the normal flow of fluids but may also cause pipeline damage or leakage, thereby triggering safety accidents and environmental pollution. Pipeline systems are widely used in fields such as water supply, drainage, oil, and natural gas transportation, and are an important part of modern urban and industrial infrastructure. Pipeline systems usually consist of main pipelines, branch pipelines, valves, joints, etc. These structures jointly ensure the efficient transportation of fluids and rely on pressure differences to drive the fluids to flow inside the pipelines, and control the flow rate and pressure through valves and regulating devices. The current practice is to use manual inspections or mechanical detectors, which are inefficient and difficult to meet the detection requirements of long-distance or complex pipeline networks. Moreover, they often rely on the experience and skills of operators, making it difficult to achieve high-precision foreign object detection, and it is easy to miss or misjudge. In addition, the use of manual inspections and mechanical detectors is not only inefficient but also costly, especially in cases where frequent detections are required, the maintenance and operation costs increase significantly. The reasons for the above problems are that traditional detection technologies have inherent limitations in accuracy and speed, making it difficult to meet the requirements of modern pipeline detection. Manual inspections and operating mechanical detectors highly rely on the skills and experience of operators, which greatly limits the consistency and reliability of detections. Also, existing pipeline detection methods lack effective automated solutions and cannot achieve precise foreign object detection.

[0003] Therefore, to further solve the above problems, the applicant has proposed a precise foreign object detection method in pipelines. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a precise foreign object detection method in pipelines, which solves the problems that in the detection of foreign objects in pipelines using mechanical detectors, it highly depends on the experience and skills of operators, it is difficult to achieve high-precision foreign object detection in manual detection, it is easy to miss or misjudge, and in cases where frequent detections are required, the maintenance and operation costs increase significantly as mentioned in the above background technique.

[0005] To achieve the above object, the present invention provides the following technical solutions: A precise foreign object detection method in pipelines, the steps of the detection method include: S1. Segment the video signal in the pipeline frame by frame to form continuous image frame signals, and manually label the foreign objects in the pipeline in the image frame signals to create a foreign object image dataset in the pipeline; S2. Use the foreign object image dataset in the pipeline as the data input, and input it into several convolutional neural networks based on multi-scale split convolutional kernels for classification, that is, each network only learns and detects a specific foreign object (such as branches, stones, plastic bottles, etc.), and classify the image dataset into pictures containing and not containing the specific foreign object; S3. Integrate and count the number of foreign object pictures of each convolutional neural network based on multi-scale split convolutional kernels to achieve foreign object detection.

[0006] As a further solution of the present invention: The specific formula of step S1 is as follows: Among them, is the pipeline closed-circuit television video signal, is the total number of frames of the video signal, is the th frame of the segmented and labeled foreign object detection image signal in the pipeline, is the created image dataset.

[0007] As a further solution of the present invention: The specific steps of step S2 include: S21. Use the foreign object image dataset in the pipeline as the data input, and input it into the multi-scale split convolutional layer for feature extraction. The formula is as follows: Among them, are respectively the convolutional kernel, convolutional kernel and convolutional kernel of is the feature of the input data, is the predefined threshold; S22. After the convolutional layer, apply two non-linear activation functions ELU and LReLU to enable the network to learn complex feature combinations. The formula is as follows: Among them, is the value input to the activation function, is a positive constant, usually with a default value of 1, is the exponential function with as the base, which maps negative values to a positive value close to 0 (because is always positive, even for negative ).

[0008] When the input value When it is greater than 0, the function directly outputs , which means that for positive values, the behavior of the function is similar to the identity function and does not modify the input value. When the input value is less than or equal to 0, the function directly outputs , for negative values close to 0, it is close to 1, so it is close to 0. For more negative values, it becomes smaller, and also becomes smaller.

[0009] Among them, is the value input to the activation function, is a positive coefficient between 0.01 and 0.3.

[0010] When the input value is less than or equal to 0, the slope of the function. When the input value is greater than 0, the function directly outputs , when the input value is less than or equal to 0, the function outputs , which means that even if the input value is negative, the neuron can still have non-zero activation, thereby allowing the gradient to flow and preventing neuron death.

[0011] As a further solution of the present invention: The specific steps of step S2 further include: S23. After the activation function, input the feature into the random pooling layer to reduce the spatial size of the feature map, reduce the number of parameters and the amount of calculation, and at the same time extract important features. The formula is as follows: Among them, is the element of the output feature map at position , is the element of the input feature map at the randomly selected position , is the randomly selected index set, is the index set the number of elements in; S24. After multiple convolutional and pooling layers, input the feature into the fully connected layer. The fully connected layer summarizes the features extracted by the previous layers to generate the final output. The formula is as follows: Among them, is the output vector of the fully connected layer, is the weight matrix of the fully connected layer, is one of the input feature maps or vectors, is a bias vector, is the matrix multiplication result of the weight matrix and the input features; S25. Input the result of the fully connected layer into the classifier to determine the classification of the input feature vector x and generate a classification result. The formula is as follows: where, is the input feature vector, is the weight vector, is the bias term, is the dot product of the weight vector and the input feature vector plus the bias term, is the output of the classifier, which is the predicted value of the class label. Different predicted values represent different classes. If two classification classes are set in the dataset and their values are 0 and 1 respectively, then the output value of the classifier, that is, the predicted value of the class label, is 0 or 1; S26. During the training process, the convolutional neural network uses the hinge loss function to measure the difference between the model's predicted value and the true value. The formula is as follows: where, is the total loss used to measure the difference between the model's predicted value and the true value, is the number of samples, is the true label of the th sample, is the feature vector of the th sample, is the predicted value of the th sample, S27. Finally, the network uses an optimization algorithm to update the network parameters according to the gradients calculated by backpropagation to minimize the loss function and thus optimize the network training. The formula is as follows: where, is the weight matrix, is the learning rate that controls the weight update step size, is the gradient of the loss function with respect to the weight , is the updated weight matrix.

[0012] As a further solution of the present invention: The specific formula of step S3 is as follows: where, For the final foreign object detection result report, It is a post - processing function based on the output of a convolutional neural network, which integrates and counts the number of each foreign object picture.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: By frame - by - frame segmenting the monitoring video signal inside the pipeline, converting the video signal into an image signal, annotating the foreign objects in the image, making a foreign object image dataset inside the pipeline, and then using it as data input and inputting it into a convolutional neural network based on a multi - scale split convolution kernel, and separately setting convolutional neural networks based on multi - scale split convolution kernels for different types of foreign objects inside the pipeline to identify and detect them, the accurate detection of foreign objects is realized, which is not easy to miss or misjudge, and in the occasions that need frequent detection, the maintenance and operation costs are significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is the flowchart of foreign object detection inside the pipeline of the device of the present invention; Figure 2 It is the block diagram of the foreign object detection algorithm inside the pipeline after improvement of the device of the present invention; Figure 3 It is the convolutional neural network model based on the multi - scale split convolution kernel of the device of the present invention.

[0015] In the figure: 1. Monitoring video signal of foreign object detection inside the pipeline; 2. Annotated image signal of foreign object detection inside the pipeline after frame - by - frame segmentation; 3. Convolutional neural network based on multi - scale split convolution kernel; 4. Classification picture result of the detection output of the convolutional neural network based on multi - scale split convolution kernel; 5. Image signal of foreign object detection inside the pipeline as the input of the convolutional neural network based on multi - scale split convolution kernel; 6. Convolution kernel; 7. Fully - connected layer; 8. SVM classifier; 9. Classification value output by the convolutional neural network based on multi - scale split convolution kernel. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0017] Embodiment Please refer to Figures 1-3, the present invention provides a precise detection method for foreign objects in pipelines. The detection method uses multiple convolutional neural network models based on multi-scale split convolutional kernels to detect the types of foreign objects in pipelines; each neural network learns a specific foreign object in the pipeline (such as branches, stones, etc.), and classifies the monitoring images in the pipeline as containing or not containing a specific type of foreign object in the pipeline for learning during the neural network training process; the detection process takes the pipeline closed-circuit television video as the input; first, the video is segmented into continuous image frames, and then each frame is sent into several convolutional neural networks based on multi-scale split convolutional kernels for classification; finally, the number of foreign object pictures of each convolutional neural network based on multi-scale split convolutional kernels is integrated and counted to determine the detection result.

[0018] The specific operation steps are as follows: Segment the video signal in the pipeline frame by frame to form continuous image frame signals and manually annotate the foreign objects in the image frame signals to produce a foreign object image dataset in the pipeline. The specific formula is as follows: Where, is the pipeline closed-circuit television video signal, is the total number of frames of the video signal, is the frame-segmented and annotated foreign object detection image signal in the pipeline, is the produced image dataset.

[0019] Take the foreign object image dataset in the pipeline as the data input and input it into the multi-scale split convolutional layer for feature extraction. The formula is as follows: Where, are the convolutional kernels of , are the convolutional kernels of , and is the feature of the input data, is the predefined threshold; After the convolutional layer, apply two non-linear activation functions ELU and LReLU to enable the network to learn complex feature combinations. The formula is as follows: Where, is the value input to the activation function, is a positive constant, usually with a default value of 1, is the exponential function with as the base, which maps negative values to a positive value close to 0 (because is always positive, even for negative ).

[0020] When the input value is greater than 0, the function directly outputs , which means that for positive values, the function behaves like an identity function and does not modify the input value. When the input value is less than or equal to 0, the function directly outputs . For negative values close to 0, is close to 1, so is close to 0. For more negative values, becomes smaller, and also becomes smaller.

[0021] Among them, is the value input to the activation function, is a positive coefficient between 0.01 and 0.3.

[0022] When the input value is less than or equal to 0, the slope of the function. When the input value is greater than 0, the function directly outputs . When the input value is less than or equal to 0, the function outputs , which means that even if the input value is negative, the neuron can still have non-zero activation, allowing the gradient to flow and preventing neuron death.

[0023] After the activation function, the feature is input into the random pooling layer to reduce the spatial size of the feature map, reduce the number of parameters and computational amount, and extract important features at the same time. The formula is as follows: Among them, is the element of the output feature map at position , is the element of the input feature map at the randomly selected position , is the randomly selected index set, is the index set the number of elements in; After multiple convolutional and pooling layers, the feature is input into the fully connected layer. The fully connected layer summarizes the features extracted by the previous layers to generate the final output. The formula is as follows: Among them, is the output vector of the fully connected layer, is the weight matrix of the fully connected layer, is one of the input feature maps or vectors, is a bias vector, is the matrix multiplication result of the weight matrix and the input features; The result of the fully connected layer is input into the classifier to determine the classification of the input feature vector x, generating a classification result. The formula is as follows: where, is the input feature vector, is the weight vector, is the bias term, is the dot product of the weight vector and the input feature vector plus the bias term, is the output of the classifier, which is the predicted value of the class label. Different predicted values represent different classes. If two classification classes are set in the dataset and their values are 0 and 1 respectively, then the output value of the classifier, that is, the predicted value of the class label, is 0 or 1; During the training process, the convolutional neural network uses the hinge loss function to measure the difference between the model's predicted value and the true value. The formula is as follows: where, is the total loss used to measure the difference between the model's predicted value and the true value, is the number of samples, is the true label of the th sample, is the feature vector of the th sample, is the predicted value of the th sample, is the calculation result of the hinge loss function for a single sample; where, is the weight matrix, is the learning rate that controls the weight update step size, is the gradient of the loss function with respect to the weight , is the updated weight matrix.

[0024] Integrate and count the number of foreign object images of each convolutional neural network based on multi-scale split convolutional kernels to achieve foreign object detection. The specific formula is as follows: where, For the final foreign object detection result report, It is a post-processing function according to the output of the convolutional neural network, which integrates and counts the number of each foreign object picture.

[0025] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for accurately detecting foreign matter in a pipeline, characterized in that: The detection method steps include: S1. Segment the video signal in the pipeline frame by frame to form continuous image frame signals and manually mark the foreign objects in the pipeline in the image frame signals to produce a dataset of images of foreign objects in the pipeline; S2. The image dataset of foreign objects in the pipeline is used as data input and input into several convolutional neural networks based on multi-scale split convolution kernels for classification. That is, each network only learns to detect a specific foreign object and classifies the image dataset into images containing and not containing the specific foreign object. S3, integrating and counting the number of foreign body images of each convolutional neural network based on multi-scale split convolution kernel to achieve foreign body detection; The specific steps of step S2 include: S21. The foreign body image dataset in the pipeline is used as data input and input into the multi-scale split convolution layer for feature extraction. The formula is as follows: in, They are The convolution kernel, The convolution kernel and The convolution kernel, is the feature of the input data, is a predefined threshold; S22. After the convolution layer, two nonlinear activation functions ELU and LReLU are applied to enable the network to learn complex feature combinations. The formula is as follows: in, is the value input to the activation function, is a positive constant, usually with a default value of 1. So An exponential function with base ; in, is the value input to the activation function, is a positive coefficient between 0.01 and 0.

3.

2. A method for accurately detecting foreign matter in a pipeline according to claim 1, characterized in that: The specific formula of step S1 is as follows: in, For pipeline CCTV video signals, is the total number of frames of the video signal, For the Image signal of foreign body detection in pipeline after frame segmentation and annotation. The image dataset is created.

3. A method for accurately detecting foreign matter in a pipeline according to claim 2, characterized in that: The specific steps of step S2 also include: S23. After the activation function, the features are input into the random pooling layer to reduce the spatial size of the feature map, reduce the number of parameters and the amount of calculation, and extract important features. The formula is as follows: in, The output feature map is at position Elements of The input feature map is at a randomly selected position Elements of is a randomly selected index set, Index Set The number of elements in ; S24. After multiple convolution and pooling layers, the features are input to the fully connected layer, which aggregates the features extracted by the previous layers to generate the final output. The formula is as follows: in, is the output vector of the fully connected layer, is the weight matrix of the fully connected layer, is one of the input feature maps or vectors, is a bias vector, is the matrix multiplication result of the weight matrix and the input feature; S25. Input the result of the fully connected layer into the classifier to determine the classification of the input feature vector x and generate the classification result. The formula is as follows: in, is the input feature vector, is the weight vector, is the bias term, Add a bias term to the dot product of the weight vector and the input feature vector, is the output of the classifier; S26. During the training process, the convolutional neural network uses the hinge loss function to measure the difference between the model prediction value and the true value. The formula is as follows: in, is the total loss used to measure the difference between the model prediction value and the true value, is the sample size, For the The true labels of samples, For the The feature vector of the samples, For the The predicted value of samples, Calculate the result for a single sample of the hinge loss function; S27. Finally, the network uses an optimization algorithm to update the network parameters according to the gradient calculated by back propagation to minimize the loss function and optimize the network training. The formula is as follows: in, is the weight matrix, The learning rate to control the weight update step size, is the loss function relative to the weight The gradient of is the updated weight matrix.

4. A method for accurately detecting foreign matter in a pipeline according to claim 3, characterized in that: The specific formula of step S3 is as follows: in, For the final foreign body detection result report, is a post-processing function based on the output of the convolutional neural network, To integrate and count the number of foreign body images.

Images