Pipeline anomaly detection method, storage medium and application

Abstract

The invention belongs to the technical field of pipe network monitoring, and particularly relates to a pipeline anomaly detection method, a storage medium and application. The invention provides a pipeline anomaly detection method. The method comprises the steps of data preprocessing, model construction and training and anomaly detection result output. Firstly, pipeline operation state data are preprocessed, and a space-time waterfall plot is generated. Then, a multi-scale parallel cascade early-quit network model is constructed, the model is composed of a plurality of parallel branches and a decision module, each branch comprises a plurality of cascaded element detection modules, and each element detection module is composed of a feature extractor, a feature flattening device, an initial classifier and a decision device; and inputting time-space waterfall plot data into the model for training to obtain final parameters of each module. And finally, inputting to-be-detected pipeline state data into the trained model, and outputting an anomaly detection result.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of pipe network monitoring, and particularly relates to a pipe anomaly detection method, a storage medium and application. BACKGROUND

[0002] With the rapid development of industry, as an important infrastructure for energy and resource transportation, the safety and reliability of pipelines are crucial for the stable operation of social economy. However, the existing pipeline detection technology has many shortcomings, which is difficult to meet the needs of modern pipeline safe operation. Manual inspection relies on the experience and visual judgment of workers, which is low in efficiency and easy to miss hidden dangers; and traditional automated detection methods, such as pressure testing and ultrasonic detection, often need to interrupt the pipeline operation, which not only increases the maintenance cost and time cost, but also is difficult to find early and small anomalies.

[0003] To solve the above problems, many algorithms have been proposed for pipeline anomaly detection. For example, patent CN119740056A proposes a pre-trained neural network model for pipeline magnetic flux detection. This method has limited ability to detect internal and external anomalies of the pipeline, and the data collection and processing time is long, which affects the efficiency. Patent CN119665164A uses particle swarm optimization algorithm for pipeline anomaly detection, but the structure is relatively complex. Patent CN118980060A uses optical fiber sensors for underground drainage pipeline anomaly detection, but the back-end algorithm processing has not been reported, and the detection accuracy needs to be improved. In addition, this method requires a large number of sensors and optical fibers to be laid, which increases the implementation difficulty and cost.

[0004] In view of the limitations of the prior art, the present application proposes a pipeline anomaly detection method based on deep learning, which has a simple structure and can not only realize fast detection of the pipeline, but also improve the detection accuracy. SUMMARY

[0005] In order to solve the problems in the prior art, the present application provides a pipeline anomaly detection method, a storage medium and application.

[0006] The technical solution adopted by the present application to solve its technical problems is as follows:

[0007] On the one hand, the present application provides a pipeline anomaly detection method, the steps of which are as follows:

[0008] S1, data receiving in the training process: real-time receiving of state data in the pipeline operation process, denoted as matrix P ij , where the element is denoted as p ij , i=1, 2,..., m, j=1, 2,..., n, m and n are the number of rows and columns of P ij respectively;

[0009] S2, data preprocessing: after preprocessing the state data obtained in step S1, a space-time waterfall chart is obtained, denoted as X ij , where the elements are denoted as x ij ; i = 1, 2,..., m, j = 1, 2,..., n;

[0010] S3, model construction and training: a multi-scale parallel cascade early exit network model is constructed, including B parallel branches and 1 decision module, and each branch is composed of s front and rear cascaded element detection modules;

[0011] Each element detection module includes:

[0012] Feature extractor: input space-time waterfall chart, output feature map a0;

[0013] Feature flattener: flatten the feature map into a one-dimensional feature vector f;

[0014] Primary classifier: classify f, output probability vector

[0015] Decision maker: compare the maximum class probability max_probs of with the threshold value u; if max_probs≥u, output to the decision module; if max_probs

[0016] The decision module sums up the of the B branches; output the abnormal class;

[0017] S4, anomaly detection: given the pipeline state data, input it into the multi-scale parallel cascade early exit network model constructed in step S3 for calculation, and finally obtain the result of anomaly detection.

[0018] Preferably, in step S1, the pipeline running process is specifically as follows: the distributed optical fiber is tightly wound on the surface of the pipeline, and the running state of the pipeline is monitored in real time through the phase change of the optical fiber to obtain a two-dimensional gray image.

[0019] Preferably, in step S2, the preprocessing specifically includes: first, normalizing the two-dimensional gray image collected in step S1, normalizing each pixel value to the range of 0-255 to obtain a space-time waterfall chart, and the specific formula is as follows:

[0020]

[0021] Where max(P ij ) and min(P ij ) are the maximum and minimum values of the matrix P ij , respectively.

[0022] Preferably, in the step S3, the multi-scale parallel cascade early-exit network model is composed of b parallel branches and 1 decision module, b is the number of branches, and is a positive integer given by a user. Each branch is composed of s front and back cascaded meta-detection modules, s is the number of cascaded modules, and is a positive integer given by a user. Each meta-detection module has 1 input and 2 outputs, and the output ports are denoted as output 1 and output 2. The input of the next level is connected to the output 1 of the previous level, and the output 2 of all meta-detection modules in each branch is connected together and output to the decision module, wherein the output 1 of the last meta-detection module is not connected. The decision module has b inputs and 1 output.

[0023] Preferably, in the step S3, each meta-detection module is composed of 1 feature extractor, 1 feature flattener, 1 initial classifier, and 1 decision maker. The feature extractor has 1 input and 2 outputs, the feature flattener has 1 input and 1 output, and the input of the feature flattener is connected to the output 2 of the feature extractor. The initial classifier has 1 input and 1 output, and the input of the initial classifier is connected to the output of the feature flattener. The decision maker has 2 inputs and 2 outputs, denoted as input 1, input 2 and output 1, output 2. The input 2 of the decision maker is connected to the output of the initial classifier, the input 1 of the decision maker is connected to the output 1 of the feature extractor, and the output 1, output 2 of the decision maker are the output 1, output 2 of the meta-detection module.

[0024] Preferably, in the step S3, the spatiotemporal waterfall diagram data set output in the step S2 is put into the constructed multi-scale parallel cascade early-exit network model for training, to obtain the final parameters of each meta-detection module and the decision module.

[0025] Preferably, in the step S3, the feature extractor specifically is a convolutional network, which includes a convolutional layer with a convolution kernel size of 3x3, a stride of 1, a padding of 1, and an output channel number of 64, for mapping the input spatiotemporal waterfall diagram to a feature space and extracting feature information related to the pipeline state; and the feature extraction formula is specifically:

[0026] α0=ReLU(W0*x+b0)

[0027] wherein x is the input spatiotemporal waterfall diagram, W0 is a convolution kernel weight, b0 is a bias term, * represents a convolution operation, and ReLU is an activation function; the size of the output feature map α0 is 64xHxW, wherein H and W are the height and width of the input image, the extracted feature F is delivered to the feature flattener as the output 2; and F is connected to the decision maker as the output 1.

[0028] Preferably, in the step S3, the feature flattener is specifically a feature flattening module, and the Flatten operation is adopted to rearrange the two-dimensional feature map into a one-dimensional vector f in element order, and the length of the vector f is 64xHxW; the specific formula is as follows:

[0029] f = Flatten(F)

[0030] wherein Flatten represents rearranging each element of the input into a one-dimensional vector in the order from left to right and from top to bottom; the operation ensures the integrity and consistency of the feature information in the dimension conversion process, and provides an adaptive input format for the subsequent fully connected layer or other processing modules.

[0031] Preferably, in the step S3, the initial classifier is specifically a random forest classifier, which receives the one-dimensional feature vector f output from the feature flattener; the random forest classifier outputs a one-dimensional vector wherein each element is denoted as wherein c is the length of the vector, b is the branch number, and b = 1, 2,..., B; specifically, the value of each element is between 0 and 1, and each element of the vector represents the prediction probability of the random forest.

[0032] Preferably, in the step S3, the decision maker is specifically a module that receives the one-dimensional vector output by the initial classifier and the output 1 of the feature extractor; the decision maker sorts all elements in from large to small, and obtains the maximum class probability max_probs, which is specifically:

[0033]

[0034] wherein max represents the maximum value, and max_probs is compared with a preset threshold u, wherein u is a decimal value between 0 and 1, which is given by human; if max_probs ≥ u, then output 1 is empty, and output 2 is If max_probs < u, then output 1 is input 1, i.e., α0, and output 2 is empty.

[0035] Preferably, in the step S3, the decision module is specifically a module that outputs B input First, the vector R k is calculated, which is a vector with a length of c, and is specifically:

[0036]

[0037] wherein β bFor each branch weight, take the value between 0 and 1, given by man; then for the vector R k Calculate the maximum value of each element, and the sequence number corresponding to the maximum value is the result of anomaly detection.

[0038] Preferably, in S3, the training process is specifically: stochastic gradient descent method, Adam algorithm, batch normalization method, etc.

[0039] On the other hand, the present application provides a storage medium storing a program capable of executing the above pipeline anomaly detection method.

[0040] Meanwhile, the present application also provides an application of the pipeline anomaly detection method, which modularly deploys the above storage medium in an embedded target platform or in an external computing module.

[0041] Compared with the existing invention, the beneficial effects of the present application are:

[0042] 1. Efficient detection and real-time performance: The present application adopts a multi-scale parallel cascade early exit network model, which can efficiently process pipeline state data and detect anomalies in real time. Compared with traditional detection methods, it does not need to interrupt the pipeline operation, greatly reducing the maintenance cost and time cost. This real-time monitoring capability ensures that potential problems can be discovered in time under normal operation of the pipeline, avoiding production interruption caused by detection, and improving the overall efficiency of pipeline operation.

[0043] 2. High precision and low false alarm rate: Through multi-stage processing of feature extractor, feature flattener, primary classifier and decision maker, the model can accurately identify pipeline anomalies. The feature extractor extracts key features, the feature flattener vectorizes the features, the primary classifier performs preliminary classification, and the decision maker outputs the results after comprehensive analysis. This multi-stage processing method effectively filters noise interference, significantly reduces the false alarm rate and the missed alarm rate, and improves the reliability of the detection results.

[0044] 3. Strong adaptability and flexibility: The model structure of the present application is flexible, and the number of branches and the number of cascade modules can be configured by users according to actual needs, which can adapt to pipeline detection tasks of different scales and complexities. Whether it is a long-distance oil pipeline or a city gas pipeline, the model can achieve optimal detection effect by adjusting parameters. In addition, the model can also adapt to complex environments inside and outside the pipeline, such as damp, greasy, corrosive environments, as well as external disturbances such as accumulations, heavy vehicles, etc.

[0045] In summary, the present application effectively solves the problems of low efficiency, poor precision and weak adaptability of traditional pipeline detection technology by proposing a multi-scale parallel cascade early exit network model, realizes efficient, accurate and real-time detection of pipeline abnormalities, and has simple structure. This technology not only significantly improves the safety and reliability of pipeline operation, but also reduces maintenance cost and time cost, and has wide application prospect and important practical significance. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions adopted by the embodiments of the present application, the drawings used in the embodiments will be briefly described. Obviously, the following listed drawings are only part of the embodiments of the present application, and other forms of drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings. Among them:

[0047] Figure 1 The present application provides a pipeline anomaly detection method, a storage medium and an overall flowchart of application;

[0048] Figure 2 The present application provides a multi-scale parallel cascade early exit network model structure diagram;

[0049] Figure 3 The present application provides a meta-detection module structure diagram. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the embodiments of the present application and the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] As shown in the drawings, a pipeline anomaly detection method, the specific steps are as follows: Figure 1

[0052] Step 1: data receiving

[0053] The distributed optical fiber is tightly wound on the surface of the pipeline, the running state of the pipeline is monitored in real time through the phase change of the optical fiber, and a two-dimensional gray image P is obtained ij , wherein the element is denoted as p ij , i=1, 2,..., m, j=1, 2,..., n, m and n are the number of rows and columns of P ij , respectively, and m=10, n=10 can be set.

[0054] Step 2: data preprocessing

[0055] ​The two-dimensional gray scale image collected in step 1 is normalized, and each pixel value is normalized to the range of 0-255 to obtain a space-time waterfall graph denoted as X ij , where the element is denoted as x ij , and the specific formula is as follows:

[0056]

[0057] , max(P ij ) and min(P ij ) are the maximum and minimum values of the matrix P ij , respectively.

[0058] The multi-scale parallel cascade early exit network model is constructed as shown in Figure 2 , which includes multiple parallel branches and one decision module, and each branch is composed of s front and rear cascaded element detection modules; the specific structure of the element detection module is shown in Figure 3 .

[0059] Step 3: Feature extraction

[0060] The space-time waterfall graph obtained above is placed in a convolutional network, which includes a convolutional layer with a convolution kernel size of 3x3, a stride of 1, a padding of 1, and an output channel number of 64, which is used to map the input space-time waterfall graph to a feature space and extract feature information related to the pipeline state. The feature extraction formula is as follows:

[0061] α0=ReLU(W0*x+b0)

[0062] , where x is the input space-time waterfall graph, W0 is the convolution kernel weight, b0 is the bias term, * represents the convolution operation, and ReLU is the activation function. The size of the output feature map α0 is 64xHxW, where H and W are the height and width of the input image. The extracted feature F is delivered to the feature expander as output 2; at the same time, F is connected to the decision maker as output 1.

[0063] Step 4: Feature expansion

[0064] The feature map α0 obtained above is placed in a Flatten module for operation, which rearranges the two-dimensional feature map into a one-dimensional vector f in element order, and the length of f is 64xHxW. The specific formula is as follows:

[0065] f=Flatten(F)

[0066] where Flatten denotes rearranging each element of the input into a one-dimensional vector from left to right and from top to bottom. This operation ensures the integrity and consistency of feature information during dimension conversion, and provides an adaptive input format for subsequent fully connected layers or other processing modules.

[0067] Step 4: Primary classification

[0068] The one-dimensional vector f obtained above is placed in a random forest classifier, which receives the one-dimensional feature vector f output from the feature flattener. The random forest classifier outputs a one-dimensional vector Each element is denoted as We can set c = 10, where c is the vector length, b is the branch number, and b = 1, 2,..., B. We can set B = 10. Specifically, the value of each element is between 0 and 1.

[0069] Step 5: Decide whether to exit early

[0070] This module receives the one-dimensional vector output by the primary classifier and the output 1 of the feature extractor. The decision maker will sort all elements in descending order and obtain the maximum class probability, denoted as max_probs, which is specifically:

[0071]

[0072] where max denotes the maximum value. Compare max_probs with a preset threshold u, where u is a decimal value between 0 and 1, and set u = 0.65. If max_probs ≥ u, output 1 as empty and output 2 as If max_probs < u, a, output 1 as input 1, i.e. α0, and output 2 as empty.

[0073] Step 7: Training

[0074] The random gradient descent method is used for training. The specific training steps are as follows:

[0075] First, initialize the model parameters θ = 0 and the learning rate α = 0.01, and randomly select a sample (x i , y i ) from the training data. Calculate the gradient of the loss function L(θ) for a single sample and update the parameters according to the gradient:

[0076]

[0077] Repeat the above steps until the maximum number of iterations is met, and set the maximum number of iterations to 10.

[0078] Step 8: Perform the test

[0079] Given the pipeline state data, it is input to the constructed multi-scale parallel cascade early retirement network model for calculation, finally, through the accurate calculation and analysis of the model, we get the result of pipeline anomaly detection. These results can clearly tell us whether there is an abnormal situation in the pipeline, and the specific location and degree of the anomaly, thereby providing strong support and basis for the maintenance and management of the pipeline.

[0080] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of pipeline anomaly detection, the method comprising: The steps are as follows: S1, in the training process, data receiving: real-time receiving of state data in the pipeline running process, denoted as matrix P ij , wherein the elements are denoted as p ij , i = 1, 2,..., m, j = 1, 2,..., n, and m and n are the number of rows and columns of P ij , respectively; S2, data preprocessing: the state data obtained in step S1 is preprocessed to obtain a space-time waterfall chart, denoted as X ij wherein the elements are denoted as x ij ; S3. Model Construction and Training: Construct a multi-scale parallel cascaded early regression network model, including B parallel branches and 1 decision module. Each branch consists of s cascaded meta-detection modules. Each meta-detection module includes: Feature extractor: Input spatiotemporal waterfall plot, output feature map α0; Feature flattener: flattens the feature map into a one-dimensional feature vector f; Primary classifier: f-classifier, outputs probability vector Decision maker: compare the maximum class probability max_probs of the with a threshold value u; if max_probs ≥ u, output to the decision module; if max_probs < u, pass the feature map α0 to the next level of meta-detection module; The decision module outputs the abnormal class category based on the weighted sum of the B branches weighted sum; output abnormal class category; S4. Anomaly Detection: Input the given pipeline state data into the trained model and output the anomaly detection result.

2. The pipeline anomaly detection method of claim 1, wherein: In step S1, the specific steps of the receiving pipeline operation process are as follows: the distributed optical fiber is tightly wound around the surface of the pipeline, and the operating status of the pipeline is monitored in real time by the phase change of the optical fiber to obtain a two-dimensional grayscale image.

3. The pipeline anomaly detection method of claim 2, wherein: In step S2, the preprocessing specifically includes: firstly, normalizing the two-dimensional grayscale image acquired in step S1, normalizing each pixel value to the range of 0 to 255, to obtain a spatiotemporal waterfall image, the specific formula of which is as follows: where max(P ij ) and min(P ij ) are the maximum and minimum values of the matrix P ij , respectively.

4. The pipeline anomaly detection method of claim 3, wherein: In step S3, the feature extractor is specifically a convolutional network containing one convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and 64 output channels. This network maps the input spatiotemporal waterfall plot to the feature space and extracts feature information related to the pipeline state. The feature extraction formula is as follows: α0 = ReLU(W0*x + b0) Where x is the input spatiotemporal waterfall plot, W0 is the convolution kernel weight, b0 is the bias term, * indicates the convolution operation, and ReLU is the activation function; the output feature map α0 has a size of 64×H×W, where H and W are the height and width of the input image, the extracted feature F is fed to the feature expander as output 2; at the same time, F is connected to the decision unit as output 1.

5. The pipeline anomaly detection method of claim 4, wherein: In step S3, the feature flattener specifically comprises: a feature flattening module that uses the Flatten operation to rearrange the two-dimensional feature map in element-wise order into a one-dimensional vector f with a length of 64×H×W; the specific formula is as follows: f = Flatten(F) Flatten means rearranging each element of the input into a one-dimensional vector in order from left to right and from top to bottom. This operation ensures that the feature information remains intact and consistent during the dimensionality transformation process, while providing an adapted input format for subsequent fully connected layers or other processing modules.

6. The pipeline anomaly detection method of claim 5, wherein: In the step S3, the primary classifier is specifically a random forest classifier, which receives a one-dimensional feature vector f output from the feature flattener; the random forest classifier outputs a one-dimensional vector Each element is denoted as k = 1, 2, …, c, where c is the vector length, b is the branch number, b = 1, 2, …, B; specifically, the value of each element is between 0 and 1.

7. The pipeline anomaly detection method of claim 6, wherein: In the step S3, the decision maker is specifically: the module receives the one-dimensional vector output by the initial classifier and the output 1 of the feature extractor, and the decision maker will sort all elements in the max_probs from large to small, and obtain the maximum category probability, denoted as max_probs, which is specifically: wherein max denotes taking the maximum value, max_probs is compared with a preset threshold value u, wherein u is a decimal number between 0 and 1 given by a human being; if max_probs≥u, then output 1 is empty and output 2 is If max_probs<u, then output 1 is input 1, i.e. α0, and output 2 is empty.

8. The pipeline anomaly detection method of claim 7, wherein: In the step S3, the decision module is specifically: according to the given B input First, the vector R is calculated k is a vector of length c, specifically: where β b is the weight of each branch, which is given by human and takes value between 0 and 1; then for vector R k The maximum value of each element is calculated, and the serial number corresponding to the maximum value is the result of anomaly detection.

9. Storage medium, characterized in that It stores a program capable of executing the pipeline anomaly detection method as described in any one of claims 1-8.

10. Use of a method for detecting anomalies in a pipeline, characterized in that, The storage medium as described in claim 9 can be modularly deployed in an embedded target platform or deployed to an external computing module.

Images