A method for identifying outliers of internal corrosion rate of natural gas pipeline based on improved DBSCAN-BPNN

Abstract

The application provides a natural gas pipeline internal corrosion rate outlier identification method based on improved DBSCAN-BPNN. The method can quickly and efficiently identify natural gas pipeline internal corrosion rate data outliers, ensuring normal evaluation of natural gas pipeline internal corrosion conditions. The method mainly includes six steps: step one, collecting natural gas pipeline internal corrosion rate data; step two, establishing an internal corrosion rate-neighborhood radius eps-density threshold Minpts database; step three, extracting sample data; step four, establishing a BPNN neural network model; step five, calculating the values of neighborhood radius eps and density threshold Minpts; and step six, DBSCAN clustering outlier identification. The step flow of the method is shown in the abstract drawing. The application proposes a new improved DBSCAN-BPNN method, which overcomes the problem that multiple tedious manual adjustments of the values of the two parameters of neighborhood radius eps and density threshold Minpts in the DBSCAN clustering method are required to obtain better outlier identification effect.

Description

Technical Field

[0001] This invention patent belongs to the field of oil and gas pipeline technology, and in particular, it is a method for identifying anomalies in corrosion rate inside natural gas pipelines based on an improved DBSCAN-BPNN. Background Technology

[0002] Natural gas pipelines are a crucial mode of transporting natural gas. With my country's ever-increasing consumption of natural gas, pipeline construction is proceeding rapidly, which will strongly guarantee national energy security and the development of the national economy. As the natural gas industry system flourishes, the issue of internal corrosion in natural gas pipeline systems is receiving increasing attention.

[0003] Because pipeline-transported natural gas may contain high levels of sulfur and other acidic substances, as well as small amounts of water, it can cause corrosion on the inner walls of natural gas pipelines. If left unchecked, this corrosion can worsen, leading to stress cracking, corrosion perforation, and ultimately pipeline failure. The oil and gas industry is a data-intensive industry, and natural gas pipelines generate a large amount of data during operation and maintenance. Corrosion rate data within natural gas pipelines can typically be obtained through methods such as pipeline inspection, placement of probes, and other techniques, providing data support for analyzing pipeline condition and assessing internal corrosion. However, outliers in the internal corrosion rate data are a significant problem, primarily caused by human factors such as negligence and misremembering, as well as objective mechanical factors such as sensor malfunction. The presence of outliers affects the assessment of corrosion within natural gas pipelines, necessitating outlier identification of the initial corrosion rate data. Currently, a commonly used method for outlier identification is the DBSCAN clustering algorithm, a density-based clustering algorithm that effectively identifies outliers.

[0004] However, when applying the DBSCAN clustering algorithm to identify outliers in corrosion rates within natural gas pipelines, it is necessary to manually adjust the neighborhood radius eps and density threshold Minpts parameters frequently to achieve good outlier identification results.

[0005] In the existing authorized patent CN107036572B, "A Method and Device for Obtaining Agricultural Machinery Operating Area," a density threshold empirical value is set to reduce the tedious operation of selecting parameters. In the authorized patent CN112345261B, "Anomaly Detection Method for Aircraft Engine Pump Adjustment System Based on Improved DBSCAN Algorithm," the dynamic time warping (DWT) distance between samples is calculated to improve the DBSCAN clustering algorithm, thus avoiding erroneous outlier identification caused by empirically determined thresholds. Meanwhile, in 2023, Zhang Lu et al. proposed a method based on simulation modeling to determine the neighborhood radius eps and density threshold Minpts using mesh generation and density ratio.

[0006] Based on the above analysis and literature review, this invention proposes a novel improved DBSCAN-BPNN method. Combining the characteristics of corrosion rate data in pipelines, BPNN can mine and learn the intrinsic relationship between corrosion rate data and neighborhood radius eps and density threshold Minpts. By inputting the corrosion rate data to be identified as outliers into BPNN, it can quickly and accurately output the specific values ​​of neighborhood radius eps and density threshold Minpts. Then, the DBSCAN clustering algorithm is used to accurately identify outliers in the corrosion rate. Summary of the Invention

[0007] Therefore, this invention provides an outlier identification method based on corrosion rate data within natural gas pipelines, BPNN, and DBSCAN clustering algorithms. This method can quickly and efficiently identify outliers in corrosion rate data within natural gas pipelines, ensuring accurate assessment of corrosion conditions. The method is characterized by the following steps: First, a database of internal corrosion rate, neighborhood radius (eps), and density threshold (Minpts) is established based on the corrosion rate data and DBSCAN clustering algorithm. Second, a BPNN neural network model is established, taking internal corrosion rate data as input and outputting neighborhood radius (eps) and density threshold (Minpts), extracting a large amount of data from the database to train the model. Third, the internal corrosion data to be identified is input, and the BPNN model calculates the neighborhood radius (eps) and density threshold (Minpts). Finally, the values ​​of these two parameters are substituted into the DBSCAN clustering algorithm to quickly and accurately identify outliers in the corrosion rate data within natural gas pipelines, obtaining the number of outliers and the number of data points.

[0008] A method for identifying anomalies in corrosion rates within natural gas pipelines based on an improved DBSCAN-BPNN mainly includes the following:

[0009] (1) Collect corrosion rate data in natural gas pipelines. Data can be collected by pipeline and may include various methods such as corrosion rate of placed clips, corrosion rate of probes, and corrosion rate of ultrasonic detection, as shown in Table 1;

[0010] Table 1. Corrosion rate data collected from pipelines.

[0011] Pipe Name Internal corrosion rate data B# [a1] B# <![CDATA[a2]]> …… …… G# <![CDATA[b1]]> G# <![CDATA[b2]]> …… …… N# <![CDATA[n n ]]>

[0012] (2) Establish an internal corrosion rate-neighborhood radius eps-density threshold Minpts database. The DBSCAN clustering algorithm was used to identify outliers in the collected internal corrosion rate data of each pipeline. The neighborhood radius eps and density threshold Minpts parameters in the algorithm were manually adjusted to complete the outlier identification of the internal corrosion rate data. The internal corrosion rate data of each pipeline, along with their corresponding neighborhood radius eps and density threshold Minpts, were integrated to establish a database, as shown in Table 2.

[0013] Table 2 Internal Corrosion Rate Data and Algorithm Parameter Database

[0014]

[0015]

[0016] (3) Extract sample data. 100 sample datasets were extracted from the established internal corrosion rate-neighborhood radius eps-density threshold Minpts database. The pipeline name, internal corrosion rate data, neighborhood radius eps, and density threshold Minpts correspond one-to-one, as shown in Table 3.

[0017] Table 3 shows the 100 sample datasets extracted from the table.

[0018] Serial Number Pipe Name Internal corrosion rate data Neighborhood radius eps Density threshold (Minpts) 1 B# <![CDATA[a1]]> <![CDATA[r1]]> <![CDATA[m1]]> 2 B# <![CDATA[a2]]> <![CDATA[r1]]> <![CDATA[m1]]> …… …… …… …… …… 10 G# <![CDATA[b1]]> <![CDATA[r2]]> <![CDATA[m2]]> 11 G# <![CDATA[b2]]> <![CDATA[r2]]> <![CDATA[m2]]> …… …… …… …… …… 99 J# <![CDATA[c1]]> <![CDATA[R1]]> <![CDATA[M1]]> 100 J# <![CDATA[c2]]> <![CDATA[R2]]> <![CDATA[M2]]>

[0019] (4) Establish a BPNN neural network model. Based on a dataset of 100 samples, the internal erosion rate data is used as input, and the neighborhood radius eps and density threshold Minpts are used as outputs to train the neural network. The input layer of the neural network has only one node for the internal erosion rate data; the output layer has two nodes for the neighborhood radius eps and density threshold Minpts; the hidden layer in the middle is one layer with 8 nodes.

[0020] (5) Calculate the neighborhood radius eps and density threshold Minpts. Based on the established BPNN neural network model, the corrosion rate data of a certain natural gas pipeline that needs to be identified as an outlier is used as input to calculate the values ​​of the neighborhood radius eps and density threshold Minpts under the corresponding conditions.

[0021] (6) DBSCAN Clustering Outlier Identification. The neighborhood radius eps and density threshold Minpts obtained in step five are substituted into the DBSCAN clustering algorithm. This eliminates the need for multiple manual adjustments of these parameters, enabling the identification of outliers in the corrosion rate data of a natural gas pipeline, as shown in Table 4.

[0022] Table 4. Results of outlier identification in pipeline corrosion rate data.

[0023] Attached Figure Description

[0024] Appendix Figure 1 Flowchart of an implementation method for identifying corrosion rate outliers in natural gas pipelines based on an improved DBSCAN-BPNN

[0025] Appendix Figure 2 Neural network structure diagram

[0026] Appendix Figure 3 Comparison chart of actual and predicted neighborhood radius values

[0027] Appendix Figure 4 Comparison chart of actual and predicted values ​​of density threshold Detailed Implementation

[0028] The specific implementation methods of the present invention will be described in detail below with reference to the accompanying drawings and examples, so that the viewpoints, features and application steps of the present invention can be easily understood and accepted by those skilled in the art, thereby making a clearer definition of the scope of protection of the present invention.

[0029] A method for identifying corrosion rate anomalies in natural gas pipelines based on an improved DBSCAN-BPNN mainly includes six steps, the process of which is shown in the attached figure. Figure 1 As shown, the specific steps are as follows:

[0030] Step 1: Collect corrosion rate data within the natural gas pipeline. Data can be collected for each pipeline section, and may include various methods such as corrosion rate measurement using inserts, probes, and ultrasonic testing, as shown in Table 1. Data can be obtained from natural gas pipeline inspection data and pipeline operation data.

[0031] Table 1. Corrosion rate data collected from pipelines.

[0032] Pipe Name Internal corrosion rate data B# <![CDATA[a1]]> B# <![CDATA[a2]]> …… …… G# <![CDATA[b1]]> G# <![CDATA[b2]]> …… …… N# <![CDATA[n n ]]>

[0033] Step 2: Establish a database of internal corrosion rate, neighborhood radius (eps), and density threshold (Minpts). The DBSCAN clustering algorithm is used to identify outliers in the collected internal corrosion rate data for each pipeline. The neighborhood radius (eps) and density threshold (Minpts) parameters in the algorithm are manually adjusted to complete the outlier identification process. The internal corrosion rate data of each pipeline, along with their corresponding neighborhood radius (eps) and density threshold (Minpts), are integrated to create a database, as shown in Table 2.

[0034] Table 2 Internal Corrosion Rate Data and Algorithm Parameter Database

[0035] Serial Number Pipe Name Internal corrosion rate data Neighborhood radius eps Density threshold (Minpts) 1 B# <![CDATA[a1]]> <![CDATA[r1]]> <![CDATA[m1]]> 2 B# <![CDATA[a2]]> <![CDATA[r1]]> <![CDATA[m1]]> …… …… …… …… …… 10 G# <![CDATA[b1]]> <![CDATA[r2]]> <![CDATA[m2]]> 11 G# <![CDATA[b2]]> <![CDATA[r2]]> <![CDATA[m2]]> …… …… …… …… …… n N# <![CDATA[n n ]]> <![CDATA[r n ]]> <![CDATA[m n ]]>

[0036] Step 3: Extract sample data. 100 sample datasets were retrieved from the established internal corrosion rate-neighborhood radius eps-density threshold Minpts database. The pipeline name, internal corrosion rate data, neighborhood radius eps, and density threshold Minpts correspond one-to-one, as shown in Table 3.

[0037] Table 3 shows the 100 sample datasets extracted from the table.

[0038] Serial Number Pipe Name Internal corrosion rate data Neighborhood radius eps Density threshold (Minpts) 1 B# <![CDATA[a1]]> <![CDATA[r1]]> <![CDATA[m1]]> 2 B# <![CDATA[a2]]> <![CDATA[r1]]> <![CDATA[m1]]> …… …… …… …… …… 10 G# <![CDATA[b1]]> <![CDATA[r2]]> <![CDATA[m2]]> 11 G# <![CDATA[b2]]> <![CDATA[r2]]> <![CDATA[m2]]> …… …… …… …… …… 99 J# <![CDATA[c1]]> <![CDATA[R1]]> <![CDATA[M1]]> 100 J# <![CDATA[c2]]> <![CDATA[R2]]> <![CDATA[M2]]>

[0039] Step four: Establish the BPNN neural network model. Based on a dataset of 100 samples, use the internal erosion rate data as input and the neighborhood radius eps and density threshold Minpts as output to train the neural network. (See attached image) Figure 2 As shown, the neural network consists of three layers. The input layer has only one node for the internal erosion rate data; the output layer has two nodes for the neighborhood radius eps and the density threshold Minpts; and the middle hidden layer has one layer with eight nodes. The first 70 sets of the 100 sample datasets were used as the training set for training the neural network; the last 30 sets were used as the test set to validate the data after the neural network training was completed. The training function was set to the Scaled Conjugate Gradient method.

[0040] Step 5: Calculate the neighborhood radius eps and density threshold Minpts. Based on the established neural network model, the corrosion rate data of a certain natural gas pipeline that needs to be identified as an outlier is used as input to calculate the values ​​of the neighborhood radius eps and density threshold Minpts under the corresponding conditions.

[0041] Step Six: DBSCAN Clustering Outlier Identification. The neighborhood radius (eps) and density threshold (Minpts) values ​​obtained in Step Five are substituted into the DBSCAN clustering algorithm. This eliminates the need for multiple manual adjustments of these parameters, enabling rapid and accurate identification of outliers in the corrosion rate data of a specific natural gas pipeline, as shown in Table 4.

[0042] Table 4. Results of outlier identification in pipeline corrosion rate data.

[0043]

[0044] The following section uses the method described in this invention to identify outliers in the corrosion rate data of a test natural gas pipeline:

[0045] The first step, following the method described in step one, is to collect corrosion rate data for multiple natural gas pipelines. Partial corrosion rate data for the pipelines are shown in Table 5.

[0046] Table 5. Internal corrosion rate data for different pipes

[0047] Pipe Name Internal corrosion rate data B# 0.0074 B# 0.0154 B# 0.0330 …… …… C# 0.0054 C# 0.0675 C# 0.0545 …… …… D# 0.0304 D# 0.0025 …… ……

[0048] The second step is to establish the internal corrosion rate-neighborhood radius eps-density threshold Minpts database according to the method described in step two.

[0049] The third step involved extracting 100 sets of Minpts datasets—corrosion rate-neighborhood radius eps-density threshold—from the database, following the method described in step three.

[0050] Fourth, following the method described in step four, train and save the neural network model. The predicted neighborhood radius obtained by the neural network model and its actual value are as follows: Figure 3 As shown, the average error is 4.85%; the predicted value of the density threshold and its true value are as follows: Figure 4 As shown, the average error is 3%, which meets the fitting accuracy requirements.

[0051] Fifth step: Following the method described in step five, the trained neural network model is used to input the internal corrosion rate data of the test pipeline, and the values ​​of the two parameters, neighborhood radius eps and density threshold Minpts, are calculated to be 1.1 and 3.19, respectively.

[0052] Step 6: Following the method described in Step 6, based on the calculated values ​​of the two parameters, the DBSCAN clustering algorithm is used to identify outliers in the corrosion rate data within the test pipeline. The identification results are shown in Table 6.

[0053] Table 6. Results of anomaly identification in corrosion rate data within the test pipeline.

[0054]

Claims

1. A method for identifying outlier corrosion rates in natural gas pipelines based on an improved DBSCAN-BPNN, characterized in that, Includes the following steps: Step 1: Collect corrosion rate data inside the natural gas pipeline. Data can be collected by pipeline and may include various methods such as corrosion rate detection by placing a clip, corrosion rate detection by probe, and ultrasonic detection of corrosion rate. Step 2: Use the DBSCAN clustering algorithm to identify outliers in the collected internal corrosion rate data of each pipeline. Manually adjust the two parameters in the algorithm, namely the neighborhood radius eps and the density threshold Minpts, to finally complete the outlier identification of the internal corrosion rate data. Integrate the internal corrosion rate data of each pipeline with its corresponding neighborhood radius eps and density threshold Minpts to establish a database. Step 3: Take 100 sets of sample datasets from the established internal corrosion rate-neighborhood radius eps-density threshold Minpts database, where the pipe name, internal corrosion rate data, neighborhood radius eps, and density threshold Minpts correspond one-to-one. Step 4: Based on the 100 sets of sample datasets, take the internal corrosion rate data as input, the neighborhood radius eps and density threshold Minpts as output, establish a BPNN neural network model, and train the neural network. Step 5: Based on the established neural network model, the corrosion rate data of a certain natural gas pipeline that needs to be identified as an outlier is used as input to calculate the values ​​of the neighborhood radius eps and density threshold Minpts under the corresponding conditions. Step 6: Substitute the two parameter values, neighborhood radius eps and density threshold Minpts, obtained from Step 5 into the DBSCAN clustering algorithm. This eliminates the need for manual adjustment of parameter values ​​multiple times, enabling rapid and accurate identification of outliers in corrosion rate data within a natural gas pipeline.

2. The method for identifying outlier corrosion rates in natural gas pipelines based on an improved DBSCAN-BPNN as described in claim 1, characterized in that, Step 4, establishing the BPNN neural network model, includes the following steps: S41: Using the internal corrosion rate data as input and the neighborhood radius eps and density threshold Minpts as output, a BPNN neural network model is established and the neural network is trained. S42: The BPNN neural network consists of three layers. The input layer has only one node for the internal erosion rate data; the output layer has two nodes for the neighborhood radius eps and the density threshold Minpts; the middle hidden layer is one layer with eight nodes; 70% of the 100 sample dataset is designated as the training set for training the neural network; and 30% of the dataset is designated as the test set to validate the data after the neural network training is completed. The training function is set to the quantized conjugate gradient method.

Images