Natural gas pipeline anomaly detection method and system based on hybrid deep neural network

Abstract

The application provides a natural gas pipeline anomaly detection method and system based on a hybrid deep neural network, wherein the method comprises normalizing feature data of a natural gas pipeline to obtain input feature values; constructing a stacked sparse denoising autoencoder deep neural network model as a first hybrid deep neural network according to the input feature values; constructing a cost function according to the input feature values, performing unsupervised feature learning of the first hybrid deep neural network by using the cost function, and obtaining a second hybrid deep neural network; adding a supervised classifier to the second hybrid deep neural network to obtain a third hybrid deep neural network; inputting the input feature values into the third hybrid deep neural network to obtain output feature values; calculating a maximum probability value of the output feature values by using the supervised classifier; and reducing a difference between the maximum probability value of the output feature values and a label. The application can improve anomaly detection accuracy and reduce the false positive rate of anomaly detection when an intrusion attack or an anomaly occurs.

Description

Technical Field

[0001] This invention belongs to the field of natural gas pipeline safety inspection technology, and particularly relates to a natural gas pipeline anomaly detection method and system based on hybrid deep neural networks. Background Technology

[0002] Natural gas energy is transported via long-distance pipelines, connecting various regions to form a complex and large-scale pipeline transportation network. Pipeline transportation offers advantages such as low cost, high safety, and resource conservation, but it is not without risk. When the monitoring, management, and control of a natural gas pipeline control system are compromised by cyberattacks, it can lead to the loss of visibility and control over the pipeline, potentially causing physical harm to the pipeline and nearby individuals in the event of an accident, and also having adverse environmental impacts. Therefore, how to employ efficient anomaly detection methods to ensure optimal safety of natural gas pipeline control systems has become a crucial issue in current industrial control system anomaly detection.

[0003] Intrusion detection systems (IDS) and anomaly detection systems help detect intrusions or anomalies in industrial control system networks, playing a crucial role in protecting the information security of data acquisition and monitoring control systems. However, IDS need to be used in conjunction with other security mechanisms, such as firewalls, vulnerability scanners, and security policy verifiers. Natural gas pipeline control systems include human-machine interfaces, compressors, pressure gauges, electromagnetic relief valves, and the sealed pipes connecting them. Natural gas pipeline control systems are affected by parameter configuration issues such as compressor operating status, gas pressure values, and proportional-integral-derivative (PID) parameters, which necessitates specific methods for intrusion or anomaly detection.

[0004] As an industrial control system, the anomaly detection problem of natural gas pipeline control systems can be summarized as an anomaly detection problem based on deep neural networks. The research challenge lies in the impracticality of manually labeling massive datasets, as this is time-consuming and prone to errors. Traditional anomaly detection methods suffer from low accuracy and high false detection rates. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a method and system for detecting anomalies in natural gas pipelines based on hybrid deep neural networks.

[0006] In a first aspect, the present invention provides a method for detecting anomalies in natural gas pipelines based on a hybrid deep neural network, comprising:

[0007] Normalize the characteristic data of the natural gas pipeline to obtain the input characteristic values;

[0008] A stacked sparse denoising autoencoder deep neural network model is constructed based on the input feature values ​​as the first hybrid deep neural network;

[0009] A cost function is constructed based on the input feature values. Unsupervised feature learning of the first hybrid deep neural network is performed using the cost function to obtain the second hybrid deep neural network.

[0010] A supervised classifier is added to the second hybrid deep neural network to obtain the third hybrid deep neural network;

[0011] The input feature values ​​are fed into a third hybrid deep neural network to obtain the output feature values;

[0012] Calculate the maximum probability value of the output feature value using a supervised classifier;

[0013] The output feature value is defined based on the label value; among the output feature values, data with a label value of 0 are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0.

[0014] A third hybrid deep neural network is trained to reduce the difference between the maximum probability value of the output feature value and the label.

[0015] Further, the construction of a stacked sparse denoising autoencoder deep neural network model based on the input feature values ​​as the first hybrid deep neural network includes:

[0016] The input feature values ​​are used as the input to the first sparse denoising autoencoder, and the decoder of the first sparse denoising autoencoder is discarded; wherein the hidden layer of the first sparse denoising autoencoder includes multiple neurons of a first hybrid deep neural network, the neurons representing the category to which normal data or abnormal data belongs.

[0017] The hidden layer of the first sparse denoising autoencoder is used as the input of the second sparse denoising autoencoder, and the decoder of the second sparse denoising autoencoder is discarded.

[0018] The process continues until the hidden layer of the (d-1)th sparse denoising autoencoder is used as the input of the dth sparse denoising autoencoder, and the decoder of the dth sparse denoising autoencoder is discarded; where d is a constant greater than 2.

[0019] By using sparse parameters, the number of hidden layer neurons in each sparse denoising autoencoder is set to be greater than the number of input layer neurons in each sparse denoising autoencoder, thus obtaining the first hybrid deep neural network.

[0020] Further, the step of constructing a final cost function based on the input feature values, and using the final cost function to perform unsupervised feature learning of the first hybrid deep neural network to obtain the second hybrid deep neural network includes:

[0021] The expression for constructing the final cost function is:

[0022]

[0023] Where, j sparsity (θ) is the final cost function after adding sparse variables; θ is the parameter variable of the sparse denoising autoencoder; m is the number of input feature values ​​used to train the first hybrid deep neural network; H w denoted as ... i For training the i-th input value of the sparse denoising autoencoder; y i To train the i-th output value of the sparse denoising autoencoder; The label is represented by β, which is the weight of the sparsity penalty factor; k is the number of neurons in the hidden layer; ρ is the sparsity parameter, which is the sparse representation of the input feature values ​​in the hidden layer. is the average activation value of the q-th neuron in the hidden layer on all unsupervised feature learning samples.

[0024] Furthermore, the addition of a supervised classifier to the second hybrid deep neural network to obtain a third hybrid deep neural network includes:

[0025] The Softmax function layer is used as a supervised classifier and as the activation function of the output layer of the second hybrid deep neural network to obtain the third hybrid deep neural network.

[0026] Furthermore, the step of calculating the maximum probability value of the output feature value using a supervised classifier includes:

[0027] Calculate the probability of the output feature value using the following formula:

[0028]

[0029] Where p is the predicted probability of the g-th class of input feature value f, i.e., the probability of the output feature value; e is the natural constant; w is the threshold for the activation of the compressor safety valve in the natural gas pipeline; t is the gas pressure value in the natural gas pipeline; l is the number of input feature value classes; w g and w j The safety valve activation thresholds associated with the input feature values ​​of classes g and j are respectively; t g and t j These are the gas pressure values ​​in the natural gas pipeline associated with the input feature values ​​of classes g and j, respectively.

[0030] The maximum value of p is calculated using the arg max function, which yields the maximum probability value of the output feature value.

[0031] Furthermore, training the third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label includes:

[0032] The first sparse denoising autoencoder of the third hybrid deep neural network is trained; the encoded representation features of the first hidden layer are used as input to the second sparse denoising autoencoder of the third hybrid deep neural network.

[0033] The second sparse denoising autoencoder of the third hybrid deep neural network is trained, and the encoded representation features of the second hidden layer are used as the input of the third sparse denoising autoencoder of the third hybrid deep neural network.

[0034] Until the u-th sparse denoising autoencoder of the third hybrid deep neural network is trained, the (u-1)-th hidden layer is used as the input of the u-th sparse denoising autoencoder of the third hybrid deep neural network;

[0035] The Softmax function layer is trained under supervision using labeled input feature values;

[0036] The difference between the maximum probability value of the output feature and the label is minimized using the cross-entropy cost function.

[0037] Furthermore, the step of minimizing the difference between the maximum probability value of the output feature value and the label using the cross-entropy cost function includes:

[0038] Construct the cross-entropy cost function expression:

[0039]

[0040] Among them, α = {γ1, γ2, ┄, γ λ} represents the true probability distribution of the label; σ = {δ1, δ2, ..., δ} λ} represents the maximum probability distribution of the output feature value; λ is the number of label categories; ε≤λ means that the output feature value belongs to the ε-th category of the label.

[0041] Secondly, the present invention provides a natural gas pipeline anomaly detection system based on a hybrid deep neural network, comprising:

[0042] The data normalization module is used to normalize the characteristic data of natural gas pipelines to obtain input characteristic values;

[0043] The first building module is used to construct a stacked sparse denoising autoencoder deep neural network model based on the input feature values ​​as the first hybrid deep neural network.

[0044] The second construction module is used to construct a cost function based on the input feature values, and to perform unsupervised feature learning of the first hybrid deep neural network using the cost function to obtain the second hybrid deep neural network.

[0045] Add a module to add a supervised classifier to the second hybrid deep neural network to obtain the third hybrid deep neural network;

[0046] The feature value input module is used to input the input feature values ​​into the third hybrid deep neural network to obtain the output feature values;

[0047] The calculation module is used to calculate the maximum probability value of the output feature value using the supervised classifier;

[0048] The label value definition module is used to define the output feature value based on the label value; among the output feature values, data with a label value of 0 are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0;

[0049] The neural network training module is used to train a third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label.

[0050] Thirdly, the present invention provides a computer device including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the natural gas pipeline anomaly detection method based on a hybrid deep neural network described in the first aspect.

[0051] Fourthly, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the natural gas pipeline anomaly detection method based on a hybrid deep neural network as described in the first aspect.

[0052] This invention provides a method and system for anomaly detection in natural gas pipelines based on hybrid deep neural networks. The method includes: normalizing the feature data of the natural gas pipeline to obtain input feature values; constructing a stacked sparse denoising autoencoder deep neural network model as a first hybrid deep neural network based on the input feature values; constructing a cost function based on the input feature values, and performing unsupervised feature learning on the first hybrid deep neural network using the cost function to obtain a second hybrid deep neural network; adding a supervised classifier to the second hybrid deep neural network to obtain a third hybrid deep neural network; inputting the input feature values ​​into the third hybrid deep neural network to obtain output feature values; defining the output feature values ​​according to the label values; wherein, data with a label value of 0 in the output feature values ​​are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0; calculating the maximum probability value of the output feature values ​​using the supervised classifier; and training the third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature values ​​and the label.

[0053] This invention preprocesses the input feature values ​​to construct a stacked sparse denoising autoencoder deep neural network as a hybrid deep neural network. On this basis, a supervised classifier is added for unsupervised feature learning training. This can improve the coordination value of anomaly detection accuracy and recall rate and reduce the false positive rate of anomaly detection when the data acquisition and monitoring control system is attacked or anomalies occur. Attached Figure Description

[0054] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a natural gas pipeline anomaly detection method based on a hybrid deep neural network, provided in an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a natural gas pipeline anomaly detection method based on a hybrid deep neural network provided in an embodiment of the present invention;

[0057] Figure 3 Hybrid deep neural network diagram provided for embodiments of the present invention;

[0058] Figure 4 A diagram of a hybrid deep neural network and a supervised classifier provided in an embodiment of the present invention;

[0059] Figure 5 This is a diagram illustrating the distributed data training process of a hybrid deep neural network provided in an embodiment of the present invention.

[0060] Figure 6 This is a diagram illustrating the types of training and testing sets for classified natural gas pipeline data provided in this embodiment of the invention.

[0061] Figure 7 A comparison chart of the reconciled average values ​​of precision and recall provided in the embodiments of the present invention;

[0062] Figure 8 A comparison chart of false positive rates provided in the embodiments of the present invention;

[0063] Figure 9 This is a comparison chart of single-machine training and distributed cluster training time provided in an embodiment of the present invention;

[0064] Figure 10 This is a schematic diagram of a natural gas pipeline anomaly detection system based on a hybrid deep neural network, provided as an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] The natural gas pipeline control system includes sealed piping connecting the compressor, pressure gauge, and electromagnetically controlled relief valve, as well as a human-machine interface. The programmable logic controller (PLC) of the natural gas pipeline control system uses a proportional-integral-derivative (PID) control scheme to maintain gas pressure in the pipeline by selecting when the compressor safety valve is activated. To maintain the required gas pressure setpoint, the natural gas pipeline system uses the compressor to add gas to the system; if the compressor status value is 1, the compressor is on; if the compressor status value is 0, the compressor is off.

[0067] Table 1 shows the normal and abnormal data categories of the natural gas pipeline control system dataset. Three types of anomalies illustrating attacks on the natural gas pipeline control system are listed below. The natural gas pipeline control system operates in both automatic and manual control modes. In automatic mode, the programmable logic controller (PLC) determines the compressor's operating state. However, sophisticated malicious response injection attacks (label 1 in Table 1) can modify compressor state values ​​as abnormal data, thus masking the compressor's actual operating state. It's important to note that in manual mode, the compressor's operating state is controlled by the manual compressor setpoint. Malicious state command injection attacks (label 2 in Table 1) can continuously or intermittently change the compressor's operating state as abnormal data. Furthermore, five proportional-integral-derivative (PID) attributes related to the PLC—gain, reset, dead zone, rate, and cycle time—affect the PLC's behavior. These attributes should be fixed during system operation, but malicious parameter command injection attacks (label 3 in Table 1) can attempt to modify these parameters, thereby disrupting normal control operations.

[0068] In one embodiment, such as Figure 1 and Figure 2 As shown, this embodiment of the invention provides a method for detecting anomalies in natural gas pipelines based on a hybrid deep neural network, including:

[0069] Step 101: Normalize the feature data of the natural gas pipeline to obtain the input feature values.

[0070] The feature data of the natural gas pipeline is scaled proportionally so that the results fall within a specific interval [0, 1]. For example, a minimum-maximum scaling method is performed using a normalization formula, which is: Where x represents the feature data; x' represents the normalized value, i.e., the input feature value; x min and x max These are the minimum and maximum values ​​of the feature data, respectively.

[0071] The input feature values ​​are proportionally divided into three non-overlapping subsets: the training dataset, the validation dataset, and the test dataset, for example, with proportions of 60%, 20%, and 20%, respectively; and quantities of 141,707, 47,236, and 47,236, respectively.

[0072] The training dataset is used to train the subsequently built hybrid deep neural network model, and the model performance is periodically evaluated using the validation dataset to avoid overfitting. If the neural network overfits, i.e., when the accuracy on the validation dataset plateaus or declines, training is stopped and the parameters and hyperparameters of the neural network itself are adjusted.

[0073] The final prediction model after the neural network training is completed is evaluated using a test dataset, and the test dataset is used only once.

[0074] Based on the data distribution, random sampling is repeatedly performed from the abnormal data samples where the characteristic data of natural gas pipelines account for a relatively small proportion, and the resulting samples are expanded into the feature value dataset to achieve class balance.

[0075] When representing feature classification, the feature values ​​of natural gas pipeline data are encoded into a vector composed of bit information, with only one column set to 1 and the rest to 0. This vector is then compared with the probability vector output by the hybrid deep neural network anomaly detection model to calculate the loss score. A Python program creates a `one_hot_encoding()` function to perform one-hot encoding on the feature data. This function has two parameters: the first is the array to be encoded, and the second is the number of categories. The function uses a for loop to retrieve each element of the array, creates an array of all zeros (as per the second parameter), and then calls the Python program's `np.put()` function to set the position of each array element to 1. This completes the one-hot encoding.

[0076] Table 1 shows the feature data divided into one category of normal behavior (label 0) and five categories of aggressive behavior (labels 1-5) using a multivariate classification method. To avoid unnecessary training bias, the training set, which accounts for 60% of the total data, was balanced, while the test set, which accounts for 20% of the total data, used the original system data. Figure 6 To perform multivariate classification of abnormal data, the training and test sets of natural gas pipeline data are used to create type graphs, and finally, all data labels are individually encoded using thermal encoding.

[0077] Table 1. Data Record Classification Table

[0078] Label type describe 0 NORMAL Non-aggressive behavior 1 CMRI Complex malicious response injection attack 2 MSCI Malicious state command injection attack 3 MPCI Malicious parameter command injection attack 4 MFCI Malicious function code command injection attack 5 RECONNAISSANCE Reconnaissance attack

[0079] Step 102: Construct a stacked sparse denoising autoencoder deep neural network model based on the input feature values ​​as the first hybrid deep neural network.

[0080] like Figure 3 As shown; the input feature values ​​are used as the input of the first sparse denoising autoencoder, using z = (z1, z2, ..., z...). n The decoder of the first sparse denoising autoencoder is discarded; wherein, the hidden layer h1 of the first sparse denoising autoencoder includes multiple nodes, which are neurons of the first hybrid deep neural network, and the neurons represent the category to which normal or abnormal data belongs. For example, the number of input layer nodes is 24, denoted by z = (z1, z2, ..., z...). 24 The first sparse denoising autoencoder's hidden layer h1 consists of 58 nodes.

[0081] The hidden layer h1 of the first sparse denoising autoencoder is used as the input to the second sparse denoising autoencoder, and the decoder of the second sparse denoising autoencoder is discarded. For example,

[0082] Until the hidden layer h of the (d-1)th sparse denoising autoencoder is... d-1 The d-th sparse denoising autoencoder is used as input, and the decoder of the d-th sparse denoising autoencoder is discarded; where d is a constant greater than 2.

[0083] Output layer nodes are represented by y = (y1, y2, ..., y s ) represents the weight matrix and bias from the input layer to the hidden layer in a stacked sparse denoising autoencoder deep neural network, respectively. n×d and b = (b1, b2, ..., b d-1 The output layer has 6 nodes, representing 6 different types of labeled datasets, denoted by y = (y1, y2, ..., y6). The weight matrix and bias from the input layer to the hidden layer of the stacked sparse denoising autoencoder deep neural network are represented by w. 6×2 b1 represents.

[0084] When setting the number of layers in a deep neural network, the deeper the network, the longer it takes to train. Therefore, by using sparse parameters to set the number of hidden layer neurons in each sparse denoising autoencoder to be greater than the number of input layer neurons, a first hybrid deep neural network is obtained.

[0085] When designing a stacked sparse denoising autoencoder deep neural network, the training parameters, hyperparameters, number of hidden layers, training epochs, and number of nodes in each layer are defined as shown in Table 2. When using multivariate classification to partition the dataset, the number of nodes in the first hidden layer is set to 58, and the number of nodes in the second hidden layer is set to 34.

[0086] Table 2 Information on Stacked Sparse Denoising Autoencoder Deep Neural Network

[0087]

[0088] Step 103: Construct a cost function based on the input feature values, and use the cost function to perform unsupervised feature learning of the first hybrid deep neural network to obtain the second hybrid deep neural network.

[0089] In this step, the parameters of the autoencoder are first optimized using the mean squared error cost function. The basic cost function expression is:

[0090]

[0091] Where θ represents the parameter variables of the sparse denoising autoencoder; m represents the number of input feature values ​​used to train the first hybrid deep neural network; H w denoted as ... i To train the ith input value of the sparse denoising autoencoder; y i To train the i-th output value of the sparse denoising autoencoder; Indicates a label.

[0092] Then, unsupervised feature learning is performed on the first hybrid deep neural network using the basic cost function. For example, a weight decay is added for regularization to the basic cost function expression, and a sparsity parameter is added to control the number of activated neurons in each layer of the first hybrid deep neural network. The cost function is adjusted accordingly to j sparsity (θ)=j(θ)+βΩ sparsity , where j sparsity (θ) is the final cost function after adding sparse variables; β is the weight of the sparsity penalty factor; Ω sparsity This is a sparse expression, used to allow only a small number of neurons in the hidden layer to be in an active state. Its mathematical expression is:

[0093]

[0094] Where β is the weight of the sparsity penalty factor; k is the number of neurons in the hidden layer; ρ is the sparsity parameter, which is the sparse representation of the input feature values ​​in the hidden layer, and is set to 0.05 in this embodiment; Let be the average activation value of the q-th neuron in the hidden layer across all unsupervised feature learning samples. In summary, the final cost function expression is:

[0095]

[0096] Step 104: Add a supervised classifier to the second hybrid deep neural network to obtain the third hybrid deep neural network.

[0097] The Softmax function layer is used as a supervised classifier and as the activation function of the output layer of the second hybrid deep neural network to obtain the third hybrid deep neural network.

[0098] Step 105: Input the input feature values ​​into the third hybrid deep neural network to obtain the output feature values.

[0099] Step 106: Define output feature values ​​based on the label values; where data with a label value of 0 are considered normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0.

[0100] Step 107: Calculate the maximum probability value of the output feature value using a supervised classifier.

[0101] For example, input feature values ​​are defined based on the label values; where data with a label value of 0 are considered normal data, and data with a label value of r are defined as abnormal data.

[0102] Calculate the probability of the output feature value using the following formula:

[0103]

[0104] Where p is the predicted probability of the g-th class of input feature value f, i.e., the probability of the output feature value; e is the natural constant; w is the threshold for the activation of the compressor safety valve in the natural gas pipeline; t is the gas pressure value in the natural gas pipeline; l is the number of input feature value classes; w g and w j Let t be the safety valve activation threshold associated with the input feature values ​​of classes g and j, respectively; g and t j Let g and j be the gas pressure values ​​in the natural gas pipeline associated with the input feature values ​​of the g and j classes, respectively.

[0105] The maximum value of p is calculated using the arg max function to obtain the maximum probability value of the output feature value; the maximum probability value of the output feature value is close to the value of the label.

[0106] For example, the actual output of the second hybrid deep neural network is [-0.124, -0.62, 0.899, 6.925, -0.332, 2.064], and the probability distribution of the class calculated by the Softmax function layer is [0.001, 0.001, 0.002, 0.987, 0.001, 0.008].

[0107] The Softmax function layer acts as a supervised classifier, calculating the probability distribution of all classes. Typically, recall, precision, false positive rate (FPR), false negative rate (FNR), and the concordance rate between precision and recall (F1-score) are used as evaluation metrics for anomaly detection methods. The formulas for these metrics are as follows: Recall is the proportion of correctly detected anomalies; precision measures the accuracy of identifying the percentage of anomalies; false positive rate is the proportion of normal behavior incorrectly identified as anomalous behavior; false negative rate is the proportion of anomalous behavior incorrectly identified as normal behavior. Since precision and recall are mutually restrictive, a harmonic mean of precision and recall is used to comprehensively weigh the two. TP represents the number of anomalous behaviors correctly labeled by the classifier; TN represents the number of normal behaviors correctly labeled by the classifier; FN represents the number of anomalous behaviors incorrectly labeled as normal by the classifier; and FP represents the number of normal behaviors incorrectly labeled as anomalous by the classifier.

[0108] This embodiment uses the concordant average of precision and recall (F1-score) and false positive rate (FPR) as evaluation metrics for anomaly detection methods. An anomaly detection method based on a hybrid deep neural network is compared with other anomaly detection algorithms. Standard anomaly detection algorithms used for comparison include Decision Tree, Naive Bayes, and Random Forest. The comparison results are as follows: Figure 7 As shown, the proposed anomaly detection method achieves a reconciled average precision and recall that is significantly higher than several other standard anomaly detection algorithms, both in detecting normal behavioral features and in detecting five types of anomalous attacks. Its advantages are particularly pronounced compared to decision tree and Naive Bayes algorithms. Furthermore, as... Figure 8 As shown, further verification shows that the proposed method has a lower false positive rate than other standard anomaly detection algorithms in detecting five types of abnormal attack features.

[0109] Step 108: Train a third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label.

[0110] For example, the first sparse denoising autoencoder of the third hybrid deep neural network is trained; the encoded representation features of the first hidden layer are used as input to the second sparse denoising autoencoder of the third hybrid deep neural network.

[0111] The second sparse denoising autoencoder of the third hybrid deep neural network is trained, and the encoded representation features of the second hidden layer are used as input to the third sparse denoising autoencoder of the third hybrid deep neural network.

[0112] The process continues until the u-th sparse denoising autoencoder of the third hybrid deep neural network is trained, with the (u-1)-th hidden layer used as the input to the u-th sparse denoising autoencoder of the third hybrid deep neural network.

[0113] The Softmax function layer is trained under supervision using labeled input feature values.

[0114] Construct the cross-entropy cost function expression:

[0115]

[0116] Among them, α = {γ1, γ2, ┄, γ λ} represents the true probability distribution of the label; σ = {δ1, δ2, ..., δ} λ} represents the maximum probability distribution of the output feature value; λ is the number of label categories; ε≤λ means that the output feature value belongs to the ε-th category of the label.

[0117] The difference between the maximum probability value of the output feature and the label is minimized using the cross-entropy cost function.

[0118] like Figure 4 As shown, the parameters of the third hybrid deep neural network are determined through unsupervised feature learning and classification training. These parameters are then applied to gradient descent to further fine-tune the network. The anomaly detection capability of the final prediction model after training is evaluated using a test set until the network converges and the error is minimized.

[0119] Furthermore, a distributed training method with parallel data is used to reduce the time required to compute gradient descent, thereby reducing the time required to detect anomalies in natural gas pipelines.

[0120] For example, such as Figure 5As shown, based on the TensorFlow framework, the training set is transferred back and forth between the variable server and working points A, B, and C. The variable server stores the weights Ω of the third hybrid deep neural network and assigns them to the working points as workload; the weights Ω represent the number of dead zones in the programmable logic controller (PLC) of the natural gas pipeline. The working points are responsible for calculating the gradients in the backpropagation training algorithm. The gradient calculation results from the working points are fed back to the variable server, representing the difference in the number of dead zones in the PLCs. This difference is used to optimize the weights Ω of the hybrid deep neural network and then propagated back to the working points, thereby reducing the time spent calculating gradient descent.

[0121] like Figure 9 This paper presents the time consumption of training a hybrid deep neural network anomaly detection model using both single-machine and distributed cluster methods, under different training iterations. The results are as follows... Figure 9 As shown, training a hybrid deep neural network model using distributed methods takes significantly less time than training a single machine, thereby improving computational efficiency and accelerating the anomaly detection process.

[0122] This invention preprocesses the input feature values ​​to construct a stacked sparse denoising autoencoder deep neural network as a hybrid deep neural network. On this basis, a supervised classifier is added for unsupervised feature learning training. This can improve the coordination value of anomaly detection accuracy and recall rate and reduce the false positive rate of anomaly detection when the data acquisition and monitoring control system is attacked or anomalies occur.

[0123] Based on the same inventive concept, this invention also provides a natural gas pipeline anomaly detection system based on a hybrid deep neural network. Since the principle of this system in solving the problem is similar to that of the aforementioned natural gas pipeline anomaly detection method based on a hybrid deep neural network, the implementation of this system can refer to the implementation of the natural gas pipeline anomaly detection method based on a hybrid deep neural network, and the repeated parts will not be described again.

[0124] In another embodiment, one embodiment of the present invention provides a natural gas pipeline anomaly detection system based on a hybrid deep neural network, such as... Figure 10 As shown, it includes:

[0125] The data normalization module 10 is used to normalize the characteristic data of the natural gas pipeline to obtain the input characteristic values.

[0126] The first building module 20 is used to build a stacked sparse denoising autoencoder deep neural network model as the first hybrid deep neural network based on the input feature values.

[0127] The second construction module 30 is used to construct a cost function based on the input feature values, and to perform unsupervised feature learning of the first hybrid deep neural network using the cost function to obtain the second hybrid deep neural network.

[0128] Add module 40 to add a supervised classifier to the second hybrid deep neural network to obtain the third hybrid deep neural network.

[0129] The feature value input module 50 is used to input the input feature values ​​into the third hybrid deep neural network to obtain the output feature values.

[0130] The calculation module 60 is used to calculate the maximum probability value of the output feature value using a supervised classifier.

[0131] The label value definition module 70 is used to define the output feature value based on the label value; among the output feature values, data with a label value of 0 are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0.

[0132] The neural network training module 80 is used to train a third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label.

[0133] Optionally, the first building module includes:

[0134] The first building unit is used to take the input feature value as the input of the first sparse denoising autoencoder and discard the decoder of the first sparse denoising autoencoder; wherein the hidden layer of the first sparse denoising autoencoder includes a plurality of neurons of a first hybrid deep neural network, the neurons representing the category to which normal data or abnormal data belongs.

[0135] The second building block is used to take the hidden layer of the first sparse denoising autoencoder as the input of the second sparse denoising autoencoder and discard the decoder of the second sparse denoising autoencoder.

[0136] The third building unit is used to take the hidden layer of the (d-1)th sparse denoising autoencoder as the input of the dth sparse denoising autoencoder and discard the decoder of the dth sparse denoising autoencoder; where d is a constant greater than 2.

[0137] The fourth building unit is used to set the number of hidden layer neurons of each sparse denoising autoencoder to be greater than the number of input layer neurons of each sparse denoising autoencoder by using sparse parameters, so as to obtain the first hybrid deep neural network.

[0138] Optionally, the second building module includes:

[0139] The fifth building block, used to construct the final cost function, is expressed as follows:

[0140]

[0141] Where, j sparsity(θ) is the final cost function after adding sparse variables; θ is the parameter variable of the sparse denoising autoencoder; m is the number of input feature values ​​used to train the first hybrid deep neural network; H w denoted as ... i To train the ith input value of the sparse denoising autoencoder; y i To train the i-th output value of the sparse denoising autoencoder; The label is represented by β, which is the weight of the sparsity penalty factor; k is the number of neurons in the hidden layer; ρ is the sparsity parameter, which is the sparse representation of the input feature values ​​in the hidden layer. is the average activation value of the q-th neuron in the hidden layer on all unsupervised feature learning samples.

[0142] Optionally, the added module includes:

[0143] A unit is added to use the Softmax function layer as a supervised classifier and as the activation function of the output layer of the second hybrid deep neural network, resulting in a third hybrid deep neural network.

[0144] Optionally, the computing module includes:

[0145] The first calculation unit is used to calculate the probability of outputting the feature value according to the following formula:

[0146]

[0147] Where p is the predicted probability of the g-th class of input feature value f, i.e., the probability of the output feature value; e is the natural constant; w is the threshold for the activation of the compressor safety valve in the natural gas pipeline; t is the gas pressure value in the natural gas pipeline; l is the number of input feature value classes; w g and w j The safety valve activation thresholds associated with the input feature values ​​of classes g and j are respectively; t g and t j These are the gas pressure values ​​in the natural gas pipeline associated with the input feature values ​​of classes g and j, respectively.

[0148] The second calculation unit is used to calculate the maximum value of p using the arg max function, thereby obtaining the maximum probability value of the output feature value.

[0149] Optionally, the neural network training module includes:

[0150] The first training unit is used to train the first sparse denoising autoencoder of the third hybrid deep neural network; the encoded representation features of the first hidden layer serve as the input to the second sparse denoising autoencoder of the third hybrid deep neural network.

[0151] The second training unit is used to train the second sparse denoising autoencoder of the third hybrid deep neural network, and the encoded representation features of the second hidden layer are used as input to the third sparse denoising autoencoder of the third hybrid deep neural network.

[0152] The third training unit is used to train the u-th sparse denoising autoencoder of the third hybrid deep neural network, taking the (u-1)-th hidden layer as the input of the u-th sparse denoising autoencoder of the third hybrid deep neural network.

[0153] The fourth training unit is used to supervise the training of the Softmax function layer using labeled input feature values.

[0154] The difference minimization unit is used to minimize the difference between the maximum probability value of the output feature value and the label using the cross-entropy cost function.

[0155] The sixth building block is used to construct the cross-entropy cost function expression:

[0156]

[0157] Among them, α = {γ1, γ2, ┄, γ λ} represents the true probability distribution of the label; σ = {δ1, δ2, ..., δ} λ} represents the maximum probability distribution of the output feature value; λ is the number of label categories; ε≤λ means that the output feature value belongs to the ε-th category of the label.

[0158] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0159] In another embodiment, the present invention also provides a computer device including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the natural gas pipeline anomaly detection method based on a hybrid deep neural network disclosed in the foregoing embodiments.

[0160] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0161] In another embodiment, the present invention also provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the aforementioned disclosed method for detecting anomalies in natural gas pipelines based on hybrid deep neural networks.

[0162] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0163] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0164] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0165] The present invention has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present invention. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the scope of the present invention. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A method for detecting anomalies in natural gas pipelines based on hybrid deep neural networks, characterized in that, include: Normalize the characteristic data of the natural gas pipeline to obtain the input characteristic values; A stacked sparse denoising autoencoder deep neural network model is constructed based on the input feature values ​​as the first hybrid deep neural network; A cost function is constructed based on the input feature values. Unsupervised feature learning of the first hybrid deep neural network is performed using the cost function to obtain the second hybrid deep neural network. A supervised classifier is added to the second hybrid deep neural network to obtain the third hybrid deep neural network; The input feature values ​​are fed into a third hybrid deep neural network to obtain the output feature values; The output feature value is defined based on the label value; among the output feature values, data with a label value of 0 are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0. Calculate the maximum probability value of the output feature value using a supervised classifier; A third hybrid deep neural network is trained to reduce the difference between the maximum probability value of the output feature value and the label.

2. The natural gas pipeline anomaly detection method according to claim 1, characterized in that, The construction of a stacked sparse denoising autoencoder deep neural network model based on input feature values ​​as the first hybrid deep neural network includes: The input feature values ​​are used as the input to the first sparse denoising autoencoder, and the decoder of the first sparse denoising autoencoder is discarded; wherein the hidden layer of the first sparse denoising autoencoder includes multiple neurons of a first hybrid deep neural network, the neurons representing the category to which normal data or abnormal data belongs. The hidden layer of the first sparse denoising autoencoder is used as the input of the second sparse denoising autoencoder, and the decoder of the second sparse denoising autoencoder is discarded. The process continues until the hidden layer of the (d-1)th sparse denoising autoencoder is used as the input of the dth sparse denoising autoencoder, and the decoder of the dth sparse denoising autoencoder is discarded; where d is a constant greater than 2. By using sparse parameters, the number of hidden layer neurons in each sparse denoising autoencoder is set to be greater than the number of input layer neurons in each sparse denoising autoencoder, thus obtaining the first hybrid deep neural network.

3. The natural gas pipeline anomaly detection method according to claim 1, characterized in that, The step of constructing a final cost function based on the input feature values, and using the final cost function to perform unsupervised feature learning of the first hybrid deep neural network to obtain the second hybrid deep neural network includes: The expression for constructing the final cost function is: Where, j sparsity (θ) is the final cost function after adding sparse variables; θ is the parameter variable of the sparse denoising autoencoder; m is the number of input feature values ​​used to train the first hybrid deep neural network; H w denoted as ... i For training the i-th input value of the sparse denoising autoencoder; y i To train the i-th output value of the sparse denoising autoencoder; The label is represented by β, which is the weight of the sparsity penalty factor; k is the number of neurons in the hidden layer; ρ is the sparsity parameter, which is the sparse representation of the input feature values ​​in the hidden layer. is the average activation value of the q-th neuron in the hidden layer on all unsupervised feature learning samples.

4. The natural gas pipeline anomaly detection method according to claim 1, characterized in that, The process of adding a supervised classifier to the second hybrid deep neural network to obtain a third hybrid deep neural network includes: The Softmax function layer is used as a supervised classifier and as the activation function of the output layer of the second hybrid deep neural network to obtain the third hybrid deep neural network.

5. The natural gas pipeline anomaly detection method according to claim 4, characterized in that, The calculation of the maximum probability value of the output feature value using a supervised classifier includes: Calculate the probability of the output feature value using the following formula: Where p is the predicted probability of the g-th class of input feature value f, i.e., the probability of the output feature value; e is the natural constant; w is the threshold for the activation of the compressor safety valve in the natural gas pipeline; t is the gas pressure value in the natural gas pipeline; l is the number of input feature value classes; w g and w j The safety valve activation thresholds associated with the input feature values ​​of classes g and j are respectively; t g and t j These are the gas pressure values ​​in the natural gas pipeline associated with the input feature values ​​of classes g and j, respectively. The maximum value of p is calculated using the arg max function, which yields the maximum probability value of the output feature value.

6. The natural gas pipeline anomaly detection method according to claim 5, characterized in that, The training of the third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label includes: The first sparse denoising autoencoder of the third hybrid deep neural network is trained; the encoded representation features of the first hidden layer are used as input to the second sparse denoising autoencoder of the third hybrid deep neural network. The second sparse denoising autoencoder of the third hybrid deep neural network is trained, and the encoded representation features of the second hidden layer are used as the input of the third sparse denoising autoencoder of the third hybrid deep neural network. Until the u-th sparse denoising autoencoder of the third hybrid deep neural network is trained, the (u-1)-th hidden layer is used as the input of the u-th sparse denoising autoencoder of the third hybrid deep neural network; The Softmax function layer is trained under supervision using labeled input feature values; The difference between the maximum probability value of the output feature and the label is minimized using the cross-entropy cost function.

7. The natural gas pipeline anomaly detection method according to claim 6, characterized in that, The method of minimizing the difference between the output feature value and the label value using the cross-entropy cost function includes: Construct the cross-entropy cost function expression: Among them, α = {γ1, γ2, ┄, γ λ } represents the true probability distribution of the label; σ = {δ1, δ2, ..., δ} λ } represents the maximum probability distribution of the output feature value; λ is the number of label categories; ε≤λ means that the output feature value belongs to the ε-th category of the label.

8. A natural gas pipeline anomaly detection system based on a hybrid deep neural network, characterized in that, include: The data normalization module is used to normalize the characteristic data of natural gas pipelines to obtain input characteristic values; The first building module is used to construct a stacked sparse denoising autoencoder deep neural network model based on the input feature values ​​as the first hybrid deep neural network. The second construction module is used to construct a cost function based on the input feature values, and to perform unsupervised feature learning of the first hybrid deep neural network using the cost function to obtain the second hybrid deep neural network. Add a module to add a supervised classifier to the second hybrid deep neural network to obtain the third hybrid deep neural network; The feature value input module is used to input the input feature values ​​into the third hybrid deep neural network to obtain the output feature values; The calculation module is used to calculate the maximum probability value of the output feature value using the supervised classifier; The label value definition module is used to define the output feature value based on the label value; among the output feature values, data with a label value of 0 are normal data, and data with a label value of r are defined as abnormal data, where r is an integer greater than 0; The neural network training module is used to train a third hybrid deep neural network to reduce the difference between the maximum probability value of the output feature value and the label.

9. A computer device, characterized in that, It includes a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the natural gas pipeline anomaly detection method based on a hybrid deep neural network as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; when the computer programs are executed by a processor, they implement the steps of the natural gas pipeline anomaly detection method based on a hybrid deep neural network as described in any one of claims 1-7.

Images