Abnormal data detection method for industrial sensor networks based on GCN-LSTM

Abstract

The present invention discloses a method for detecting abnormal data in an industrial sensor network based on GCN-LSTM. The method first obtains historical data in an industrial sensor network and uses it as a training set; uses the training set and adopts the Spearman correlation analysis method to obtain an adjacency matrix; uses the T-distributed stochastic neighbor embedding (TSNE) method to obtain the two-dimensional coordinates of sensor data; draws an industrial sensor network logic diagram through the adjacency matrix and the two-dimensional coordinates of the sensor; proposes a deep learning model that embeds a long short-term memory network (LSTM) in a graph convolutional network (GCN), and trains it using a standardized training set; and uses the trained model to detect abnormal data in an industrial sensor network. The present invention gives a specific algorithm description using a real industrial sensor network data set as test data, and obtains experimental results through a series of experiments.

Description

Technical Field

[0001] The present invention relates to abnormal data detection in industrial sensor networks, and in particular to an abnormal data detection method in industrial sensor networks based on GCN-LSTM. Background Art

[0002] With the close integration of IoT technology and industrial control systems, industrial control systems are developing in the direction of informatization, automation, and intelligence, but also exposing many existing security issues of industrial control systems. The safe operation of industrial control systems requires the joint assistance of a large number of IoT devices, and the reliable operation of IoT devices depends on a large number of sensors, which constitute the industrial sensor network. There are a large number of vulnerabilities in IoT devices, which can be exploited by attackers and penetrated layer by layer, and malicious behaviors such as issuing wrong instructions or blocking alarm signals will be taken, eventually causing large-scale damage to the physical equipment of the industrial control system, posing a major security threat to people's livelihood, social stability, and national security. For attackers, no matter what means of attack they use, the ultimate goal is to change the operating state of the industrial control system and destroy the physical equipment. The change of sensor data is closely related to the operating state of the industrial control system. Therefore, timely and accurate detection of abnormal sensor data is a key measure to ensure the safe, reliable and stable operation of the industrial control system.

[0003] In industrial sensor networks, there are many types of sensor data, such as pH sensors, temperature sensors, speed sensors, and water depth measurement sensors, and there are complex temporal and spatial coupling relationships, which makes it extremely difficult to analyze and detect abnormal sensor data. The spatial coupling relationship of sensors means that there are certain connections and mutual influences between sensor data. For example, if a large amount of tap water is added to a water tank, then as the depth of the water increases, the pH and temperature will inevitably change. At the same time, sensor data is time series data and there is a temporal coupling relationship. For example, the data value at time T must change based on the data value at time T-1. Therefore, fully extracting the temporal and spatial characteristics of sensors is of great significance for realizing accurate detection of abnormal sensor data. However, most of the existing research work only focuses on analyzing the spatial characteristics or temporal characteristics of sensors. Therefore, how to comprehensively consider the spatial and temporal characteristics of sensor data has become an important problem in realizing abnormal data detection in industrial sensor networks. Summary of the invention

[0004] The purpose of this invention is to solve the problem of how to fully extract the temporal and spatial characteristics of sensor data of industrial sensor networks, and proposes a complete analysis method to address the deficiencies in the existing research on anomaly detection of sensor data of industrial sensor networks; the proposed GCN-LSTM-based industrial sensor network anomaly data detection model has guiding significance for the research on anomaly detection of industrial control systems.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention discloses a method for detecting abnormal data in an industrial sensor network based on GCN-LSTM, comprising the following steps:

[0007] (1) Obtain historical data from industrial sensor networks and use it as a training set, label it as normal or abnormal, and perform standardization;

[0008] (2) Using the training set of the industrial sensor network, the Spearman correlation analysis and t-distributed stochastic neighbor embedding (TSNE) dimensionality reduction method are used to construct the adjacency matrix and logical graph of the industrial sensor network;

[0009] (3) The GCN and LSTM-based detection model is trained using the training set of the industrial sensor network to obtain a trained classifier;

[0010] (4) Collecting industrial sensor network data in real time and performing standardized processing, inputting the industrial sensor network adjacency matrix obtained in step (2) and the standardized real-time data into the classifier trained in step (3), and outputting normal or abnormal classification labels.

[0011] Preferably, the step (1) is specifically:

[0012] The formula for normalizing sensor data in industrial sensor networks is as follows:

[0013]

[0014] In the formula, x j is the jth sensor value, Max train,j is the maximum value of the jth sensor value in the training set, Min train,j is the minimum value of the jth sensor value in the training set, x' j is the normalized value of the jth sensor.

[0015] Preferably, the step (2) is specifically:

[0016] (a) Using the training set, the TSNE method is used to cluster the sensor data and obtain the two-dimensional coordinates of each sensor. The formula is as follows:

[0017] (aix j ,aiy j ) = TSNE(x' j )

[0018] In the formula, (aix j ,aiyj ) is the two-dimensional coordinate of the jth sensor obtained by the TSNE method. TSNE(·) expresses the TSNE clustering function.

[0019] (b) The Spearman correlation analysis method is used to calculate the correlation between sensors using the training set of the industrial sensor network, and the adjacency matrix is ​​constructed. The formula is as follows:

[0020]

[0021]

[0022]

[0023] In the formula, q z,w is the correlation coefficient between sensor z and sensor w. The data of sensor z and sensor w are sorted from small to large, d z,w,k represents the difference between the values ​​of the kth pair of sensor z and sensor w in this sorted data set; n z,w Indicates the number of values ​​of sensor z or sensor w in the training set. z,w It means that when the correlation coefficient is q z,w The size of the P value is used to facilitate P value verification. A represents the adjacency matrix, which records the sensor numbers with correlation. q represents the correlation coefficient threshold. When the correlation coefficient is greater than q, it means that there is a correlation between the two sensors. β represents the significance level. Only when p z,w When the value is less than β, the hypothesis can be accepted that there is a correlation between the two sensors.

[0024] (c) Based on the two-dimensional coordinates of each sensor obtained in step (a) and the adjacency matrix obtained in step (b), a logical graph of the industrial sensor network is constructed. The adjacency matrix indicates which sensors should be connected.

[0025] Preferably, the step (3) is specifically:

[0026] (a) Use GCN to extract spatial features. The formula is as follows:

[0027]

[0028] Where l represents the l-th layer of neural network, including the l-th graph convolution layer, the l-th LSTM layer and the l-th residual layer; Where A represents the adjacency matrix, I represents the identity matrix; δ(·) represents the activation function; is the degree matrix of A; W (l) is the weight matrix of GCN in layer l; y G,l is the output of the l-th layer of GCN.

[0029] (b) Use LSTM to extract time series features. The LSTM process is as follows:

[0030] y L,l =LSTM l (y G,l )

[0031] In the formula, LSTM l (·) represents the LSTM function in the lth layer, y L,l is the output of the lth layer LSTM.

[0032] (c) The residual layer is used to add the input and output of the lth layer to prevent the gradient from disappearing or exploding. The specific formula is as follows:

[0033] y L+G,l =y L,l +y G,l

[0034] In the formula, y L+G,l Represents the output of layer l.

[0035] Preferably, the step (4) is specifically:

[0036] Collect industrial sensor network data in real time and standardize it, input the logic diagram and standardized real-time data into the trained classifier, and output the classification results in real time.

[0037] Industrial sensor networks are very different from ordinary Internet and ordinary wireless sensor networks. Industrial sensor networks have complex temporal and spatial coupling relationships. Accurate anomaly detection in industrial sensor networks is of great significance for ensuring the safe and stable operation of industrial control systems, maintaining social stability and ensuring national security. However, existing anomaly detection modeling methods are difficult to fully extract the temporal and spatial characteristics of industrial sensor networks at the same time.

[0038] In view of the characteristics of sensor data in industrial sensor networks, the present invention proposes a method for detecting abnormal data in industrial sensor networks based on GCN-LSTM. Its innovation and technical contribution are mainly reflected in the following aspects: (1) A deep learning detection model that embeds LSTM into GCN is proposed to fully extract the temporal and spatial characteristics of sensor data. (2) The Spearman correlation analysis and TSNE method are used to construct the logical graph of the industrial sensor network to replace the topological graph of the sensor network, and it is used as the input of the GCN-LSTM detection model to fully extract the spatial characteristics of the sensor data. (3) Using a real industrial sensor network dataset, it is proved that the detection model proposed in the present invention improves the detection accuracy and is more in line with the requirements of the industrial control system for the accuracy of abnormal detection.

[0039] The present invention solves the difficult problem of how to establish an anomaly detection model by combining the temporal and spatial characteristics of industrial sensor networks. The industrial sensor network anomaly detection model based on GCN and LSTM can fully extract the temporal information and spatial information of the sensor and output accurate detection results, which provides an important information basis for the subsequent reconstruction of the attack chain and the provision of corresponding countermeasures. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2(a) is the logic diagram of the industrial sensor network when q = 0.4;

[0043] Figure 2(b) is the logic diagram of the industrial sensor network when q = 0.5;

[0044] Figure 2(c) is the logic diagram of the industrial sensor network when q = 0.6;

[0045] Figure 3 This is the classification result diagram of the detection model under different q values;

[0046] Figure 4 It is a comparison chart of the classification results of the present invention and different detection models. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below with reference to the accompanying drawings, and the purpose and effects of the present invention will become more apparent.

[0048] like Figure 1 As shown, the GCN-LSTM-based industrial sensor network abnormal data detection method of the present invention includes the following steps:

[0049] (1) Obtain historical data from industrial sensor networks and use it as a training set, label it as normal or abnormal, and perform standardization;

[0050] Standardization processing, specifically:

[0051] (a) Standardize the sensor data in the industrial sensor network. The formula is as follows:

[0052]

[0053] In the formula, x j is the jth sensor value, Max train,j is the maximum value of the jth sensor value in the training set, Min train,j is the minimum value of the jth sensor value in the training set, x' jis the normalized value of the jth sensor.

[0054] (2) Using the training set of the industrial sensor network, the Spearman correlation analysis and t-distributed stochastic neighbor embedding (TSNE) dimensionality reduction method are used to construct the adjacency matrix and logical graph of the industrial sensor network;

[0055] (a) Using the training set, the TSNE method is used to reduce the dimension of the sensor data and obtain the two-dimensional coordinates of each sensor. The formula is as follows:

[0056] (aix j ,aiy j ) = TSNE(x' j )

[0057] In the formula, (aix j ,aiy j ) is the two-dimensional coordinate of the jth sensor obtained by the TSNE method; TSNE(·) expresses the TSNE clustering function;

[0058] (b) The Spearman correlation analysis method is used to calculate the correlation between sensors using the training set of the industrial sensor network, and the adjacency matrix is ​​constructed. The formula is as follows:

[0059]

[0060]

[0061]

[0062] In the formula, q z,w is the correlation coefficient between sensor z and sensor w; the data of sensor z and sensor w are sorted from small to large, d z,w,k represents the difference between the values ​​of the kth pair of sensor z and sensor w in this sorted data set; n z,w represents the number of values ​​of sensor z or sensor w in the training set; p z,w It means that when the correlation coefficient is q z,w , so as to facilitate P value verification; A represents the adjacency matrix, which records the sensor numbers with correlation; q represents the correlation coefficient threshold. When the correlation coefficient is greater than q, it means that there is a correlation between the two sensors; β represents the significance level. z,w When the value is less than β, the hypothesis can be accepted, that is, there is a correlation between the two sensors;

[0063] (c) constructing a logical graph of the industrial sensor network based on the two-dimensional coordinates of each sensor obtained in step (a) and the adjacency matrix obtained in step (b); wherein the adjacency matrix indicates which sensors should be connected.

[0064] (3) The GCN and LSTM-based detection model is trained using the training set of the industrial sensor network to obtain a trained classifier;

[0065] (a) Use GCN to extract spatial features. The formula is as follows:

[0066]

[0067] Where l represents the l-th layer of neural network, including the l-th graph convolution layer, the l-th LSTM layer and the l-th residual layer; Where A represents the adjacency matrix, I represents the identity matrix; δ(·) represents the activation function; is the degree matrix of A; W (l) is the weight matrix of GCN in layer l; y G,l is the output of the lth layer of GCN;

[0068] (b) Using LSTM to extract time series features, the process is as follows:

[0069] y L,l =LSTM l (y G,l )

[0070] In the formula, LSTM l (·) represents the LSTM function in the lth layer, y L,l is the output of the lth layer LSTM;

[0071] (c) The residual layer is used to add the input and output of the lth layer to prevent the gradient from disappearing or exploding. The specific formula is as follows:

[0072] y L+G,l =y L,l +y G,l

[0073] In the formula, y L+G,l Represents the output of layer l.

[0074] (4) Collecting industrial sensor network data in real time and performing standardized processing, inputting the industrial sensor network adjacency matrix obtained in step (2) and the standardized real-time data into the classifier trained in step (3), and outputting normal or abnormal classification labels.

[0075] Among them, after the classifier is trained with a certain training set, the test set of the industrial sensor network (part of the historical data in the industrial sensor network) can be used, and the standardized test set and the adjacency matrix are input into the trained classifier, and the classification effect is evaluated using the following evaluation indicators:

[0076]

[0077]

[0078]

[0079] In the formula, TP is the number of abnormal samples correctly classified as abnormal samples; FP is the number of normal samples incorrectly classified as abnormal samples, and FN is the number of abnormal samples incorrectly classified as normal samples; F1-score is the harmonic mean of Precision and Recall, and is also an indicator for balancing the two. After the classifier meets the evaluation index requirements, the classifier is used in the real-time abnormal data detection application of step (4).

[0080] The present invention is further described below with reference to a specific example.

[0081] Through the real industrial sensor network data set, an industrial sensor network abnormal data detection model based on GCN-LSTM with appropriate parameters is established. The original data is first divided into a training set and a test set, and then standardized; then the training set is used to construct the adjacency matrix and logical graph of the industrial sensor network, and the classifier based on GCN-LSTM is trained; finally, the test set is used to test the detection effect of the classification model.

[0082] Different logic diagrams can be drawn according to different correlation coefficient thresholds q, where the significance level β = 0.05. Figure 2 (a) is the logic diagram of the industrial sensor network when q = 0.4; Figure 2 (b) is the logic diagram of the industrial sensor network when q = 0.5; Figure 2 (c) is the logic diagram of the industrial sensor network when q = 0.6; Figure 2(a)-Figure 2(c) It can be seen that when the q value is different, the obtained logical graph of the industrial sensor network is also different. The larger the q is, the sparser it is.

[0083] The GCN-LSTM structural parameters used in the present invention are as follows:

[0084] Table 1

[0085]

[0086]

[0087] The experimental environment of the present invention is shown in the following table:

[0088] Table 2

[0089]

[0090] The method proposed in the present invention is evaluated using the Precision, Recall and F1-score indicators. In the experiment, the learning rate is set to 0.001, the batch_size is 128, the optimizer is Adam, and the loss function is the criterion function.

[0091] The q value affects the adjacency matrix, which is one of the inputs of the GCN-LSTM-based industrial sensor network abnormal data detection model proposed in this paper. Therefore, the q value will affect the classification results of the detection model. Using the sensor data set, different correlation coefficient thresholds q are first selected for comparative experiments, and finally the q parameter value with the highest F1-score is selected. Figure 3 It is the classification result diagram of the detection model under different q values. Figure 3 It can be concluded that when q = 0.6, the maximum F1-score is 73.86%. Therefore, q = 0.6 is selected as the final q parameter.

[0092] Finally, the GCN- and LSTM-based detection models are compared with the GCN- and LSTM-based detection models. Figure 4 The results show that when the detection model based on GCN and LSTM is used, the Precision, Recall and F1-score are the largest, which are 95.45%, 60.24% and 73.86% respectively. This also proves that the deep learning detection model based on GCN and LSTM proposed in the present invention can fully extract the time and feature space of industrial sensor data and improve the detection accuracy.

[0093] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting abnormal data in industrial sensor networks based on GCN-LSTM, characterized by: The following steps are included (1) Obtain historical data from industrial sensor networks and use it as a training set, label it as normal or abnormal, and perform standardization; (2) Using the training set of the industrial sensor network, the Spearman correlation analysis and t-distributed stochastic neighbor embedding (TSNE) dimensionality reduction method are used to construct the adjacency matrix and logical graph of the industrial sensor network; (3) The GCN and LSTM-based detection model is trained using the training set of the industrial sensor network to obtain a trained classifier; The step (3) is specifically: (a) Use GCN to extract spatial features. The formula is as follows: Where l represents the l-th layer of neural network, including the l-th graph convolution layer, the l-th LSTM layer and the l-th residual layer; Where A represents the adjacency matrix, I represents the identity matrix; δ(·) represents the activation function; is the degree matrix of A; W (l) is the weight matrix of GCN in layer l; y G,l is the output of the l-th layer GCN; (b) Using LSTM to extract time series features, the process is as follows: y L,l =LSTM l (y G,l ) In the formula, LSTM l (·) represents the LSTM function in the lth layer, y L,l is the output of the lth layer LSTM; (c) The residual layer is used to add the input and output of the lth layer to prevent the gradient from disappearing or exploding. The specific formula is as follows: and L+G,l =and L,l +y G,l In the formula, y L+G,l represents the output of the lth layer; (4) Collecting industrial sensor network data in real time and performing standardized processing, inputting the industrial sensor network adjacency matrix obtained in step (2) and the standardized real-time data into the classifier trained in step (3), and outputting normal or abnormal classification labels.

2. According to claim 1, a method for detecting abnormal data in industrial sensor networks based on GCN-LSTM is characterized in that: The standardization process in step (1) is specifically as follows: The formula for normalizing sensor data in industrial sensor networks is as follows: In the formula, x j is the jth sensor value, Max train,j is the maximum value of the jth sensor value in the training set, Min train,j is the minimum value of the jth sensor value in the training set, x' j is the normalized value of the jth sensor.

3. According to claim 1, a method for detecting abnormal data in industrial sensor networks based on GCN-LSTM is characterized in that: The step (2) is specifically: (a) Using the training set, the TSNE method is used to reduce the dimension of the sensor data and obtain the two-dimensional coordinates of each sensor. The formula is as follows: (oh my) j oh my j )=TSNE(x' j ) In the formula, (aix j ,aiy j ) is the two-dimensional coordinate of the jth sensor obtained by the TSNE method; TSNE(·) expresses the TSNE clustering function; (b) The Spearman correlation analysis method is used to calculate the correlation between sensors using the training set of the industrial sensor network, and the adjacency matrix is ​​constructed. The formula is as follows: In the formula, q z,w is the correlation coefficient between sensor z and sensor w; the data of sensor z and sensor w are sorted from small to large, d z,w,k represents the difference between the values ​​of the kth pair of sensor z and sensor w in this sorted data set; n z,w represents the number of values ​​of sensor z or sensor w in the training set; p z,w It means that when the correlation coefficient is q z,w , so as to facilitate P value verification; A represents the adjacency matrix, which records the sensor numbers with correlation; q represents the correlation coefficient threshold. When the correlation coefficient is greater than q, it means that there is a correlation between the two sensors; β represents the significance level. z,w When the value is less than β, the hypothesis can be accepted, that is, there is a correlation between the two sensors; (c) constructing a logical graph of the industrial sensor network based on the two-dimensional coordinates of each sensor obtained in step (a) and the adjacency matrix obtained in step (b); wherein the adjacency matrix indicates which sensors should be connected.

4. According to claim 1, a method for detecting abnormal data in industrial sensor networks based on GCN-LSTM is characterized in that: The step (4) is specifically: Collect industrial sensor network data in real time and standardize it, input the logic diagram and standardized real-time data into the trained classifier, and output the classification results in real time.

Images