Industrial control network situation prediction method and system based on deep learning, storage medium and equipment

Abstract

The invention discloses an industrial control network situation prediction method and system based on deep learning, a storage medium and equipment, and belongs to the field of industrial control network security. The method comprises the following steps: preprocessing data, standardizing the data and dividing a time window; generating a node feature matrix and an adjacent matrix according to the preprocessed data, and determining a graph structure and a weight; secondly, spatial feature extraction is carried out; and processing the spatial dependency relationship between the nodes by utilizing a self-attention mechanism through the GAT layer, learning the mutual relationship between the devices, and updating the node feature representation. And then time feature extraction is carried out. The forward and backward dependency relationship of the time sequence data is captured by means of the forward and backward LSTM of the BiLSTM layer, and the gating mechanism controls the information flow, so that the time sequence modeling capability is improved. And then prediction and output are carried out. And finally, training and optimizing the model. According to the method, topology and time characteristics are fully fused, and the accuracy and efficiency of abnormal flow detection are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial control network security, and specifically relates to an industrial control network situation prediction method, system, storage medium and equipment based on deep learning. Background Art

[0002] With the digitization and networking of industrial control systems, industrial control networks are increasingly used in critical infrastructure and production processes. However, the openness, heterogeneity and complexity of industrial control networks also make them face more and more network security threats. Traditional network security protection methods, such as rule-based intrusion detection systems (IDS) and firewalls, are difficult to cope with complex and changeable attack methods, especially in industrial environments. In order to improve the security of industrial control networks, situation prediction becomes particularly important. Security situation prediction not only involves understanding the current network status, but also includes the prediction of potential threats in the future. Traditional security situation awareness methods mainly rely on expert rules or statistical analysis. Although this method is simple, it has limited ability to capture complex attack patterns, massive traffic data and time series relationships. In recent years, detection methods based on machine learning and deep learning have gradually become mainstream, but the existing methods generally have the following problems:

[0003] (1) It is difficult to capture complex topological features: Industrial control networks usually have a graph structure, and existing methods are difficult to fully explore the correlation and dependency between nodes.

[0004] (2) Insufficient utilization of time characteristics: Industrial control network traffic has strong time dependence, and traditional methods fail to effectively utilize this characteristic, resulting in limited detection effects.

[0005] (3) Difficulty in model training and optimization: Deep learning models usually require a large number of parameters, and existing methods are prone to slow convergence or local optimality during the optimization process.

[0006] Based on the above problems, the present invention proposes a method combining a graph attention neural network (GAT) and a time series modeling method (BiLSTM), which can perform deep fusion of topological features and temporal characteristics, and use the Adam optimization algorithm to improve the model training effect and detection efficiency. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the present invention provides a method for detecting abnormal traffic in an industrial control network based on the combination of GAT and BiLSTM. Through this method, the topological structure and time characteristics of the industrial control network traffic can be fully utilized to improve the accuracy and efficiency of abnormal traffic detection.

[0008] The technical solution of the present invention:

[0009] A method for industrial control network situation prediction based on deep learning, comprising:

[0010] S1. Collect and preprocess multi-dimensional data of industrial control network traffic;

[0011] S2. Build a deep learning model for industrial control network security situation prediction;

[0012] The industrial control network security situation prediction deep learning model includes five parts: input layer, GAT layer, BiLSTM layer, output layer, training and optimization;

[0013] S3, spatial feature extraction, processing the spatial dependencies between nodes in the graph through the GAT layer;

[0014] S4, temporal feature extraction, capturing the temporal dependencies of time series through BiLSTM;

[0015] S5, Situation prediction and response: The output layer converts the output result H of the BiLSTM layer into BiLSTM The output is sent to the fully connected layer, which uses a linear activation function to output continuous prediction values.

[0016] S6, model training and optimization, optimizing the parameters of the GAT layer and BiLSTM layer through loss function and optimization algorithm;

[0017] S7, detection result output, is used to output the prediction result of the model and complete the industrial control network situation prediction function.

[0018] Furthermore, the collected data comes from key devices such as programmable logic controllers, remote terminal units, switches, etc. in the industrial control network. The GAT layer calculates the attention weights of nodes through the attention mechanism to capture topological information, and the BiLSTM layer captures the bidirectional features of the time series through bidirectional processing. During the training process, the Adam algorithm updates the model parameters according to the first-order moment estimation and second-order moment estimation of the gradient, including the steps of calculating the gradient, updating the first-order moment estimation, updating the second-order moment estimation, deviation correction, and parameter updating. The detection result output can predict the traffic of the industrial control network and ensure the security of the industrial control network.

[0019] A deep learning-based industrial control network situation prediction system includes a data acquisition and preprocessing module, an industrial control network security situation prediction deep learning module, and a detection result output module; the data acquisition and preprocessing module is used to collect and preprocess multi-dimensional data of industrial control network traffic, the industrial control network security situation prediction deep learning module predicts the industrial control network situation by constructing an industrial control network security situation prediction deep learning model, and the detection result output module is used to output the prediction result of the model to complete the prediction of the industrial control network situation.

[0020] A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned industrial control network situation prediction method based on deep learning.

[0021] A server comprises a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement an industrial control network situation prediction method based on deep learning.

[0022] The beneficial effects of the present invention are as follows: by combining GAT and BiLSTM, the topological structure and traffic time series characteristics of the industrial control network are simultaneously utilized. Compared with the traditional method that only focuses on the characteristics of a single dimension, the present invention can more comprehensively analyze the traffic situation, accurately identify abnormal traffic, and reduce the false alarm rate and missed alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the process of the present invention.

[0024] Figure 2 This is the overall structure of the industrial control network security situation prediction deep learning model of the present invention. DETAILED DESCRIPTION

[0025] A method for industrial control network situation prediction based on deep learning, comprising:

[0026] S1. Collect and preprocess multi-dimensional data of industrial control network traffic

[0027] S1.1. Collect multi-dimensional data of industrial control network traffic, including network node traffic data, network node feature data, network topology information, and security log data;

[0028] Furthermore, the network node traffic data includes the data packet size, protocol type, port information (source IP, destination IP, port number) and timestamp at each moment.

[0029] Furthermore, the node characteristic data is status information of each node (ie, device), such as whether the device is normal or has a fault.

[0030] Furthermore, the network topology information is the topological structure of the industrial control network, including the connection relationship between devices, the communication frequency of each node, etc.

[0031] Furthermore, the security log data is historical security event data, such as IDS alerts, intrusion detection reports, etc.

[0032] S1.2. The collected data are sorted in time series, cleaned, standardized, and graph structured;

[0033] Further, the time series is sorted:

[0034] According to the traffic characteristics of the industrial control network, the data is divided into multiple fixed time windows. The data collected at each fixed time interval is used as a sample. The data in each time window will be used as the input of the deep learning model for industrial control network security situation prediction.

[0035] Furthermore, data standardization is performed:

[0036] Use Z-score standardization to process the numerical features of industrial control network traffic (packet size, network topology information). Ensure that all input features have similar scales to avoid certain features dominating the model learning process.

[0037]

[0038] Among them, μ is the mean of the data, σ is the standard deviation of the data, x is the original data, and x ′ is the standardized data.

[0039] Furthermore, we construct graph structure data:

[0040] The industrial control network node graph structure is generated based on the standardized data, where the node features are the source IP and the destination IP, and a feature matrix and an adjacency matrix are generated for each network node.

[0041] S2. Build a deep learning model for industrial control network security situation prediction

[0042] The industrial control network security situation prediction deep learning model includes five parts: input layer, GAT layer, BiLSTM layer, output layer, training and optimization;

[0043] Furthermore, the input layer is used to input multi-dimensional data of industrial control network traffic;

[0044] Furthermore, the GAT layer is responsible for processing the topological information in the industrial control network and generating a graph structure representation of each node.

[0045] Furthermore, the BiLSTM layer is a bidirectional LSTM network that can simultaneously capture the forward and reverse dependencies in time series data and process traffic data with a long time span.

[0046] Furthermore, the output layer is used to output the output results of the BiLSTM layer to the fully connected layer, use a linear activation function to output continuous prediction values, and optimize using an optimization algorithm.

[0047] Furthermore, training and optimization utilize optimizers and loss functions to optimize and train the industrial control network security situation prediction deep learning model.

[0048] S3, spatial feature extraction, processing the spatial dependencies between nodes in the graph through the GAT layer;

[0049] The GAT layer is used to capture the spatial relationship between nodes in the industrial control network. The topological structure information of the industrial control network is represented by a graph structure, in which nodes represent devices or systems in the network, and edges represent connections between nodes.

[0050] In the GAT layer, the input is the node feature matrix X and the adjacency matrix A of the network, and its output is the graph representation H of the node GAT In each graph convolution operation, a node uses a self-attention mechanism to adjust the representation of the node according to its relationship with neighboring nodes.

[0051] In the self-attention mechanism: for each node, the attention coefficient between it and its neighboring nodes is calculated to weight the sum of the features of the neighboring nodes, thereby updating the feature representation of the node. The specific formula is:

[0052] e ij =LeakyReLU(a T [Wh i ||Wh j ])

[0053]

[0054] Among them, e ij represents the original attention coefficient between nodes i and j, h i ,h j is the feature vector of node i,j, α ij is the attention coefficient, LeakyReLU is the ReLU function with a negative slope, Indicates that W is the weight matrix and a represents the weight vector.

[0055] Use the updated attention coefficient α ij , α ij represents the relative importance of node j to node i. The features of each node are weighted and aggregated by the features of its neighboring nodes. Specifically, the new feature representation of node i is h′ i for:

[0056]

[0057] Among them, W σi is a trainable weight matrix used to adjust the dimension of the feature. σ(·) is the activation function, specifically the LeakyReLU function.

[0058] The output of the GAT layer H GAT It is expressed as:

[0059] H GAT =(h′ 1 ,h′ 2 ,h′ 3 …h′ m )

[0060] Through the GAT layer, the industrial control network security situation prediction deep learning model can learn the relationships between various devices in the network and form a more accurate representation of each node, thus providing a basis for subsequent time series modeling.

[0061] S4, temporal feature extraction, capturing the temporal dependencies of time series through BiLSTM;

[0062] The BiLSTM layer is a key part in processing short-term and long-term dependencies in time series data. BiLSTM consists of two parts: a forward LSTM and a reverse LSTM. The forward LSTM processes data from front to back, and the reverse LSTM processes data from back to front. The combination of the two enables the model to capture the dependencies between the previous and next time steps at the same time.

[0063] The core of LSTM lies in the gating mechanism, which controls the flow of information through the forget gate, input gate and output gate. The formula of LSTM is:

[0064] Input gate: i t =σ(W xi x t +W hi h t-1 +W ci c t-1 +b i )

[0065] Forget gate: f t =σ(W xf x t +W hf h t-1 +W cf c t-1 +b f )

[0066] Cell status update:

[0067] Cell status update:

[0068] Output gate: o t =σ(W xo x t +Who h t-1 +W co c t +b o )

[0069] Hidden state update: h t =o t ·tanh(c t )

[0070] Among them, i t and f t are the input gate and the forget gate, respectively, h t is the hidden state, x t is the current input.

[0071] W xi ,W hi ,W ci ,W xf ,W hf ,W cf ,W xc ,W hc ,W xo ,W ho ,W co ,b i ,b f ,b c ,b o are related network parameters.

[0072] Through the BiLSTM layer, the industrial control network security situation prediction deep learning model can capture the bidirectional dependency of time series in the entire sequence range, thereby improving the timing modeling capability of industrial control network traffic data.

[0073] S5. Situation prediction and response:

[0074] The output layer converts the output result H of the BiLSTM layer into BiLSTM The output is sent to the fully connected layer, which uses a linear activation function to output continuous prediction values. Specifically, the output layer is enlarged as follows:

[0075]

[0076] Among them, W out and b out are the weights and biases of the fully connected layer, is the final prediction result.

[0077] S6, model training and optimization, optimizing the parameters of the GAT layer and BiLSTM layer through loss function and optimization algorithm;

[0078] Furthermore, during the training process, the model uses the standard backpropagation algorithm (Backpropagation) and the gradient descent optimizer (Adam) to update the weights. At the same time, the loss function is used to supervise the model to ensure that the model can be gradually optimized in each training cycle.

[0079] Furthermore,

[0080] S6.1. Loss Function

[0081] In order to optimize the performance of the model, an appropriate loss function must be defined. For the security situation prediction of industrial control network traffic data, it is usually necessary to select different loss functions according to different tasks. Since the model task in the present invention is mainly to perform regression problems (predicting traffic trends or network status), the mean square error loss function (MSE) is selected.

[0082] The mean square error loss function is used to measure the gap between the model prediction value and the true value. The formula is as follows:

[0083]

[0084] Among them, y i is the true label of the sample, is the value predicted by the model, and N is the number of samples. The purpose of the MSE loss function is to minimize the difference between the predicted result and the true value, thereby improving the accuracy of the model.

[0085] During the model training process, the loss function is used to guide the optimization algorithm to perform gradient descent to update parameters, helping the model to gradually learn the traffic patterns and security situations in the industrial control network.

[0086] S6.2. Optimization algorithm selection: Adam optimizer

[0087] The choice of optimization algorithm is crucial for the training of deep learning models. Therefore, the Adam optimizer (Adaptive Moment Estimation) which combines AdaGrad and RMSProp is selected.

[0088] The Adam optimizer dynamically adjusts the learning rate of each parameter by estimating the first-order moment (mean of the gradient) and the second-order moment (variance of the gradient). The update rule of Adam is as follows

[0089] First moment estimate (mean) m t =β 1 m t-1 +(1-β 1 ) t

[0090] Second moment estimate (variance):

[0091] Bias correction:

[0092] Parameter update:

[0093] Among them, m t and v t are estimates of the mean and variance of the gradient, g t is the current gradient, α is the learning rate, β 1 and β 2 is the attenuation coefficient, and ∈ is a constant to avoid division by zero.

[0094] The advantage of the Adam optimizer is that it can adaptively adjust the learning rate, is suitable for sparse data and non-stationary targets, and can effectively improve training efficiency and stability. When processing industrial control network traffic data, the Adam optimizer can help the model converge better.

[0095] S6.3. Regularization and overfitting prevention

[0096] To prevent the model from overfitting, we can add regularization methods during the training process to ensure that the model does not overfit the training data, but performs better on the test data.

[0097] L2 regularization: Add L2 regularization term to the loss function to reduce the situation where the weight is too large. The formula is:

[0098]

[0099] Among them, ω i is the weight in the model and λ is the regularization coefficient.

[0100] S7. Detection result output: used to output the prediction results of the model and complete the industrial control network situation prediction function.

[0101] in conclusion

[0102] The GAT-BiLSTM neural network model proposed in this paper combines the advantages of graph neural networks and bidirectional LSTM, and can effectively process traffic data, topological structure information and time series data in industrial control networks, improving the accuracy and real-time performance of security situation prediction. The model has strong adaptability, can cope with complex and changeable industrial control network environments, and provides reliable support for real-time security protection.

Claims

1. A method for industrial control network situation prediction based on deep learning, characterized in that: include: S1. Collect and preprocess multi-dimensional data of industrial control network traffic S2. Build a deep learning model for industrial control network security situation prediction The industrial control network security situation prediction deep learning model includes five parts: input layer, GAT layer, BiLSTM layer, output layer, training and optimization; S3, spatial feature extraction, processing the spatial dependencies between nodes in the graph through the GAT layer; S4, temporal feature extraction, capturing the temporal dependencies of time series through BiLSTM; S5. Situation prediction and response: S6, model training and optimization, optimizing the parameters of the GAT layer and BiLSTM layer through loss function and optimization algorithm; S7. Detection result output: used to output the prediction results of the model and complete the industrial control network situation prediction function.

2. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: The step 1 comprises: S1.

1. Collect multi-dimensional data of industrial control network traffic, including network node traffic data, network node feature data, network topology information, and security log data; The network node traffic data includes the data packet size, protocol type, port information and timestamp at each moment; The node characteristic data is the status information of each node; The network topology information is the topological structure of the industrial control network, including the connection relationship between devices and the communication frequency of each node; The security log data is historical security event data; S1.

2. The collected data is sorted in time series, cleaned, standardized, and graph structured; Time series sorting: According to the traffic characteristics of the industrial control network, the data is divided into multiple fixed time windows; the data collected at each fixed time interval is used as a sample; the data in each time window will be used as the input of the deep learning model for industrial control network security situation prediction; Data standardization: Use Z-score standardization to process the numerical features of industrial control network traffic; ensure that all input features have similar scales to avoid certain features dominating the model learning process; Among them, μ is the mean of the data, σ is the standard deviation of the data, x is the original data, and x ′ It is the standardized data; Constructing graph structure data: The industrial control network node graph structure is generated based on the standardized data, where the node features are the source IP and the destination IP, and a feature matrix and an adjacency matrix are generated for each network node.

3. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: In step 2: The input layer is used to input multi-dimensional data of industrial control network traffic; The GAT layer is responsible for processing the topological information in the industrial control network and generating a graph structure representation of each node; The BiLSTM layer is a bidirectional LSTM network that can simultaneously capture the forward and reverse dependencies in time series data and process long-span traffic data; The output layer is used to output the output results of the BiLSTM layer to the fully connected layer, use the linear activation function to output continuous prediction values, and use the optimization algorithm for optimization; Training and optimization is the process of optimizing and training the deep learning model for industrial control network security situation prediction using optimizers and loss functions.

4. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: In step 3: The GAT layer is used to capture the spatial relationship between nodes in the industrial control network. The topological structure information of the industrial control network is represented by a graph structure, in which nodes represent devices or systems in the network, and edges represent connections between nodes. In the GAT layer, the input is the node feature matrix X and the adjacency matrix A of the network, and its output is the graph representation H of the node GAT ; In each graph convolution operation, the node uses a self-attention mechanism to adjust the node’s representation based on its relationship with neighboring nodes; In the self-attention mechanism: for each node, the attention coefficient between it and its neighboring nodes is calculated to weight the sum of the features of the neighboring nodes, thereby updating the feature representation of the node; The specific formula is: e ij =LeakyReLU(a T [Wh i ||Wh j ]) Among them, e ij represents the original attention coefficient between nodes i and j, h i ,h j is the feature vector of node i,j, α ij is the attention coefficient, LeakyReLU is the ReLU function with a negative slope, Indicates that W is the weight matrix and a is the weight vector; Use the updated attention coefficient α ij , α ij represents the relative importance of node j to node i. The features of each node are weighted and aggregated by the features of its neighboring nodes. Specifically, the new feature representation of node i is h i i for: Among them, W σi is a trainable weight matrix used to adjust the dimension of the feature; σ(·) is the activation function, specifically the LeakyReLU function; The output of the GAT layer H GAT It is expressed as: H GAT =(h′1,h′2,h′3…h′ m ) Through the GAT layer, the industrial control network security situation prediction deep learning model can learn the relationships between various devices in the network and form a more accurate representation of each node, thus providing a basis for subsequent time series modeling.

5. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: In step 4: The BiLSTM layer is a key part in processing short-term and long-term dependencies in time series data. BiLSTM consists of two parts: a forward LSTM and a reverse LSTM. The forward LSTM processes data from front to back, and the reverse LSTM processes data from back to front. The combination of the two enables the model to capture the dependencies between the previous and next time steps at the same time. The core of LSTM lies in the gating mechanism, which controls the flow of information through the forget gate, input gate and output gate; the formula of LSTM is: Input gate: i t =σ(W xi x t +W hi h t-1 +W ci c t-1 +b i ) Forget gate: f t =σ(W xf x t +W hf h t-1 +W cf c t-1 +b f ) Cell status update: Cell status update: Output gate: o t =σ(W xo x t +W ho h t-1 +W co c t +b o ) Hidden state update: h t =o t ·tanh(c t ) Among them, i t and f t are the input gate and the forget gate respectively, h t is the hidden state, x t is the current input; W xi ,W hi ,W ci ,W xf ,W hf ,W cf ,W xc ,W hc ,W xo ,W ho ,W co ,b i ,b f ,b c ,b o are relevant network parameters; Through the BiLSTM layer, the industrial control network security situation prediction deep learning model can capture the bidirectional dependency of time series in the entire sequence range, thereby improving the timing modeling capability of industrial control network traffic data.

6. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: In step 5: the output layer converts the output result H of the BiLSTM layer into BiLSTM The output is sent to the fully connected layer, which uses a linear activation function to output continuous prediction values. Specifically, the output layer is enlarged as follows: Among them, W out and b out are the weights and biases of the fully connected layer, is the final prediction result.

7. The method for predicting industrial control network situation based on deep learning according to claim 1, characterized in that: In step 6: Furthermore, during the training process, the model uses the standard back-propagation algorithm and gradient descent optimizer to update the weights; at the same time, the loss function is used to supervise the model to ensure that the model can be gradually optimized in each training cycle; Furthermore, S6.

1. Loss Function In order to optimize the performance of the model, an appropriate loss function must be defined; for the security situation prediction of industrial control network traffic data, it is usually necessary to select different loss functions according to different tasks; since the model task in the present invention is mainly to perform regression problems, the mean square error loss function is selected; The mean square error loss function is used to measure the gap between the model prediction value and the true value. The formula is as follows: Among them, y i is the true label of the sample, is the value predicted by the model, and N is the number of samples; the role of the MSE loss function is to minimize the difference between the predicted result and the true value, thereby improving the accuracy of the model; During the model training process, the loss function is used to guide the optimization algorithm to perform gradient descent to update parameters, helping the model to gradually learn the traffic patterns and security situations in the industrial control network; S6.

2. Optimization algorithm selection: Adam optimizer The choice of optimization algorithm is crucial for the training of deep learning models; therefore, the Adam optimizer, which combines AdaGrad and RMSProp, is selected. The Adam optimizer dynamically adjusts the learning rate of each parameter by estimating the first-order moment and the second-order moment; the update rule of Adam is as follows First moment estimate: m t =β1m t-1 +(1-β1)g t Second moment estimate: Deviation Calibration: Parameter update: Among them, m t and v t are estimates of the mean and variance of the gradient, g t is the current gradient, α is the learning rate, β1 and β2 are the decay coefficients, and ∈ is a constant to avoid division by zero; The advantage of the Adam optimizer is that it can adaptively adjust the learning rate, is suitable for sparse data and non-stationary targets, and can effectively improve training efficiency and stability. When processing industrial control network traffic data, the Adam optimizer can help the model converge better. S6.

3. Regularization and overfitting prevention To prevent the model from overfitting, we can add regularization methods during the training process to ensure that the model does not overfit the training data, but performs better on the test data; L2 regularization: Add L2 regularization term to the loss function to reduce the situation where the weight is too large. The formula is: Among them, ω i is the weight in the model and λ is the regularization coefficient.

8. A deep learning-based industrial control network situation prediction system, characterized in that: It includes a data acquisition and preprocessing module, an industrial control network security situation prediction deep learning module, and a detection result output module; the data acquisition and preprocessing module is used to collect and preprocess multi-dimensional data of industrial control network traffic, the industrial control network security situation prediction deep learning module predicts the industrial control network situation by constructing an industrial control network security situation prediction deep learning model, and the detection result output module is used to output the prediction results of the model to complete the prediction of the industrial control network situation.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a method for industrial control network situation prediction based on deep learning as described in any one of claims 1-7.

10. A server, characterized in that: The server includes a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement an industrial control network situation prediction method based on deep learning as described in any one of claims 1-7.

Images