An Industrial Control System Intrusion Detection Method Based on BiLSTM and Attention Mechanism

Abstract

This invention provides an intrusion detection method for industrial control networks based on BiLSTM and an attention mechanism, comprising the following steps: preprocessing network traffic data to obtain preprocessed network features; applying a sparse attention layer to process the preprocessed network features; passing the features obtained from the sparse attention layer to a Bidirectional Long Short-Term Memory (BiLSTM) layer; inputting the outputs of the sparse attention layer and the BiLSTM layer into a bidirectional attention layer; further processing the output of the bidirectional attention layer using a fully connected layer; and outputting the classification result of network attack detection. This invention, through comprehensive preprocessing, a sparse attention mechanism, and the efficient combination of BiLSTM and bidirectional attention mechanisms, can effectively capture and process complex network traffic features, improve the accuracy and real-time performance of intrusion detection, and is applicable to various network security scenarios, providing strong protection for networks and industrial control systems.

Description

Technical Field

[0001] This invention relates to the field of industrial control system security technology, and in particular to an industrial control network intrusion detection method based on BiLSTM and attention mechanism. Background Technology

[0002] In the field of Industrial Control Systems (ICS) and critical infrastructure protection, Intrusion Detection Systems (IDS) are a key component for maintaining network security. Traditional methods mainly rely on rule-based systems and signature-based detection. While these methods are effective in identifying known attack patterns, their ability to detect unknown attacks, polymorphic attacks, or advanced persistent threats (APTs) is limited. As ICS network environments become increasingly complex and attack methods continue to evolve, the need for smarter and more efficient intrusion detection technologies is becoming increasingly urgent.

[0003] 1. Feature-based intrusion detection: Early IDS primarily relied on predefined attack signatures, matching network activity with known attack patterns to detect attacks. This method is simple and direct, but it falls short when facing mutated or customized attacks.

[0004] 2. Statistical and Anomaly Detection Methods: With technological advancements, statistical analysis and machine learning have begun to be applied to intrusion detection. By analyzing the statistical characteristics of network traffic or learning normal behavioral patterns, deviant behaviors are identified as potential attack signals. However, these methods are susceptible to high false positive rates, especially in dynamically changing industrial control network environments.

[0005] 3. Applications of Deep Learning in Intrusion Detection (IDS): In recent years, deep learning technologies, especially recurrent neural networks (RNNs) and their variants, such as long short-term memory networks (LSTM) and bidirectional LSTM (Bi-LSTM), have shown great potential in the field of intrusion detection. These models can automatically learn complex time-series features, effectively capture long-term dependencies in network traffic, and improve detection accuracy and adaptability. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide an industrial control network intrusion detection method based on BiLSTM and attention mechanisms, which solves the problems of large scale, diverse types, and noise redundancy in complex network data in industrial control systems, thus improving model performance. For intelligent models based on neural network structures, sufficient high-quality data is provided to fully train the model parameters, thereby avoiding overfitting. Through a network structure based on sparse attention, BiLSTM, and bidirectional attention mechanisms, and by comprehensively utilizing the feature importance and temporal relationship of network data packets, accurate network intrusion detection is achieved for different industrial scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: an intrusion detection method for industrial control networks based on BiLSTM and attention mechanism, comprising the following steps:

[0008] Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data;

[0009] Step 2: Apply a sparse attention layer to process the features of the preprocessed network traffic data;

[0010] Step 3: Pass the network traffic data features obtained from the sparse attention layer to the Bidirectional Long Short-Term Memory (BiLSTM) network layer;

[0011] Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer;

[0012] Step 5: Use a fully connected layer to further process the output of the bidirectional attention layer and output the classification results of network attack detection.

[0013] In a preferred embodiment, step 1 includes the following steps:

[0014] Step 11: Outlier Filtering;

[0015] Step 12: Impute missing data values;

[0016] Step 13: Merge duplicate values ​​and correct invalid values.

[0017] In a preferred embodiment, step 2 specifically involves: providing a network traffic data feature matrix. Where R represents the real number field, It is the sequence length. It is the feature dimension; calculate the attention weight A:

[0018] Where softmax() is the normalized exponential function, T denotes the transpose, and d k The dimension of the key matrix K is represented by the query matrix. Key matrix , It is a trainable weight matrix; the sparsification strategy sparsifies the attention matrix by combining Top-k selection and thresholding; the sparse attention output is calculated. The value matrix It is a trainable weight matrix, d v This represents the dimension of the value matrix V.

[0019] In a preferred embodiment, step 3 specifically involves: calculating the output of the feedforward LSTM based on the sparse attention output Z. And the output of the backward LSTM: ,connect and Obtain the output of BiLSTM ; where h t h represents the output of the feedforward LSTM at time t. t-1 h represents the output of the feedforward LSTM at time t-1. t+1 This represents the output of the forward LSTM at time t+1.

[0020] In a preferred embodiment, step 4 specifically involves: calculating the forward attention weights based on the BiLSTM layer output H and the sparse attention output Z. and backward attention weights ,in, This represents the transpose of the sparse attention output Z. Represent the transpose of the BiLSTM output H; calculate the forward attention output. and backward attention output The features of bidirectional attention fusion are formed by fusing forward and backward attention. ,in, This indicates that attention is being directed towards Quanzhou. This represents the backward attention weights.

[0021] In a preferred embodiment, step 5 includes the following steps:

[0022] Step 51: Fusing features after bidirectional attention Passed to the fully connected layer:

[0023] ,

[0024] Among them W and It consists of the weight matrix and bias terms, where P is the weight vector for each class;

[0025] Step 52: Calculate the scores for each category :

[0026] ,

[0027] Where c is the total number of categories, e is the base of the natural logarithm, and p i p represents the weight of class i in p. j This represents the weight of category j in p;

[0028] Step 53: Find the category corresponding to the highest score and output the classification result of network attack detection:

[0029] ,

[0030] The category i with the highest score is the final classification result y.

[0031] In a preferred embodiment, the raw network traffic data is preprocessed, assuming... For complex network time series data packets, where, This represents the eigenvector of the observed values ​​at time 1. This represents the eigenvector of the observed values ​​at time 2. This represents the feature vector of the observed value at time n. Step 11, outlier screening, includes the following steps:

[0032] Step 111: Calculate the angle and direction measurement :

[0033]

[0034] Step 112: Construct the nearest neighbor sort: For each Calculate its eigenvectors with those of other observations. Between , Let m be the eigenvector of the observed value at time m, and then according to... For other observation eigenvectors Sort the data points and select the L nearest data points as their neighbor set. ;

[0035] Step 113: Adaptive Synthesis of Samples: For ,according to Middle neighbor node and distance New representative points are synthesized using a weighted average method. Weight Based on the reciprocal of the distance:

[0036]

[0037] in, Represents a node neighboring nodes The subscript k;

[0038] Then synthesize new points:

[0039]

[0040] Step 114: Outlier Detection: Calculation With the synthesis point Distance between:

[0041]

[0042] If the distance is greater than the preset threshold Then it is believed It is an outlier; Adjustments should be made according to the specific scenario.

[0043] In a preferred embodiment, the raw network traffic data is preprocessed, assuming... For complex network time series data packets, where To indicate missing values, step 12 includes the following steps:

[0044] Step 121: Select non-missing values ​​to construct training sequences , used to train the LSTM model M;

[0045] Step 122: For each missing time t, use M-prediction. ,Right now , where k is the number of historical time steps considered, which can be adjusted according to the specific scenario.

[0046] In a preferred embodiment, the raw network traffic data undergoes preprocessing, including duplicate value merging and invalid value correction. A duplicate value merging algorithm based on neighbor-based record sorting and a regular expression method are used to batch process the dataset; this includes the following steps:

[0047] Step 131: For each row of data in the specified format attribute column, perform duplicate value filtering and regular expression matching. If duplicate values ​​exist, set them to empty values. For data that does not meet the preset conditions of the regular expression, also set them to empty values.

[0048] Step 132: For the data in the specified attribute column, restrict the size range; data outside the range will also be set to null.

[0049] In a preferred embodiment, the sparse attention layer, BiLSTM layer, bidirectional attention layer and fully connected layer each employ Dropout to prevent network overfitting and gradient explosion, with the Dropout parameter set to 0.4.

[0050] Compared with existing technologies, this invention has the following advantages: From data preprocessing to feature extraction and classification prediction, this invention forms a complete solution that can effectively address the unique data challenges of industrial control networks, such as large-scale complex network traffic, high noise, and numerous anomalies. Through anomaly filtering, missing value supplementation, duplicate value merging, and invalid value correction, it effectively solves common quality problems in industrial control network data. Addressing the temporal characteristics of industrial control network traffic, it first extracts the importance relationships of features within data packets based on sparse attention, then applies BiLSTM to capture the temporal relationships between data packets, and finally integrates features through bidirectional attention, forming a deep understanding of network behavior. Through the combination of refined data processing and multi-layered deep learning models, it improves the detection capability of industrial systems against network intrusions. Attached Figure Description

[0051] Figure 1 This is a flowchart of an industrial control network intrusion detection method based on BiLSTM and attention mechanism in a preferred embodiment of the present invention.

[0052] Figure 2 This is a structural diagram of an intrusion detection model for an industrial control network intrusion detection method based on BiLSTM and attention mechanism, according to a preferred embodiment of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0055] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0056] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0057] The basic idea of ​​this invention is to preprocess network traffic data to obtain preprocessed network features, apply a sparse attention layer to process these features, and then pass the features obtained from the sparse attention layer to a Bidirectional Long Short-Term Memory (BiLSTM) layer. The outputs of the sparse attention layer and the BiLSTM layer are then input into the bidirectional attention layer, and a fully connected layer is used to further process the output of the bidirectional attention layer, outputting the classification result of network attack detection. Through meticulous data preprocessing steps (such as outlier filtering, missing value imputation, duplicate value merging, and invalid value correction), data quality is effectively improved, providing a solid dataset foundation for subsequent model training. By combining the attention mechanism and the BiLSTM deep learning model, and learning the micro-level importance features of data packets and the macro-level temporal features of data flows respectively, this method can more comprehensively and deeply understand network behavior, improve the ability to identify complex intrusion behaviors, and protect the secure operation of the system.

[0058] refer to Figure 1-2 An intrusion detection method for industrial control networks based on BiLSTM and attention mechanism includes the following steps:

[0059] Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data;

[0060] Step 2: Apply a sparse attention layer to process the features of the preprocessed network traffic data;

[0061] Step 3: Pass the network traffic data features obtained from the sparse attention layer to the Bidirectional Long Short-Term Memory (BiLSTM) network layer;

[0062] Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer;

[0063] Step 5: Use a fully connected layer to further process the output of the bidirectional attention layer and output the classification results of network attack detection.

[0064] Step 1 includes the following steps:

[0065] Step 11: Outlier Filtering;

[0066] Step 12: Impute missing data values;

[0067] Step 13: Merge duplicate values ​​and correct invalid values.

[0068] Step 2 specifically involves: providing a network traffic data feature matrix. Where R represents the real number field, It is the sequence length. It is the feature dimension; calculate the attention weight A:

[0069] Where softmax() is the normalized exponential function, T denotes the transpose, and d k The dimension of the key matrix K is represented by the query matrix. Key matrix , It is a trainable weight matrix; the sparsification strategy sparsifies the attention matrix by combining Top-k selection and thresholding; the sparse attention output is calculated. The value matrix It is a trainable weight matrix, d v This represents the dimension of the value matrix V.

[0070] Step 3 specifically involves: calculating the output of the forward LSTM based on the sparse attention output Z. And the output of the backward LSTM: ,connect and Obtain the output of BiLSTM ; where h t h represents the output of the feedforward LSTM at time t. t-1 h represents the output of the feedforward LSTM at time t-1. t+1 This represents the output of the forward LSTM at time t+1.

[0071] Step 4 specifically involves calculating the forward attention weights based on the BiLSTM layer output H and the sparse attention output Z. and backward attention weights ,in, This represents the transpose of the sparse attention output Z. Represent the transpose of the BiLSTM output H; calculate the forward attention output. and backward attention output The features of bidirectional attention fusion are formed by fusing forward and backward attention. ,in, This indicates that attention is being directed towards Quanzhou. This represents the backward attention weights.

[0072] Step 5 includes the following steps:

[0073] Step 51: Fusing features after bidirectional attention Passed to the fully connected layer:

[0074] ,

[0075] Among them W and It consists of the weight matrix and bias terms, where P is the weight vector for each class;

[0076] Step 52: Calculate the scores for each category :

[0077] ,

[0078] Where c is the total number of categories, e is the base of the natural logarithm, and p i p represents the weight of class i in p. j This represents the weight of category j in p;

[0079] Step 53: Find the category corresponding to the highest score and output the classification result of network attack detection:

[0080] ,

[0081] The category i with the highest score is the final classification result y.

[0082] The raw network traffic data is preprocessed, and then... For complex network time series data packets, where, This represents the eigenvector of the observed values ​​at time 1. This represents the eigenvector of the observed values ​​at time 2. This represents the feature vector of the observed value at time n. Step 11, outlier screening, includes the following steps:

[0083] Step 111: Calculate the angle and direction measurement :

[0084]

[0085] Step 112: Construct the nearest neighbor sort: For each Calculate its eigenvectors with those of other observations. Between , Let m be the eigenvector of the observed value at time m, and then according to... For other observation eigenvectors Sort the data points and select the L nearest data points as their neighbor set. ;

[0086] Step 113: Adaptive Synthesis of Samples: For ,according to Middle neighbor node and distance New representative points are synthesized using a weighted average method. Weight Based on the reciprocal of the distance:

[0087]

[0088] in, Represents a node neighboring nodes The subscript k;

[0089] Then synthesize new points:

[0090]

[0091] Step 114: Outlier Detection: Calculation With the synthesis point Distance between:

[0092]

[0093] If the distance is greater than the preset threshold Then it is believed It is an outlier; Adjustments should be made according to the specific scenario.

[0094] The raw network traffic data is preprocessed, and then... For complex network time series data packets, where To indicate missing values, step 12 includes the following steps:

[0095] Step 121: Select non-missing values ​​to construct training sequences , used to train the LSTM model M;

[0096] Step 122: For each missing time t, use M-prediction. ,Right now , where k is the number of historical time steps considered, which can be adjusted according to the specific scenario.

[0097] The raw network traffic data undergoes preprocessing, including duplicate value merging and invalid value correction. A duplicate value merging algorithm based on proximity record sorting and regular expression methods are used to batch process the dataset; this includes the following steps:

[0098] Step 131: For each row of data in the specified format attribute column, perform duplicate value filtering and regular expression matching. If duplicate values ​​exist, set them to empty values. For data that does not meet the preset conditions of the regular expression, also set them to empty values.

[0099] Step 132: For the data in the specified attribute column, restrict the size range; data outside the range will also be set to null.

[0100] The sparse attention layer, BiLSTM layer, bidirectional attention layer, and fully connected layer all employ Dropout in the middle of each layer to prevent overfitting and gradient explosion. The Dropout parameter is set to 0.4.

Claims

1. An intrusion detection method for industrial control networks based on BiLSTM and attention mechanism, characterized in that... It includes the following steps: Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data; Step 2: Apply a sparse attention layer to process the features of the preprocessed network traffic data; Step 3: Pass the network traffic data features obtained from the sparse attention layer to the Bidirectional Long Short-Term Memory (BiLSTM) network layer; Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer; Step 5: Use a fully connected layer to further process the output of the bidirectional attention layer, and output the classification results of network attack detection; Step 2 specifically involves: providing a network traffic data feature matrix. Where R represents the real number field, It is the sequence length. It is the feature dimension; calculate the attention weights. : Where softmax() is the normalized exponential function, T denotes the transpose, and d k The dimension of the key matrix K is represented by the query matrix. Key matrix , It is a trainable weight matrix; the sparse attention layer sparsifies the attention matrix by combining Top-k selection and thresholding; the sparse attention output is calculated. : The value matrix , It is a trainable weight matrix; Step 3 specifically involves: based on the output of the sparse attention layer... Calculate the output of the forward LSTM: And the output of the backward LSTM: ,connect and Obtain the output of BiLSTM ; where h t h represents the output of the feedforward LSTM at time t. t-1 h represents the output of the feedforward LSTM at time t-1. t+1 This represents the output of the forward LSTM at time t+1; Step 4 specifically involves: based on the output of the BiLSTM layer... and sparse attention output Calculate the forward attention weights and backward attention weights ,in, This represents the transpose of the sparse attention output Z. Represent the transpose of the BiLSTM output H; calculate the forward attention output. and backward attention output The features of bidirectional attention fusion are formed by fusing forward and backward attention. ,in, This indicates that attention is being directed towards Quanzhou. Indicates the backward attention weights; Step 5 includes the following steps: Step 51: Fusing features after bidirectional attention Passed to the fully connected layer: ， in and These are the weight matrix and the bias terms. Weight vectors for each category; Step 52: Calculate the scores for each category : ， in It is the total number of categories. p is the base of the natural logarithm. i p represents the weight of class i in p. j This represents the weight of category j in p, where j represents the weight of each enumerated category. Step 53: Find the category corresponding to the highest score and output the classification result of network attack detection: ， The category with the highest score That is the final classification result .

2. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 1, characterized in that, Step 1 includes the following steps: Step 11: Outlier Filtering; Step 12: Imput missing data values; Step 13: Merge duplicate values ​​and correct invalid values.

3. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 2, characterized in that, The network traffic data is preprocessed, and then... For complex network time series data packets, where, This represents the eigenvector of the observed values ​​at time 1. This represents the eigenvector of the observed values ​​at time 2. Indicates the first The feature vector of the observed value at time 11, and the outlier screening in step 11, include the following steps: Step 111: Calculate the angle and direction measurement : ， Step 112: Construct the nearest neighbor sort: For each Calculate its eigenvectors with those of other observations. Between , Let m be the eigenvector of the observed value at time m, and then according to For other observation eigenvectors Sort the results and select the closest ones. Each data point is considered as its neighboring node set. ; Step 113: Adaptive Synthesis of Samples: For ,according to Middle neighbor node and distance New representative points are synthesized using a weighted average method. Weight Based on the reciprocal of the distance: ,in, Represents node s i neighboring nodes s k The subscript k; Then synthesize new points: Step 114: Outlier Detection: Calculation With the synthesis point Distance between: If the distance is greater than the preset threshold Then it is believed It is an outlier; Adjustments should be made according to the specific scenario.

4. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 2, characterized in that, The network traffic data is preprocessed, and then... For complex network time series data packets, where To indicate missing values, step 12 includes the following steps: Step 121: Select non-missing values ​​to construct training sequences Used to train LSTM models ; Step 122: For each missing time point ,use predict ,Right now ,in It takes into account the number of historical time steps and adjusts them according to the specific scenario.

5. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 2, characterized in that, The network traffic data undergoes preprocessing, including duplicate value merging and invalid value correction. A duplicate value merging algorithm based on proximity record sorting and regular expression methods are used to batch process the dataset; this includes the following steps: Step 131: For each row of data in the specified format attribute column, perform duplicate value filtering and regular expression matching. If duplicate values ​​exist, set them to empty values. For data that does not meet the preset conditions of the regular expression, also set them to empty values. Step 132: For the data in the specified attribute column, restrict the size range; data outside the range will also be set to null.

6. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 1, characterized in that, The sparse attention layer, BiLSTM layer, bidirectional attention layer, and fully connected layer all employ Dropout in the middle of each layer to prevent overfitting and gradient explosion. The Dropout parameter is set to 0.4.

Images