Industrial control network intrusion detection method based on BiLSTM and attention mechanism

Abstract

The invention provides an industrial control network intrusion detection method based on BiLSTM and an attention mechanism, and the method comprises the following steps: carrying out the preprocessing of network flow data, obtaining the preprocessed network features, carrying out the processing of the preprocessed network features through employing a sparse attention layer, and obtaining the network flow data; and transmitting features obtained by the sparse attention layer to a bidirectional long short-term memory network (BiLSTM) layer, inputting the output of the sparse attention layer and the output of the BiLSTM layer to the bidirectional attention layer, further processing the output of the bidirectional attention layer by using a full connection layer, and outputting a classification result of network attack detection. Through efficient combination of comprehensive preprocessing, a sparse attention mechanism, a BiLSTM and a bidirectional attention mechanism, complex network traffic characteristics can be effectively captured and processed, the accuracy and the real-time performance of intrusion detection are improved, the method is suitable for various network security scenes, and powerful protection is provided for a network and an industrial control system.

Description

Technical Field

[0001] The present invention relates to the field of industrial control system security technology, and in particular to an industrial control network intrusion detection method based on BiLSTM and attention mechanism. Background Art

[0002] In the field of industrial control systems (ICS) and critical infrastructure protection, network intrusion detection systems (IDS) are key components for maintaining network security. Traditional methods mainly rely on rule-based systems and signature detection, which are effective in identifying known attack patterns, but have limited detection capabilities for unknown attacks, deformed attacks, or advanced persistent threats (APTs). As the industrial control network environment becomes increasingly complex and attack methods continue to evolve, the demand for smarter and more efficient intrusion detection technology becomes more urgent.

[0003] 1. Signature-based intrusion detection: Early IDS was mainly based on pre-defined attack signatures, and detected attacks by matching network activities with known attack patterns. This method is simple and direct, but it is incapable of dealing with variant attacks or customized attacks.

[0004] 2. Statistical and anomaly detection methods: With the development of technology, statistical analysis and machine learning have begun to be applied to intrusion detection, by analyzing the statistical characteristics of network traffic or learning normal behavior patterns, identifying behaviors that deviate from the norm as potential attack signals. However, such methods are prone to high false alarm rates, especially in dynamically changing industrial control network environments.

[0005] 3. Application of deep learning in IDS: In recent years, deep learning technology, especially recurrent neural network (RNN) and its variants, such as long short-term memory network (LSTM) and bidirectional LSTM (Bi-LSTM), has shown great potential in the field of intrusion detection. These models can automatically learn complex time series features, effectively capture long-term dependencies in network traffic, and improve detection accuracy and adaptability. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide an industrial control network intrusion detection method based on BiLSTM and attention mechanism, which solves the problem of large scale, multiple types of complex network data and redundant noise interfering with model performance in industrial control systems. For intelligent models based on neural network structures, sufficient high-quality data is provided to fully train model parameters, thereby avoiding overfitting of the model. Through the network structure based on sparse attention, BiLSTM and bidirectional attention mechanism, the feature importance and temporal relationship of network data packets are comprehensively utilized to achieve accurate network intrusion detection for different industrial scenarios.

[0007] To achieve the above purpose, the present invention adopts the following technical solution: an industrial control network intrusion detection method based on BiLSTM and attention mechanism, comprising the following steps: Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data; Step 2: Apply the sparse attention layer to process the preprocessed network traffic data features; Step 3: Pass the network traffic data features obtained by the sparse attention layer to the bidirectional long short-term memory network layer BiLSTM; Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer; Step 5: Use the fully connected layer to further process the output of the bidirectional attention layer and output the classification result of network attack detection.

[0008] In a preferred embodiment, the step 1 comprises the following steps: Step 11: Outlier screening; Step 12: Supplement missing data values; Step 13: Merge duplicate values ​​and correct invalid values.

[0009] In a preferred embodiment, the step 2 is specifically as follows: given a network traffic data feature matrix , where R represents the real number field, is the sequence length, is the feature dimension; calculate the attention weight A: , where softmax() is the normalized exponential function, T represents the transpose, and d k represents the dimension of the key matrix K, the query matrix , the bond matrix , is a trainable weight matrix; the sparsification strategy sparses the attention matrix by combining Top-k selection and threshold; calculates the sparse attention output , where the value matrix is a trainable weight matrix, d v Represents the dimensions of the value matrix V.

[0010] In a preferred embodiment, the step 3 is specifically: based on the sparse attention output Z, the output of the forward LSTM is calculated: And the output of the backward LSTM: ,connect and Get the output of BiLSTM ; Among them, h trepresents the output of the forward LSTM at time t, h t-1 represents the output of the forward LSTM at time t-1, h t+1 Represents the output of the forward LSTM at time t+1.

[0011] In a preferred embodiment, step 4 is specifically as follows: based on the BiLSTM layer output H and the sparse attention output Z, calculate the forward attention weight and the backward attention weights ,in, represents the transpose of the sparse attention output Z, Represents the transpose of the BiLSTM output H; computes the forward attention output and the backward attention output , fusion of forward and backward attention to form bidirectional attention fusion features ,in, represents the forward attention Quanzhou, represents the backward attention weight.

[0012] In a preferred embodiment, the step 5 comprises the following steps: Step 51: Fusing the features after bidirectional attention Passed to the fully connected layer: , Where W and is the weight matrix and bias term, P is the weight vector for each category; Step 52: Calculate the scores for each category : , where c is the total number of categories, e is the base of the natural logarithm, and p i represents the weight of category i in p, p j represents the weight of category j in p; Step 53: Find the category corresponding to the maximum score and output the classification result of network attack detection: , The category i with the highest score is the final classification result y.

[0013] In a preferred embodiment, the raw network traffic data is preprocessed. is a complex network time series data packet, where represents the observed value feature vector at the first moment, represents the observed value feature vector at the second moment, Represents the observed value feature vector at the nth moment. Step 11 outlier screening includes the following steps: Step 111: Calculate the Angle and direction metrics : Step 112: Construct adjacent node sorting: For each , calculate its eigenvector with other observations Between , Represents the observed value feature vector at the mth moment, and then according to For other observations, the feature vector Sort and select L data points with a closer distance as its neighboring node set ; Step 113: Adaptive synthesis samples: ,according to Middle adjacent node and Distance , a new representative point is synthesized by weighted average , weight Based on the inverse of distance: in, Representation Node The neighboring nodes The subscript k of Then synthesize new points: Step 114: Outlier Detection: Calculation With synthesis point Distance between: If the distance is greater than the preset threshold , then it is believed that It is an outlier; Adjust according to the specific scenario.

[0014] In a preferred embodiment, the raw network traffic data is preprocessed. is a complex network time series data packet, where Indicates missing values, step 12 includes the following steps: Step 121: Select non-missing values ​​to construct training sequences , used to train the LSTM model M; Step 122: For each missing time t, use M to predict ,Right now , where k is the number of historical time steps considered, which is adjusted according to the specific scenario.

[0015] In a preferred embodiment, the raw network traffic data is preprocessed, duplicate values ​​are merged and invalid values ​​are corrected, and a duplicate value merging algorithm based on adjacent record sorting and a regular expression method are used to batch process the data set; the following steps are included: Step 131: for each row of data in the specified format attribute column, duplicate value screening and regular expression matching are performed. If there are duplicate values, they are set to null values. For data that does not meet the preset conditions of the regular expression, they are also set to null values. Step 132: For the data in the specified attribute column, a size range is restricted, and data outside the range is also set to a null value.

[0016] In a preferred embodiment, Dropout is used in the middle of each of the sparse attention layer, BiLSTM layer, bidirectional attention layer and fully connected layer to prevent network overfitting and gradient explosion, and the Dropout parameter value is selected to be 0.4.

[0017] Compared with the prior art, the present invention has the following beneficial effects: From data preprocessing to feature extraction to classification prediction, the present invention forms a complete set of solutions, which can effectively cope with the data challenges unique to industrial control networks, such as large-scale complex network traffic, large noise, and many anomalies. Through the screening of abnormal points, the supplementation of missing data values, the merging of duplicate values, and the correction of invalid values, the common quality problems of industrial control network data are effectively solved. In view of the temporal characteristics of industrial control network traffic, the importance relationship of the internal features of the data packet is first extracted based on sparse attention, and then BiLSTM is applied to capture the temporal relationship between data packets, and then the features are integrated through bidirectional attention to form a deep understanding of network behavior. Through the combination of refined data processing and multi-level deep learning models, the detection capability of industrial systems for network intrusions is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The present invention is a flowchart of an industrial control network intrusion detection method based on BiLSTM and attention mechanism in a preferred embodiment of the present invention.

[0019] Figure 2 This is a structural diagram of an intrusion detection model of an industrial control network intrusion detection method based on BiLSTM and attention mechanism in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0022] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0023] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0024] The basic idea of ​​the present invention is to preprocess the network traffic data to obtain the preprocessed network features, apply the sparse attention layer to process the preprocessed network features, pass the features obtained by the sparse attention layer to the bidirectional long short-term memory network (BiLSTM) layer, input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer, use the fully connected layer to further process the output of the bidirectional attention layer, and output the classification results of network attack detection. Through the sophisticated data preprocessing steps (such as outlier screening, missing value filling, duplicate value merging and invalid value correction), the data quality is effectively improved, providing a solid data set foundation for subsequent model training and learning. By combining the attention mechanism and the deep learning model of BiLSTM, the importance characteristics of micro-data packets and the timing characteristics of macro-data flows are learned respectively. This method can understand network behavior more comprehensively and deeply, improve the ability to identify complex intrusion behaviors, and protect the safe operation of the system.

[0025] refer to Figure 1-2 , an industrial control network intrusion detection method based on BiLSTM and attention mechanism, including the following steps: Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data; Step 2: Apply the sparse attention layer to process the preprocessed network traffic data features; Step 3: Pass the network traffic data features obtained by the sparse attention layer to the bidirectional long short-term memory network layer BiLSTM; Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer; Step 5: Use the fully connected layer to further process the output of the bidirectional attention layer and output the classification result of network attack detection.

[0026] The step 1 comprises the following steps: Step 11: Outlier screening; Step 12: Supplement missing data values; Step 13: Merge duplicate values ​​and correct invalid values.

[0027] The step 2 is specifically as follows: given a network traffic data feature matrix , where R represents the real number field, is the sequence length, is the feature dimension; calculate the attention weight A: , where softmax() is the normalized exponential function, T represents the transpose, and d k represents the dimension of the key matrix K, the query matrix , the bond matrix , is a trainable weight matrix; the sparsification strategy sparses the attention matrix by combining Top-k selection and threshold; calculates the sparse attention output , where the value matrix is a trainable weight matrix, d v Represents the dimensions of the value matrix V.

[0028] The step 3 is specifically as follows: based on the sparse attention output Z, the output of the forward LSTM is calculated: And the output of the backward LSTM: ,connect and Get the output of BiLSTM ; Among them, h t represents the output of the forward LSTM at time t, h t-1 represents the output of the forward LSTM at time t-1, h t+1 Represents the output of the forward LSTM at time t+1.

[0029] The step 4 is specifically as follows: based on the BiLSTM layer output H and the sparse attention output Z, calculate the forward attention weight and the backward attention weights ,in, represents the transpose of the sparse attention output Z, Represents the transpose of the BiLSTM output H; computes the forward attention output and the backward attention output , fusion of forward and backward attention to form bidirectional attention fusion features ,in, represents the forward attention Quanzhou, represents the backward attention weight.

[0030] The step 5 comprises the following steps: Step 51: Fusing the features after bidirectional attention Passed to the fully connected layer: , Where W and is the weight matrix and bias term, P is the weight vector for each category; Step 52: Calculate the scores for each category : , where c is the total number of categories, e is the base of the natural logarithm, and p i represents the weight of category i in p, p j represents the weight of category j in p; Step 53: Find the category corresponding to the maximum score and output the classification result of network attack detection: , The category i with the highest score is the final classification result y.

[0031] The original network traffic data is preprocessed. is a complex network time series data packet, where represents the observed value feature vector at the first moment, represents the observed value feature vector at the second moment, Represents the observed value feature vector at the nth moment. Step 11 outlier screening includes the following steps: Step 111: Calculate the Angle and direction metrics : Step 112: Construct adjacent node sorting: For each , calculate its eigenvector with other observations Between , Represents the observed value feature vector at the mth moment, and then according to For other observations, the feature vector Sort and select L data points with a closer distance as its neighboring node set ; Step 113: Adaptive synthesis samples: ,according to Middle adjacent node and Distance , a new representative point is synthesized by weighted average , weight Based on the inverse of distance: in, Representation Node The neighboring nodes The subscript k of Then synthesize new points: Step 114: Outlier Detection: Calculation With synthesis point Distance between: If the distance is greater than the preset threshold , then it is believed that It is an outlier; Adjust according to the specific scenario.

[0032] The original network traffic data is preprocessed. is a complex network time series data packet, where Indicates missing values, step 12 includes the following steps: Step 121: Select non-missing values ​​to construct training sequences , used to train the LSTM model M; Step 122: For each missing time t, use M to predict ,Right now , where k is the number of historical time steps considered, which is adjusted according to the specific scenario.

[0033] The raw network traffic data is preprocessed, duplicate values ​​are merged and invalid values ​​are corrected, and a duplicate value merging algorithm based on adjacent record sorting and a regular expression method are used to batch process the data set; the steps include: Step 131: for each row of data in the specified format attribute column, duplicate value screening and regular expression matching are performed. If there are duplicate values, they are set to null values. For data that does not meet the preset conditions of the regular expression, they are also set to null values. Step 132: For the data in the specified attribute column, a size range is restricted, and data outside the range is also set to a null value.

[0034] The sparse attention layer, BiLSTM layer, bidirectional attention layer and fully connected layer use Dropout in the middle of each layer to prevent network overfitting and gradient explosion, and the Dropout parameter value is 0.4.

Claims

1. An industrial control network intrusion detection method based on BiLSTM and attention mechanism, characterized in that The following steps are involved: Step 1: Preprocess the network traffic data to obtain the characteristics of the preprocessed network traffic data; Step 2: Apply the sparse attention layer to process the preprocessed network traffic data features; Step 3: Pass the network traffic data features obtained by the sparse attention layer to the bidirectional long short-term memory network layer BiLSTM; Step 4: Input the output of the sparse attention layer and the output of the BiLSTM layer into the bidirectional attention layer; Step 5: Use the fully connected layer to further process the output of the bidirectional attention layer and output the classification result of network attack detection.

2. According to claim 1, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The step 1 comprises the following steps: Step 11: Outlier screening; Step 12: Supplement missing data values; Step 13: Merge duplicate values ​​and correct invalid values.

3. According to claim 1, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The step 2 is specifically as follows: given a network traffic data feature matrix , where R represents the real number field, is the sequence length, is the feature dimension; calculate the attention weight A: , where softmax() is the normalized exponential function, T represents the transpose, and d k represents the dimension of the key matrix K, the query matrix , the bond matrix , is a trainable weight matrix; the sparsification strategy sparses the attention matrix by combining Top-k selection and threshold; calculates the sparse attention output , where the value matrix is a trainable weight matrix, d v Represents the dimensions of the value matrix V.

4. According to claim 3, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The step 3 is specifically as follows: based on the sparse attention output Z, the output of the forward LSTM is calculated: And the output of the backward LSTM: ,connect and Get the output of BiLSTM ; Among them, h t represents the output of the forward LSTM at time t, h t-1 represents the output of the forward LSTM at time t-1, h t+1 Represents the output of the forward LSTM at time t+1.

5. According to claim 4, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The step 4 is specifically as follows: based on the BiLSTM layer output H and the sparse attention output Z, calculate the forward attention weight and the backward attention weights ,in, represents the transpose of the sparse attention output Z, Represents the transpose of the BiLSTM output H; computes the forward attention output and the backward attention output , fusion of forward and backward attention to form bidirectional attention fusion features ,in, represents the forward attention Quanzhou, represents the backward attention weight.

6. According to claim 1, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The step 5 comprises the following steps: Step 51: Fusing the features after bidirectional attention Passed to the fully connected layer: ， Where W and is the weight matrix and bias term, P is the weight vector for each category; Step 52: Calculate the scores for each category : ， where c is the total number of categories, e is the base of the natural logarithm, and p i represents the weight of category i in p, p j represents the weight of category j in p; Step 53: Find the category corresponding to the maximum score and output the classification result of network attack detection: ， The category i with the highest score is the final classification result y.

7. According to claim 2, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The original network traffic data is preprocessed. is a complex network time series data packet, where represents the observed value feature vector at the first moment, represents the observed value feature vector at the second moment, Represents the observed value feature vector at the nth moment. Step 11 outlier screening includes the following steps: Step 111: Calculate the Angle and direction metrics : Step 112: Construct adjacent node sorting: For each , calculate its eigenvector with other observations Between , Represents the observed value feature vector at the mth moment, and then according to For other observations, the feature vector Sort and select L data points with a closer distance as its neighboring node set ; Step 113: Adaptive synthesis samples: ,according to Middle adjacent node and Distance , a new representative point is synthesized by weighted average , weight Based on the inverse of distance: in, Representation Node The neighboring nodes The subscript k of Then synthesize new points: Step 114: Outlier Detection: Calculation With synthesis point Distance between: If the distance is greater than the preset threshold , then it is believed that It is an outlier; Adjust according to the specific scenario.

8. According to claim 2, a method for industrial control network intrusion detection based on BiLSTM and attention mechanism is characterized in that: The original network traffic data is preprocessed. is a complex network time series data packet, where Indicates missing values, step 12 includes the following steps: Step 121: Select non-missing values ​​to construct training sequences , used to train the LSTM model M; Step 122: For each missing time t, use M to predict ,Right now , where k is the number of historical time steps considered, which is adjusted according to the specific scenario.

9. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 2 is characterized in that: The raw network traffic data is preprocessed, duplicate values ​​are merged and invalid values ​​are corrected, and a duplicate value merging algorithm based on adjacent record sorting and a regular expression method are used to batch process the data set; the steps include: Step 131: for each row of data in the specified format attribute column, duplicate value screening and regular expression matching are performed. If there are duplicate values, they are set to null values. For data that does not meet the preset conditions of the regular expression, they are also set to null values. Step 132: For the data in the specified attribute column, a size range is restricted, and data outside the range is also set to a null value.

10. The industrial control network intrusion detection method based on BiLSTM and attention mechanism according to claim 1 is characterized in that: The sparse attention layer, BiLSTM layer, bidirectional attention layer and fully connected layer use Dropout in the middle of each layer to prevent network overfitting and gradient explosion, and the Dropout parameter value is selected to be 0.4.

Images