A network intrusion detection method based on LSTM and attention mechanism

Abstract

The application discloses a network intrusion detection method based on LSTM and an attention mechanism, and comprises the following steps: 1) data preprocessing and target sample construction; 2) WGAN model establishment and new training sample merging; 3) network intrusion detection model establishment based on LSTM and the attention mechanism; 4) training in the obtained network intrusion detection model; and 5) model performance evaluation. The method can improve the detection accuracy and the detection rate of minority class samples.

Description

Technical Field

[0001] This invention relates to the field of network information security, specifically a network intrusion detection method based on LSTM and attention mechanisms. Background Technology

[0002] With the rapid development of network technology and big data, increasingly sophisticated attacks are emerging. According to the latest "2022 Cost of Data Breach Report," the average cost of a data breach is as high as $435, and 83% of surveyed organizations have experienced data breaches before. In recent years, cyberattacks and privacy breaches have occurred frequently in critical infrastructure organizations such as those in finance, healthcare, and the military, causing incalculable losses and seriously threatening daily life. Intrusion Detection Systems (IDS) are an effective means of detecting behaviors that compromise computer system security, effectively preventing and stopping unauthorized intrusions. Traditional intrusion detection methods are no longer sufficient to meet the demands of increasingly complex attack methods. Therefore, effective models are needed to improve the ability to detect network attacks, increase accuracy, and reduce false positive rates. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a network intrusion detection method based on LSTM and attention mechanisms. This method can improve detection accuracy and the detection rate of minority class samples.

[0004] The technical solution to achieve the objective of this invention is:

[0005] A network intrusion detection method based on LSTM and attention mechanism includes the following steps:

[0006] 1) Download the NSL-KDD network intrusion dataset and perform data preprocessing and construct target samples: The data preprocessing involves integrating the data, quantifying the character-type attributes, and then performing normalization operations, including:

[0007] 1-1) Preprocess the original samples: Specifically:

[0008] 1-1-1) Preprocessing the feature data involves first deleting data with empty values ​​in the dataset, and then converting non-numerical data into numerical data.

[0009] 1-1-2) Normalization is used to process the dataset, and the last column of the dataset is extracted as the classification label. The dataset is divided into training set: validation set ratio of 8:2. The normalization formula is shown in formula (1):

[0010]

[0011] Formula (1)x_max x is the maximum value among the eigenvalues. _min x is the minimum value among the eigenvalues. _norm The output value is between [0, 1].

[0012] 1-2) WGAN is used to oversample minority class samples, and random undersampling technique is used to undersample majority class samples to balance the original samples;

[0013] 2) Build a WGAN model to augment the data obtained in steps 1-2). Perform random undersampling on the normal flow data and merge it into new training samples. Specifically:

[0014] 2-1) First, define the generator network as G and the discriminator network as D. The input is the generator random noise z and the real sample x. For each training step, the same number of samples are randomly drawn from the real dataset and the noise z generated by the generator network.

[0015] 2-2) Input the real sample x and the sample G(z) generated by the generator into the discriminator network respectively to obtain the discriminator's prediction results D(x) and D(G(z)) for the real sample and the generated sample, respectively. Calculate the Wasserstein distance, i.e., the difference between D(x) and D(G(z)), as shown in Equation (2):

[0016] Wasserstein=E[D(x)]-E[D(G(z))] (2),

[0017] Where E[] represents the expected value, which is the average value of the sample;

[0018] 2-3) Update the parameters of the generator and discriminator based on the value of the Wasserstein distance. To maximize the Wasserstein distance, it is necessary to minimize the discriminator's prediction result D(G(z)) for the generated image and maximize the discriminator's prediction result D(x) for the real image. Repeat the steps until the model converges.

[0019] 2-4) In the NSL-KDD dataset, 50,000 normal stream data were randomly selected from the original 67,343 normal stream data using the random undersampling technique;

[0020] 2-5) Merge normal flow data, enhanced minority class data, and attack data of unselected types as training samples;

[0021] 3) Establish a network intrusion detection model based on LSTM and attention mechanisms: including:

[0022] 3-1) Long Short-Term Memory (LSTM) network is used for feature extraction from the dataset, specifically:

[0023] 3-1-1) The input gate, forget gate, and output gate of the Long Short-Term Memory (LSTM) network are used to extract temporal features from the data;

[0024] 3-1-2) Stack the extracted features along the depth direction;

[0025] 3-2) Employing a self-attention mechanism to update features: Specifically:

[0026] Define each input as in The output O for each time step is calculated in the LSTM through memory cells. i = [O1, O2, ..., O t ] and the hidden state H at the last time step t Let the function g represent the internal operation of the LSTM model, as shown in formula (3):

[0027]

[0028] Then, establish O for each time step. i For H t The weights are calculated using formula (4) at time step t to obtain the weight matrix Key. t Value t Query, score e t and weight a t :

[0029]

[0030] In formula (4), the output O at each time step t After a linear transformation, it becomes Key and Value, ω K ω V and ω Q These are the parameters of the neural network, updated after each backpropagation, and then the weights a are adjusted using formula (5). t and Value t By performing a weighted summation, we obtain the vector Z:

[0031]

[0032] Formula (5) is updated h times to obtain z1, z2, ..., z t Let z1, z2, ..., z t After concatenation and a linear transformation, the output is as shown in formula (6):

[0033] MultiHead(Q,K,V)=Concat(z1,z2,...,zt )ω z (6);

[0034] 3-3) Softmax is used to classify the data: The predicted values ​​for the classification task are obtained after passing through a fully connected layer, as shown in formula (7):

[0035] y = Softmax(w i ·MultiHead(Q,K,V)+b) (7);

[0036] 4) Input the training samples merged in step 2) into the network intrusion detection model obtained in step 3) for training: When inputting the training set into the network intrusion detection model for training, the Adam optimizer is used, the learning rate is 0.0001, the batch size is 64, and all data are trained to completion as one loop, and the loop is repeated 100 times.

[0037] 5) Evaluate the model's performance using accuracy (acc) and F-measure values:

[0038] The definitions of the various performance indicators are shown in formulas (8) and (9):

[0039]

[0040]

[0041] Compared with existing technologies, the beneficial effects of this technical solution are:

[0042] This technical solution uses generative adversarial networks to address the imbalance problem of datasets and improve the detection of rare attack samples. It also uses the Adam optimizer to optimize the training process, avoiding local optima while accelerating the training convergence speed.

[0043] This method can improve detection accuracy and the detection rate of minority samples. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the discriminator network of WGAN in the embodiment;

[0045] Figure 2 This is a schematic diagram of the network intrusion detection model structure based on LSTM and attention mechanism in the embodiment;

[0046] Figure 3 This is a schematic diagram illustrating the F1 scores of the NSL-KDD dataset on network intrusion detection models using different processing methods in the examples, specifically LSTM and attention mechanisms. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this is not intended to limit the scope of the invention.

[0048] Example:

[0049] A network intrusion detection method based on LSTM and attention mechanism includes the following steps:

[0050] 1) Download the NSL-KDD network intrusion dataset and perform data preprocessing and construct target samples: The data preprocessing involves integrating the data, quantifying the character-type attributes, and then performing normalization operations, including:

[0051] 1-1) Preprocess the original samples: Specifically:

[0052] 1-1-1) Preprocessing the feature data involves first deleting data with empty values ​​in the dataset, and then converting non-numerical data into numerical data.

[0053] 1-1-2) Normalization is used to process the dataset, and the last column of the dataset is extracted as the classification label. The dataset is divided into training set: validation set ratio of 8:2. The normalization formula is shown in formula (1):

[0054]

[0055] Formula (1)x _max x is the maximum value among the eigenvalues. _min x is the minimum value among the eigenvalues. _norm The output value is between [0, 1].

[0056] 1-2) WGAN is used to oversample the minority class samples, and random undersampling is used to undersample the majority class samples to balance the original samples. In this example:

[0057] The dataset used is the NSL-KDD dataset. Since deep learning inputs must be numerical data, the non-numerical data was first converted into numerical data. Then, the merged dataset was processed using MinMaxScaler normalization in sklearn, and the last column of the dataset was extracted as the classification label. The NSL-KDD dataset suffers from a severe class imbalance; some attack datasets have only a few dozen (minority class) samples, while others have tens of thousands. This leads to a very low minority class detection rate. Therefore, WGAN is used to generate minority class samples. The WGAN discriminator network first passes through a convolutional neural network and batch normalization, then through a fully connected layer, finally yielding an output. The generator is similar to the discriminator, as shown below. Figure 1As shown in Table 1, a comparison of samples before and after generation was performed. Normal flow samples were removed using random undersampling techniques, and a comparison of samples before and after deletion is shown in Table 2. During model training, to continuously monitor model performance, the training and validation sets were divided in an 8:2 ratio.

[0058] Table 1 Comparison of samples before and after data augmentation.

[0059]

[0060] Table 2 Comparison of samples after random undersampling

[0061] Dataset Normal stream raw data After undersampling NSL-KDD 67343 50000 ;

[0062] 2) Build a WGAN model to augment the data obtained in steps 1-2). Perform random undersampling on the normal flow data and merge it into new training samples. Specifically:

[0063] 2-1) First, define the generator network as G and the discriminator network as D. The input is the generator random noise z and the real sample x. For each training step, the same number of samples are randomly drawn from the real dataset and the noise z generated by the generator network.

[0064] 2-2) Input the real sample x and the sample G(z) generated by the generator into the discriminator network respectively to obtain the discriminator's prediction results D(x) and D(G(z)) for the real sample and the generated sample, respectively. Calculate the Wasserstein distance, i.e., the difference between D(x) and D(G(z)), as shown in Equation (2):

[0065] Wasserstein=E[D(x)]-E[D(G(z))] (2),

[0066] Where E[] represents the expected value, which is the average value of the sample;

[0067] 2-3) Update the parameters of the generator and discriminator based on the value of the Wasserstein distance. To maximize the Wasserstein distance, it is necessary to minimize the discriminator's prediction result D(G(z)) for the generated image and maximize the discriminator's prediction result D(x) for the real image. Repeat the steps until the model converges.

[0068] 2-4) In the NSL-KDD dataset, 50,000 normal stream data were randomly selected from the original 67,343 normal stream data using the random undersampling technique;

[0069] 2-5) Merge normal flow data, enhanced minority class data, and attack data of unselected types as training samples;

[0070] 3) Establish a network intrusion detection model based on LSTM and attention mechanisms: including:

[0071] 3-1) Long Short-Term Memory (LSTM) network is used for feature extraction from the dataset, specifically:

[0072] 3-1-1) The input gate, forget gate, and output gate of the Long Short-Term Memory (LSTM) network are used to extract temporal features from the data;

[0073] 3-1-2) Stack the extracted features along the depth direction;

[0074] 3-2) Employing a self-attention mechanism to update features: Specifically:

[0075] Define each input as in The output O for each time step is calculated in the LSTM through memory cells. i = [O1, O2, ..., O t ] and the hidden state H at the last time step t Let the function g represent the internal operation of the LSTM model, as shown in formula (3):

[0076]

[0077] Then, establish O for each time step. i For H t The weights are calculated using formula (4) at time step t to obtain the weight matrix Key. t Value t Query, score e t and weight a t :

[0078]

[0079] In formula (4), the output O at each time step t After a linear transformation, it becomes Key and Value, ω K ω V and ω Q These are the parameters of the neural network, updated after each backpropagation, and then the weights a are adjusted using formula (5). t and Value t By performing a weighted summation, we obtain the vector Z:

[0080]

[0081] Formula (5) is updated h times to obtain z1, z2, ..., z t Let z1, z2, ..., zt After concatenation and a linear transformation, the output is as shown in formula (6):

[0082] MultiHead(Q,K,V)=Concat(z1,z2,...,z t )ω z (6);

[0083] 3-3) Softmax is used to classify the data: The predicted values ​​for the classification task are obtained after passing through a fully connected layer, as shown in formula (7):

[0084] y = Softmax(w i ·MultiHead(Q, K, V)+b) (7), in this example:

[0085] The classic LSTM model is a recurrent neural network used to process sequential data, especially in natural language processing tasks. The main purpose of LSTM is to solve the vanishing or exploding gradient problems that ordinary RNNs encounter when processing long sequences. LSTM has three gates: the input gate, the output gate, and the forget gate. These gates control the amount and direction of information flow, thus better capturing the dependencies between long sequences. Attention mechanisms are mechanisms that dynamically assign different importance weights to different parts based on input and context. In natural language processing, attention mechanisms can be used to weight and summarize key information in a sentence, thereby better understanding the meaning of the sentence. In attention mechanisms, the input sequence is first mapped to feature vectors, then the similarity between each input vector and a specific context vector is calculated, and finally, the input vectors are weighted and summarized. This weighted summarization method makes the model more flexible and can better capture important information in the input sequence. Combining these two techniques, we can create a model based on LSTM and attention mechanisms, where each time step uses the attention mechanism to weight each vector in the input sequence, thereby better capturing important information in the input sequence and generating better output. Figure 2The model structure shown is a multi-layer LSTM model structure that obtains temporal feature values ​​of the data and then uses an attention mechanism to perform deeper fusion of the temporal features. LSTM is used to process the temporal dimension of the network flow. Each data point initially passes through an LSTM layer to extract preliminary features. Then, it passes through the attention mechanism module, which, based on the parameter values ​​set at the time step, calculates an output vector and the state of the current time step based on the current input vector and the state of the previous time step. This state is passed to the next time step so that the next time step can use the information of the previous time step for calculation. This further extracts features and prevents gradient vanishing. This technical solution is for a classification problem, and uses accuracy, recall, and F-measure values ​​to measure the classification performance. Generally speaking, the higher the accuracy, the better the overall classification performance of the classification system.

[0086] 4) Input the training samples merged in step 2) into the network intrusion detection model obtained in step 3) for training: When inputting the training set into the network intrusion detection model for training, the Adam optimizer is used, the learning rate is 0.0001, the batch size is 64, and all data are trained to completion as one loop, and the loop is repeated 100 times.

[0087] 5) Evaluate the model's performance using accuracy (acc) and F-measure values:

[0088] The definitions of the various performance indicators are shown in formulas (8) and (9):

[0089]

[0090] In this example, by comparing commonly used deep learning models and data augmentation using the SMOTE algorithm, we analyze the F1 score. The accuracy varies depending on the algorithm and also differs across different attack types, thus validating the effectiveness of this example. Table 3 shows the comparison of F1 scores for each model before and after NSL-KDD data balancing. Then, we compare the F1 scores of different processing methods on the NSL-KDD dataset for network intrusion detection models using LSTM and attention mechanisms. Figure 3 As shown, considering all the indicators, the method in this example is superior. The SMOTE algorithm process is as follows:

[0091] Step 1: For each sample x in the minority class i Calculate its Euclidean distance with all minority class samples;

[0092] Step 2: Select the k minority class samples that are closest to the current sample, where k is a specified parameter, and randomly select one sample from them;

[0093] Step 3: For each randomly selected sample, construct a new sample according to formula (10):

[0094] x new =x i +rand(0,1)*(x j -x i )j=1, 2, 3, ....., M (10),

[0095] Table 3 F1 values ​​before and after NSL-KDD data balancing

[0096]

[0097] Experimental results show that, compared with existing technologies, the proposed method can improve detection accuracy and detection rate of minority class samples.

Claims

1.A network intrusion detection method based on LSTM and attention mechanism, characterized in that, Comprising the following steps: 1) download NSL-KDD network intrusion dataset and data preprocessing, build target sample: the data preprocessing is to integrate the data, and the character type attribute is numerically operated, and then normalized, including: 1-1) preprocessing of original sample: specifically: 1-1-1) preprocessing of feature data, that is, first deleting the data column with null value in the data set, and then converting the non-numeric data into numeric data by one-hot encoding; 1-1-2) normalizing the data set, extracting the last column of the data set as the classification label, and dividing the data set into training set: validation set according to the ratio of 8:2, wherein the normalization formula is shown in formula (1): Equation (1) x _max is the maximum value among the eigenvalues, x _min is the minimum value among the eigenvalues, x _norm is the output value, the output value being between [0, 1]; 1-2) separate the data with less than 1000 data in NSL-KDD dataset as minority class data, that is, U2R attack class and R2L attack class, use WGAN to oversample the minority class sample, and use random undersampling technology to undersample the majority class sample to balance the original sample; 2) establish WGAN model to enhance the data obtained in step 1-2), randomly undersample the normal flow data, and combine into new training samples, specifically: 2-1) first define the generator network as G and the discriminator network as D, the input is the random noise z of the generator and the real sample x, for each training step, randomly select the same number of samples from the real data set and the noise z generated by the generator network; 2-2) input the real sample x and the sample G(z) generated by the generator into the discriminator network respectively, get the prediction results D(x) and D(G(z)) of the discriminator for the real sample and the generated sample, calculate the Wasserstein distance, that is, the difference between D(x) and D(G(z)), as shown in formula (2): Wasserstein=E[D(x)]-E[D(G(z))] (2), Where E[] represents the expected value, that is, the average calculation of the sample; 2-3) update the parameters of the generator and the discriminator according to the value of the Wasserstein distance, in order to maximize the Wasserstein distance, minimize the prediction result D(G(z)) of the discriminator for the generated image, and maximize the prediction result D(x) of the discriminator for the real image, repeat the steps until the model converges; 2-4) in NSL-KDD dataset, use random undersampling technology to randomly select 50000 normal flow data from the original 67343 normal flow data; 2-5) merge the normal flow data, the enhanced minority class data and the unselected type attack data into the training sample; 3) establish network intrusion detection model based on LSTM and attention mechanism: including: 3-1) using long short-term memory network LSTM for feature extraction in data set, specifically: 3-1-1) using input gate, forget gate and output gate in long short-term memory network LSTM to extract time sequence features in data; 3-1-2) stack the extracted features in depth direction; 3-2) Update the features using self-attention mechanism: Specifically: Define the input of each time as where The output O of each time step is calculated by the memory unit in LSTM i = [O1, O2,..., O t ] and the hidden state H of the last time step t The operation inside the LSTM model is represented by the function g, as shown in equation (3): Then, establish each time step O i For H t The weight of the time step t, the weight matrix Key t , Value t , Query, score e t And weight a t : In formula (4), the output O of each time step t After linear transformation, ω K , ω V and ω Q are the parameters of the neural network, which are updated after each back propagation, and then the weights a t and Value t are weighted and summed to obtain the vector Z: Formula (5) is updated h times to obtain z1, z2, ..., z t Let z1, z2, ..., z t After concatenation and a linear transformation, the output is as shown in formula (6): MultiHead(Q, K, V) = Concat(z1, z2,..., z t )ω z (6); 3-3) Classify the data using softmax: Get the prediction value of the classification task through the full connection layer, as shown in equation (7): y = Softmax(w i • MultiHead(Q, K, V) + b (7); 4) Input the training samples combined in step 2) into the network intrusion detection model obtained in step 3) for training: When inputting the training set into the network intrusion detection model for training, the Adam optimizer is used, the learning rate is 0.0001, the batch is 64, all data training is completed as one cycle, and the cycle is 100 times; 5) Evaluate the performance of the model using accuracy acc and F-measure value: The definitions of various indicators for performance evaluation are shown in equations (8) and (9):

Images