Network intrusion detection method based on gated convolution and feature pyramid

Abstract

The present invention belongs to the field of intrusion detection, and relates to a network intrusion detection method based on gated convolution and feature pyramid. The method comprises the following steps: S1) preprocessing the used data; S2) using a gated convolutional neural network to extract semantic features of network data at different levels according to depth; S3) using a feature pyramid network to fuse the feature graphs of the deep layer of the model; S4) applying a fully connected neural network to the fused deep feature graphs of each model for classification judgment; S5) using a FocalLoss loss function to perform multi-supervisory training on the overall model. The present invention can achieve high-performance intrusion detection. Compared with other methods, the present invention has better results in precision, recall and comprehensive index F1-score in multi-classification intrusion detection, and achieves a huge improvement in the F1-score of minority class samples while improving the performance of the intrusion detection system.

Description

Technical Field

[0001] The invention belongs to the technical field of network intrusion detection, and relates to a network intrusion detection method based on gated convolution and feature pyramid. Background Art

[0002] Network intrusion detection technology is an important research direction in the field of information security in modern society. In today's network environment, various types of network attacks emerge in an endless stream, and intrusion detection technology plays an important role in maintaining network security. The task of network intrusion detection is to identify various abnormal traffic in the network and determine which type of attack it belongs to, so as to take corresponding defense mechanisms. At present, classic machine learning classifiers such as decision trees, support vector machines, logistic regression and multi-layer perceptron models are widely used in the classification task of network intrusion detection.

[0003] Traditional network data detection methods based on port identification, deep packet detection categories, and classical machine learning techniques are widely used in the field of intrusion detection. However, with the continuous expansion and development of the scale of network data, current network attacks have undergone earth-shaking changes in both quantity and category, and deep learning is more and more widely used in the field of intrusion detection. It is recognized as an important research method for key technologies of future intrusion detection, which is significantly different from traditional intrusion detection methods. In a real network environment, there are a large number of multi-dimensional, label-less, complex and changeable data, which puts higher requirements on the construction of intrusion detection models. Deep learning, with its excellent feature extraction capabilities, brings new ideas to the processing of complex intrusion data. Many researchers apply various deep learning to the field of intrusion detection, such as convolutional neural networks, recurrent neural networks, deep belief networks, etc. Therefore, the present invention adopts a method based on deep learning to construct an intrusion detection model.

[0004] Convolutional neural networks can automatically extract rich local abstract features and mine potential connections between data. However, when the convolutional neural network in deep learning extracts features from the input original data set, there are some similarities between the multiple feature maps extracted in each layer. These similar features are a certain degree of information redundancy for the next layer of neural network. The redundant feature information accumulated in each layer will consume a lot of processing time, causing the processing speed of the entire model to decrease and the final detection accuracy to decrease. Therefore, the features extracted by each convolutional layer are screened and filtered to obtain more effective feature information, and the amount of feature information can be relatively reduced. The method to achieve selective feature extraction in current research is to add a gating mechanism to the convolutional neural network. The gating mechanism can control the path of information flow in the network, and has been proven to be a very effective means in recurrent neural networks. LSTM controls the separated cells through input and forget gates to enable LSTM to obtain long-term memory. This allows information to flow unimpeded for multiple time steps. Without these gates, information will easily disappear when transferring through time steps. However, convolutional neural networks do not encounter such gradient vanishing phenomena, so the model only needs an output gate, which can control whether information should pass through these layers.

[0005] Since the size of the feature map extracted by each convolution layer and the semantic information it contains are different, for example, the information extracted by the shallow convolution layer is generally larger in size and richer in information, while the information extracted by the shallow convolution layer is generally smaller in size and contains more concentrated semantic information. Therefore, fusing the feature information of different network layers to form a more comprehensive feature map can enable the model to make more effective classification decisions. Existing neural network models are usually trained in a single-supervised manner. At present, many studies have applied multi-supervised learning to model training. Compared with single-supervised learning, multi-supervision uses multiple or multiple labels of data to compare the errors of the prediction results of multiple levels of the model, realizing a wider range of supervised learning. Although the training steps and costs have become higher, the model's final feature extraction ability and classification decision ability have become stronger.

[0006] Therefore, designing a network intrusion detection method based on gated convolution and feature pyramid is of great significance to improving the performance of intrusion detection systems. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a network intrusion detection method based on gated convolution and feature pyramid. The gated convolution is to add a gated linear unit in each convolution layer to simulate the forget gate of LSTM, or the idea of ​​judging weights, and also make a filter with the same parameters as CNN convolution, with values ​​between 0 and 1, to judge which of the extracted features should be paid attention to and which should be ignored. The output is the element-by-element multiplication of the outputs of two standard convolutional layers, one layer is followed by any activation function, and the other layer is followed by a Sigmoid activation function.

[0008] The operation of gated convolution is a standard convolution layer followed by a Sigmoid function as soft gating, which weights the output of the current convolution layer before inputting it to the next convolution layer. Hard gating only uses 0 or 1 for weighting, but for soft gating, any number between 0 and 1 can be used for weighting, which is more flexible and can be learned through training. Drawing on the Gate Mechanism idea of ​​LSTM, based on the Relu activation function and the Tanh activation function, combined with gateunit, activation units such as GTU units and GLU units are generated.

[0009] Because the pooling operation will continuously reduce the size of the feature map, the resolution of the convolutional neural network from shallow to deep becomes coarser and the feature map becomes smaller and smaller, but the higher the convolution layer, the richer the semantic information contained in the feature map. The feature pyramid network uses both the high resolution of low-level features and the high semantic information of high-level features, and achieves the prediction effect by fusing the features of these different layers. The feature pyramid network consists of two parts, bottom-up and top-down. The bottom-up part is the traditional convolutional network for feature extraction. As the convolution goes deeper, the spatial resolution decreases, the spatial information is lost, but more high-level semantic information is detected. The feature pyramid network connects the high-level features with low resolution and high semantic information with the low-level features with high resolution and low semantic information from top to bottom, so that the features at all scales have rich semantic information.

[0010] The general structure of the feature pyramid network is a bottom-up line, a top-down line, and a horizontal connection line. The bottom-up process is the normal forward propagation process of the neural network. In the forward process, the size of the feature map will change after passing through some layers, but will not change when passing through other layers. The layers that do not change the size of the feature map are classified as a stage, so each extracted feature is the last layer output of each stage, so that the feature pyramid can be formed. The top-down process is to upsample the high-level feature map and then connect the feature to the previous layer laterally, so that the high-level features are strengthened. Upsampling uses the interpolation method, that is, based on the original image pixels, a suitable interpolation algorithm is used to insert new elements between pixels, thereby expanding the size of the original image. By upsampling the feature map, the upsampled feature map has the same size as the feature map of the next layer. This is mainly to utilize the positional details of the bottom layer. The role of the horizontal connection is to splice the feature map of the previous layer with the upsampled feature map of the next layer to obtain the feature map with the richest information, so as to make classification decisions for the final fully connected neural network.

[0011] And the classification decision of the present invention is performed once on each fused feature layer, that is, predictions are made separately according to the different degrees of semantic information extracted by the model, and then the gaps between these multiple prediction results and the actual category labels of the data are compared. Therefore, the model proposed by the present invention is a multi-output model that adopts a multi-supervised training method.

[0012] Therefore, the present invention proposes a network intrusion detection method based on gated convolution and feature pyramid. The method uses the GLU gated mechanism to improve the conventional convolution, and combines the feature pyramid network to fuse the various feature maps in the deep layer of the model, so as to obtain more comprehensive and rich feature information for classification and judgment, and realize high-performance network intrusion detection. The method includes the following steps:

[0013] Step 1) preprocessing the data used;

[0014] Step 2) Using a gated convolutional neural network to extract semantic features of network data at different levels according to depth;

[0015] Step 3) Use feature pyramid network to fuse the deep feature maps of the model;

[0016] Step 4) Applying a fully connected neural network to the fused deep feature maps of each model for classification and judgment;

[0017] Step 5) Use the Focal Loss loss function to perform multi-supervisory training on the overall model.

[0018] Further, the step 1) specifically comprises the following steps:

[0019] Step 11) Network intrusion data usually has character features, which are inconvenient to input into the model for calculation, so they need to be encoded into numerical values. For example, network data usually has a protocol feature protocol, which has three attribute values: "TCP", "UDP", and the attribute values ​​are encoded into '0', '1', '2' using LabelEncoder label encoding. In this way, different attribute values ​​can be distinguished by the size of the numerical value, and it is easy to input into the model for calculation.

[0020] Step 12) Each dimension of network intrusion data has a different range of values. If these ranges can be proportionally shrunk to around 0 and the variance is 1, the characteristic attribute value of each piece of data will be greatly reduced, and the calculation speed in the model can be greatly improved. The present invention uses the Z-score standardization method, and the formula is:

[0021]

[0022] x * is the transformed output value, x is the attribute value of each dimension of the original data, u, σ are the mean and variance of the attribute value of each dimension of the original data.

[0023] Further, the step 2) specifically includes the following steps:

[0024] Step 21) Send the preprocessed data x to the gated convolution layer for feature extraction. The model has a total of 5 layers of gated convolution layers, each of which is a standard convolution layer followed by a standard convolution output activated by a Sigmoid function as a soft gate to construct a GLU unit. The gated convolution layer uses three types of convolution kernels: 1*1, 3*3, and 5*5. The 5*5 convolution kernel is used in the shallow layer of the network, the 3*3 convolution kernel is used in the middle layer of the network, and the 1*1 convolution kernel is used in the deep layer of the network.

[0025] The expression of GLU is as follows:

[0026]

[0027] Where X is the input data of each convolution layer, W and V are the weight parameters of two different convolution kernels, b and c are their respective bias parameters, and δ is the Sigmoid activation function. In fact, the expression δ(X*V+c) is the "activation function" of the standard convolution output (X*W+b). Compared with the conventional activation function, using GLU's soft gating as the activation layer of the standard convolution layer can determine the components of the extracted features that should be focused on and those that can be ignored. Compared with the Tanh activation function, GLU has a linear channel, the gradient is relatively stable, and it will not drop sharply during back propagation. Therefore, the convergence speed is faster when using GLU as the activation training.

[0028] Further, step 3) specifically comprises the following steps:

[0029] Step 31) The outputs of the three gated convolutional layers at the end of the network in step 2) are subjected to feature fusion using a feature pyramid network. The feature map output by the end network is expanded to the same size as the feature map output by the previous network layer by upsampling, and then the two feature maps are merged to obtain a comprehensive feature map containing different degrees of semantic information, which is output to the decision network part of the model for more accurate decision making. Upsampling uses bilinear interpolation, that is, linear interpolation is performed in the two dimensional directions of the feature map.

[0030] Further, the step 4) specifically comprises the following steps:

[0031] Step 41) applies Flatten operation to the multi-layer comprehensive feature map obtained in step 3) to compress it into a one-dimensional vector, and sends it to the fully connected neural network for dimensionality reduction extraction, and finally the Softmax layer outputs the predicted multi-classification results.

[0032] The Softmax formula is as follows:

[0033]

[0034] where y i It indicates that the sample is predicted to be the output value of category i, n represents the total number of categories in the data set, and the Softmax function can be used to convert the output value of multiple categories into a probability distribution ranging from [0, 1] and 1.

[0035] Further, the step 5) specifically comprises the following steps:

[0036] Step 51) compares the multi-classification results output by the multi-layer comprehensive feature map in step 4) with the actual data category labels, applies the Focal Loss loss function to record the prediction error loss value in real time and feed it back to the model, and continuously optimizes and updates the weights and biases of the network nodes through multiple rounds of data iterative training, so that the final classification results are closer to the actual labels. After completing the above parameter training, make a decision response to the test data.

[0037] Because network intrusion detection datasets are usually data with unbalanced class proportions, the Focal Loss loss function introduces a weight factor α in the cross entropy loss to scale the losses of samples of different classes, so that the model focuses on the loss of a certain class of samples during training. Generally, the inverse of the number of samples in each class is taken as the weight factor of the class of samples in FocalLoss. The expression of the Focal Loss loss function is as follows:

[0038] FL(p t )=-α t (1-p t ) γ log(p t )

[0039] where p t represents the probability that the sample is classified as category t, α t It indicates the weight of samples of category t in calculating the loss function, and γ is the modulation factor, also known as the focusing parameter. It can be seen from the formula that setting α t The value of can adjust the model's attention to the t-th class sample during training. t ) γ The role is: when the sample classification is wrong, p t tends to 0, the modulation factor tends to 1, making the loss function almost unaffected; when the sample is correctly classified, p t will tend to 1 and the modulation factor will tend to 0, making the loss very close to 0, thereby reducing the weight of this particular example. The focusing parameter γ serves to smoothly adjust the rate at which easily classified examples are weighted downward.

[0040] Beneficial effects:

[0041] 1. The proposed gated convolutional neural network layer can filter out useless information in the features, retain useful information, and obtain a more accurate feature map when performing feature extraction.

[0042] 2. The feature pyramid network can be used to fuse the information of each feature map deep in the model to obtain a comprehensive feature map with richer semantics.

[0043] 3. Use multi-supervised training to optimize the classification judgment results output by each comprehensive feature map, so that the final classification performance of the model is greatly improved compared with single-supervised training.

[0044] 4. The Focal Loss loss function training model can adjust the weight of the loss of each category of samples and accelerate the convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to express the purpose, technical solutions and advantages of the present invention more clearly, the present invention will be described in more detail below with reference to the accompanying drawings, in which:

[0046] Figure 1 Schematic diagram of the intrusion detection model structure based on gated convolution and feature pyramid

[0047] Figure 2 Schematic diagram of the gated convolution module structure DETAILED DESCRIPTION

[0048] The present invention provides an intrusion detection method based on gated convolution and feature pyramid, such as Figure 1 As shown, the method comprises the following steps:

[0049] Step 1: Since network data is high-dimensional linear data, if it is used directly as a neural network model, it is necessary to use a data preprocessing module to preprocess the original data. First, the symbolic features in the data need to be digitized, and then the data needs to be normalized and mapped to the [0,1] interval.

[0050] Step 2: According to the structural characteristics of the gated convolutional network, the one-dimensional linear data is converted into matrix data for input. The NSL-KDD dataset is used. The original data size is 41 features. It is expanded to 42 dimensions using a column of all 0s, which is convenient for conversion into a 6*7 two-dimensional matrix as the input of the model.

[0051] Step 3: First, each gated convolutional layer uses the GLU unit to extract features from the data and performs data dimension reduction in the pooling layer to obtain multi-layer feature maps. Then, each feature map in the deep layer of the model is upsampled to the same size as the feature map output by the previous layer of the network, and these feature maps are merged to obtain a comprehensive feature map with more channels and richer feature semantics.

[0052] Step 4: Use a fully connected neural network to classify and judge each comprehensive feature graph obtained at the output end, predict the probability that the sample belongs to each attack category, compare the output result with the true category of the sample, and use the Focal Loss function to calculate the loss of the prediction result. The prediction result of each comprehensive feature graph is trained with the true category label to continuously reduce the prediction loss, realize the function of multi-supervised learning, and finally obtain a higher performance network intrusion detection model.

[0053] The above description is only a specific implementation mode of the present application. It should be pointed out that for ordinary technicians in this field, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A network intrusion detection method based on gated convolution and feature pyramid, characterized by: This method uses the GLU gating mechanism to improve conventional convolution, and combines the deep feature maps of the feature pyramid network fusion model to obtain more comprehensive and rich feature information for classification and judgment, thereby realizing network intrusion detection. The method includes the following steps: Step 1) preprocessing the data used; Step 2) Using a gated convolutional neural network to extract semantic features of network intrusion data at different levels according to depth; Step 3) Use feature pyramid network to fuse the deep feature maps of the model; Step 4) Applying a fully connected neural network to the fused deep feature maps of each model for classification and judgment; Step 5) Use FocalLoss loss function to perform multi-supervision training on the overall model; The step 1) specifically comprises the following steps: Step 11) encoding the character features of the network intrusion data into numerical values; Step 12) normalizing the data converted into numerical values ​​using the Z-score standardization method; The step 2) specifically comprises the following steps: Step 21) The preprocessed data is sent to the gated convolution layer for feature extraction; the model has a total of 5 layers of gated convolution layers, each of which is a standard convolution layer followed by a standard convolution output activated by a Sigmoid function as a soft gate, thereby constructing a GLU unit; the gated convolution layer uses a total of three types of convolution kernels: 1*1, 3*3, and 5*5; a 5*5 convolution kernel is used in the shallow layer of the network, a 3*3 convolution kernel is used in the middle layer of the network, and a 1*1 convolution kernel is used in the deep layer of the network; The expression of GLU is as follows: Where X is the input data of each convolution layer, W and V are the weight parameters of two different convolution kernels, b and c are the bias parameters of W and V, δ is the Sigmoid activation function, and the expression δ(X*V+c) is the "activation function" of (X*W+b); The step 3) specifically comprises the following steps: Step 31) The outputs of the three gated convolutional layers at the end of the network in step 2) are subjected to feature fusion using a feature pyramid network; the feature map output by the end network is expanded to the same size as the feature map output by the previous network layer by upsampling, and then the two feature maps are merged to obtain a comprehensive feature map containing semantic information of different degrees, which is output to the decision network part of the model for more accurate decision; upsampling adopts a bilinear interpolation method, that is, linear interpolation is performed in the two dimensional directions of the feature map respectively; The step 4) specifically comprises the following steps: Step 41) applies Flatten operation to the multi-layer comprehensive feature map obtained in step 3) to compress it into a one-dimensional vector, and sends it to a fully connected neural network for dimensionality reduction extraction, and finally outputs the predicted multi-classification result by the Softmax layer; The Softmax formula is as follows: where y i Indicates that the sample is predicted to be the output value of category i, n represents the total number of categories in the data set, and the Softmax function is used to convert the output value of the multi-classification into a probability distribution ranging from [0, 1] and 1; The step 5) specifically comprises the following steps: Step 51) compares the multi-classification results output by the multi-layer comprehensive feature map in step 4) with the real data category labels, applies the Focal Loss loss function to record the prediction error loss value in real time and feed it back to the model, and continuously updates the weights and biases of the network nodes through multiple rounds of data iterative training, so that the final classification results are close to the real labels. After completing the above parameter training, make a decision response to the test data; The expression of FocalLoss loss function is as follows: FL(P t )=-a t (1-p t ) γ log(p t ) where p t represents the probability that the sample is classified as category t, a t It represents the weight of samples of category t in calculating the loss function, and γ is the modulation factor.

Images