Intrusion detection method based on combination of multilayer deep neural network model and time attention mechanism

Abstract

The invention discloses an intrusion detection method based on a multi-layer deep neural network model in combination with a time attention mechanism, and the method comprises the steps: inputting a data set into a time sliding window algorithm, and obtaining the data of retention time features; capturing the continuous correlation between the time dynamics and the network behavior in the data; by combining a mixed architecture of 1DCNN and Multi-BIGRU, the model can extract local and global time features from input data at the same time, introduces a time attention mechanism, and automatically pays attention to a key time node which has the greatest influence on a classification result. According to the method, key features are extracted from multiple dimensions, accurate recognition of complex attack behaviors is realized, and the attack detection efficiency of network intrusion is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of information security, and specifically relates to an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism. Background Art

[0002] Traditional intrusion detection methods are mainly based on rule matching and statistical analysis. Although these methods are effective in specific scenarios, their detection effectiveness and real-time performance often cannot meet the needs when faced with increasingly complex and changeable network attacks.

[0003] The rise of deep learning technology has provided a new solution for intrusion detection. Deep learning models can automatically learn and extract features from massive data, thereby effectively detecting unknown attacks. However, network intrusion behavior has temporal characteristics, that is, the attack behavior usually continues to occur over a period of time or presents a certain pattern. If the temporal feature dependency is not considered, the model may not be able to accurately capture these patterns, resulting in reduced detection accuracy. In real-time intrusion detection scenarios, capturing temporal features is particularly important. Not considering temporal feature dependency may cause the model to fail to respond in time to rapidly changing network threats. Models that do not consider temporal feature dependency may lack sufficient generalization capabilities when facing different types of network intrusions. Because different attack modes may have different temporal features, ignoring these features may cause the model to be unable to effectively respond to diverse attack methods. In recent years, deep neural network structures that consider temporal features have emerged, such as the CNN-BILSTM model.

[0004] The CNN-BiLSTM model combines the advantages of CNN in feature extraction and classification with the strengths of BiLSTM in processing time series data and capturing time dependencies. However, the CNN-BiLSTM model has high computational complexity and requires a long training time and high computing resources. At the same time, the data dependency is strong, and the performance of the model depends largely on the quality and quantity of the input data. If the data is noisy, missing, or unbalanced, it may affect the training and prediction effects of the model. This is a common disadvantage of multi-layer deep neural networks.

[0005] How to effectively improve the performance of intrusion detection systems, how to achieve real-time response to network attacks, and provide strong support for network security protection are technical issues that need to be solved urgently. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, which improves the attack detection efficiency of network intrusion by first processing a one-dimensional convolutional neural network and then a multi-layer bidirectional gated recurrent unit.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, comprising the following steps: Preprocess the original network traffic data set to obtain the network traffic data set; Using the time sliding window with a unified step size, the network traffic data set is processed to obtain coarse-grained time features; The coarse-grained time features are processed using a one-dimensional convolutional neural network to obtain a high-level feature sequence, which is then input into a multi-layer bidirectional gated recurrent unit for time series processing to obtain multi-level time features. The time attention mechanism is used to perform weighted processing on multi-level time features to obtain weighted time features, and the multi-level time features and weighted time features are fed back to the intrusion detection model to dynamically adjust and update the model parameters. Test the intrusion detection model and obtain multi-classification results of intrusion detection data.

[0008] As a preferred technical solution, the feature processing of the network traffic data set is specifically as follows: Initialize an empty list, store the network traffic data in the empty list, obtain the data table, loop through the data table, and complete a cycle to move a window with a fixed time step; Calculate the statistical features of the data in each window. The statistical features are used for feature analysis and model training, including the mean, maximum, minimum, and mean square error. The statistical features of each window are packaged, the network traffic data is randomly divided into training data and test data according to the set ratio, and the time series data is kept, and the original data structure is restored in each window.

[0009] As a preferred technical solution, the one-dimensional convolutional neural network is connected to the multi-layer bidirectional gated recurrent unit through a batch normalization layer, and the one-dimensional convolutional neural network includes multiple convolution kernels.

[0010] As a preferred technical solution, the one-dimensional convolutional neural network is used to process the coarse-grained time features, including: Multiple convolution kernels are used to extract features from the training set. Each convolution kernel extracts multiple different features. Each convolution kernel slides on the input data through convolution operations to obtain multiple feature maps.

[0011] As a preferred technical solution, the multi-layer bidirectional gated recurrent unit includes a multi-layer BiGRU network, a temporal attention layer and a Reshape layer, and the multi-layer BiGRU networks are connected through the temporal attention layer and the Reshape layer.

[0012] As a preferred technical solution, the high-level feature sequence is input into a multi-layer bidirectional gated recurrent unit for time series processing, specifically: The BiGRU network is used to process the features in the high-level feature sequence to obtain the time series features. The temporal attention layer is used to calculate the attention score to obtain the key temporal features. After feature processing by multiple layers of BiGRU networks and temporal attention layers, multi-level temporal features are obtained.

[0013] As a preferred technical solution, the intrusion detection model includes a pooling layer and a dropout layer, and the pooling layer is arranged between the one-dimensional convolutional neural network and the multi-layer bidirectional gated recurrent unit, and within the multi-layer bidirectional gated recurrent unit; The pooling layer is used to learn the spatial hierarchical structure information in the data through the pooling operation, and the dropout layer is used to randomly select a part of the neurons of the multi-layer bidirectional gated recurrent unit and set the outputs of these neurons to zero.

[0014] As a preferred technical solution, the method of using the time attention mechanism to perform weighted processing on multi-level time features includes: The energy value of each time step is calculated, and the energy value is converted into probability through the softmax function to obtain the attention weight of the multi-level time feature; the multi-level time feature has multiple time steps, and each time step includes the feature vector of the respective time step; according to the attention weight, the feature vectors of all time steps are weighted and summed to obtain the weighted time feature.

[0015] As a preferred technical solution, the multi-level time features and weighted time features are fed back to the intrusion detection model, specifically: Calculate various potential interferences using multi-level time features and weighted time features, obtain loss values, adjust the intrusion detection model parameters through back propagation, and obtain the probability values ​​of each round of prediction; The probability value is compared with the actual label to quantify the loss of the evaluation model, and the evaluation result is used to guide the optimization of the intrusion detection model.

[0016] As a preferred technical solution, the preprocessing of the original network traffic data set includes: Random oversampling randomly copies data points from the minority class; Adjust the amount of data; Use one-hot encoding to convert features into numerical features; Normalize and rescale the data to a specific range.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention uses a one-dimensional convolutional neural network 1D-CNN and a multi-layer deep neural network Multi-BiGRU. 1D-CNN is responsible for extracting local features, while Multi-BiGRU is responsible for capturing deeper temporal dependencies. This combination simulates the real process and helps to more comprehensively analyze and understand attack behaviors. The real attack behavior is simulated from coarse-grained to fine-grained extraction, and key features are extracted from multiple dimensions to achieve accurate recognition of complex attack behaviors and improve the robustness of the model.

[0018] (2) The present invention introduces a temporal attention mechanism, which enables the model to flexibly process time series data. Weights can be automatically assigned according to the importance of the data, allowing the model to focus on time points that have a significant impact on the detection results, thereby improving the efficiency and accuracy of the detection.

[0019] (3) The present invention uses a time sliding window algorithm to preserve the time dependency between attack traffic data, efficiently track the time dependency in attack traffic data, and ensure sensitive capture of dynamic changes in traffic data. This not only improves the speed of data processing, but also enhances the real-time and accuracy of attack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to an embodiment of the present invention; Figure 2 This is a structural diagram of a multi-layer deep neural network model combined with a temporal attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0023] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0024] One-dimensional convolutional neural network (1D-CNN) is a variant of convolutional neural network (CNN), which is mainly used to process one-dimensional sequence data, such as audio, text, time series, etc. 1D-CNN extracts local features in sequence data through convolutional layers, which reflect local patterns in the input data. Compared with traditional fully connected neural networks, 1D-CNN can more effectively utilize local relationships in data when processing sequence data, so it performs well in tasks such as speech recognition, natural language processing, and time series prediction.

[0025] See also Figure 1 , this embodiment provides an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, comprising the following steps: S1. Preprocess the original network traffic data set to obtain the network traffic data set.

[0026] For the preprocessing method of the original network traffic data set, this embodiment provides the following steps: S101. Oversampling technique: Random oversampling randomly copies data points from the minority class, which reduces data imbalance and improves the prediction accuracy of the minority class.

[0027] S102. Adjust the number of items: The time sliding window algorithm is particularly suitable for flow time series data, and can effectively capture the dynamic nature of data changing over time. This method can help extract meaningful patterns from continuous traffic data. First, an empty list is initialized to store the processed data. Then, this embodiment traverses the data set through a loop that moves a fixed step size at a time and calculates the required statistical features within each window, such as the mean, maximum, minimum, and mean square error. These statistical features are considered to represent each window and can be used for subsequent analysis or machine learning model training.

[0028] S103.One-hot encoding: Use one-hot encoding to convert features into digital features.

[0029] S10, Normalization: Normalization rescales the data to a specific range, reducing redundancy and shortening model training time. This article uses Min-Max normalization to adjust the data range to [0,1], where the formula used is: .

[0030] S2. Use the time sliding window to unify the step size, use the time sliding window to unify the step size, perform feature processing on the network traffic data set, and obtain coarse-grained time features.

[0031] Assume that you already have a dataset D that contains timestamps and multiple features. At the same time, set the window size W to 60 (for example, representing a time window of 60 data points) and the step size S to 10 (indicating that the window moves 10 data points each time).

[0032] Initialize an empty list to store the statistical features of the processed data window. Traverse the data set, starting from index=0, and traverse the data set D. As long as index plus the window size W does not exceed the length of data set D, perform the following operations: From the data set D, starting from the current index index, extract W consecutive data points as a data window; For this data window, this embodiment calculates its statistical characteristics, including mean, maximum, minimum and standard deviation, and stores these statistical characteristics in a temporary variable stats; Add stats to the previously initialized empty list; Update the index index to increase the step size S, that is, index = index + S, thereby moving the data window to the next position.

[0033] After traversing the entire data set D, a list containing the statistical features of all data windows is obtained. This list is the data preprocessed by the time sliding window and can be used for subsequent data analysis or model training. Each of the window instances is packaged together to ensure that they are randomly divided into training data and test data at a ratio of 98%, and the original data structure is restored in each window.

[0034] S3. Use a one-dimensional convolutional neural network to process the coarse-grained time features to obtain a high-level feature sequence, and input the high-level feature sequence into a multi-layer bidirectional gated recurrent unit for time series processing to obtain multi-level time features.

[0035] See also Figure 2In this embodiment, a one-dimensional convolutional neural network 1D-CNN and a multi-layer bidirectional gated recurrent unit are successively connected in series to form an intrusion detection model, wherein the one-dimensional convolutional neural network 1D-CNN and the multi-layer bidirectional gated recurrent unit are connected through a batch normalization layer, the one-dimensional convolutional neural network 1D-CNN includes multiple convolution kernels, the multi-layer bidirectional gated recurrent unit includes a multi-layer BiGRU network, a temporal attention layer and a Reshape layer, and the multi-layer BiGRU networks are connected through the temporal attention layer and the Reshape layer.

[0036] One-dimensional convolutional neural network 1D-CNN is used to process long sequence data of input. These long sequence data may contain time series features. In order to avoid losing important features, multiple convolution kernels are used to extract multiple features. Each kernel can extract different features from the input data. Each kernel slides on the input data through convolution operation and calculates a series of feature maps.

[0037] During the training of deep neural networks, the input distribution of each layer will change due to the change of parameters in the previous layers, which makes the training process complicated. Batch normalization layer (BN) can suppress overfitting and accelerate convergence training, allowing the use of higher learning rates and less stringent requirements on initialization parameters.

[0038] The BiGRU layer has better temporal sensitivity and is able to use past and future information to capture patterns that GRU may miss. It consists of a forward GRU and a reverse GRU, so it can use not only past information in forward order, but also future information in reverse order.

[0039] 1D-CNN is able to process long sequences and is insensitive to time series features. RNN structures such as GRU or LSTM are well suited for streaming long sequence signals, but are computationally expensive when processing longer sequences. In contrast, 1D-CNN has lower complexity and requires less computational time when processing the same data. Therefore, the 1D-CNN and BiGRU architectures are integrated, and 1D-CNN with multiple convolutional kernels is used to process long sequence data. 1D-CNN is able to extract multiple features from the input data and convert these features into shorter high-level feature sequences. The high-level feature sequences are then input into the Multi-BiGRU network for further time series feature extraction. Between the two layers of the BiGRU network, a time attention layer is set. This layer calculates attention scores to capture key temporal features, thereby reducing the running time of the model and preventing overfitting.

[0040] At the same time, in order to further improve the performance of the model, this embodiment adds a pooling layer and a dropout layer to the network structure.

[0041] The main function of the pooling layer is to reduce the spatial size of the data, that is, dimensionality reduction, which not only reduces the amount of calculation, but also effectively prevents overfitting. Through the pooling operation, the model can learn the spatial hierarchical structure information in the data, which is especially important for hierarchical data such as images or speech. At the same time, the pooling layer also has translation invariance, that is, for a small translation change of the data, the output after pooling will not change significantly, which further enhances the robustness of the model.

[0042] The dropout layer is another effective technique to prevent overfitting. During the training process, the dropout layer randomly "turns off" some of the neurons in the network (that is, sets their output to 0), which is equivalent to training a different network structure in each iteration. In this way, the dropout layer can force the network to learn more robust feature representations because the model cannot rely on any single neuron or feature set to make predictions. In addition, the dropout layer can also be regarded as a model averaging technique, which improves the generalization ability of the model by training multiple different models (actually subsets of models that share parameters) and averaging their predictions.

[0043] S4. Use the temporal attention mechanism to perform weighted processing on multi-level temporal features to obtain weighted temporal features, feed the multi-level temporal features and weighted temporal features back to the intrusion detection model, and dynamically adjust and update the model parameters.

[0044] This embodiment provides a model that integrates a temporal attention mechanism, which allows the model to assign weights to each temporal feature to reflect the importance of each time point for completing the task. The specific steps are as follows: This mechanism calculates the energy value of each time step, which is used to determine the attention weight. After calculating the energy value, it is converted into a probability through a softmax function to ensure that the sum of the weights of all time steps is 1. This conversion not only provides a soft selection mechanism to decide which time steps are more critical, but also maintains the stability of the gradient, which helps model training.

[0045] Based on their respective attention weights, the feature vectors of all time steps are weighted and summed as follows: ; ; in, It represents energy or score, which is used to measure the importance of the feature vector at a specific time step for the current task; represents the weight matrix, which represents the parameters of the linear transformation. It generates a new feature representation by linearly combining the input feature vector x;i represents the feature vector representing the input, usually some representation at time step t, such as the output of RNN or LSTM, which contains important information about the input data; b represents the bias term, which is used to adjust the output of the linear transformation. It allows the model to introduce some fixed offsets to change the range or direction of the feature representation; represents the attention weight vector, which is obtained by normalizing e in some way (such as softmax function) to ensure that the sum of all weights is 1. These weights reflect the importance of the feature vector of each time step for the current task; j represents a specific time step and is usually used to index the feature vectors of different time steps.

[0046] This resulting vector emphasizes features that are more relevant to the task. The temporal attention mechanism allows the model to dynamically adjust the weight distribution based on the characteristics of the input data. This means that the model can adaptively focus on those time points that are more predictive of the task, rather than treating all time steps equally.

[0047] The model cleverly uses the features of the intermediate hidden layers to calculate the probabilities of various potential interferences. These calculated loss values ​​are then used to fine-tune the model's parameters through the back-propagation algorithm to continuously improve its performance. At the same time, the model's predicted probabilities are compared with the actual labels to quantify the model's losses and guide subsequent optimization work. This step ensures that the model can continue to learn and improve to better adapt to complex and changing data environments.

[0048] S5. Test the intrusion detection model and obtain multi-classification results of the intrusion detection data.

[0049] The intrusion detection classification performance indicators mentioned include accuracy, recall, precision and F1 score.

[0050] The experimental results on the UNSW-NB15 dataset verified the effectiveness of the model in abnormal traffic detection. The multi-classification accuracy of the experiment reached 80.8% and the AUC value reached 0.96.

[0051] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0052] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, characterized in that: The steps include: Preprocess the original network traffic data set to obtain the network traffic data set; Using the time sliding window with a unified step size, the network traffic data set is processed to obtain coarse-grained time features; The coarse-grained time features are processed using a one-dimensional convolutional neural network to obtain a high-level feature sequence, which is then input into a multi-layer bidirectional gated recurrent unit for time series processing to obtain multi-level time features. The time attention mechanism is used to perform weighted processing on multi-level time features to obtain weighted time features, and the multi-level time features and weighted time features are fed back to the intrusion detection model to dynamically adjust and update the model parameters. Test the intrusion detection model and obtain multi-classification results of intrusion detection data.

2. According to claim 1, the intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism is characterized in that: The feature processing of the network traffic data set is specifically as follows: Initialize an empty list, store the network traffic data in the empty list, obtain the data table, loop through the data table, and complete a cycle to move a window with a fixed time step; Calculate the statistical features of the data in each window. The statistical features are used for feature analysis and model training, including the mean, maximum, minimum, and mean square error. The statistical features of each window are packaged, the network traffic data is randomly divided into training data and test data according to the set ratio, and the time series data is kept, and the original data structure is restored in each window.

3. According to claim 1, the intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism is characterized in that: The one-dimensional convolutional neural network is connected to the multi-layer bidirectional gated recurrent unit through a batch normalization layer, and the one-dimensional convolutional neural network includes multiple convolution kernels.

4. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 3 is characterized in that: The method of processing the coarse-grained time features by using a one-dimensional convolutional neural network includes: Multiple convolution kernels are used to extract features from the training set. Each convolution kernel extracts multiple different features. Each convolution kernel slides on the input data through convolution operations to obtain multiple feature maps.

5. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1 is characterized in that: The multi-layer bidirectional gated recurrent unit includes a multi-layer BiGRU network, a temporal attention layer and a Reshape layer, and the multi-layer BiGRU networks are connected through the temporal attention layer and the Reshape layer.

6. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 5 is characterized in that: The high-level feature sequence is input into a multi-layer bidirectional gated recurrent unit for time series processing, specifically: The BiGRU network is used to process the features in the high-level feature sequence to obtain the time series features. The temporal attention layer is used to calculate the attention score to obtain the key temporal features. After feature processing by multiple layers of BiGRU networks and temporal attention layers, multi-level temporal features are obtained.

7. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1 is characterized in that: The intrusion detection model includes a pooling layer and a dropout layer, wherein the pooling layer is arranged between the one-dimensional convolutional neural network and the multi-layer bidirectional gated recurrent unit, and within the multi-layer bidirectional gated recurrent unit; The pooling layer is used to learn the spatial hierarchical structure information in the data through the pooling operation, and the dropout layer is used to randomly select a part of the neurons of the multi-layer bidirectional gated recurrent unit and set the outputs of these neurons to zero.

8. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1 is characterized in that: The method of using the time attention mechanism to perform weighted processing on multi-level time features includes: The energy value of each time step is calculated, and the energy value is converted into probability through the softmax function to obtain the attention weight of the multi-level time feature; the multi-level time feature has multiple time steps, and each time step includes the feature vector of the respective time step; according to the attention weight, the feature vectors of all time steps are weighted and summed to obtain the weighted time feature.

9. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1 is characterized in that: The multi-level time features and weighted time features are fed back to the intrusion detection model, specifically: Calculate various potential interferences using multi-level time features and weighted time features, obtain loss values, adjust the intrusion detection model parameters through back propagation, and obtain the probability values ​​of each round of prediction; The probability value is compared with the actual label to quantify the loss of the evaluation model, and the evaluation result is used to guide the optimization of the intrusion detection model.

10. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that: The preprocessing of the original network traffic data set includes: Random oversampling randomly copies data points from the minority class; Adjust the amount of data; Use one-hot encoding to convert features into numerical features; Normalize and rescale the data to a specific range.

Images