Intrusion Detection Method Based on Multi-Layer Deep Neural Network Model Combined with Temporal Attention Mechanism

Abstract

This invention discloses an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism. The method includes: inputting a dataset into a time-sliding window algorithm to obtain data that retains temporal features; capturing the temporal dynamics and persistent correlations of network behavior within the data; and, through a hybrid architecture combining 1DCNN and Multi-BIGRU, enabling the model to simultaneously extract local and global temporal features from the input data. The introduction of a temporal attention mechanism automatically focuses on key time nodes that have the greatest impact on the classification results. This invention extracts key features from multiple dimensions, achieving accurate identification of complex attack behaviors and improving the efficiency of network intrusion detection.

Description

Technical Field

[0001] This invention belongs to the technical field of information security, specifically relating to an intrusion detection method based on a multi-layer deep neural network model combined with a time attention mechanism. Background Technology

[0002] Traditional intrusion detection methods are mainly based on rule matching and statistical analysis. Although these methods are effective in specific scenarios, their detection effectiveness and real-time performance often fail to meet the needs when facing increasingly complex and ever-changing network attacks.

[0003] The rise of deep learning technology has provided new solutions for intrusion detection. Deep learning models can automatically learn and extract features from massive amounts of data, thereby achieving effective detection of unknown attacks. However, network intrusion behavior has a temporal characteristic, meaning that attacks typically occur continuously over a period of time or exhibit certain patterns. If temporal dependencies are not considered, the model may fail to accurately capture these patterns, leading to decreased detection accuracy. Capturing temporal features is particularly important in real-time intrusion detection scenarios. Ignoring temporal dependencies may prevent the model from responding promptly to rapidly changing network threats. Models that do not consider temporal dependencies may lack sufficient generalization ability when facing different types of network intrusions. This is because different attack patterns may have different temporal characteristics, and ignoring these characteristics may cause the model to be unable to effectively cope with diverse attack methods. In recent years, deep neural network architectures that consider temporal features have emerged, such as the CNN-BILSTM model.

[0004] The CNN-BiLSTM model combines the advantages of CNNs in feature extraction and classification with the strengths of BiLSTMs in handling temporal data and capturing temporal dependencies. However, the CNN-BiLSTM model has high computational complexity, requiring long training times and significant computational resources. Furthermore, it is highly dependent on the data; the model's performance largely depends on the quality and quantity of the input data. Problems such as noise, missing data, or imbalance can negatively impact the model's training and prediction results. This is a common drawback of multi-layered deep neural networks.

[0005] How to effectively improve the performance of intrusion detection systems and how to achieve real-time response to network attacks, so as to provide strong support for network security protection, are urgent technical problems to be solved. Summary of the Invention

[0006] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide an intrusion detection method based on a multi-layer deep neural network model combined with a time attention mechanism. By first processing a one-dimensional convolutional neural network and then a multi-layer bidirectional gated recurrent unit, the attack detection efficiency of network intrusion is improved.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, comprising the following steps:

[0009] Preprocess the raw network traffic dataset to obtain a new network traffic dataset;

[0010] By using a time sliding window with a uniform step size, feature processing is performed on the network traffic dataset to obtain coarse-grained temporal features;

[0011] One-dimensional convolutional neural networks are used to process coarse-grained time features to obtain high-level feature sequences. These high-level feature sequences are then input into multi-layer bidirectional gated recurrent units for time series processing to obtain multi-level time features.

[0012] The multi-level time features are weighted using a time attention mechanism to obtain weighted time features. The multi-level time features and the weighted time features are then fed back into the intrusion detection model to dynamically adjust and update the model parameters.

[0013] Test the intrusion detection model and obtain multi-classification results of the intrusion detection data.

[0014] As a preferred technical solution, the feature processing of the network traffic dataset specifically includes:

[0015] Initialize an empty list, store network traffic data in the empty list, retrieve the data table, loop through the data table, and move the window by a fixed time step after completing one loop.

[0016] Calculate the statistical characteristics of the data within each window. These statistical characteristics are used for feature analysis and model training, including the mean, maximum, minimum, and mean squared error.

[0017] The statistical features of each window are packaged, and the network traffic data is randomly divided into training data and test data according to a set ratio, while maintaining the time series data, and the original data structure is restored within each window.

[0018] As a preferred technical solution, the one-dimensional convolutional neural network and the multi-layer bidirectional gated recurrent unit are connected through a batch normalization layer, and the one-dimensional convolutional neural network includes multiple convolutional kernels.

[0019] As a preferred technical solution, the process of using a one-dimensional convolutional neural network to process coarse-grained temporal features includes:

[0020] Multiple convolutional kernels are used to extract features from the training set. Each convolutional kernel extracts multiple different features. Each convolutional kernel slides on the input data through convolution operations to obtain multiple feature maps.

[0021] As a preferred technical solution, the multi-layer bidirectional gated recurrent unit includes a multi-layer BiGRU network, a temporal attention layer, and a reshape layer, wherein the multi-layer BiGRU networks are connected to each other through the temporal attention layer and the reshape layer.

[0022] As a preferred technical solution, the step of inputting the high-level feature sequence into a multi-layer bidirectional gated cyclic unit for time series processing specifically involves:

[0023] The BiGRU network is used to perform feature processing on the high-level feature sequence to obtain time series features. Attention scores are calculated using a time attention layer to obtain key time features. After feature processing through multiple layers of BiGRU network and time attention layer, multi-level time features are obtained.

[0024] As a preferred technical solution, the intrusion detection model includes a pooling layer and a dropout layer, with the pooling layer positioned between a one-dimensional convolutional neural network and a multi-layer bidirectional gated recurrent unit, as well as within the multi-layer bidirectional gated recurrent unit;

[0025] The pooling layer is used to learn spatial hierarchical structure information in the data through pooling operations, and the dropout layer is used to randomly select a portion of neurons in a multi-layer bidirectional gated recurrent unit and set the output of these neurons to zero.

[0026] As a preferred technical solution, the weighted processing of multi-level temporal features using a time attention mechanism includes:

[0027] The energy value of each time step is calculated, and the energy value is converted into a probability using the softmax function to obtain the attention weights of the multi-level time features. The multi-level time features have multiple time steps, and each time step includes a feature vector of its own time step. Based on the attention weights, the feature vectors of all time steps are weighted and summed to obtain the weighted time features.

[0028] As a preferred technical solution, the step of feeding multi-level temporal features and weighted temporal features back into the intrusion detection model specifically involves:

[0029] Various potential interferences are calculated using multi-level time features and weighted time features to obtain loss values. The loss values ​​are then used to adjust the parameters of the intrusion detection model through backpropagation, and the probability values ​​of each round of prediction are obtained.

[0030] The probability values ​​are compared with the actual labels to quantify the loss of the model, and the evaluation results are used to guide the optimization of the intrusion detection model.

[0031] As a preferred technical solution, the preprocessing of the original network traffic dataset includes:

[0032] Random oversampling randomly copies data points from the minority class;

[0033] Adjust the amount of data;

[0034] One-hot encoding is used to convert features into digital features;

[0035] Normalize the data and rescale it to a specific range.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] (1) This invention utilizes a one-dimensional convolutional neural network (1D-CNN) and a multi-layer deep neural network (Multi-BiGRU). 1D-CNN is responsible for extracting local features, while Multi-BiGRU is responsible for capturing deeper temporal dependencies. This combination simulates the real process and helps to analyze and understand attack behavior more comprehensively. Simulating real attack behavior, it extracts features from coarse to fine granular levels and from multiple dimensions, enabling accurate identification of complex attack behavior and improving the robustness of the model.

[0038] (2) This invention introduces a time attention mechanism, which gives the model the ability to flexibly process time series data. It can automatically assign weights according to the importance of the data, so that the model can focus on the time points that have an important impact on the detection results, thereby improving the efficiency and accuracy of detection.

[0039] (3) This invention utilizes a time-sliding window algorithm, which can preserve the temporal dependencies between attack traffic data and efficiently track the temporal dependencies in attack traffic data, ensuring sensitive capture of dynamic changes in traffic data. This not only improves the speed of data processing but also enhances the real-time performance and accuracy of attack detection. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, according to an embodiment of the present invention.

[0042] Figure 2 This is a structural diagram of a multi-layer deep neural network model incorporating a time attention mechanism, as shown in an embodiment of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0044] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0045] One-dimensional convolutional neural networks (1D-CNNs) are a variant of convolutional neural networks (CNNs) primarily used to process one-dimensional sequential data, such as audio, text, and time series data. 1D-CNNs extract local features from sequential data through convolutional layers; these features reflect local patterns in the input data. Compared to traditional fully connected neural networks, 1D-CNNs can more effectively utilize local relationships in sequential data, thus performing superiorly in tasks such as speech recognition, natural language processing, and time series prediction.

[0046] Please see Figure 1 This embodiment provides an intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, including the following steps:

[0047] S1. Preprocess the original network traffic dataset to obtain the network traffic dataset.

[0048] This embodiment provides the following steps for preprocessing raw network traffic datasets:

[0049] S101. Oversampling technique: Random oversampling randomly copies data points from the minority class, which reduces data imbalance and improves the prediction accuracy of the minority class.

[0050] S102. Adjusting the Number of Records: The time-sliding window algorithm is particularly suitable for serial data with varying flow durations, effectively capturing the dynamic properties of data over time. This method helps extract meaningful patterns from continuous flow data. First, an empty list is initialized to store the processed data. Then, in this embodiment, a loop iterates through the dataset, moving a fixed step at a time, and calculates the required statistical features, such as mean, maximum, minimum, and mean squared error, within each window. These statistical features are considered representative of each window and can be used for subsequent analysis or machine learning model training.

[0051] S103. One-hot coding: Use one-hot coding to convert features into digital features.

[0052] S10. Normalization: Normalization rescales the data to a specific range, reducing redundancy and shortening model training time. This paper uses Min-Max normalization to adjust the data range to [0,1], where the formula is:

[0053] .

[0054] S2. Using a time sliding window with a uniform step size, feature processing is performed on the network traffic dataset to obtain coarse-grained time features.

[0055] Suppose we have a dataset D containing timestamps and multiple features. We also set the window size W to 60 (e.g., representing a time window of 60 data points) and the step size S to 10 (meaning the window moves 10 data points at a time).

[0056] Initialize an empty list to store the statistical characteristics of the processed data window. Iterate through the dataset, starting from index=0, and proceed through dataset D. As long as the sum of the index and the window size W does not exceed the length of dataset D, perform the following operations:

[0057] From dataset D, starting from the current index, extract W consecutive data points as a data window;

[0058] For this data window, this embodiment calculates its statistical characteristics, including mean, maximum, minimum and standard deviation, and stores these statistical characteristics in a temporary variable stats;

[0059] Add stats to the previously initialized empty list;

[0060] Update the index by increasing the step size S, i.e., index = index + S, thereby moving the data window to the next position.

[0061] After traversing the entire dataset D, a list containing the statistical features of all data windows is obtained. This list represents the data after time-sliding window preprocessing and can be used for subsequent data analysis or model training. Each window instance is then packaged together, ensuring they are randomly divided into training and testing data at a 98% ratio, thus restoring the original data structure within each window.

[0062] S3. Use a one-dimensional convolutional neural network to process coarse-grained time features to obtain high-level feature sequences. Input the high-level feature sequences into a multi-layer bidirectional gated recurrent unit for time series processing to obtain multi-level time features.

[0063] Please see Figure 2 In this embodiment, a one-dimensional convolutional neural network (1D-CNN) and a multi-layer bidirectional gated recurrent unit are sequentially connected to form an intrusion detection model. The one-dimensional convolutional neural network (1D-CNN) and the multi-layer bidirectional gated recurrent unit are connected through a batch normalization layer. The one-dimensional convolutional neural network (1D-CNN) includes multiple convolutional kernels, and the multi-layer bidirectional gated recurrent unit includes a multi-layer BiGRU network, a temporal attention layer, and a reshape layer. The multi-layer BiGRU networks are connected to each other through the temporal attention layer and the reshape layer.

[0064] One-dimensional convolutional neural networks (1D-CNNs) are used to process long input sequences of data. These long sequences may contain time-series features, and to avoid losing important features, multiple convolutional kernels are used to extract multiple features. Each kernel can extract different features from the input data. Each kernel slides across the input data through convolution operations and computes a series of feature maps.

[0065] During the training of deep neural networks, the input distribution of each layer changes due to variations in the parameters of the first few layers, which complicates the training process. Batch normalization (BN) layers can suppress overfitting and accelerate convergence training, allowing for higher learning rates and less stringent requirements on initialization parameters.

[0066] The BiGRU layer has better temporal sensitivity, enabling it to capture patterns that the GRU might miss by utilizing both past and future information. It consists of a forward GRU and a backward GRU, so it can use past information not only in forward order but also future information in reverse order.

[0067] 1D-CNNs can handle long sequences but are insensitive to time-series features. RNN structures such as GRU or LSTM are well-suited for handling long-sequence signals, but are computationally expensive when processing longer sequences. In contrast, 1D-CNNs have lower complexity and require less computation time to process the same data. Therefore, an 1D-CNN and BiGRU architecture is integrated, using multiple convolutional kernels in a 1D-CNN to process long-sequence data. 1D-CNNs can extract multiple features from the input data and transform these features into shorter, high-level feature sequences. These high-level feature sequences are then fed into a Multi-BiGRU network for further time-series feature extraction. A temporal attention layer is placed between two BiGRU layers. This layer computes attention scores to capture key temporal features, thereby reducing model runtime and preventing overfitting.

[0068] In addition, to further improve the performance of the model, this embodiment adds a pooling layer and a dropout layer to the network structure.

[0069] The primary function of pooling layers is to reduce the spatial size of data, i.e., dimensionality reduction. This not only reduces computational cost but also effectively prevents overfitting. Through pooling operations, the model can learn the spatial hierarchical structure information within the data, which is particularly important for hierarchical data such as images or speech. Furthermore, pooling layers exhibit translation invariance, meaning that small translational changes in the data do not significantly alter the output after pooling, further enhancing the model's robustness.

[0070] Dropout layers are another effective technique to prevent overfitting. During training, dropout layers randomly "turn off" a subset of neurons in the network (i.e., set their output to 0), which is equivalent to training a different network structure in each iteration. In this way, dropout layers force the network to learn more robust feature representations because the model cannot rely on any single neuron or feature set to make predictions. Furthermore, dropout layers can also be viewed as a model averaging technique, improving the model's generalization ability by training multiple different models (actually subsets of models sharing parameters) and averaging their predictions.

[0071] S4. Use the time attention mechanism to weight the multi-level time features to obtain weighted time features. Feed the multi-level time features and weighted time features into the intrusion detection model to dynamically adjust and update the model parameters.

[0072] This embodiment provides a model integrating a time attention mechanism, which allows the model to assign weights to each time feature to reflect the importance of each time point in completing the task. The specific steps are as follows:

[0073] This mechanism calculates an energy value for each time step, which is used to determine the attention weights. After calculating the energy values, they are converted into probabilities using a softmax function to ensure that the sum of the weights across all time steps is 1. This transformation not only provides a soft selection mechanism to determine which time steps are more critical but also maintains gradient stability, which is beneficial for model training.

[0074] Based on their respective attention weights, the feature vectors of all time steps are weighted and summed as follows:

[0075] ;

[0076] ;

[0077] in, This represents energy or a score, used to measure the importance of a feature vector at a specific time step to the current task. The weight matrix represents the parameters of the linear transformation, which generates a new feature representation by linearly combining the input feature vector x; x i 'b' represents the feature vector that represents the input, usually a representation at time step t, such as the output of an RNN or LSTM. These feature vectors contain important information about the input data; 'b' represents the bias term, which is used to adjust the output of the linear transformation. It allows the model to introduce some fixed offset to change the range or direction of the feature representation. The attention weight vector is obtained by normalizing e in some way (such as the softmax function) to ensure that the sum of all weights is 1. These weights reflect the importance of the feature vector at each time step to the current task; j represents a specific time step and is usually used to index the feature vector at different time steps.

[0078] This result vector emphasizes features that are more relevant to the task. The temporal attention mechanism allows the model to dynamically adjust the weight distribution based on the features of the input data. This means that the model can adaptively focus on those time steps that are more predictive of the task, rather than treating all time steps equally.

[0079] This model cleverly utilizes features from the intermediate hidden layers to calculate the probabilities of various potential disturbances. These calculated loss values ​​are then used to fine-tune the model's parameters through backpropagation, continuously improving its performance. Simultaneously, the probabilities predicted by the model are compared with the actual labels to quantitatively evaluate the model's loss, thereby guiding subsequent optimization efforts. This step ensures that the model can continuously learn and improve, better adapting to complex and ever-changing data environments.

[0080] S5. Test the intrusion detection model and obtain multi-classification results of the intrusion detection data.

[0081] The intrusion detection classification performance metrics mentioned above include accuracy, recall, precision, and F1 score.

[0082] Experimental results on the UNSW-NB15 dataset validated the effectiveness of the model in detecting abnormal traffic, with a multi-class classification accuracy of 80.8% and an AUC value of 0.96.

[0083] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0084] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism, characterized in that, Includes the following steps: Preprocess the raw network traffic dataset to obtain a new network traffic dataset; By using a time sliding window with a uniform step size, feature processing is performed on the network traffic dataset to obtain coarse-grained temporal features; A one-dimensional convolutional neural network is used to process coarse-grained temporal features to obtain high-level feature sequences. These high-level feature sequences are then input into a multi-layer bidirectional gated recurrent unit (BRU) for time series processing to obtain multi-level temporal features. The one-dimensional convolutional neural network and the BRU are connected through a batch normalization layer. The one-dimensional convolutional neural network includes multiple convolutional kernels. The BRU includes a multi-layer BiGRU network, a temporal attention layer, and a reshape layer. The multi-layer BiGRU networks are connected through the temporal attention layer and the reshape layer. The multi-level time features are weighted using a time attention mechanism to obtain weighted time features. The multi-level time features and the weighted time features are then fed back into the intrusion detection model to dynamically adjust and update the model parameters. Test the intrusion detection model and obtain multi-classification results of the intrusion detection data.

2. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The feature processing of the network traffic dataset specifically includes: Initialize an empty list, store network traffic data in the empty list, retrieve the data table, loop through the data table, and move the window by a fixed time step after completing one loop. Calculate the statistical characteristics of the data within each window. These statistical characteristics are used for feature analysis and model training, including the mean, maximum, minimum, and mean squared error. The statistical features of each window are packaged, and the network traffic data is randomly divided into training data and test data according to a set ratio. The time series data is preserved, and the statistical features within each window are restored to the original data structure.

3. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The process of using a one-dimensional convolutional neural network to process coarse-grained temporal features includes: Multiple convolutional kernels are used to extract features from the training set. Each convolutional kernel extracts multiple different features. Each convolutional kernel slides on the input data through convolution operations to obtain multiple feature maps.

4. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The step of inputting the high-level feature sequence into a multi-layer bidirectional gated cyclic unit for time series processing specifically involves: The BiGRU network is used to process high-level feature sequences to obtain time series features. Attention scores are calculated using a time attention layer to obtain key time features. After feature processing through multiple layers of BiGRU network and time attention layer, multi-level time features are obtained.

5. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The intrusion detection model includes a pooling layer and a dropout layer. The pooling layer is placed between a one-dimensional convolutional neural network and a multi-layer bidirectional gated recurrent unit, as well as within the multi-layer bidirectional gated recurrent unit. The pooling layer is used to learn spatial hierarchical structure information in the data through pooling operations, and the dropout layer is used to randomly select a portion of neurons in a multi-layer bidirectional gated recurrent unit and set the output of these neurons to zero.

6. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The weighted processing of multi-level temporal features using a time attention mechanism includes: The energy value of each time step is calculated, and the energy value is converted into a probability using the softmax function to obtain the attention weights of the multi-level time features. The multi-level time features have multiple time steps, and each time step includes a feature vector of its own time step. Based on the attention weights, the feature vectors of all time steps are weighted and summed to obtain the weighted time features.

7. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The feeding of multi-level temporal features and weighted temporal features into the intrusion detection model specifically involves: Various potential interferences are calculated using multi-level time features and weighted time features to obtain loss values. The loss values ​​are then used to adjust the parameters of the intrusion detection model through backpropagation, and the probability values ​​of each round of prediction are obtained. The probability values ​​are compared with the actual labels to quantify the loss of the model, and the evaluation results are used to guide the optimization of the intrusion detection model.

8. The intrusion detection method based on a multi-layer deep neural network model combined with a temporal attention mechanism according to claim 1, characterized in that, The preprocessing of the raw network traffic dataset includes: Random oversampling randomly copies data points from the minority class; Adjust the amount of data; One-hot encoding is used to convert features into digital features; Normalize the data and rescale it to a specific range.

Images