DNN and BiGRU-based hierarchical attention network intrusion detection method and device

Abstract

The invention discloses a DNN and BiGRU-based hierarchical attention network intrusion detection method and device, and relates to the field of network security. The method comprises the steps of performing spatial feature extraction on traffic data through two first models connected in series, and performing time sequence feature extraction on the traffic data through two second models connected in series; wherein the first model comprises a deep neural network embedded with self-attention, and the second model comprises a bidirectional gating circulation unit embedded with self-attention; fusing the spatial features and the time sequence features through splicing operation to obtain fused features; and inputting the fusion feature into a multi-layer perceptron to obtain a network intrusion detection classification result of the flow data. The method shows obvious advantages in the aspects of feature extraction, anomaly recognition, data imbalance coping and the like, and shows wide applicability and huge potential in actual network security application.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a layered attention network intrusion detection method and device based on DNN and BiGRU. Background Art

[0002] In today's rapidly evolving information landscape, the ease of information exchange has brought numerous opportunities to the world, but it has also posed significant challenges to information security. The frequent occurrence of security incidents such as data leaks and system sabotage, even escalating to national security threats, has made network security a global concern. As a key defensive measure for ensuring network security, network intrusion detection technology (IDS) has garnered significant attention in both theoretical research and practical applications. Traditional machine learning methods, such as K-nearest neighbor, support vector machines, naive Bayesian, and decision trees, have played a significant role in early intrusion detection systems. However, their performance is insufficient in today's massive, high-dimensional, and complex network data, making it difficult to effectively identify increasingly complex attack behaviors. In recent years, deep learning, with its efficient extraction of deep data features and nonlinear modeling capabilities, has demonstrated significant advantages in the field of intrusion detection, becoming a new approach to improving detection accuracy and reducing false alarm rates.

[0003] Although intrusion detection systems based on deep neural networks (DNNs) show great potential for performance improvement, with the increasing complexity of network environments and the continuous evolution of attack methods, they still face numerous challenges in adapting to new threats, processing massive amounts of heterogeneous data, and implementing them under resource-constrained conditions. For example, DNN models are often criticized for their "black box" architecture and lack of interpretability, which limits their widespread application, particularly in the protection of critical infrastructure. Furthermore, their lack of robustness against adversarial attacks has become a research hotspot. Therefore, building efficient and scalable detection systems while ensuring model interpretability, improving robustness against adversarial attacks, and optimizing resource efficiency remains a key direction for future development. Summary of the Invention

[0004] Based on this, it is necessary to provide a layered attention network intrusion detection method and device based on DNN and BiGRU that can effectively improve classification accuracy, stability and generalization ability to address the above technical problems.

[0005] In the first aspect, the present application provides a layered attention network intrusion detection method based on DNN and BiGRU. The method includes:

[0006] The spatial features of the traffic data are extracted using two first models connected in series, and the temporal features of the traffic data are extracted using two second models connected in series; wherein the first model includes a deep neural network embedded with self-attention, and the second model includes a bidirectional gated recurrent unit embedded with self-attention;

[0007] The spatial features and temporal features are fused through splicing operations to obtain fused features;

[0008] The fused features are input into the multi-layer perceptron to obtain the network intrusion detection classification results of the traffic data.

[0009] In one embodiment, each first model includes a fully connected layer, a BN layer, a Dropout layer, and a self-attention mechanism layer; each second model includes a bidirectional GRU layer, a BN layer, a Dropout layer, and a self-attention mechanism layer.

[0010] In one embodiment, after obtaining the fusion features, the method further includes:

[0011] The Flatten layer is used to process the fused features and convert the fused vector into a one-dimensional vector format to eliminate the time step dimension in the fused features.

[0012] In one embodiment, the deep neural network uses 32 neurons, adopts the ReLU activation function, performs BatchNormalization for normalization, and the Dropout rate is set to 0.5.

[0013] In one embodiment, the bidirectional gated recurrent unit uses 32 neurons and sets a return sequence.

[0014] In one embodiment, the multilayer perceptron uses 60 neurons, and employs ReLU activation function, BatchNormalization, and Dropout to optimize feature representation.

[0015] In a second aspect, the present application also provides a layered attention network intrusion detection device based on DNN and BiGRU. The device includes:

[0016] A feature extraction module, configured to extract spatial features from traffic data using two first models connected in series, and to extract temporal features from traffic data using two second models connected in series; wherein the first model includes a deep neural network embedded with self-attention, and the second model includes a bidirectional gated recurrent unit embedded with self-attention;

[0017] The feature fusion module is used to fuse spatial features and temporal features through splicing operations to obtain fused features;

[0018] The classification module is used to input the fusion features into the multi-layer perceptron to obtain the network intrusion detection classification results of the traffic data.

[0019] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps in the above-mentioned layered attention network intrusion detection method based on DNN and BiGRU.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-mentioned layered attention network intrusion detection method based on DNN and BiGRU.

[0021] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the above-mentioned layered attention network intrusion detection method based on DNN and BiGRU.

[0022] The above-mentioned layered attention network intrusion detection method and device based on DNN and BiGRU uses two serially connected first models to extract spatial features from traffic data and two serially connected second models to extract temporal features from traffic data. The first model comprises a deep neural network embedded with self-attention, and the second model comprises a bidirectional gated recurrent unit embedded with self-attention. The spatial and temporal features are fused through a concatenation operation to obtain fused features. The fused features are input into a multi-layer perceptron to obtain network intrusion detection classification results for the traffic data. This invention proposes a network intrusion detection model that integrates DNN, BiGRU, and a self-attention mechanism to optimize feature extraction and classification performance from multiple dimensions. By introducing a bidirectional gated recurrent unit and a self-attention mechanism, the model's ability to capture temporal dependencies and key features in network traffic is effectively enhanced. The bidirectional gated recurrent unit can model complex pattern relationships in bidirectional time series, while the self-attention mechanism dynamically assigns attention weights after each bidirectional gated recurrent unit layer, improving discrimination and feature representation. A self-attention mechanism is systematically embedded within each layer of the deep neural network and bidirectional gated recurrent unit (BGRU), enabling the model to automatically focus on key attributes in a multidimensional feature space, overcoming the limitations of traditional methods that rely on manually selected features. By fusing static features extracted by the deep neural network with dynamic features extracted by the BGRU, multi-level complementary learning of spatial and temporal features is achieved, significantly improving the ability to recognize complex intrusion patterns. Finally, the model incorporates a multi-layer perceptron (MLP) as a classifier, effectively improving classification accuracy, stability, and generalization, demonstrating its strong potential in network intrusion detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 1 is a model framework diagram of a DNN in one embodiment;

[0024] Figure 2 1 is an architecture diagram of a BiGRU in one embodiment;

[0025] Figure 3 Schematic diagram of the calculation process of the self-attention mechanism layer in one embodiment;

[0026] Figure 4 FIG1 is a structural diagram of a layered attention network intrusion detection model based on DNN and BiGRU in one embodiment;

[0027] Figure 5 This is a diagram of an overall detection framework in one embodiment;

[0028] Figure 6 for Figure 4 Performance graph of the model's training and validation accuracy on the NSL-KDD dataset;

[0029] Figure 7 for Figure 4 Training and validation loss performance graphs of the model on the NSL-KDD dataset;

[0030] Figure 8 for Figure 4 Performance graph of the model's training and validation accuracy on the UNSW-NB15 dataset;

[0031] Figure 9 for Figure 4 Plot of the model's training and validation loss performance on the UNSW-NB15 dataset. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0033] The present invention provides a layered attention network intrusion detection method based on DNN and BiGRU, including the following steps:

[0034] In step 102, spatial features of the traffic data are extracted using two first models connected in series, and temporal features of the traffic data are extracted using two second models connected in series; wherein the first model includes a deep neural network embedded with self-attention, and the second model includes a bidirectional gated recurrent unit embedded with self-attention.

[0035] Among them, the basic unit of deep neural network (DNN) is the neuron. Each neuron receives input from other neurons and changes the influence of the input on the neuron by adjusting the weight. Figure 1As shown in Figure 1, in a DNN, data starts at the input layer, passes through the hidden layers, and finally reaches the output layer. The output of each layer of neurons serves as the input of the neurons in the next layer. Neurons in each layer are connected through activation functions, thus achieving complex nonlinear mapping capabilities, which is used to improve the accuracy of intrusion behavior recognition.

[0036] Specifically, the input layer receives an input vector x=(x1,x2,...,x m ), where each x i Represents an original feature. The input vector x is passed to the first hidden layer of the DNN and is weighted by the inter-layer weight matrix W1 and the bias vector b1. The calculation formula is as follows:

[0037]

[0038] Subsequently, an activation function such as ReLU or Sigmoid is used to perform nonlinear processing on the weighted sum to obtain the output of the first hidden layer, as shown in formula (2)

[0039]

[0040] For the jth hidden layer (j∈{2,...,n}), the output of each neuron is formed by weighted summing the output of the previous layer and the weight of the current layer, plus the bias term, to form a weighted input:

[0041]

[0042] Among them, n j-1 represents the number of neurons in the j-1th layer, represents the connection weight from the i-th neuron in the j-1th layer to the k-th neuron in the j-th layer, is the corresponding bias.

[0043] The output layer is located after the last hidden layer. The number of neurons in the output layer depends on the specific task requirements. This layer receives the output signals from the nth hidden layer, performs a weighted summation, and applies an activation function to generate the model's prediction results.

[0044]

[0045] Among them, w (o) and b (o) They represent the weight matrix and bias term of the output layer respectively, and f is the activation function of the output layer.

[0046] During the training process, in order to measure the gap between the model output and the true label, the mean square error (MSE) is often used as the loss function:

[0047]

[0048] Where T is the number of training samples, p is the output dimension, and Represent the true value and predicted value of the t-th sample respectively.

[0049] To improve model performance and reduce the loss function value, the network optimizes and adjusts the weight parameters through the backpropagation algorithm. Taking the weight update connecting the j-1 layer and the j layer as an example, the gradient calculation is as follows:

[0050]

[0051] Among them, δ is the derivative of the loss function with respect to the weighted input of the current layer (error term), which is multiplied by the output of the previous layer to form the gradient propagation under the chain rule.

[0052] The final weight update formula is:

[0053]

[0054] Here, α represents the learning rate, which controls the step size of weight updates. During the continuous iterations of forward and backward propagation, the DNN model gradually adjusts network weights, thereby improving the model's ability to discriminate between input samples. After multiple rounds of training, the model is able to effectively classify and recognize data, demonstrating promising application potential and performance in scenarios such as network intrusion detection.

[0055] The Bidirectional Gated Recurrent Unit (BiGRU) is a bidirectional structure developed from the Gated Recurrent Unit (GRU). It consists of two GRU layers: forward and backward. Each GRU unit has an update gate and a reset gate. These gating mechanisms are implemented by neurons and their connections. Neurons in the two directional GRU layers perform calculations independently, and their outputs at each time step are concatenated or otherwise combined. This allows the model to capture both past and future information of the sequence, thereby improving its ability to process sequential data.

[0056] like Figure 2 As shown in Figure 1, BiGRU processes the input sequence in both the forward and reverse directions, extracting the current context features from the past and future information, and fusing the forward and reverse hidden states to provide a richer and more comprehensive semantic representation for each time step. Its current hidden state is determined by the current input x t , the forward hidden state h of the previous time step t-1 and the reverse hidden state h at the next time step t+1Together, this structure enables the model to better capture temporal dependencies and contextual information in sequences.

[0057] The forward derivation equation is as follows:

[0058]

[0059]

[0060] The reverse derivation equation is as follows:

[0061]

[0062] The results from the forward and reverse directions are linearly fused as follows:

[0063]

[0064] in:

[0065] and is the reset gate vector that determines the influence of the previous hidden state on the current candidate hidden state.

[0066] and is the update gate vector that controls how the previous hidden state is combined with the current candidate hidden state to update the hidden state.

[0067] is a candidate hidden state, computed based on the current input, the previous hidden state, and the value of the reset gate.

[0068] and are the forward and backward hidden states at the current time step, determined by the update gate and the candidate hidden state.

[0069] is the concatenation of the forward and backward hidden states. This concatenated vector contains information from both directions and can be used as input to the output layer to generate the final prediction.

[0070] σ is the sigmoid activation function.

[0071] ⊙ represents element-wise multiplication.

[0072] By integrating information from two temporal directions, BiGRU enriches the semantic representation of each time step and enhances the model's ability to understand complex sequence dynamics. This makes it particularly suitable for applications such as network intrusion detection, where both historical and future patterns may provide critical context clues.

[0073] The self-attention mechanism effectively captures the complex internal dependencies in a sequence by calculating the similarities between elements in the sequence. At the same time, its structure supports highly parallel operations, significantly improving the modeling capabilities of long text and long sequence data, and providing important support for building high-performance deep learning models. The calculation process is as follows: Figure 3 shown.

[0074] The basic idea of ​​the self-attention mechanism is to calculate the correlation between elements in the sequence and dynamically adjust the information contribution of different positions to obtain a more expressive representation. The calculation process of the self-attention mechanism adopts the QKV (Query-Key-Value) structure. The specific principles are as follows:

[0075] Assume that the input sequence is Where X represents the sequence length, D x Represents the input feature dimension. The input sequence is first linearly transformed into three different representations: query, key, and value, namely:

[0076]

[0077] Among them, W Q , W K , W V are respectively trainable weight matrices used to project input features. i ∈Q, whose corresponding attention weight can be obtained by comparing it with all key vectors k j ∈K similarity is calculated, that is, the scaled dot product attention is used:

[0078]

[0079] Among them, Q T Represents the dot product similarity between the query and the key, which is used to measure their degree of association; scaling factor This is used to mitigate gradient instability issues that can arise from excessively large dot products. The softmax function normalizes the similarities across all positions to generate attention weights. The final output, H, is the weighted sum of these weights applied to the value, representing the updated representation of each position in the sequence. This mechanism allows the model to flexibly emphasize relevant contextual information at each time step, enabling effective long-range dependency modeling in various sequence learning tasks, such as machine translation, language modeling, and intrusion detection.

[0080] In this embodiment, a deep neural network is combined with a self-attention mechanism to form a DA (DNN + Attention) structure, the first model, which is used to extract spatial features. A bidirectional gated recurrent unit is combined with the self-attention mechanism to form a BA (BiGRU + Attention) structure, the second model, which is used to extract temporal features. These first and second models fully leverage the DNN's ability to extract static high-dimensional features and the BiGRU's advantage in modeling bidirectional temporal dependencies. Combined with the self-attention mechanism's ability to focus on key dimensions and key time points, they effectively improve the accuracy of identifying abnormal behavior.

[0081] On this basis, a multi-path feature extraction structure consisting of "dual DA" and "dual BA" was designed. This structure combines static feature modeling with dynamic time series modeling capabilities, and exhibits significant advantages in multiple dimensions compared to the traditional single-path DNN+BiLSTM tandem structure.

[0082] First, the multi-path parallel mechanism significantly enhances the model's feature expression capabilities. The dual DNN branch focuses on mining static global features in the input data, suitable for capturing non-temporal patterns such as traffic statistics. The dual BiGRU branch, through a bidirectional recurrent structure, models contextual dependencies in sequence features from both the forward and backward dimensions, suitable for revealing behavioral evolution patterns during attacks. The parallel combination of the four submodules constructs a multi-angle, complementary feature learning path, improving the model's overall perception of complex attack samples.

[0083] Secondly, the introduction of the self-attention mechanism further strengthens the model's ability to focus on key features. In the DNN branch, the Attention module dynamically adjusts the importance of different neuron outputs, improving global semantic understanding. In the BiGRU branch, the self-attention mechanism effectively extracts the time-step information most critical for classification tasks, enhancing the model's ability to model long-range dependencies. The combined use of spatial and temporal attention enables the model to demonstrate stronger intelligent control capabilities in information screening and feature focusing.

[0084] Finally, the multi-branch structure offers enhanced robustness and generalization capabilities. The independent learning of different attack signatures by different submodules creates a redundant and complementary representation system. This allows the model to consistently perform high-quality intrusion identification even in the face of inputs with abnormal data, attack variations, or strong background noise.

[0085] In contrast, the traditional single DNN and single BiLSTM series structures often find it difficult to fully cover complex intrusion features due to their single path and limited information fusion dimension, which can easily lead to information loss or unstable classification performance.

[0086] Therefore, the dual DA + dual BA multi-path fusion structure proposed in this embodiment not only effectively improves the multi-dimensional feature expression capability of the model, but also enhances the key feature focusing capability and the overall robustness of the model. It shows better detection performance when processing high-dimensional, unbalanced, and complex intrusion data, and provides a solid foundation for building an efficient and stable network intrusion detection system.

[0087] In the multi-path network intrusion detection model, dual-DA and dual-BA appear structurally symmetrical, but in reality, they possess both independence and differentiation. While the two DNN branches are identical in terms of network hierarchy and input data, each branch's network layers are independently instantiated, and their weights are not shared during training. This allows each to learn static features from different perspectives, resulting in complementary representations. Similarly, the two BiGRU branches share the same structural configuration and input sources, but their weights are initialized and updated independently, enabling the model to extract sequential behavioral features from different time-dependent paths. This "structurally consistent, parameter-independent" parallel modeling approach allows the dual-DA and dual-BA substructures to achieve redundant feature extraction while enhancing representational diversity, significantly improving the model's stability, generalization, and robustness in the face of high-dimensional, complex attack data. This mechanism is a crucial foundation for the model's superior performance.

[0088] Step 104: The spatial features and the temporal features are fused through a splicing operation to obtain fused features.

[0089] The features extracted by the first module and the second module are fused through a splicing operation to form a comprehensive feature representation and obtain fused features.

[0090] In one embodiment, the fused features are input into the Flatten layer to convert the multi-dimensional fused features into one-dimensional vector features to eliminate the time step dimension and facilitate subsequent processing.

[0091] Step 106: Input the fusion features into a multi-layer perceptron to obtain a network intrusion detection classification result of the traffic data.

[0092] The fused features are input into a multi-layer perceptron (MLP). The MLP includes a fully connected layer with 60 neurons. After ReLU activation and Dropout operations, the classification probability is finally output through the Sigmoid activation function.

[0093] In one embodiment, Figure 4Figure 1 shows the architecture of a layered attention network intrusion detection model based on a dense neural network (DNN) and a BiGRU (BiGRU-Attention-MLP), namely, a DNN-BiGRU-Attention-MLP. This model consists of two serially connected direct-attachment (DA) structures and two serially connected batch-attention (BA) structures. Each DA structure comprises a fully connected layer, a batch normalization (BN) layer, a dropout layer, and a self-attention mechanism layer to extract spatial features. Each BA structure comprises a bidirectional GRU layer, a batch normalization (BN) layer, a dropout layer, and a self-attention mechanism layer to capture temporal features. In both the DA and BA structures, features undergo batch normalization and dropout before being fed into the self-attention mechanism, which weights the features and highlights the importance of key features. Both the DNN in the DA model and the BiGRU in the BA model use 32 neurons. The DNN module uses the ReLU activation function, batch normalization, and a dropout rate of 0.5. The BiGRU module uses a bidirectional GRU with 32 units and a return sequence. Furthermore, a self-attention mechanism is introduced to enhance feature attention. Finally, a multi-layer perceptron (MLP) module was used for feature fusion and nonlinear mapping. The MLP consisted of 60 neurons, and the feature representation was further optimized using the ReLU activation function, batch normalization, and dropout. The entire model was trained using the Adam optimizer for 100 epochs with a batch size of 32 and binary cross entropy as the loss function. Finally, metrics such as accuracy were used as evaluation criteria.

[0094] The hierarchical attention network intrusion detection model based on DNN and BiGRU effectively integrates spatial information, time series features and attention mechanism through a multi-level combination of DA and BA modules, thereby enhancing the ability to focus on important features and significantly improving the classification effect of network intrusion detection.

[0095] The layered attention network intrusion detection model based on DNN and BiGRU proposed in the above embodiment was experimentally tested. The experiment was run on Windows 10 operating system, the code was based on Python 3.9.19, TensorFlow and PyTorch were used as deep learning frameworks, the CPU was Intel(R) Core(TM) i5-8500@3.00GHZ, the memory was 8GB, and the graphics card was NVIDIA GeForce GT 730. The detection method is based on the public datasets NSL-KDD and UNSW-NB15. The overall detection process includes three processes: data preprocessing, model training, and prediction and classification. The overall detection framework is as follows: Figure 5 shown.

[0096] Among them, the KDD CUP 99 dataset was originally generated by simulating the U.S. Air Force local area network environment and combining network connection and system audit data. It has played an important role in promoting network intrusion detection (NIDS) research. However, this dataset contains a large number of redundant records (approximately 78% in the training set and approximately 75% in the test set), which can easily lead to model learning bias. To overcome this problem, researchers proposed an improved version of the NSL-KDD dataset, which removes duplicate samples, reconstructs training and test data, alleviates data maldistribution and missing issues, and significantly improves data quality. It has now become a commonly used benchmark dataset in intrusion detection research. The NSL-KDD dataset includes four subsets (KDDTrain+, KDDTest+, KDDTrain+_20Percent, and KDDTest-21), containing a total of 43 features, of which 41 are numerical features and 2 are label items. The data covers normal traffic and four types of attack behaviors: Denial of Service (DoS), Probe, User-to-Root (U2R), and Remote-to-Local (R2L). The training set, KDDTrain+, contains 125,973 records covering 22 attack types, while the test set, KDDTest+, contains 22,544 records covering 39 attack types. Features include three categorical variables: ICMP, UDP, and TCP, representing different network protocols and used to describe network connection behavior. The remaining numerical features comprehensively reflect network traffic characteristics. This dataset provides a reliable foundation for evaluating the effectiveness of intrusion detection models.

[0097] The UNSW-NB15 dataset, designed and publicly released by the University of New South Wales in Australia, is generated based on actual network traffic. It exhibits high fidelity and complexity and is widely used in network intrusion detection research. This experiment primarily analyzes the training and test sets of this dataset. The dataset covers nine attack types and one normal traffic type, encompassing approximately 80,000 network connection records: 45,743 in the training set, 22,361 in the test set, and 11,089 in the validation set. Each record contains 49 features, covering key attributes such as source / destination IP addresses, port numbers, protocol types, and connection status.

[0098] To obtain high-quality datasets, public datasets need to be preprocessed. Data preprocessing is a key step in deep learning model training. Before model building, the original dataset is systematically preprocessed. This mainly includes data cleaning, feature digitization and encoding, feature selection, and data normalization. The specific steps are as follows:

[0099] Data cleaning: Missing values ​​or incorrect character types may exist in the dataset, which can prevent the model from training properly. First, clean the data. By examining all features in the dataset column by column, we identify and address missing values ​​or unusual characters. For example, in the service column, we replace invalid characters (such as "-") with missing values ​​(NaN). We then delete all rows containing missing values ​​to ensure data integrity and consistency.

[0100] Feature digitization and encoding: Because deep learning models cannot directly process text-based features, non-numeric features must be digitized. In the dataset, features such as protocol_type, service, flag, and state are categorical character variables. We use one-hot encoding to convert them into numeric form and further vectorize them to enhance the model's expressiveness and learning effectiveness.

[0101] Feature selection: To further improve model training efficiency and accuracy, we used the Pearson correlation coefficient to analyze the correlation between each feature and the label. We selected important features with a correlation coefficient greater than 0.3 and eliminated redundant or irrelevant features to construct a more representative feature set for the input data.

[0102] Data normalization: Because the value ranges of different features may vary greatly, some features may dominate model learning during training, affecting training effectiveness. Therefore, this study uses the Min-Max normalization method to normalize all numerical features. This method scales each feature to the range [0, 1]. The normalization formula is as follows:

[0103]

[0104] Among them, X is the original eigenvalue, X min and X max are the minimum and maximum values ​​of the feature, respectively, X ' is the normalized eigenvalue. This process helps to accelerate the convergence of the model and improve the generalization ability of the model.

[0105] Next, the pre-processed dataset is divided into a training set and a test set. The training set is input into the network model proposed in the above embodiment for training. The test set is input into the trained model to obtain classification prediction results and observe the model performance.

[0106] When evaluating network intrusion detection models, metrics such as accuracy, precision, recall, and F1-score are commonly used to measure model performance. These evaluation criteria reflect the model's performance in classifying positive and negative samples from different dimensions, helping to assess its recognition ability and generalization effect, while also providing guidance for optimizing training strategies. Accuracy represents the proportion of samples correctly classified by the model out of the total number of samples; precision reflects the proportion of samples identified as positive that are truly positive; recall represents the proportion of samples successfully identified as positive by the model; and the F1-score combines precision and recall, providing a measure of overall classification performance through a harmonic average.

[0107] Accuracy (ACC) represents the ratio of the number of samples correctly predicted by the model to the total number of samples. It is a basic indicator for measuring the overall classification effect of the model. The higher the accuracy, the better the model performs in the overall classification task. The calculation formula is:

[0108]

[0109] Precision (P) measures the proportion of samples that are actually positive among those predicted by the model. It reflects the reliability of the model in judging the positive class. A higher precision means fewer false positives. Its calculation formula is:

[0110]

[0111] Recall (R) represents the proportion of samples that are actually positive that are successfully identified as positive by the model. An increase in recall means that the model has fewer missed cases and stronger recognition ability. Its calculation formula is:

[0112]

[0113] The F1-score is a weighted harmonic mean of precision and recall, which comprehensively considers the model's ability to accurately identify positive samples and reduce false positives. A higher F1 score indicates a more balanced model in classifying positive and negative samples. Its calculation formula is:

[0114]

[0115] Figure 6The figure shows the accuracy trend of the model during training on the NSL-KDD dataset, with the horizontal axis representing the model training cycle and the vertical axis representing the accuracy. With each training epoch, the model's training and validation accuracy continue to rise, reaching convergence around the 80th epoch. Ultimately, the model achieved an accuracy of approximately 99.20% on the training set and 99.48% on the validation set.

[0116] At the beginning of training, the loss value is high, about 0.12. As the number of training rounds increases, the training loss decreases rapidly. At about 20 epochs, the loss value has dropped to about 0.04. After that, the training loss continues to decrease slowly, and finally stabilizes at a lower level, close to 0.02. The validation loss is also high at the beginning, close to the training loss. During the training process, the validation loss also shows a downward trend, and is basically consistent with the downward trend of the training loss. In the end, the validation loss stabilizes at a level slightly lower than the training loss, about 0.018. The results are as follows Figure 7 As shown, Figure 7 The horizontal axis is the model training cycle, and the vertical axis is the loss.

[0117] Table 1 shows the accuracy values ​​and loss performance, where the training and validation accuracy values ​​are 0.9948 and 0.9920, respectively, and the training and validation loss values ​​are 0.02 and 0.018, respectively.

[0118] Table 1 Accuracy and loss performance

[0119] Evaluation Metrics Performance Value Training accuracy 0.9920 Verification accuracy 0.9948 Training loss 0.020 Validation loss 0.018

[0120] To validate the effectiveness and superiority of the proposed DNN-BiGRU-Attention-MLP intrusion detection model in real-world network environments, we designed and conducted a series of systematic comparative experiments based on the widely used NSL-KDD and UNSW-NB15 datasets. This experimental approach, divided into four categories, comprehensively and multi-layeredly evaluates the model's detection performance and generalization capabilities across four dimensions: classification methods, feature extraction approaches, model architecture combinations, and comparisons with existing representative intrusion detection algorithms. In the first set of experiments, the proposed DNN-BiGRU-Attention feature extraction module was combined with various mainstream classifiers (e.g., Naive Bayes, Support Vector Machine, Decision Tree, K-Nearest Neighbor, and Random Forest) to explore the impact of different classification strategies on overall detection performance. The second set of experiments, focusing on feature selection, compared the effectiveness of LDA, ICA, RFE, and a random forest-based importance assessment method for feature extraction. These methods were then combined with an MLP classifier to analyze the impact of different feature processing methods on model performance. The third set of experiments focused on the optimization and evolution of the model structure. By gradually introducing BiGRU, attention mechanisms, and MLP layers, multiple structurally differentiated deep model versions were constructed, further demonstrating the contribution of each module in modeling temporal features, enhancing key feature weights, and improving classification accuracy. The fourth set of experiments compared representative algorithms, including SDAE-ELM, BDNN, BCNN, and PV-DM, to verify the overall performance leadership of our model, comparing it to classic models and current mainstream deep learning architectures. Table 2 shows the comparative experiments on the NSL-KDD dataset.

[0121] Table 2 Comparative experimental results of NSL-KDD

[0122]

[0123] Under the premise of fixed feature extraction and network structure, this paper integrates different classifiers into the DNN-BiGRU-Attention model to explore their impact on model performance. Experimental results show that the random forest (RF) classifier performs best across all metrics, with an accuracy of 98.73% and an F1-score of 98.69%, demonstrating its strong modeling capabilities for handling nonlinear boundaries and high-dimensional features. KNN and decision tree (DT) follow closely behind, with F1-scores of 98.44% and 98.17%, respectively, also demonstrating good classification capabilities. Although SVM performs well in terms of precision (97.33%), its recall is relatively low (90.28%), indicating limitations in identifying minority attack samples. In contrast, Naive Bayes (NB) performs significantly worse than other classifiers due to its assumption of conditional independence, making it difficult to effectively model relationships between complex features. In summary, the choice of classifier has a significant impact on intrusion detection performance.

[0124] To evaluate the impact of feature engineering on intrusion detection performance, this paper employed LDA, ICA, RFE, and random forest (RF) methods for feature selection, combined with a multi-layer perceptron (MLP) for classification. Experimental results showed that random forest-based feature selection (RF-MLP) performed best across all metrics, achieving an accuracy of 95.17% and an F1-score of 94.86%, demonstrating its superior ability to retain key features and eliminate redundant information. ICA-MLP and RFE-MLP achieved F1-scores of 93.59% and 93.08%, respectively, slightly lower than RF, but still stable. In contrast, while LDA-MLP achieved relatively balanced overall performance, it was inferior to RF in handling complex nonlinear feature relationships. These experimental results demonstrate the importance of appropriately selecting feature extraction methods for improving model performance in high-dimensional intrusion detection data.

[0125] In terms of model architecture evolution, this paper begins with a basic DNN model and gradually introduces the BiGRU, Attention mechanism, and MLP classification layer to evaluate the impact of each module on model performance. Experimental results show that while the DNN model alone achieves high precision (98.09%), its recall is relatively low (89.85%), with an F1-score of only 93.79%, indicating that it still has shortcomings in attack sample identification. The introduction of the BiGRU architecture significantly enhances the model's ability to model sequence features, with the F1-score rising to 99.00%. Further incorporating the self-attention mechanism effectively improves the model's ability to perceive key features, enhancing both precision and generalization. Finally, when combined with the MLP classifier, the model achieves optimal performance, reaching 99.14% accuracy and 99.11% F1-score, demonstrating the synergistic benefits of the multi-module combination. This experiment fully validates the significant impact of model architecture optimization on performance improvement and demonstrates the effectiveness and feasibility of the DNN-BiGRU-Attention-MLP combination for intrusion detection tasks.

[0126] In a comparative analysis with existing mainstream intrusion detection methods, the proposed model performed exceptionally well across all evaluation metrics, achieving 99.48% accuracy and 99.43% F1-score, significantly outperforming traditional methods and demonstrating superior detection performance. Although the PV-DM model achieved a similar F1-score (96.77%), it lagged slightly behind in terms of recall and overall balance. In contrast, the SDAE-ELM and BDNN models achieved recall rates of only 64.12% and 87.00%, respectively, reflecting their limited ability to identify complex or minority attack behaviors. While the BCNN model achieved relatively balanced performance across all metrics, its overall accuracy and F1-score were still lower than our proposed model. These experimental results further validate the proposed model's comprehensive advantages in recognition rate, robustness, and adaptability to complex network environments.

[0127] From classifier selection, feature extraction, model architecture optimization, to comparative analysis with existing methods, the proposed DNN-BiGRU-Attention-MLP model consistently achieved leading performance on the NSL-KDD dataset, demonstrating superior detection accuracy and generalization. In particular, while maintaining high accuracy, the recall rate and F1-score also remained high, demonstrating the model's enhanced ability to identify various attack behaviors and its high practical application value.

[0128] Experiments were conducted on the UNSW-NB15 dataset. Figure 8This figure shows the accuracy trends of the model during training on the UNSW-NB15 dataset. Both training and validation accuracy improve with increasing iterations, but convergence is slow. After approximately 100 iterations, the model performance stabilizes. Ultimately, the training accuracy stabilizes at approximately 98.0%, while the validation accuracy stabilizes at approximately 98.43%.

[0129] like Figure 9 As shown, the initial loss is approximately 0.11, then rapidly decreases, reaching around 0.07 after 20 epochs. After that, the rate of decline in training loss slows, with an overall fluctuating downward trend, ultimately stabilizing at a level close to 0.05. Initially, the validation loss is comparable to the training loss. During training, the validation loss fluctuates downward, with the gap between the validation loss and the training loss becoming more pronounced at certain stages. Ultimately, the validation loss stabilizes at approximately 0.045, slightly lower than the training loss.

[0130] Table 3 shows the accuracy values ​​and loss performance, where the training and validation accuracy values ​​are 0.980 and 0.9843, and the training and validation loss values ​​are 0.05 and 0.045, respectively.

[0131] Table 3 Accuracy and loss performance

[0132] Evaluation Metrics Performance Value Training accuracy 0.980 Verification accuracy 0.9843 Training loss 0.05 Validation loss 0.045

[0133] We conducted systematic comparative experiments on the UNSW-NB15 dataset. Four sets of experiments were conducted to comprehensively evaluate the performance of the DNN-BiGRU-attention-MLP approach. Table 4 shows the comparative experiments on the UNSW-NB15 dataset.

[0134] Table 4 Comparative experimental results of UNSW-NB15

[0135]

[0136]

[0137] Based on fixed feature extraction and network structure, different traditional classifiers were integrated into the DNN-BiGRU-Attention model for performance comparison experiments. The results showed that the choice of classifier had a significant impact on the model's final performance. Among them, the random forest (RF) classifier performed best, with an accuracy of 98.14% and an F1-score of 97.16%. All indicators were well-balanced, demonstrating its strong modeling capabilities for handling high-dimensional and nonlinear data. KNN and decision tree (DT) followed closely behind, with F1-scores of 96.45% and 96.08%, respectively, also demonstrating good classification results. Although the SVM achieved a high accuracy of 99.95%, its low recall (91.46%) caused its F1-score to drop to 95.54%, indicating that it still has some shortcomings in identifying minority attack samples. In contrast, the Naive Bayes (NB) model performed the worst. Although it had a high recall rate (98.31%), its precision was as low as 48.58% and its F1-score was only 65.03%, indicating that it was not effective in dealing with complex dependencies between features.

[0138] This paper compares the performance of four feature selection methods (LDA, ICA, RFE, and RF) in intrusion detection tasks, using a fixed MLP classifier structure. Experimental results show that the RF-MLP model achieves the strongest overall performance, with an accuracy of 98.09% and an F1-score of 98.89%, demonstrating its significant advantages in effectively identifying key features and eliminating redundant information. ICA-MLP and RFE-MLP follow closely behind, with F1-scores of 95.63% and 95.61%, respectively, demonstrating good stability and generalization capabilities while maintaining high accuracy. Although LDA-MLP performs slightly worse, its F1-score still reaches 95.37%, demonstrating its applicability when processing linearly separable data. Overall, feature selection strategies have a significant impact on improving model performance.

[0139] In a model architecture comparison experiment, this paper analyzed model performance by gradually introducing the BiGRU, Attention mechanism, and MLP classification layer. The results showed that while the basic DNN model achieved high accuracy (96.13%), its recall was low (86.93%), with an F1-score of only 91.30%, indicating significant deficiencies in identifying attack samples. The introduction of the BiGRU enhanced the model's temporal modeling capabilities, significantly improving its F1-score to 96.58%. Further incorporating the Attention mechanism enhanced the model's ability to capture key features, achieving an F1-score of 96.15%. Finally, combining the MLP classification layer achieved optimal overall performance, with accuracy increasing to 98.41% and an F1-score of 96.64%. These experimental results demonstrate that the multi-module architecture exhibits a significant synergistic enhancement effect, effectively improving the model's classification and generalization capabilities.

[0140] In comparative experiments with different algorithms, the proposed DNN-BiGRU-Attention-MLP model achieved the best performance across all metrics, achieving an accuracy of 98.43% and an F1-score of 96.68%, demonstrating excellent stability and generalization. In comparison, the BCNN model, a representative of traditional convolutional architectures, achieved an F1-score of 90.45%. While this performance is relatively good, it still lags significantly behind our proposed model. While SDAE-ELM and BDNN exhibited some advantages in recall, their precision was relatively low, with F1-scores of 79.49% and 79.00%, respectively, suggesting possible overfitting or insufficient ability to distinguish between normal and abnormal samples. The PV-DM model performed mediocrely in this experiment, achieving an F1-score of only 80.20%. In summary, this experiment fully validated the robustness, adaptability, and practicality of our proposed model in processing complex, high-dimensional network intrusion data, demonstrating its significant advantages over other mainstream algorithms.

[0141] Through multi-dimensional experimental comparisons, the proposed DNN-BiGRU-Attention-MLP deep fusion model demonstrates superior performance across key metrics such as accuracy, precision, recall, and F1-score, particularly in attack identification, model stability, and generalization. By effectively integrating a multi-module structure and rationally selecting feature extraction methods and classifiers, this model constructs a powerful and robust intrusion detection system. It exhibits significant advantages in feature extraction, anomaly identification, and handling data imbalance, demonstrating its broad applicability and great potential in practical network security applications. Compared with existing mainstream methods, this model improves network intrusion detection performance by hierarchically learning multi-layered spatiotemporal features of traffic data, enhancing accuracy and generalization. Experimental results demonstrate that the proposed DNN-BiGRU-based hierarchical attention mechanism intrusion detection model demonstrates superior performance on the NSL-KDD and UNSW-NB15 datasets. Compared with traditional detection methods, this model achieves significant improvements in both detection accuracy and classification performance, effectively enhancing the ability to identify network attack behaviors.

[0142] In summary, in response to the limitations of traditional intrusion detection methods in high-dimensional traffic data feature modeling and cross-modal fusion, this application proposes a modular intrusion detection model that integrates deep neural networks (DNNs), bidirectional gated recurrent units (BiGRUs) and self-attention mechanisms. This model forms a hierarchical feature extraction architecture by constructing a dual DA structure (two DNN modules embedded with self-attention) and a dual BA structure (two BiGRU modules embedded with self-attention) to achieve refined mining of spatial and temporal features of network traffic; the model fully utilizes the DNN's ability to extract static high-dimensional features and the BiGRU's advantages in modeling bidirectional temporal dependencies, combined with the self-attention mechanism's ability to focus on key dimensions and key time points, effectively improving the accuracy of identifying abnormal behaviors. The deep features output by each substructure are spliced ​​and fused to form a global representation, and the multi-layer perceptron (MLP) finally completes the classification task. Compared with traditional models, this model is more innovative in terms of structural design and feature modeling: on the one hand, the hierarchical fusion of DA and BA substructures overcomes the limitations of a single network structure in spatial or temporal modeling, and improves its adaptability to complex and heterogeneous network traffic; on the other hand, the modular embedding strategy of the attention mechanism avoids information redundancy and improves the dynamics and sophistication of feature selection. Experimental results show that this method achieves excellent performance on both the NSL-KDD and UNSW-NB15 datasets, significantly outperforming existing methods in key indicators such as accuracy and F1 score, demonstrating good detection results and generalization capabilities. Future research will further focus on the lightweight design of the model and real-time detection optimization to better adapt to the processing needs of large-scale, high-dimensional intrusion data in complex network environments.

[0143] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0144] Based on the same inventive concept, the embodiment of the present application also provides a layered attention network intrusion detection device based on DNN and BiGRU for implementing the above-mentioned layered attention network intrusion detection method based on DNN and BiGRU. The implementation solution provided by the device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of one or more embodiments of the layered attention network intrusion detection device based on DNN and BiGRU provided below can be found in the above-mentioned limitations of the layered attention network intrusion detection method based on DNN and BiGRU, and will not be repeated here.

[0145] In one embodiment, a layered attention network intrusion detection device based on a DNN and a BiGRU is provided, comprising: a feature extraction module for extracting spatial features from traffic data using two first models connected in series, and extracting temporal features from traffic data using two second models connected in series; wherein the first model comprises a deep neural network embedded with self-attention, and the second model comprises a bidirectional gated recurrent unit embedded with self-attention;

[0146] The feature fusion module is used to fuse spatial features and temporal features through splicing operations to obtain fused features;

[0147] The classification module is used to input the fusion features into the multi-layer perceptron to obtain the network intrusion detection classification results of the traffic data.

[0148] Each module in the aforementioned DNN and BiGRU-based hierarchical attention network intrusion detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0149] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in all the above method embodiments when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in all the above method embodiments are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in all the above method embodiments when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0153] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to these.

[0154] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A hierarchical attention network intrusion detection method based on DNN and BiGRU, characterized in that: The method comprises: Performing spatial feature extraction on traffic data using two first models connected in series, and performing temporal feature extraction on the traffic data using two second models connected in series; wherein the first model includes a deep neural network embedded with self-attention, and the second model includes a bidirectional gated recurrent unit embedded with self-attention; fusing the spatial features and the temporal features through a splicing operation to obtain a fused feature; The fusion features are input into a multi-layer perceptron to obtain a network intrusion detection classification result of the traffic data.

2. The method according to claim 1, wherein: Each of the first models includes a fully connected layer, a BN layer, a Dropout layer and a self-attention mechanism layer; each of the second models includes a bidirectional GRU layer, a BN layer, a Dropout layer and a self-attention mechanism layer.

3. The method according to claim 1, characterized in that After obtaining the fusion feature, the method further includes: The fused features are processed using a Flatten layer to convert the fused vector into a one-dimensional vector format to eliminate the time step dimension in the fused features.

4. The method according to claim 1, wherein The deep neural network uses 32 neurons, adopts ReLU activation function, BatchNormalization for normalization, and the Dropout rate is set to 0.

5.

5. The method according to claim 4, characterized in that The bidirectional gated recurrent unit uses 32 neurons and sets a return sequence.

6. The method according to claim 1, characterized in that The multilayer perceptron uses 60 neurons and adopts ReLU activation function, BatchNormalization and Dropout to optimize feature representation.

7. A hierarchical attention network intrusion detection model based on DNN and BiGRU, characterized by: The device comprises: a feature extraction module, configured to extract spatial features from traffic data using two first models connected in series, and to extract temporal features from the traffic data using two second models connected in series; wherein the first model comprises a deep neural network embedded with self-attention, and the second model comprises a bidirectional gated recurrent unit embedded with self-attention; A feature fusion module, configured to fuse the spatial features and the temporal features through a splicing operation to obtain a fused feature; The classification module is used to input the fusion features into a multi-layer perceptron to obtain the network intrusion detection classification results of the traffic data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images