Adaptive network anomaly detection method based on two-stage deep learning

Abstract

The invention discloses a self-adaptive network anomaly detection method based on two-stage deep learning, relates to a network anomaly detection method, and provides an innovative and collaborative deep learning framework for network traffic anomaly detection, and the framework effectively utilizes the advantages of a convolutional neural network (CNN), a long short-term memory network (LSTM) and a Transformer. The method has the remarkable characteristic that the system can automatically adapt to a continuously evolved network threat environment due to the seamlessly integrated self-learning capability. The adaptive attribute is crucial in the background of a dynamic network environment. The framework provided by the invention provides a promising way for developing a more efficient and more autonomous network security solution, and the method represents a great leap in the field of network traffic anomaly detection, promotes the boundary of the field, and paves a way for developing a self-adaptive and high-performance network security system. And the method has a far-reaching influence on protecting digital assets and ensuring the integrity of network infrastructures when facing increasingly complex network threats.

Description

Technical Field

[0001] The present invention relates to a network anomaly detection method, and in particular to an adaptive network anomaly detection method based on dual-stage deep learning. Background Art

[0002] In the digital age, cybersecurity has become a critical issue in protecting national infrastructure, business operations, and personal privacy. With the popularization of Internet of Things (IoT) devices, the promotion of 5G technology, and the widespread application of cloud computing, the complexity of network connections and data traffic has increased exponentially, posing unprecedented challenges to network security. Traditional advanced machine learning techniques, such as generative adversarial networks (GANs), graph neural networks (GNNs), and Transformers, represent a major leap in our ability to detect and mitigate cyber threats. Although these technologies have made significant progress, current research still faces many challenges. As the scale and complexity of networks expand, how to effectively process and analyze large-scale, high-dimensional network traffic data remains a difficult problem. Real-time demands also put tremendous pressure on anomaly detection systems. How to respond accurately in a very short time is a key issue.

[0003] Faced with ever-evolving attack vectors, designing adaptive detection models to address unknown threats has become a research focus. This paper proposes an innovative, synergistic deep learning architecture to address contemporary challenges in anomaly detection. This approach integrates the key features of CNNs, LSTMs, and Transformer models, enhancing their detection capabilities with a self-learning mechanism. This fusion aims to improve the model's generalization and operational efficiency, thereby advancing the state-of-the-art in identifying anomalies in complex and evolving networks. Summary of the Invention

[0004] The purpose of this invention is to provide an adaptive network anomaly detection method based on two-stage deep learning. This method proposes an innovative and collaborative deep learning framework for network traffic anomaly detection, which effectively utilizes the advantages of convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and transformers. The seamlessly integrated self-learning capability enables the system to autonomously adapt to the evolving network threat environment. This adaptive property is crucial in the context of a dynamic network environment. It has far-reaching implications for protecting digital assets and ensuring the integrity of network infrastructure in the face of increasingly complex network threats.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] An adaptive network anomaly detection method based on two-stage deep learning, the method comprising the following components:

[0007] (1) CNN-LSTM module:

[0008] A model structure that combines CNN and LSTM. It leverages CNN's strengths in extracting spatial features to identify local characteristics of network traffic data. A pooling layer reduces the data dimension while retaining key information. An LSTM layer then memorizes long-term dependencies in time series and detects anomalous patterns that rely on historical information, such as distributed denial of service (DDoS) attacks. Finally, the classification results output by the fully connected layer are mapped to probability values ​​for binary classification.

[0009]

[0010] in is the predicted label, y pred is the probability value output by the model;

[0011] (2) LSTM-Transformer module:

[0012] A hybrid model combining LSTM and Transformer for multi-classification tasks; a two-pronged approach is adopted to model temporal dependencies; the long short-term memory (LSTM) layer is used to extract and encapsulate the long-term sequence patterns inherent in time series data, and its gating mechanism is used to selectively retain relevant information over long time spans; a Transformer module is also integrated to establish comprehensive cross-temporal relationships through its self-attention mechanism, facilitating the capture of global dependencies at different time steps in the input sequence; a fully connected layer is used to output the predicted classification probability for each category;

[0013] (3) Multi-head self-attention layer:

[0014] A custom multi-headed self-attention module; this architectural component performs a complex series of transformations on the input tensor, projecting it into three different representation subspaces: query, key, and value domains. The essence of this attention paradigm is to compute the temporal correlation coefficient of each element in the sequence through a scaled dot product operation; these coefficients are normalized by the softmax function to obtain a probability distribution of attention weights; these weights are then used to calculate a weighted aggregation of the values, ultimately obtaining a context-rich output;

[0015] (4) Transformer block layer:

[0016] The LSTM-Transformer model processes the anomalous network traffic data identified by the CNN-LSTM model in the first phase. By focusing on this small portion of data, the LSTM-Transformer applies its capabilities to learning more subtle patterns that distinguish different types of anomalies.

[0017] (5) Self-learning module:

[0018] A self-learning and model-updating system that maintains high performance and generalization capabilities by regularly checking for new data, detecting data drift, and retraining and reevaluating the model when needed;

[0019] (6) Self-learning and model updating:

[0020] The system will periodically check if there are new data files in the set path according to the set update cycle. If new data is detected, it will be merged with the current data to generate a new data set.

[0021] (7) Data drift detection:

[0022] Data drift detection is a step in model effectiveness. For each feature, its distribution is calculated in the training data and the new data. Changes in data distribution are detected using JS divergence calculation, KS test, and changes in mean and variance. If the JS divergence of any feature exceeds a threshold, the p-value of the KS test is less than 0.05, or the mean and variance change exceed a threshold, data drift is detected.

[0023] (8) Experimental evidence

[0024] The data drift detection mechanism uses a multifaceted approach to identify significant changes in the statistical properties of network traffic features. The two-sample Kolmogorov-Smirnov (KS) test, a nonparametric method capable of detecting changes in the shape and scale of distributions, is used. The KS test is applied independently to each feature with a significance level of α ≈ 0.05. To mitigate the increased Type I error rate associated with multiple comparisons, a Bonferroni correction is implemented, adjusting the significance level to α′ = α / m, where m is the number of features.

[0025] In the adaptive network anomaly detection method based on two-stage deep learning, the calculation formula for each classification probability is as follows:

[0026] MModel(x)=Dense(Flatten(TransformerBlock(LSTM(x))))

[0027] The adaptive network anomaly detection method based on two-stage deep learning is described. The Transformer block consists of a multi-head self-attention layer and a feedforward neural network. Each sublayer is followed by layer normalization and dropout regularization to prevent overfitting and stabilize the training process. The core implementation formula is as follows:

[0028] TransformerBlock(x)=LayerNorm(x+Dropout(MultiHeadSelfAttention(x))).

[0029] The advantages and effects of the present invention are:

[0030] This paper proposes an innovative, collaborative deep learning framework for network traffic anomaly detection that effectively leverages the strengths of convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and transformers. Its notable feature is its seamlessly integrated self-learning capability, enabling the system to autonomously adapt to the evolving cyber threat landscape. This adaptive property is crucial in the context of dynamic network environments.

[0031] 2. This paper demonstrates the tremendous potential of combining complementary deep learning architectures and incorporating self-learning capabilities to build robust, adaptive anomaly detection systems. The proposed framework offers a promising path to developing more efficient and autonomous network security solutions that can effectively cope with the ever-changing threat landscape. In summary, this research represents a significant leap forward in the field of network traffic anomaly detection, pushing the boundaries of the field and paving the way for the development of adaptive, high-performance network security systems. The new techniques introduced in this paper have far-reaching implications for protecting digital assets and ensuring the integrity of network infrastructure in the face of increasingly sophisticated cyber threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a diagram showing the impact of CNN-LSTM model parameters on accuracy.

[0033] Figure 2 This is a diagram showing the impact of packet loss rate on accuracy in the present invention;

[0034] Figure 3 This is the overall framework diagram of the model of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the embodiments shown in the accompanying drawings.

[0036] Overview of the Model of the Invention

[0037] This paper proposes an innovative deep learning method that combines the strengths of CNN, LSTM, and Transformer and introduces a self-learning mechanism to achieve efficient detection of network traffic anomalies. It consists of two core modules: a classification output module (containing a CNN-LSTM binary classifier and an LSTM-Transformer multi-classifier) ​​and a self-learning module.

[0038] The overall framework of the model of the present invention is as follows Figure 3 shown.

[0039] CNN-LSTM module

[0040] To address the information loss introduced by pooling operations in standard convolution, this paper proposes a model structure that combines CNN and LSTM. This model leverages CNN's strengths in spatial feature extraction to identify local features in network traffic data. Pooling layers reduce the data dimensionality while retaining key information. An LSTM layer then memorizes long-term dependencies in the time series and detects anomalous patterns that rely on historical information, such as distributed denial of service (DDoS) attacks. Finally, the classification results output by the fully connected layer are mapped to probability values ​​for binary classification.

[0041]

[0042] in is the predicted label, y pred is the probability value output by the model.

[0043] In this section, we systematically explain the optimization and design of the CNN-LSTM model from two perspectives: parameter modification and module combination. In the CNN part, the parameters of the convolutional layer (Conv1D) and the pooling layer (MaxPooling1D) were modified. Specifically, in the convolutional layer, the number of filters was increased from [32, 64] to [64, 128] to extract richer and more diverse features. At the same time, the kernel_size was kept at 3 to control computational complexity while expanding the receptive field. In the pooling layer, the pool_size was set to 2 to maintain sufficient feature resolution while reducing computational overhead. By adjusting these parameters, the model's feature extraction capability can be adjusted to a relatively high level.

[0044] like Figure 1 As shown in the impact of CNN-LSTM model parameters on accuracy, increasing the filter value does not necessarily significantly improve model performance. At the same time, it is necessary to balance the time and equipment investment required for high performance.

[0045] In the LSTM model, we focused on the impact of the Dropout layer parameters. By introducing Dropout regularization, we hoped to alleviate the model's overfitting problem and improve its generalization performance. To find the optimal Dropout rate, we conducted experiments at three levels: 0.2, 0.3, and 0.4.

[0046] like Figure 2 The effect of dropout rate on accuracy shows that as the dropout rate increases, the model's performance on the test set first increases and then decreases, reaching a peak at approximately 0.3. This suggests that moderate dropout can effectively suppress overfitting and enable the model to learn more robust and generalizable features. However, excessively high dropout rates can also overly suppress feature learning, leading to underfitting. Therefore, choosing the dropout rate requires striking a balance between reducing the risk of overfitting and preserving key features.

[0047] CNNs and LSTMs each have their own strengths in network anomaly detection. Their combination allows them to learn from each other and leverage synergistic advantages. In the model, CNN serves as the front-end, mapping raw network traffic data into more abstract and semantically rich feature sequences. LSTM, serving as the back-end, further models and encodes the feature sequences extracted by the CNN as time series. This cascaded combination enables the model to simultaneously consider the spatial and temporal characteristics of network traffic data and comprehensively describe its internal patterns.

[0048] LSTM-Transformer module

[0049] In order to better handle the abnormal network traffic obtained by CNN-LSTM, the present invention designs and implements a hybrid model for multi-classification tasks that combines LSTM and Transformer. This design adopts a two-pronged approach to modeling temporal dependencies. The long short-term memory (LSTM) layer is used to extract and encapsulate the long-term sequence patterns inherent in time series data, and its gating mechanism is used to selectively retain relevant information over a long time span. At the same time, a Transformer module is integrated to establish comprehensive cross-temporal relationships through its self-attention mechanism, which promotes the capture of global dependencies at different time steps in the input sequence. This synergistic combination enables our model to simultaneously process local temporal dynamics and overall sequence structure. Finally, a fully connected layer is used to output the predicted probability of each category. The calculation formula for each classification probability is as follows:

[0050] MModel(x)=Dense(Flatten(TransformerBlock(LSTM(x))))

[0051] Multi-head self-attention layer

[0052] To enhance the model's contextual learning capabilities, a custom multi-head self-attention module was designed and implemented. This innovative architectural component performs a complex series of transformations on the input tensor, projecting it into three distinct representation subspaces: query, key, and value domains. The essence of this attention paradigm lies in computing the temporal correlation coefficients of each element in the sequence via a scaled dot product operation. These coefficients are normalized using a softmax function to produce a probability-distributed attention weight. These weights are then used to compute a weighted aggregation of the values, ultimately yielding a context-rich output.

[0053] Transformer Block Layer

[0054] The LSTM-Transformer model is designed to process the anomalous network traffic data identified by the CNN-LSTM model in the first phase. By focusing on this small portion of data, the LSTM-Transformer can leverage its capabilities to learn more subtle patterns that can distinguish different types of anomalies. The Transformer block consists of a multi-head self-attention layer and a feedforward neural network. Each sublayer is followed by layer normalization and dropout regularization to prevent overfitting and stabilize the training process. The core implementation formula is as follows:

[0055] TransformerBlock(x)=LayerNorm(x+Dropout(MultiHeadSelfAttention(x)))

[0056] The feedforward neural network architecture consists of a densely connected, two-layer configuration of neurons. The initial layer uses a rectified linear unit (ReLU) as its activation function, promoting a nonlinear transformation of the input data. Subsequently, the terminal layer generates an output vector with the same dimensionality as the input data, preserving the spatial properties of the original data representation throughout the forward propagation of the network.

[0057] Self-learning module

[0058] To adapt to the ever-changing network environment and reduce the need for frequent human intervention, this invention adds a self-learning and model update system to maintain high performance and generalization ability by regularly checking for new data, detecting data drift, and retraining and evaluating the model when necessary.

[0059] Self-learning and model updating

[0060] The system will periodically check for new data files in the specified path based on the specified update interval. If new data is detected, it will be merged with the current data to generate a new dataset. This process ensures that the model has timely access to the latest data so that the latest data changes can be reflected during training. After incorporating the new data, the system will detect any significant changes in the data distribution, known as data drift. Data drift detection uses a variety of methods, including JS divergence, Kolmogorov-Smirnov test, and changes in mean and variance. If data drift is detected, the model will be triggered to retrain. When data drift is detected or when the regular update interval is reached, the system will retrain the model on the training set and evaluate it on the test set. Metrics such as precision, recall, and F1 score are calculated to determine whether the new model performs better than the old one. By comparing the performance metrics of the old and new models, the system decides whether to save the new model. If the new model performs better, the old model will be saved and replaced. Simultaneously, a performance improvement graph is plotted to visually demonstrate the change in model performance.

[0061] Data drift detection

[0062] Data drift detection is a critical step in ensuring model effectiveness. For each feature, its distribution is calculated in both the training data and the new data. Changes in data distribution are detected using the JS divergence calculation, the KS test, and changes in the mean and variance. Data drift is detected if the JS divergence for any feature exceeds a threshold, the KS test p-value is less than 0.05, or the mean and variance change exceed a threshold.

[0063] 1) JS divergence calculation: JS divergence is used to measure the similarity between two probability distributions. For each feature, the distribution of the training data and the new data is calculated, and then the JS divergence is calculated using the following formula:

[0064]

[0065] in P represents the frequency counting and normalization of the values ​​of a column in the training dataset to obtain a probability distribution, and Q represents the probability distribution of the same column in the new dataset. KL is the Kullback-Leibler divergence.

[0066] The JS divergence is calculated as follows:

[0067]

[0068] JS divergence values ​​range between 0 and 1, with higher values ​​indicating greater differences between the two distributions.

[0069] 2) Kolmogorov-Smirnov test: The KS test is used to compare whether two samples have the same distribution. The KS statistic and its p-value are calculated as follows:

[0070]

[0071] where F1 and F2 are the empirical distribution functions of the two samples, respectively. The P value is used to determine whether the two samples come from the same distribution; if the P value is less than 0.05, the data distributions are considered significantly different.

[0072] 3) Change in mean and variance: Calculate the mean and variance of the training data and the new data and compare the differences between them. If the change in mean or variance exceeds the set threshold, the data distribution is considered to have changed. The mean and variance are calculated using the following formula:

[0073]

[0074] Experimental evidence

[0075] The data drift detection mechanism uses a multifaceted approach to identify significant changes in the statistical properties of network traffic features. A two-sample Kolmogorov-Smirnov (KS) test, a nonparametric method capable of detecting changes in the shape and scale of distributions, is used. The KS test is applied independently to each feature with a significance level of α ≈ 0.05. To mitigate the increased Type I error rate associated with multiple comparisons, a Bonferroni correction is implemented, adjusting the significance level to α' = α / m, where m is the number of features.

[0076] To further enhance the robustness of data drift detection, an effect size threshold is introduced using Cohen's d statistic. Drift is signaled only when the KS test is significant (p < α') and Cohen's d > 0.2, ensuring that not only statistically significant changes but also practically meaningful changes are captured. The reference distribution for data drift detection is dynamically updated using an exponential moving average (EMA) with a smoothing factor of 0.01, which allows it to gradually adapt to slowly changing patterns while remaining sensitive to sudden changes.

[0077] The self-learning mechanism aims to maintain model performance over time through adaptive retraining. The system operates based on a dual trigger mechanism: performance degradation or detection of data drift. Performance is monitored using accuracy and F1 score metrics, with thresholds set at 0.95 and 0.90, respectively.

Claims

1. An adaptive network anomaly detection method based on two-stage deep learning, characterized in that: The method comprises the following components: (1) CNN-LSTM module: A model structure that combines CNN and LSTM. It uses CNN's advantage in extracting spatial features to extract features, identify local features of network traffic data, and reduce data dimensions and retain key information through the pooling layer. Then, the LSTM layer memorizes long-term dependencies in time series and detects abnormal patterns that rely on historical information, such as distributed denial of service attacks (DDoS). Finally, the classification results output by the fully connected layer are mapped to probability values ​​for binary classification. in is the predicted label, y pred is the probability value output by the model; (2) LSTM-Transforme module: A hybrid model combining LSTM and Transformer for multi-classification tasks; a two-pronged approach is adopted to model temporal dependencies; the long short-term memory (LSTM) layer is used to extract and encapsulate the long-term sequence patterns inherent in time series data, and its gating mechanism is used to selectively retain relevant information over a long time span; a Transformer module is also integrated to establish comprehensive cross-temporal relationships through its self-attention mechanism, which facilitates the capture of global dependencies at different time steps in the input sequence; a fully connected layer is used to output the predicted classification probability for each category; (3) Multi-head self-attention layer: A custom multi-head self-attention module; this architectural component performs a complex series of transformations on the input tensor to project it into three different representation subspaces: query, key, and value domains; the essence of this attention paradigm is to calculate the temporal correlation coefficient of each element in the sequence through a scaled dot product operation; These coefficients are normalized by the Softmax function to obtain a probability distribution of attention weights; These weights are then used to compute a weighted aggregation of the values, ultimately yielding a context-rich output; (4) Transformer block layer: The LSTM-Transformer model processes the anomalous network traffic data identified by the CNN-LSTM model in the first stage; by focusing on this small portion of data, the LSTM-Transformer puts its power to use in learning more subtle patterns that distinguish different types of anomalies; (5) Self-learning module: A self-learning and model-updating system that maintains high performance and generalization capabilities by periodically checking for new data, detecting data drift, and retraining and evaluating the model when needed; (6) Self-learning and model updating: The system will periodically check if there are new data files in the set path at the set update cycle; if new data is detected, it will be merged with the current data to generate a new data set; (7) Data drift detection: Data drift detection is a step in model effectiveness; for each feature, calculate its distribution in the training data and new data; Use JS divergence calculation, KS test, and changes in mean and variance to detect changes in data distribution; if the JS divergence of any feature exceeds the threshold, the p-value of the KS test is less than 0.05, or the mean and variance changes exceed the threshold, data drift is detected; (8) Experimental evidence The data drift detection mechanism uses a multifaceted approach to identify significant changes in the statistical properties of network traffic features; the two-sample Kolmogorov-Smirnov (KS) test is used, which is a non-parametric method that can detect changes in the shape and scale of the distribution; the KS test is applied to each feature independently with a significance level of α≈0.05; to mitigate the increased type I error rate associated with multiple comparisons, a Bonferroni correction is implemented, adjusting the significance level to α'=α / m, where m is the number of features.

2. According to claim 1, a method for adaptive network anomaly detection based on two-stage deep learning is characterized in that: The calculation formula for each classification probability is as follows: MModel(x)=Dense(Flatten(TransformerBlock(LSTM(x)))).

3. The method for adaptive network anomaly detection based on two-stage deep learning according to claim 1, characterized in that: The Transformer block consists of a multi-head self-attention layer and a feed-forward neural network; each sub-layer is followed by layer normalization and Dropout regularization to prevent overfitting and stabilize the training process; the core implementation formula is as follows: TransformerBlock(x)=LayerNorm(x+Dropout(MultiHeadSelfAttention(x))).

Images