A dual-channel multi-scale network traffic anomaly detection method

Abstract

The application discloses a kind of dual-channel multiscale network traffic anomaly detection methods, belong to network security technical field.Aiming at the problem that existing method cannot comprehensively analyze the time-space characteristics of network traffic to carry out anomaly detection, first, all the characteristics of network traffic data are divided into spatial characteristics and time characteristics, the spatial characteristics are refined and expanded by multiple different scale hole convolutions, and the spatial information within different distances is captured, the outputs of multiple hole convolution blocks are combined, and the time characteristics are fused through the fully connected layer, the time sequence dependence relationship is modeled through BIGRU when the time characteristics, the attention mechanism allocates different weights for each time step in the sequence, and the capture of important time sequence characteristics is enhanced;Then the fusion of the two channel characteristics is completed through the fusion layer;Finally, the linear layer is mapped to the output space to generate the probability distribution of each category.The experimental results show that the application achieves good results on real-world datasets.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security, and particularly relates to a dual-channel multi-scale network traffic anomaly detection method. BACKGROUND

[0002] With the increasing popularity of the Internet, the increasing network applications, and the increasing role of Internet technology, the Internet technology is developing at a high speed. Various aspects of daily life will generate a large amount of network traffic data. In these data, there is a large amount of valuable information, which also includes sensitive organization or personal information. With the increase of the scale of computer network, this huge network traffic data is vulnerable to attacks and intrusions, and the network security problem is increasingly serious, even to the level of national security, so the network security problem needs to be solved urgently.

[0003] With the rapid development of information technology, network traffic anomaly detection technology is also evolving and innovating. Network traffic anomaly detection is the process of analyzing and modeling traffic data to identify traffic data that does not conform to normal behavior. First, there is the rule-based approach, which detects anomalies based on pre-defined rules. These rules are developed based on the understanding and experience of network administrators about the characteristics of normal network space. For example, it is stipulated that internal network hosts can only communicate with specific external IP address ranges, or that certain ports can only be used for specific services. When network traffic violates these rules, it is considered abnormal. However, the formulation of rules requires network administrators to have a deep understanding of the network environment and potential attacks, and the rules are often static. For new and complex attacks, especially those that can disguise as normal traffic space characteristics, there may be false positives or false negatives. To solve the above problems, machine learning and deep learning methods are widely used in anomaly detection and have achieved remarkable results. Machine learning-based detection methods can automatically build traffic normal behavior models by learning from a large amount of historical network traffic data, effectively identifying abnormal traffic that deviates significantly from the normal model. Deep learning technology, with its powerful feature automatic extraction and complex model construction capabilities, has shown excellent performance in handling large-scale, high-dimensional network traffic data, and can more accurately capture subtle abnormal features and complex nonlinear relationships in traffic data. Among these features, there are mainly time features and space features. Network traffic data is actually a kind of sequential data that changes over time, and has obvious time correlation in some features. The network traffic anomaly detection method based on time features uses this characteristic to analyze the correlation and rules of sequence data in the time dimension to detect anomalies. However, when network traffic appears complex and nonlinear abnormal patterns (such as sudden and large-scale DDoS attacks, whose traffic growth is exponential), traditional time series models may not be able to fit and predict well. Because these models are mostly based on linear assumptions, they have weak processing capabilities for nonlinear relationships. On the other hand, if the pattern of traffic data does not have a specific time period, the predicted traffic data cannot correctly classify the actual traffic data. In the context of network traffic anomaly detection, spatial features refer to the characteristics of network traffic from the spatial dimension of network topology and traffic distribution. Unlike time series-based methods, it focuses on analyzing the relationship and distribution of traffic between different nodes (such as hosts, servers, routers, etc.) in the network at the same time. For example, the allocation of traffic between different subnets, different server groups or different geographic regions, and the connection status of each node in the network. In a complex network environment, relying solely on time series data to detect anomalies is not enough.Network attacks and abnormal traffic often exhibit specific patterns in space. Therefore, considering spatial features can provide a more comprehensive perspective for anomaly detection. However, few existing methods effectively integrate spatial and temporal features for anomaly detection. SUMMARY

[0004] To solve the problem that existing methods cannot comprehensively analyze the spatial and temporal features of network traffic for anomaly detection, the present application provides a dual-channel multi-scale network traffic anomaly detection method. The detection method uses multiple different scales of dilated convolution (MC) in deep learning, a bidirectional gated recurrent unit (Bi-GRU), and a self-attention mechanism (Self-Attention) to realize network traffic anomaly detection.

[0005] Dilated convolution (Dilated Convolution) is an important convolution operation variant in the field of convolutional neural networks (CNN) in deep learning. Its core is to introduce a key parameter called dilation rate. In traditional convolution, the convolution kernel slides over the input data (such as feature maps) with a fixed stride and closely arranged manner for calculation. Dilated convolution inserts a certain number of "holes" between convolution kernel elements through dilation rate. While increasing the receptive field, it maintains the computational complexity. By adjusting the dilation rate, features can be extracted at different scales. Small dilation rate dilated convolution can be used to capture detailed features of the target, and large dilation rate dilated convolution can focus on long-term features of the target.

[0006] Bidirectional gated recurrent unit (Bi-GRU) is a powerful architecture in the field of recurrent neural networks (RNN). Bi-GRU constructs a bidirectional information processing channel based on GRU, processes input sequences from both forward and reverse directions, and effectively extracts long-distance dependencies in sequences using complete context information of sequences, exhibiting excellent performance in processing sequence data.

[0007] Self-attention mechanism (Self-Attention) is based on the quantitative evaluation of the association between input data elements. By calculating the importance weight of each element, the model can focus on the most relevant part of the input data for the current task. It effectively cooperates with the bidirectional gated recurrent unit to process network traffic data in parallel, thereby better capturing the features in the data.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] A dual-channel multi-scale network traffic anomaly detection method, the method comprising the following steps:

[0010] Step 1, data preprocessing; using CICFlowMeter to extract features from the collected pcap file, using the data processing function in the Python library to preprocess the collected network traffic data (data cleaning, missing value processing, normalization), obtaining the corresponding network traffic features, and classifying these traffic features to obtain spatial features and time features;

[0011] Further, the specific operation of step 1 is:

[0012] Step 1.1: Packet analysis: network attack through fixed IP address at a certain time, capture network traffic data packet, get pcap file, parse the pcap file through CICFlowMeter, get the relevant feature information of network traffic data packet, such as: the number of network traffic data packets in the outflow direction, the number of network traffic data packets in the inflow direction, the maximum size of network traffic data packets in the forward direction, the minimum size of network traffic data packets in the forward direction, flow duration, flow byte rate, etc. At the same time, the IP address of the attacker is labeled to generate real labels of abnormal traffic and normal traffic as input for step 1.2;

[0013] Step 1.2: Data cleaning: missing value processing and normalization processing are performed on the feature information obtained in step 1.1 to obtain 78-dimensional features for training the network traffic anomaly detection model DMCG;

[0014] Step 1.3: Time and space feature division: divide the processed feature information into spatial features and time features as input for step 2.

[0015] Step 2: Construction and training of network traffic anomaly detection model DMCG;

[0016] Model construction: using PyTorch and Scikit-learn to build a network traffic anomaly detection model DMCG composed of a multi-scale dilated convolution module, a bidirectional gated recurrent unit module and a self-attention module;

[0017] The multi-scale dilated convolution module is built using the Scikit-learn library, which includes a dilated convolution layer, a max pooling layer and a linear layer;

[0018] The bidirectional gated recurrent unit module and the self-attention module are built based on the PyTorch framework;

[0019] Model training: set the training rounds of the model in advance, and input the corresponding space-time features into the corresponding channel for training;

[0020] Training process: divide the spatial features and temporal features obtained in step 1 into training set and validation set according to the proportion, input the training set into the model, use the multi-scale hollow convolution module to extract the features of the spatial features, and then perform down-sampling and data dimension reduction on the extracted features through the maximum pooling layer, and aggregate the spatial features of different scales; input the time feature into the bidirectional gate recurrent unit module, capture the time sequence dependence and hierarchical structure information therein, and then assign different weights to each time step in the sequence through the self-attention mechanism to enhance the capture of important time sequence features; aggregate the time feature and the spatial feature, and then pass through the classifier composed of two linear layers and a softmax function to generate the class probability of the sample, and then calculate the cross-entropy loss between the predicted result and the real label; when the loss is lower than the optimal model, the current model is saved as the optimal model; when the loss is higher than the optimal model, a new round of training is performed, and the early stopping mechanism is used to stop training when the loss of the trained model no longer decreases, so that the model converges to the minimum cross-entropy loss while avoiding overfitting problem; during the training process, a model is saved after each round of training, and the model with the minimum loss on the validation set is taken as the optimal model. If the loss of the model in the next round of training is smaller, it is the optimal model.

[0021] Further, the specific operation of step 2 is:

[0022] Step 2.1: Data sample loading: using the deep learning framework PyTorch 2.0, inheriting the Dataset class, and reloading the __getitem__ method, classifying the 78-dimensional features obtained in step 1, and packaging them into batches with a size of 256 respectively as inputs for steps 2.2 and 2.3;

[0023] Step 2.2: Spatial feature channel module construction: constructing three different scales of different dilation rates of hollow convolution, with convolution kernel sizes of 3, 5, and 7, and dilation rates of 1, 2, and 3 respectively; capture information on different scales of network traffic, respectively output 256x64 size feature matrix, through splicing, become 256x192 size feature matrix, then through the maximum pooling layer and the linear layer for integration, form high-level feature representation, and fuse with the time feature;

[0024] The receptive field formula of the hollow convolution is as follows:

[0025] k' = k + (k-1)*(d-1)

[0026] In the formula, k' represents the equivalent size of the convolution kernel after the hollow convolution; k represents the convolution kernel size; d represents the dilation rate;

[0027] Step 2.3: Time channel module construction: use three-layer bidirectional gate recurrent unit, the number of hidden units in each layer is 128; then assign different weights to each time step in the sequence through self-attention mechanism; the relevant formula is as follows:

[0028] z t = σ(W z · [h t-1 , x t ] + b z )

[0029] r t = σ(W r · [h t-1 , x t ] + b r )

[0030]

[0031] In the formula, z t represents the update gate; r t represents the reset gate; h t represents the hidden state passed to the next moment; h z represents the candidate hidden state; σ represents the sigmoid function, which converts data to a value in the range of 0-1; W r represents the weight matrix of the update gate; W h represents the weight matrix of the reset gate; W t represents the weight matrix used to calculate the candidate hidden state; x z represents the input at the current moment t; b r represents the bias vector of the update gate, used to adjust the linear transformation result in the update gate calculation; b h represents the bias vector of the reset gate; b i represents the bias vector of the reset gate; Q represents the query vector obtained by transforming the current moment traffic data or its features; K represents the vector generated by linearly transforming the historical traffic data or its features; V represents the vector containing actual traffic information or its features; represents the scaling of the dimension of the K vector;

[0032] The output of step 2.2 and step 2.3 is used as the input of step 2.4;

[0033] Step 2.4: Model training: high-level features from the spatial channel and the temporal channel training set are spliced into a unified feature vector, two linear layers are stacked in sequence, used to combine and convert the spliced results of statistical features and time features, the first linear layer expands the feature space, the second linear layer reduces it to 256 dimensions, maps the fused 256-dimensional features to the output space, generates the probability distribution of each class, compares with the real label, then calculates the cross-entropy loss between the predicted result and the real label, when the loss is lower than the optimal model, the current model is saved as the optimal model, when the loss is higher than the optimal model, a new round of training is carried out, and the early stopping mechanism is used to stop training when the loss of the trained model no longer decreases;

[0034] Further, the calculation formula of the cross-entropy loss of the step 2 network traffic anomaly detection model DMCG training process is:

[0035]

[0036] Wherein, y i represents the label of sample i, the positive class is 1 and the negative class is 0, p i represents the probability that sample i is predicted as a positive class; L i represents the loss of the i-th sample.

[0037] Step 3, network traffic anomaly detection model DMCG performance evaluation; the macro average of precision, recall and F1 score is used as the evaluation index of the model to evaluate the performance of the model;

[0038] Further, the calculation formula of the step 3 network traffic anomaly detection model DMCG evaluation index is:

[0039]

[0040] Wherein, ACC represents the accuracy, P represents the precision, R represents the recall; TP (true positive) represents the number of abnormal traffic detected as abnormal; FP (false positive) represents the number of normal traffic detected as abnormal; FN (false negative) represents the number of abnormal traffic detected as normal; TN (true negative) represents the number of normal traffic detected as normal.

[0041] Step 4, network traffic detection: using CICFlowMeter tool to collect network traffic from real network environment, data cleaning, missing value processing, normalization, etc. are carried out on the collected traffic, and sent into the network traffic anomaly detection model for detection;

[0042] Further, the specific operation of step 4 is:

[0043] Step 4.1: first parse and feature extraction by step 1.1; then data cleaning, missing value processing, normalization processing by step 1.2 to obtain corresponding network traffic features, and classification of these network traffic features by step 1.3, classified as spatial features and temporal features;

[0044] Step 4.2: spatial feature and temporal feature extraction: first reshape the features obtained in step 4.1 by step 2.1, and pack them into a batch with a size of 256; then extract spatial features and temporal features by steps 2.2 and 2.3 respectively, and finally fuse the extracted features by step 2.4 to obtain the features as the input of step 4.3;

[0045] Step 4.3: anomaly detection: input the obtained features into a classifier composed of two linear layers and a softmax function to obtain the abnormal traffic prediction category of the feature sample.

[0046] Compared with the prior art, the present application has the following advantages:

[0047] (1) By classifying network traffic data features, the feature extraction is more targeted, the feature information is effectively extracted, and efficient feature representation of network traffic data is realized.

[0048] (2) By using multiple different kernel sizes and different dilation rates of the hole convolution, the spatial features of the network traffic data are extracted in multiple scales in the short-term mutation and long-term trend, and the features are more comprehensive. By combining self-attention and bidirectional GRU, the model can focus on the most relevant part of the sequence, enhance the capture of important time series features, and help identify abnormal behaviors that appear as time bias.

[0049] (3) A complex fusion network is proposed. It can process different feature information in parallel and capture global dependencies in order, which enables network security practitioners to more comprehensively and accurately identify abnormal traffic in complex network environments. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is the architecture diagram of network traffic anomaly detection in the present application;

[0051] Figure 2 is a detailed diagram of the spatial features extracted in the present application;

[0052] Figure 3 is a detailed diagram of the temporal features extracted in the present application;

[0053] Figure 4 is a parameter setting diagram of DMCG in the present application

[0054] Figure 5 Algorithm for training phase and execution phase of DMCG in the present application. DETAILED DESCRIPTION

[0055] For a more complete understanding of the present application, we will make a comprehensive and detailed description of it. However, the present application has multiple implementations and is not limited to the specific examples listed herein. The presentation of these examples aims to deepen the overall understanding of the disclosure of the present application.

[0056] As shown in the accompanying Figure 1 The network traffic anomaly detection model DMCG used by the present application is composed of 2 modules, which are spatial feature channel and time feature channel respectively. DMCG first divides all features of network traffic data into spatial features and time features. Spatial features are refined and expanded through multiple different scale cavity convolutions to capture spatial information within different distances. The outputs of multiple cavity convolution blocks are combined and fused with time features through a fully connected layer. Time features model the time sequence dependency relationship through BIGRU. The attention mechanism assigns different weights to each time step in the sequence to enhance the capture of important time sequence features. Then the fusion of the features of the two channels is completed through the fusion layer. Finally, the linear layer is mapped to the output space to generate the probability distribution of each category.

[0057] A dual-channel multi-scale network traffic anomaly detection method, the method comprising the following steps:

[0058] Step 1, data preprocessing; using CICFlowMeter to extract features from the collected pcap file, using the data processing function in the Python library to preprocess the collected network traffic data (data cleaning, missing value processing, normalization), obtaining the corresponding network traffic features, and classifying these traffic features to obtain spatial features and time features;

[0059] Further, the specific operation of step 1 is:

[0060] Step 1.1: packet analysis: network attack through fixed IP address at a certain time, capture network traffic data packets, get pcap file, parse pcap file through CICFlowMeter to get relevant feature information of network traffic data packets, such as: number of network traffic data packets in the outflow direction, number of network traffic data packets in the inflow direction, maximum size of network traffic data packets in the forward direction, minimum size of network traffic data packets in the forward direction, flow duration, flow byte rate, etc. At the same time, the IP address of the attacker is labeled to generate real labels of abnormal traffic and normal traffic as input for step 1.2;

[0061] Step 1.2: Data cleaning: missing value processing and normalization processing are performed on the feature information obtained in step 1.1 to obtain 78-dimensional features for training the network traffic anomaly detection model DMCG;

[0062] Step 1.3: Time-space feature division: the processed feature information is divided into spatial features and temporal features as inputs for step 2, as shown in FIG. 2A and FIG. 2B. Figure 2 As shown in FIG. 2A and FIG. 2B. Figure 3

[0063] Step 2: Construction and training of the network traffic anomaly detection model DMCG; important parameters of each part are shown in FIG. 3. Figure 4

[0064] Model construction: a network traffic anomaly detection model DMCG composed of a multi-scale dilated convolution module, a bidirectional gated recurrent unit module, and a self-attention module is built using PyTorch and Scikit-learn;

[0065] The multi-scale dilated convolution module is built using the Scikit-learn library, which includes a dilated convolution layer, a max pooling layer, and a linear layer;

[0066] The bidirectional gated recurrent unit module and the self-attention module are built based on the PyTorch framework;

[0067] Model training: the training rounds of the model are set in advance, and the corresponding spatial and temporal features are input into the corresponding channels for training;

[0068] Training process: the spatial and temporal features obtained in step 1 are divided into training and validation sets in proportion. The training set is input into the model, the multi-scale dilated convolution module is used to extract features from the spatial features, the extracted features are downsampled and dimensionally reduced by the max pooling layer, and the spatial features of different scales are aggregated. The temporal features are input into the bidirectional gated recurrent unit module to capture the temporal dependence and hierarchical structure information. Then, through the self-attention mechanism, different weights are assigned to each time step in the sequence to enhance the capture of important temporal features. The temporal features and spatial features are aggregated, and then passed through a classifier composed of two linear layers and a softmax function to generate the class probability of the sample. Then the predicted result and the true label are used to calculate the cross-entropy loss. When the loss is lower than the optimal model, the current model is saved as the optimal model. When the loss is higher than the optimal model, a new round of training is performed. The early stopping mechanism is used to stop training when the loss of the trained model no longer decreases, so that the model converges to the minimum cross-entropy loss while avoiding overfitting. During the training process, a model is saved after each round of training, and the model with the smallest loss on the validation set is considered as the optimal model. If the loss of the model in the next round of training is smaller, it is considered as the optimal model. ​​

[0069] Further, the specific operation of step 2 is:

[0070] Step 2.1: Data sample loading: using the deep learning framework PyTorch 2.0, inheriting the Dataset class, and reloading its __getitem__ method, classifying the 78-dimensional features obtained in step 1, and respectively packaging them into batches with a size of 256, as the input of steps 2.2 and 2.3;

[0071] Step 2.2: Spatial feature channel module construction: construct three different scales of different dilation rate hole convolutions, with convolution kernel sizes of 3, 5, and 7, and dilation rates of 1, 2, and 3, respectively; capture information at different scales of network traffic, respectively output 256x64 size feature matrix, through splicing, become 256x192 size feature matrix, then through the maximum pooling layer and linear layer for integration, form high-level feature representation, and fuse with time characteristics;

[0072] The receptive field formula of the hole convolution is as follows:

[0073] k' = k + (k-1)*(d-1)

[0074] In the formula, k' represents the equivalent size of the convolution kernel after the hole convolution; k represents the convolution kernel size; d represents the dilation rate;

[0075] Step 2.3: Time channel module construction: use three layers of bidirectional gated recurrent units, with 128 hidden units in each layer; then assign different weights to each time step in the sequence through the self-attention mechanism; the related formula is as follows:

[0076] z t = σ(W z ·[h t-1 ,x t ]+b z )

[0077] r t = σ(W r ·[h t-1 ,x t ]+b r )

[0078]

[0079] In the formula, z t represents the update gate; r t represents the reset gate; h t represents the hidden state passed to the next time; denotes the candidate hidden state; σ denotes the sigmoid function, which converts the data into a value in the range of 0-1; W z denotes the weight matrix of the update gate; W r denotes the weight matrix of the reset gate; W h denotes the weight matrix for calculating the candidate hidden state; x t denotes the input at the current time t; b z denotes the bias vector of the update gate, used to adjust the linear transformation result in the update gate calculation; b r denotes the bias vector of the reset gate; b h denotes the bias vector of the reset gate; Q denotes the query vector obtained by transforming the current time traffic data or its features; K denotes the vector generated by linearly transforming the historical traffic data or its features; V denotes the vector containing the actual traffic information or its features; denotes the scaling representing the dimension of the K vector;

[0080] The outputs of steps 2.2 and 2.3 are used as inputs of step 2.4;

[0081] Step 2.4: Model training: high-level features from the training set of the spatial channel and the temporal channel are spliced into a unified feature vector, two linear layers are stacked in sequence to combine and convert the spliced results of the statistical features and the temporal features, the first linear layer expands the feature space, and the second linear layer reduces it to 256 dimensions, maps the fused 256-dimensional features to the output space, generates the probability distribution of each class, compares it with the real label, then calculates the cross-entropy loss between the predicted result and the real label, saves the current model as the optimal model when the loss is lower than the optimal model, and when the loss is higher than the optimal model, a new round of training is performed, and the early stopping mechanism is used to stop training when the loss of the trained model no longer decreases;

[0082] Further, the calculation formula of the cross-entropy loss of the step 2 network traffic anomaly detection model DMCG training process is:

[0083]

[0084] wherein y i denotes the label of sample i, the positive class is 1, and the negative class is 0; p i denotes the probability that sample i is predicted as a positive class; L i denotes the loss of the i-th sample.

[0085] Step 3, performance evaluation of the network traffic anomaly detection model DMCG; the macro average of precision, recall, and F1 score is used as the evaluation index of the model to evaluate the performance of the model;

[0086] Further, the calculation formula of the step 3 network traffic anomaly detection model DMCG evaluation index is:

[0087]

[0088] Wherein, ACC represents accuracy, P represents precision, R represents recall; TP (true positive) represents the number of abnormal traffic detected as abnormal; FP (false positive) represents the number of normal traffic detected as abnormal; FN (false negative) represents the number of abnormal traffic detected as normal; TN (true negative) represents the number of normal traffic detected as normal, and the DMCG anomaly detection algorithm is as shown in the accompanying Figure 5

[0089] Step 4, network traffic detection: using CICFlowMeter tool to collect network traffic from real network environment, carrying out data cleaning, missing value processing, normalization and the like on the collected traffic, and sending into the network traffic anomaly detection model for detection;

[0090] Further, the specific operation of the step 4 is:

[0091] Step 4.1: first, analyzing and extracting features thereof through step 1.1; then, carrying out data cleaning, missing value processing, normalization processing through step 1.2 to obtain corresponding network traffic features, and classifying the network traffic features through step 1.3, the classification being spatial features and time features;

[0092] Step 4.2: spatial feature and time feature extraction: first, remodeling the features obtained in step 4.1 through step 2.1, and packing into a batch with a size of 256; then, extracting spatial features and time features through steps 2.2 and 2.3 respectively, and finally, carrying out feature fusion on the extracted features through step 2.4, and taking the obtained features as the input of step 4.3;

[0093] Step 4.3: anomaly detection: inputting the obtained features into a classifier composed of two linear layers and a softmax function to obtain the abnormal traffic prediction category of the feature sample.

[0094] In order to verify the effectiveness of the model DMCG, a large number of experiments are carried out for evaluating the DMCG. The used data set and data set division mode are as shown in Table 1. The evaluation results of the present application and other detection models are as shown in Table 2, and the recall, precision and F1 Score indexes on the data set are all better than most of the existing methods, proving the superiority and effectiveness of the DMCG model in traffic analysis.​

[0095] Table 1 dataset and dataset division method

[0096]

[0097] Table 2 comparison of evaluation results of the present application and other detection models

[0098]

[0099] The above shows and describes the main features and advantages of the present application, and it is obvious to those skilled in the art that the specific embodiments of the present application are not limited to the details of the above exemplary embodiments, and the inventive idea and design idea of the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application, and should be equivalent to the protection scope disclosed in the technical solutions of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application.

[0100] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

[0101] The contents not described in detail in the specification of the present application belong to the prior art known to those skilled in the art. Although the above describes the specific embodiments of the present application for the purpose of facilitating those skilled in the art to understand the present application, it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that all kinds of changes within the spirit and scope of the present application defined and determined by the appended claims are obvious, and all the inventions utilizing the concept of the present application are within the scope of protection.

Claims

1. A dual-channel multi-scale network traffic anomaly detection method, characterized by: The method comprises the following steps: Step 1: Data preprocessing: Use CICFlowMeter to extract features from the collected pcap files. Use the data processing functions in the Python library to preprocess the collected network traffic data to obtain the corresponding network traffic features. These traffic features are then classified to obtain spatial and temporal features. Step 2: Construction and training of the network traffic anomaly detection model DMCG; Model construction: Use PyTorch and Scikit-learn to build a network traffic anomaly detection model DMCG, which consists of a multi-scale dilated convolution module, a bidirectional gated recurrent unit module, and a self-attention module; The multi-scale dilated convolution module is built using the Scikit-learn library, which includes dilated convolution layers, maximum pooling layers, and linear layers. The bidirectional gated recurrent unit module and the self-attention module are built based on the PyTorch framework; Model training: Set the model training rounds in advance and input the corresponding spatial and temporal features into the corresponding channels for training; Training process: The spatial features and temporal features obtained in step 1 are divided into training sets and validation sets in proportion, the training set is input into the model, and the spatial features are extracted using a multi-scale dilated convolution module. The extracted features are downsampled and the data dimension is reduced through the maximum pooling layer, and spatial features of different scales are aggregated; the temporal features are input into the bidirectional gated recurrent unit module to capture the temporal dependency and hierarchical structure information therein, and then a different weight is assigned to each time step in the sequence through the self-attention mechanism to enhance the capture of important temporal features; the temporal features are aggregated with the spatial features, and then a classifier consisting of two linear layers and a softmax function is used to generate the category probability of the sample. The cross entropy loss is then calculated between the predicted results and the true labels. When the loss is lower than the optimal model, the current model is saved as the optimal model. When the loss is higher than the optimal model, a new round of training is performed, and the early stopping mechanism is used to stop training when the loss of the trained model no longer decreases; Step 3: Evaluate the performance of the DMCG network traffic anomaly detection model. Use the macro-average of precision, recall, and F1 score as evaluation metrics to assess the model's performance. Step 4: Detect network traffic; use the CICFlowMeter tool to collect network traffic from a real network environment, clean the collected traffic, handle missing values, normalize it, and then send it to the network traffic anomaly detection model for detection.

2. The dual-channel multi-scale network traffic anomaly detection method according to claim 1, characterized in that: The specific operations of step 1 are: Step 1.1: Packet parsing: At a certain moment, a network attack is carried out from a fixed IP address, and network traffic packets are captured to obtain a pcap file. The pcap file is parsed using CICFlowMeter to obtain relevant characteristic information of the network traffic packets. At the same time, the attacker's IP address is annotated to generate the actual labels of abnormal and normal traffic, which serve as the input for step 1.

2. Step 1.2: Data cleaning: Perform missing value processing and normalization on the feature information obtained in step 1.1 to obtain 78-dimensional features for training the network traffic anomaly detection model DMCG; Step 1.3: Split the time and space features: Split the processed feature information into spatial features and temporal features as the input of step 2.

3. The dual-channel multi-scale network traffic anomaly detection method according to claim 2, characterized in that: The specific operations of step 2 are: Step 2.1: Loading data samples: Using the deep learning framework PyTorch 2.0, inherit the Dataset class, overload its __getitem__ method, classify the 78-dimensional features obtained in step 1, and package them into batches of 256 as input for steps 2.2 and 2.

3. Step 2.2: Spatial feature channel module construction: Construct three dilated convolutions at different scales and dilation rates, with kernel sizes of 3, 5, and 7, and dilation rates of 1, 2, and 3, respectively. Capture information at different scales of network traffic and output feature matrices of 256×64. After concatenation, they are transformed into feature matrices of 256×192. These are then integrated through a maximum pooling layer and a linear layer to form a high-level feature representation, which is then fused with the temporal features. The receptive field formula of the dilated convolution is as follows: k'=k+(k-1)*(d-1) Where k' represents the equivalent size of the convolution kernel after the dilated convolution; k represents the convolution kernel size; d represents the dilation rate; Step 2.3: Time channel module construction: Use three layers of bidirectional gated recurrent units, with 128 hidden units in each layer; then use the self-attention mechanism to assign different weights to each time step in the sequence; the relevant formula is as follows: z t =σ(W z ·[h t-1 ,x t ]+b z ) r t =σ(W r ·[h t-1 ,x t ]+b r ) Where z t represents the update gate; r t Represents the reset gate; h t Represents the hidden state passed to the next moment; represents the candidate hidden state; σ represents the sigmoid function, which converts the data into a value in the range of 0-1; W z Represents the weight matrix of the update gate; W r Represents the weight matrix of the reset gate; W h represents the weight matrix used to calculate the candidate hidden state; x t represents the input at the current time t; b z Represents the bias vector of the update gate, which is used to adjust the linear transformation result in the update gate calculation; b r represents the bias vector of the reset gate; b h represents the bias vector for resetting the gate; Q represents the query vector obtained by transforming the current traffic data or its characteristics; K represents the vector generated by linear transformation of historical traffic data or its characteristics; V represents the vector containing actual traffic information or its characteristics; represents the scaling of the dimension representing the K vector; The outputs of steps 2.2 and 2.3 serve as the inputs of step 2.4; Step 2.4: Model training: The high-level features of the training sets from the spatial channel and the temporal channel are concatenated into a unified feature vector. Two linear layers are stacked in sequence to combine and transform the concatenation results of statistical features and temporal features. The first linear layer expands the feature space, and the second linear layer reduces its dimension to 256 dimensions. The fused 256-dimensional features are mapped to the output space, and the probability distribution of each category is generated and compared with the true label. The cross entropy loss is then calculated between the predicted results and the true label. When the loss is lower than the optimal model, the current model is saved as the optimal model. When the loss is higher than the optimal model, a new round of training is performed. The early stopping mechanism is used to stop training when the loss of the trained model no longer decreases.

4. The dual-channel multi-scale network traffic anomaly detection method according to claim 3, characterized in that: The calculation formula for the cross entropy loss in the DMCG training process of the network traffic anomaly detection model in step 2 is: Among them, y i Indicates the label of sample i, the positive class is 1, the negative class is 0, p i Indicates the probability that sample i is predicted to be positive; L i represents the loss of the i-th sample.

5. The dual-channel multi-scale network traffic anomaly detection method according to claim 4, characterized in that: The calculation formula of the DMCG evaluation index of the network traffic anomaly detection model in step 3 is: Among them, ACC represents accuracy, P represents precision, and R represents recall; TP represents the number of abnormal traffic flows detected as abnormal; FP represents the number of normal traffic flows detected as abnormal; FN represents the number of abnormal traffic flows detected as normal; and TN represents the number of normal traffic flows detected as normal.

6. The dual-channel multi-scale network traffic anomaly detection method according to claim 5, characterized in that: The specific operations of step 4 are: Step 4.1: First, perform analysis and feature extraction according to step 1.

1. Then, perform data cleaning, missing value processing, and normalization according to step 1.2 to obtain the corresponding network traffic features. Finally, perform classification according to step 1.3 to classify these network traffic features into spatial features and temporal features. Step 4.2: Extract spatial and temporal features. First, reshape the features obtained in step 4.1 through step 2.1 and package them into a batch of 256. Then, extract spatial and temporal features through steps 2.2 and 2.3 respectively. Finally, fuse the extracted features through step 2.4 and use the obtained features as input for step 4.

3. Step 4.3: Anomaly detection: Input the obtained features into a classifier consisting of two linear layers and a softmax function to obtain the abnormal traffic prediction category of the feature sample.

Images