Multi-scale fusion fast network traffic anomaly detection method based on comparative learning

Abstract

The invention discloses a multi-scale fusion fast network traffic anomaly detection method based on comparative learning, and the method comprises the steps: carrying out the preprocessing of traffic data, extracting features, converting the traffic data into a traffic image, carrying out the normalization processing of the traffic image, and dividing the traffic image into small blocks; constructing a multi-granularity multi-layer perceptron model, inputting traffic images after normal traffic data and abnormal traffic data division into the multi-granularity multi-layer perceptron model, and extracting feature information; defining a comparative learning loss function, training the multi-granularity multi-layer perceptron model by comparing the characteristics of the normal traffic and the abnormal traffic, and iteratively adjusting the model parameters of the multi-granularity multi-layer perceptron model to reduce the comparative learning loss function value until convergence; and inputting the preprocessed traffic image into the trained multi-granularity multi-layer perceptron model, wherein the multi-granularity multi-layer perceptron model can distinguish normal traffic from abnormal traffic.

Description

Technical Field

[0001] The invention relates to a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning, and belongs to the field of network security and traffic monitoring. Background Art

[0002] With the rapid development of information technology, the Internet has become the infrastructure of modern society. Network traffic anomaly detection plays a vital role in ensuring network security and stable operation. The goal of network traffic anomaly detection is to timely discover and identify potential security threats, such as DDoS attacks, malware propagation, unauthorized access, and data leakage, by monitoring and analyzing network traffic. However, with the increasing complexity of the network environment, traditional anomaly detection methods have problems such as low detection accuracy, high computational complexity, and slow response speed when facing high-dimensional and heterogeneous network traffic data. In the modern network environment, traffic data presents the characteristics of high dimensionality and heterogeneity. Network traffic not only includes traditional TCP / IP packets, but also involves data from various application layer protocols and services, such as HTTP, HTTPS, DNS, FTP, etc. In addition, with the popularization of IoT devices and mobile Internet, network traffic also contains a large amount of data from smart devices and mobile terminals. These data come from different sources, have complex structures, and are of various types, which brings huge challenges to anomaly detection. Traditional statistical methods and rule-based detection methods often require a lot of feature engineering and preprocessing when dealing with high-dimensional and heterogeneous data, which not only increases the computational complexity, but also easily misses important abnormal features. In recent years, machine learning and deep learning technologies have made significant progress in the field of anomaly detection, improving detection accuracy and efficiency through automated feature extraction and learning. However, these methods still have certain limitations when facing complex network environments. The relationship structure in network traffic data is complex and changeable, and there may be significant correlations between different types of traffic. Knowledge graph technology provides an intuitive and effective way to understand and manage complex network traffic data by systematically representing diverse elements and their relationships. The nodes and edges in the knowledge graph can represent various entities and their associations in network traffic, allowing the system to flexibly adapt to changes in relationships and provide a clear and unified view, which helps to improve the accuracy and comprehensiveness of anomaly detection. In actual network environments, the quality of data directly affects the effect of anomaly detection. Network traffic data often contains a large amount of noise and anomalies, which may come from fluctuations in the network environment, equipment failures, or malicious attacks. If effective data cleaning is not performed, the noise data will seriously interfere with the training and detection process of the model, resulting in high false positive and false negative rates. Data cleaning technology improves the accuracy and consistency of data by filtering and correcting noise data. Modern machine learning algorithms can automatically identify and process anomalies in data, improving the efficiency and effectiveness of data cleaning. By adopting data-driven anomaly detection algorithms, the system can detect and process new noise and anomalies in real time, adapt to the dynamic changes of the network environment, and improve overall detection performance. The characteristics of network traffic may show significant dynamic changes in different time periods and environments.Effective anomaly detection methods need to be able to flexibly adjust and optimize the feature extraction process to accurately reflect the real-time status of traffic. Traditional feature extraction methods are often based on fixed rules and templates, which are difficult to adapt to the dynamic changes of traffic characteristics. To meet this challenge, modern anomaly detection methods use adaptive feature extraction and optimization techniques. By analyzing and counting traffic features in real time, the attribute constraints of nodes, such as traffic size, delay, bandwidth, etc., are dynamically adjusted to ensure that the system can accurately characterize the dynamic characteristics of network traffic. The feature extraction process combines statistical tools and machine learning algorithms to improve the response speed and adaptability of the system while ensuring high detection accuracy. Contrastive learning is a method of training models by comparing the similarities and differences between samples. Compared with traditional supervised learning, contrastive learning has significant advantages in dealing with problems with large differences between classes. By designing a contrastive learning loss function, the difference between normal and abnormal traffic can be effectively amplified, and the model's ability to distinguish can be enhanced. In network traffic anomaly detection, the contrastive learning strategy can help the model better understand the feature differences between normal and abnormal traffic, and improve the accuracy and robustness of detection. Through contrastive learning, the system can not only improve the generalization ability of the model during the training phase, but also maintain efficient detection performance in complex network environments. In the field of network security, fast response and efficient detection are important requirements for anomaly detection systems. Existing anomaly detection methods (such as Transformer-based network traffic classification models) often require long training and reasoning times due to their high computational complexity, making it difficult to meet the needs of real-time detection. In order to achieve efficient anomaly detection, it is necessary to improve the training and reasoning speed of the system by optimizing the algorithm and network structure. Summary of the invention

[0003] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and provide a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning, which realizes high-precision and fast network traffic anomaly detection by designing a multi-scale feature extraction mechanism, feature extraction and block processing, contrastive learning training, and efficient training and deployment, and can obtain better detection effect.

[0004] The technical solution of the present invention is: a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning, wherein:

[0005] Collect and preprocess traffic data, extract features and convert them into traffic images, normalize the traffic images and divide them into small blocks;

[0006] Construct a multi-granularity multi-layer perceptron model, input the traffic image divided by normal traffic data and abnormal traffic data into it, and extract feature information;

[0007] Define the contrastive learning loss function, train the multi-granularity multi-layer perceptron model by comparing the characteristic information of normal traffic and abnormal traffic, and iteratively adjust the model parameters of the multi-granularity multi-layer perceptron model to make the contrastive learning loss function value smaller until convergence;

[0008] The preprocessed traffic image is input into the trained multi-granularity multi-layer perceptron model, and the multi-granularity multi-layer perceptron model can distinguish between normal traffic and abnormal traffic.

[0009] Preferably, when preprocessing the flow data:

[0010] Collect traffic data at each key node and device, and record the obtained network traffic data as x∈R N×D , where N is the number of collected data and D is the feature dimension; if there are fewer missing values ​​in the data, the corresponding missing values ​​are discarded.

[0011] Preferably, after extracting features, converting them into flow images:

[0012] Bit-level, packet-level and flow-level features are extracted from traffic data and converted into images to form multi-channel traffic images.

[0013] Preferably, the converted flow image data is normalized to the maximum and minimum values, that is:

[0014]

[0015] Among them, x new is the normalized traffic image data, x min is the minimum value of the flow image data, x max is the maximum value of the flow image data.

[0016] Preferably, the traffic image is divided into several small blocks, each of which is represented by X∈R S×C , assuming that the size of the input image is (H, W) and the size of each small block is (P, P), then the number of small blocks is S = HW / P 2 , all small blocks are linearly projected through the same projection matrix.

[0017] Preferably, the multi-granularity multi-layer perceptron model is composed of multiple multi-granularity perceptron modules of the same size, each module including a token mixing fully connected layer and a channel mixing fully connected layer, wherein:

[0018] The token mixing fully connected layer extracts the time series features of traffic at different times. Each token mixing fully connected layer acts on the column direction of the input;

[0019] The channel mixing fully connected layer is used to extract channel features of different channel flows, acting in the row direction of the input;

[0020] The multi-granularity perceptron module composed of the above two fully connected layers can simultaneously extract the correlation of traffic data at different times and in different channels, and achieve a fusion effect by continuously stacking fully connected layers in the timing direction and the channel direction.

[0021] Preferably, the time series features of traffic at different times are extracted through a token mixing fully connected layer, specifically:

[0022] U *i =X *i +W2σ(W1 LayerNorm(X *i ))

[0023] In the above formula, W1 and W2 represent the trainable parameters in the two token mixing fully connected layers, X *i represents the i-th column of the input to the token mixing fully connected layer, U *i It represents the output of the ith column of the input through the token mixing fully connected layer. LayerNorm represents the layer normalization unit, which is used to accelerate training convergence and avoid gradient vanishing and gradient exploding problems. σ represents the element-by-element nonlinear activation function.

[0024] Preferably, the channel mixing fully connected layer extracts channel features of different channel flows, specifically:

[0025] Y j* =X j* +W4σ(W3 LayerNorm(U j* ))

[0026] Where W3 and W4 represent the trainable parameters in the channel mixing fully connected layer, X j* represents the jth row of the input to the token mixing fully connected layer, U j* Y represents the jth row of the output features of the token mixing fully connected layer, that is, the jth row of the input of the channel mixing fully connected layer; j* It represents the output of the j-th row of the input of this layer through the channel mixing fully connected layer, LayerNorm represents the layer normalization unit, and σ represents the element-by-element nonlinear activation function.

[0027] Preferably, the contrastive learning loss function is:

[0028]

[0029] Among them, sim represents the metric for evaluating the similarity between features, τ represents the temperature hyperparameter, which is used to control the smoothness of the distribution; z i ,z j ,z k is the fusion feature extracted based on the input flow image, where z i and zj The positive sample pairs are fused feature pairs extracted from the same type of traffic images. The same type of traffic images refers to traffic images with the same normal traffic or the same abnormal traffic. i ,z k The negative sample pairs are represented by fused feature pairs extracted based on traffic images of different types; N represents the batch size of traffic images input into the model each time during training.

[0030] Preferably, when training a multi-granularity multi-layer perceptron model using the designed contrastive learning loss function:

[0031] The ADAM optimizer is used for reverse gradient propagation. During the training process, iterative optimization is performed to adjust the model parameters based on the product of the momentum term and the learning rate in ADAM. The learning rate is adjusted using an exponential decay strategy to reduce the value of the contrastive learning loss function until convergence. At this time, the features obtained by the normal traffic through the model are very similar to those obtained by the abnormal traffic through the model, and the model can effectively distinguish between normal traffic and abnormal traffic.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The present invention significantly improves the accuracy and speed of anomaly detection through multi-scale feature extraction and contrastive learning strategies. Through the multi-scale feature extraction mechanism, the details and abstract features in the traffic data can be fully captured, and the sensitivity of the model to anomalies is improved. The channel fully connected layer and the token mixed fully connected layer are adopted, and only a small number of parameters are introduced to achieve efficient feature extraction and reduce the computational complexity. The contrastive learning loss function is designed to enhance the model's ability to distinguish between normal traffic and abnormal traffic, and improve the detection accuracy. The algorithm and network structure are optimized, the training and reasoning time are significantly shortened, and efficient network traffic anomaly detection is achieved. The proposed method has certain reference significance for the classification and detection of network traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the pretreatment process of the present invention;

[0035] Figure 2 Schematic diagram of the multi-granularity perceptron module of the present invention. DETAILED DESCRIPTION

[0036] A multi-scale fusion fast network traffic anomaly detection method based on contrastive learning, the specific process is as follows:

[0037] The first step is to collect network traffic data and preprocess the data, including filling missing values, extracting features of different scales, converting them into traffic images, and normalizing them.

[0038] In the second step, channels and tokens are used to model the correlation of data in different channels and at different times of the traffic image in a multi-granularity multi-layer perceptron model. In addition, compared with the existing Transformer-based model, this structure avoids the high computational complexity caused by the complex structure and can achieve better modeling results with fewer parameters.

[0039] In the third step, the feature differences between normal traffic and abnormal traffic are amplified by combining the designed contrast loss to conduct effective model training and solution. During testing, the features are directly used to calculate the similarity with the existing category features for classification.

[0040] 1. Step 1 of the present invention is specifically:

[0041] (1-1) Figure 1 As shown, firstly, traffic data is collected in each key node and device. Assume that the acquired network traffic data is x∈R N×D , where N is the number of collected data, D is the feature dimension, and if there are fewer missing values ​​in the data, the corresponding missing values ​​are discarded.

[0042] (1-2) Extract traffic features at multiple scales, such as bit level, packet level and flow level, from the original network traffic data and convert it into a multi-channel traffic image so as to extract multi-scale features using image processing technology. By mapping the traffic data into the image space, the details and abstract features in the data can be captured more intuitively.

[0043] The bit-level, packet-level and stream-level feature extraction methods are as follows: First, the original data is divided into different streams based on the IP address, port number, and protocol type; then the packet header is removed from the data, the port number is set to 0, the IP address is set to a random value, and the adjacent packets in the data stream are converted into images. However, when converting to images, they are constructed as follows:

[0044] (a) Bit-level traffic features: Each row of the traffic image contains only one type of network byte, which can be divided into two types: packet header and payload, thereby distinguishing different types of byte information and providing a basic structure for subsequent network-based feature extraction;

[0045] (b) Packet-level traffic features: Each data packet consists of two parts: a header and a payload. The two parts of data are converted into a matrix to integrate the information of the header and payload in the packet and represent the characteristics of the traffic at the packet level;

[0046] (c) Flow-level traffic features: Since a flow is composed of ordered data packets, M adjacent packet-level matrices are concatenated to integrate the information of multiple packets in the flow and represent the traffic features at the flow level.

[0047] The traffic image obtained based on the above construction method contains feature information of different scales in the traffic data. This data construction method lays the foundation for the subsequent multi-scale feature extraction of traffic.

[0048] (1-3) Since the network traffic characteristics are complex and the numerical ranges of different characteristics vary greatly, the converted traffic image data is normalized to the maximum and minimum values, that is,

[0049]

[0050] Among them, x new is the normalized traffic image data, x min is the minimum value of the flow image data, x max is the maximum value of the flow image data.

[0051] 2. Step 2 of the present invention is specifically as follows:

[0052] (1) Figure 1 As shown in the figure, we first construct the input required for model training based on the traffic image and divide the traffic image into several small blocks, where each small block is represented by X∈R S×C , assuming that the size of the input image is (H, W) and the size of each small block is (P, P), then the number of small blocks is S = HW / P 2 All small blocks are linearly projected through the same projection matrix to capture the features of different regions. Each small block can be independently feature extracted to ensure that the model can focus on the local details of the flow image.

[0053] (2) Figure 2 As shown in the figure, a multi-granularity multi-layer perceptron model is further constructed. The model is composed of multiple multi-granularity perceptron modules of the same size connected in series (generally 3 to 5), and each multi-granularity perceptron module is composed of a token mixing fully connected layer and a channel mixing fully connected layer; the token mixing fully connected layer and the channel mixing fully connected layer have similar structures, both of which are composed of two multi-layer perceptron (MLP, Multilayer Perceptron) blocks.

[0054] First, token-mixing MLPs are applied to extract the features of traffic at different times. Specifically, each MLP module consists of two fully connected layers and a nonlinear activation layer. The token-mixing MLP acts on the columns of the layer input. The token-mixing MLPs extracts the time series features to improve the richness of feature representation and detection accuracy:

[0055] U *i =X *i +W2σ(W1 LayerNorm(X *i )).

[0056] In the above formula, W1 and W2 represent the trainable parameters in the two fully connected layers, X *i represents the i-th column of the input to the token mixing fully connected layer, U *i It represents the output of the i-th column of the input through the token mixing fully connected layer. LayerNorm represents the layer normalization unit, which is used to accelerate the training convergence and avoid the gradient vanishing and gradient exploding problems. σ represents the element-by-element nonlinear activation function. The present invention uses GELU.

[0057] Secondly, channel-mixing MLPs are applied to the output of the token-mixing MLPs to extract features of different channels of the traffic image. This process acts on the rows of the channel-mixing MLPs input to achieve feature fusion between the data of each channel and capture more information; feature fusion here refers to the continuous series stacking of token-mixing MLPs and channel-mixing MLPs to continuously extract time- and channel-wise features to achieve the effect of feature fusion. The jth row of the channel-mixing MLPs input can be expressed as:

[0058] Y j* =X j* +W4σ(W3 LayerNorm(U j* ))

[0059] Where W3 and W4 represent the trainable parameters in the channel mixing fully connected layer, U j* Y represents the jth row of the output feature of the token mixing fully connected layer (which is also the input of the channel mixing fully connected layer); j* Represents the output of the j-th row of the input of this layer through the channel-mixing fully connected layer.

[0060] Compared with the existing Transformer-based network traffic classification model, the model parameters of this scheme only require 15.26% of it. In addition, this method only uses a simple multi-layer perceptron module to simultaneously extract the characteristics of network traffic data at different times and in different channels; ultimately achieving a better traffic anomaly detection effect.

[0061] The output of the multi-granularity multi-layer perceptron model is the output of the last multi-granularity perceptron module, which is used for subsequent training using the contrastive learning loss function.

[0062] 3. Step 3 of the present invention is specifically as follows:

[0063] (1) First, design a contrastive learning loss function to amplify the difference between normal and abnormal traffic by comparing their features. The loss function is designed based on infoNCE loss. The input required in the loss is the output obtained based on the model in the second step above. The contrastive learning loss function L infoNCEfor:

[0064]

[0065] Among them, z i ,z j ,z k is the fusion feature extracted based on the model in the second step and the input flow image in the first step, where z i and z j It constitutes a positive sample pair, which represents the fusion feature pair extracted based on the same type of traffic image (i.e., the same normal traffic or the same abnormal traffic); N represents the batch size of the traffic image input into the model each time during training, and z i ,z k It constitutes a negative sample pair, which represents a fusion feature pair extracted based on different types of traffic images; sim represents a measure of similarity between evaluation features, such as cosine similarity: τ is a temperature hyperparameter that controls the smoothness of the distribution. The goal of contrastive learning is achieved by maximizing the similarity between pairs of positive samples (the numerator) and normalizing their similarity with all other samples (including negative samples) (the denominator).

[0066] (2) Use the designed contrastive learning loss function to train the model. Use the ADAM optimizer for reverse gradient propagation and train the entire data set for 100 rounds. During the training process, the learning rate is adjusted using an exponential decay strategy. Through iterative optimization, the model parameters are adjusted based on the product of the momentum term and the learning rate in ADAM, so that the infoNCE loss (the value of the contrastive learning loss function L infoNCE ) becomes smaller until convergence. At this point, the features obtained by the normal traffic through the model are very similar to the features obtained by the abnormal traffic through the model, so the model can effectively distinguish normal traffic from abnormal traffic. During the training process, monitor the performance of the model to ensure its robustness in complex network environments.

[0067] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.

Claims

1. A multi-scale fusion fast network traffic anomaly detection method based on contrastive learning, characterized by include: Collect and preprocess traffic data, extract features and convert them into traffic images, normalize the traffic images and divide them into small blocks; Construct a multi-granularity multi-layer perceptron model, input the traffic image divided by normal traffic data and abnormal traffic data into it, and extract feature information; Define the contrastive learning loss function, train the multi-granularity multi-layer perceptron model by comparing the characteristic information of normal traffic and abnormal traffic, and iteratively adjust the model parameters of the multi-granularity multi-layer perceptron model to make the contrastive learning loss function value smaller until convergence; The preprocessed traffic image is input into the trained multi-granularity multi-layer perceptron model, and the multi-granularity multi-layer perceptron model can distinguish between normal traffic and abnormal traffic.

2. According to claim 1, a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning is characterized by: When preprocessing traffic data: Collect traffic data at each key node and device, and record the obtained network traffic data as x∈R N×D , where N is the number of collected data and D is the feature dimension; if there are fewer missing values ​​in the data, the corresponding missing values ​​are discarded.

3. According to claim 1, a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning is characterized by: When converting it into a flow image after extracting features: Bit-level, packet-level and flow-level features are extracted from traffic data and converted into images to form multi-channel traffic images.

4. According to claim 2, a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning is characterized by: The converted flow image data is normalized to the maximum and minimum values, namely: Among them, x new is the normalized traffic image data, x min is the minimum value of the flow image data, x max is the maximum value of the flow image data.

5. According to claim 1, a multi-scale fusion fast network traffic anomaly detection method based on contrastive learning is characterized by: Divide the traffic image into several small blocks, each of which is represented by X∈R S×C , assuming that the size of the input image is (H, W) and the size of each small block is (P, P), then the number of small blocks is S = HW / P 2 , all small blocks are linearly projected through the same projection matrix.

6. The method for fast network traffic anomaly detection based on contrastive learning and multi-scale fusion according to claim 1 is characterized by: The multi-granularity multi-layer perceptron model consists of multiple multi-granularity perceptron modules of the same size, each of which includes a token-mixing fully connected layer and a channel-mixing fully connected layer, where: The token mixing fully connected layer extracts the time series features of traffic at different times. Each token mixing fully connected layer acts on the column direction of the input; The channel mixing fully connected layer is used to extract channel features of different channel flows, acting in the row direction of the input; The multi-granularity perceptron module composed of the above two fully connected layers can simultaneously extract the correlation of traffic data at different times and in different channels, and achieve a fusion effect by continuously stacking fully connected layers in the timing direction and the channel direction.

7. The method for fast network traffic anomaly detection based on contrastive learning and multi-scale fusion according to claim 6 is characterized by: The time series features of traffic at different times are extracted through the token mixing fully connected layer, specifically: U *i =X *i +W2σ(W1LayerNorm(X *i )) In the above formula, W1 and W2 represent the trainable parameters in the two token mixing fully connected layers, X *i represents the i-th column of the input to the token mixing fully connected layer, U *i It represents the output of the ith column of the input through the token mixing fully connected layer. LayerNorm represents the layer normalization unit, which is used to accelerate training convergence and avoid gradient vanishing and gradient exploding problems. σ represents the element-by-element nonlinear activation function.

8. The method for fast network traffic anomaly detection based on contrastive learning and multi-scale fusion according to claim 6 is characterized by: The channel mixing fully connected layer extracts the channel features of different channel flows, specifically: Y j* =X j* +W4σ(W3LayerNorm(U j* )) Where W3 and W4 represent the trainable parameters in the channel mixing fully connected layer, X j* represents the jth row of the input to the token mixing fully connected layer, U j* Y represents the jth row of the output features of the token mixing fully connected layer, that is, the jth row of the input of the channel mixing fully connected layer; j* It represents the output of the j-th row of the input of this layer through the channel mixing fully connected layer, LayerNorm represents the layer normalization unit, and σ represents the element-by-element nonlinear activation function.

9. The method for fast network traffic anomaly detection based on contrastive learning and multi-scale fusion according to claim 1 is characterized by: The contrastive learning loss function is: Among them, sim represents the metric for evaluating the similarity between features, τ represents the temperature hyperparameter, which is used to control the smoothness of the distribution; z i ,z j ,z k is the fusion feature extracted based on the input flow image, where z i and z j The positive sample pairs are fused feature pairs extracted from the same type of traffic images. The same type of traffic images refers to traffic images with the same normal traffic or the same abnormal traffic. i ,z k The negative sample pairs are represented by fused feature pairs extracted based on traffic images of different types; N represents the batch size of traffic images input into the model each time during training.

10. The method for fast network traffic anomaly detection based on contrastive learning and multi-scale fusion according to claim 1 is characterized by: When training a multi-granularity multilayer perceptron model using the designed contrastive learning loss function: The ADAM optimizer is used for reverse gradient propagation. During the training process, iterative optimization is performed to adjust the model parameters based on the product of the momentum term and the learning rate in ADAM. The learning rate is adjusted using an exponential decay strategy to reduce the value of the contrastive learning loss function until convergence. At this time, the features obtained by the normal traffic through the model are very similar to those obtained by the abnormal traffic through the model, and the model can effectively distinguish between normal traffic and abnormal traffic.

Images