A network traffic classification method based on attention mechanism

Abstract

The present invention relates to the field of network traffic classification, and in particular to a network traffic classification method based on an attention mechanism. Aiming at the characterization problem of different important features, this paper proposes a new classification model, which first pre-processes the original network traffic, then extracts spatial features through a CNN layer, extracts temporal features from an LSTM layer, then fuses the extracted multi-scale features, and uses an attention mechanism to extract multi-modal network traffic significant features to improve the output quality of dynamic structures and improve feature representation capabilities. Through the design of the present invention, the classification accuracy can be ultimately improved.

Description

Technical Field

[0001] The present invention relates to the field of network traffic classification, and in particular to a network traffic classification method based on an attention mechanism. Background Art

[0002] In recent years, with the rapid development of network technology and the sharp increase in network traffic, it is necessary to develop a fast and accurate network traffic automatic classification method. In order to better manage network traffic, optimize network quality, understand real-time network traffic distribution, and prevent network attacks, a better network traffic classification method is needed.

[0003] At the same time, due to the widespread application of network encryption algorithms, existing network traffic identification and classification methods face huge challenges. The obstacle to network traffic identification and classification is encryption. Network traffic encryption is a key technology for protecting the privacy and security of Internet users and a key technology for various privacy enhancement tools. For example, Tor, originally known as the Onion Router, is the most widely used anonymous communication software with low latency and high quality. Tor forwards traffic through three nodes to achieve anonymity. Since the three forwarding nodes are distributed in different parts of the world, the anonymous communication system provides anonymous services by transmitting TCP data streams through channels, making it difficult to track network censorship.

[0004] There are mainly the following methods in the field of network traffic classification: based on port matching, deep packet inspection, behavioral characteristics and machine learning. However, with the increasing complexity of network applications and the widespread use of encryption protocols, traditional traffic classification methods can no longer accurately identify traffic characteristics. At present, most of the improved methods of deep learning are achieved by increasing the width and depth of the network. Although the classification accuracy of network traffic can be improved, a large number of parameters need to be calculated, which increases the computational complexity of the algorithm. At the same time, there are also problems such as overfitting and low training efficiency, which are not suitable for the current application scenarios of network traffic classification and identification.

[0005] In order to understand the development status of the prior art, the existing patents and documents were searched, compared and analyzed, and the following technical information with high relevance to the present invention was screened out:

[0006] Patent Solution 1: CN111147396A A method for classifying encrypted traffic based on sequence features, providing a method for classifying encrypted traffic based on sequence features, by preprocessing the original encrypted traffic data; using the preprocessed data to learn the local features of data grouping using a one-dimensional convolutional network based on an attention mechanism; using the output of the local features to learn the global features of data grouping using a long short-term memory network. The present invention does not require manual feature extraction, and on this basis, it can ensure the original sequence features of the encrypted traffic data and assign them to specific applications; it solves the serious gradient vanishing and gradient explosion problems of traditional recurrent neural networks; it effectively solves the problem of different packet lengths of two different transport layer protocols, and filters the interference information of the network layer and the data link layer; it solves the prediction problem of long sequence data well. Disadvantages: The scheme aims to classify network traffic using a method based on sequence features. The local features of data grouping are learned by a one-dimensional convolutional network based on an attention mechanism, and the global features of data grouping are learned by a long short-term memory network using the output of step 2. However, traffic has multiple characteristics of different granularities. The characteristic representation under different modes needs to be considered in traffic feature extraction. The network architecture in this solution is a serial structure, which does not consider feature processing at multiple granularities and is not stable in complex application scenarios.

[0007] Patent Solution 2: CN111209933A A network traffic classification method and device based on a neural network and an attention mechanism discloses a network traffic classification method and device based on a neural network and an attention mechanism. The method includes: performing one-hot encoding on n data packets of the same network flow based on bytes to obtain n one-hot encoding matrices; extracting features from the n one-hot encoding matrices through a neural network to obtain n feature encoding vectors; using the data packet attention mechanism to strengthen the attention of the feature encoding vectors corresponding to important data packets to obtain the data packet attention encoding matrix of the network flow; and passing the data packet attention encoding matrix through a fully connected layer to obtain the classification result of the network flow. The present invention can be used to identify and classify network traffic. Disadvantages: This solution uses a neural network to extract features, and then uses an attention mechanism to strengthen attention to important features. However, network traffic often has diversified characteristics, and the feature extraction method in this solution does not take into account the comprehensive multi-scale features in the spatial dimension and the time dimension. At the same time, when one-hot encoding is used to encode data packets in the same flow, when many categories appear, the dimension of the feature matrix obtained will be very large, which may lead to parallelism and multicollinearity problems.

[0008] There are several main methods in the field of network traffic classification: based on port matching, deep packet inspection, behavioral features, and machine learning. However, with the increasing complexity of network applications and the widespread use of encryption protocols, traditional traffic classification methods can no longer accurately identify traffic features. In response to the problem of low accuracy in identifying network traffic, a common study is to use deep learning methods to classify network traffic. However, most solutions are implemented by increasing the depth of the network, and the feature representation capabilities extracted by the model are not comprehensive enough. Therefore, it is necessary to develop a new network traffic classification scheme with multi-scale feature fusion capabilities. Summary of the invention

[0009] Aiming at the problem of automatic identification of network traffic in anonymous communication scenarios, the present invention focuses on the three main problems of high computational complexity, low recognition accuracy and limited feature characterization capability, and designs a network traffic classification method based on the attention mechanism.

[0010] The technical solution adopted by the present invention is:

[0011] A network traffic classification method based on attention mechanism includes the following steps:

[0012] S1. Perform data preprocessing on the original traffic, extract the bidirectional flow from the original traffic file, and extract the traffic attributes to form a traffic attribute vector;

[0013] S2, normalizing the extracted traffic attributes;

[0014] S3, using the CNN+LSTM structure to extract the temporal and spatial features of traffic attributes at different granularities;

[0015] S4, performing multi-scale fusion processing on the extracted temporal and spatial features of different granularities;

[0016] S5. Use the attention mechanism to extract the salient features of the fused features;

[0017] S6. Use the fully connected layer + softmax function to classify network traffic using the significant features of the fused features.

[0018] Furthermore, step S3 specifically includes:

[0019] S31. Use convolution layers with different convolution kernel sizes to extract the spatial features of traffic attributes. The calculation formula of the CNN layer is as follows:

[0020] CNN(x)=f(w·x i+h-1 +b)

[0021] Among them, x is the normalized traffic attribute vector, x i+h-1is the i+h-1th traffic attribute in the traffic attribute vector, w is the weight matrix, b is the bias parameter, f is the activation function, and h is the convolution kernel length;

[0022] S32. Put the traffic features extracted by the CNN layer into the LSTM layer to extract the time features of the traffic. The calculation formula of the LSTM layer is as follows:

[0023] f t =σ(W xf x t +W hf h t-1 )

[0024] i t =σ(W xi x t +W hi h t-1 )

[0025] o t =σ(W xo x t +W ho h t-1 )

[0026] c_in t =tanh(W xc x t +W hc h t-1 )

[0027] c t =f t c t-1 +i t c_in t

[0028] h t =o t tanh(c t )

[0029] Among them, x t is the input of the neuron, is the spatial feature extracted by the CNN layer, W xf , W hf , W xi , W hi , W xo , W ho , W xc and W hc are weight matrices, h t-1 is the hidden layer state at the previous moment, h t is the hidden layer state at the current moment, f t For the forget gate, i t is the input gate, o tis the output gate, c t is the current cell state, c t-1 is the cell state at the previous moment, c_in t is the updated state at time t, σ is the element-wise sigmoid function, and tanh is the hyperbolic tangent function used as the activation function.

[0030] Furthermore, step S5 specifically includes:

[0031] Use the attention mechanism to extract the salient features of the fused features c i , the calculation formula is:

[0032]

[0033]

[0034]

[0035] Among them, s i-1 represents the fusion feature of the previous moment, h j represents the fusion feature at the current moment, a ij Indicates h j The weight of i Represents the context vector of the input sequence, V a , W a and U a They represent the weight matrices of the corresponding variables, tanh represents the hyperbolic tangent function used as the activation function, and m represents the length of the input sequence.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] (1) As network traffic has multiple spatiotemporal characteristics, it first uses CNN to extract local spatial features, then uses LSTM to extract temporal information, and then uses multi-scale fusion to obtain a comprehensive consideration of the characteristics of network traffic data;

[0038] (2) Compared with the existing deep learning traffic classifier based on single-mode input, the present invention designs a multi-scale feature fusion method from a multi-modal perspective, learns the spatiotemporal characteristics of traffic, effectively processes multiple different types of features, reasonably completes the information complementarity between features of different granularity, reduces the heterogeneity differences between modalities, and thus obtains richer deep fusion features, provides more information for model decision-making, and improves the accuracy of the overall classification results. ;

[0039] (3) In order to effectively extract the significant features after multimodal fusion, the present invention adopts the attention mechanism to give the model the ability to focus on high-importance features. At the same time, the parallel structure is adopted to effectively solve the gradient vanishing and explosion problems caused by the complexity of the serial model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further explained below in conjunction with the accompanying drawings.

[0042] In the example of the present invention, in order to verify the effectiveness of the multi-scale spatiotemporal feature fusion model with attention mechanism, we will identify Tor traffic in the ISCXTor2016 dataset containing normal traffic and Tor traffic, such as Figure 1 As shown, the specific steps are as follows:

[0043] S1. Perform data preprocessing on the original traffic, extract the bidirectional flow from the file storing the original traffic, and extract the attributes, generating a total of 67,828 records, including 23 columns of attributes and 1 column of label values. TOR is used to identify Tor traffic, and nonTOR is used to identify regular traffic. Step S1 specifically includes:

[0044] S11, the data packet is described by {source IP, destination IP, source port, destination port, protocol, number of bytes};

[0045] S12. A network flow is defined as a sequence of packets with the same five-tuple {source IP, destination IP, source port, destination port, protocol} value, and its attributes are {source IP, destination IP, source port, destination port, protocol, direction, number of bytes, number of packets}. The bidirectional flow is extracted from the original traffic file, and the 23 extracted attributes are shown in Table 1. The first packet transmission direction from the source to the destination is defined as the forward direction, and the packet transmission direction from the destination to the source is defined as the backward direction. ;

[0046]

[0047] Table 1 Stream related attribute description table

[0048] Among them, the maximum value, minimum value, average value and standard deviation of the Flow IAT, Fwd IAT, Fwd IAT, Active and Idle field values ​​need to be extracted at the same time.

[0049] S2. Normalize the data so that the attribute values ​​after normalization are within [0,1] to eliminate the influence of singular samples. The min-max normalization function is represented by Nor() and is defined as:

[0050]

[0051] Where x is the individual value to be normalized in set X, and min(X) and max(X) represent the minimum and maximum values ​​of set X.

[0052] S3. Network traffic is a one-dimensional byte stream with a hierarchical structure. The structure of bytes, packets, and sessions in the traffic is very similar to the structure of characters, words, and sentences in natural language processing. Therefore, a one-dimensional convolutional neural network is suitable for processing sequence data, and convolutional layers with different convolution kernel sizes are used to extract the spatial features of the traffic. The calculation formula of the CNN layer is as follows:

[0053] CNN(x)=f(w·x i+h-1 +b)

[0054] Among them, x is the normalized traffic attribute vector, x i+h-1 is the i+h-1th traffic attribute in the traffic attribute vector, w is the weight matrix, b is the bias parameter, f is the activation function, and h is the convolution kernel length;

[0055] Considering that network traffic is essentially a kind of time series data, the traffic features extracted by the CNN layer are put into the LSTM layer to extract the time features of the traffic, and the Dropout layer is added to prevent the model from overfitting. The calculation formula of the LSTM layer is as follows:

[0056] f t =σ(W xf x t +W hf h t-1 )

[0057] i t =σ(W xi x t +W hi h t-1 )

[0058] o t =σ(W xo x t +W ho h t-1 )

[0059] c_in t =tanh(W xc x t +W hc h t-1 )

[0060] c t =f t c t-1 +i t c_in t

[0061] ht =o t tanh(c t )

[0062] Among them, x t is the input of the neuron, is the spatial feature extracted by the CNN layer, W xf , W hf , W xi , W hi , W xo , W ho , W xc and W hc are weight matrices, h t-1 is the hidden layer state at the previous moment, h t is the hidden layer state at the current moment, f t For the forget gate, i t is the input gate, o t is the output gate, c t is the current cell state, c t-1 is the cell state at the previous moment, c_in t is the updated state at time t, σ is the element-wise sigmoid function, and tanh is the hyperbolic tangent function used as the activation function.

[0063] This embodiment uses three one-dimensional convolutional layers with kernel sizes of 2, 3, and 4 in the table below to extract spatial features in parallel and obtain 16 feature maps with dimensions of 22, 21, and 20 respectively, and inputs the features of the three different granularities into a long short-term memory network with a hidden layer size of 64, and obtains 64 features with dimensions of 22, 21, and 20 respectively, completing the temporal feature extraction part.

[0064] Number of convolution kernels Convolution kernel size Step Length FlowDuration 16 2 1 FlowBytes / s 16 3 1 FlowPackets / s 16 4 1

[0065] Table 2 Convolution information

[0066] S4. Fusion of features of different granularities to enrich feature representation. Multi-scale feature fusion refers to the effective processing of multiple different types of features, the reasonable completion of information complementarity between features of different granularities, and the resolution of possible redundancy problems, thereby obtaining richer deep fusion features. In the fusion process, the detailed features and overall features of network traffic are fused, and the output vector obtained is a fusion feature with 64 dimensions and 63 dimensions.

[0067] S5. Use the attention mechanism to handle the importance differences of spatiotemporal features ignored by the existing structure and extract important fine-grained features. Due to the different importance of the extracted spatiotemporal features, important salient features often contain more information and have a greater impact on the trend of the actual classification effect. After multi-scale feature fusion, if the model is given the ability to pay more attention to high-importance features, effective features that contribute to classification can be optimized after feature fusion. Use the attention mechanism to extract multimodal salient features to improve the output quality of the dynamic structure, and add a Dropout layer to prevent the model from overfitting.

[0068] The calculation formula is:

[0069]

[0070]

[0071]

[0072] Among them, s i-1 represents the fusion feature of the previous moment, h j represents the fusion feature at the current moment, a ij Indicates h j The weight of i Represents the context vector of the input sequence, V a、 W a and U a They represent the weight matrices of the corresponding variables, tanh represents the hyperbolic tangent function used as the activation function, and m represents the length of the input sequence; after processing, a 128-dimensional attention vector is obtained.

[0073] S6. Finally, the obtained attention vector is classified using the fully connected layer and softmax function. The accuracy of the model on the training set is 95.39% and the accuracy on the test set is 95.58%.

Claims

1. A network traffic classification method based on attention mechanism, characterized in that: The following steps are involved: S1. Perform data preprocessing on the original traffic, extract the bidirectional flow from the original traffic file, and extract the traffic attributes to form a traffic attribute vector; S2, normalizing the extracted traffic attributes; S3, using the CNN+LSTM structure to extract the temporal and spatial features of traffic attributes at different granularities; S4, performing multi-scale fusion processing on the extracted temporal and spatial features of different granularities; S5. Use the attention mechanism to extract the salient features of the fused features; S6, using the fully connected layer + softmax function, classify the network traffic using the significant features of the fused features; Wherein, step S3 specifically includes: S31. Use convolution layers with different convolution kernel sizes to extract the spatial features of traffic attributes. The calculation formula of the CNN layer is as follows: CNN(x)=f(w·x i+h-1 +b) Among them, x is the normalized traffic attribute vector, x i+h-1 is the i+h-1th traffic attribute in the traffic attribute vector, w is the weight matrix, b is the bias parameter, f is the activation function, and h is the convolution kernel length; S32. Put the traffic features extracted by the CNN layer into the LSTM layer to extract the time features of the traffic. The calculation formula of the LSTM layer is as follows: f t =σ(W xf x t +W hf h t-1 ) i t =σ(W xi x t +W hi h t-1 ) o t =σ(W xo x t +W ho h t-1 ) c_in t =tanh(W xc x t +W hc h t-1 ) c t =f t c t-1 +i t c_in t h t =o t fishy t ) Among them, x t is the input of the neuron, is the spatial feature extracted by the CNN layer, W xf , W hf , W xi , W hi、 W xo , W ho , W xc and W hc are the corresponding weight matrices, h t-1 is the hidden layer state at the previous moment, h t is the hidden layer state at the current moment, f t For the forget gate, i t is the input gate, o t is the output gate, c t is the current cell state, c t-1 is the cell state at the previous moment, c_in t is the updated state at time t, σ is the element-wise sigmoid function, and tanh is the hyperbolic tangent function used as the activation function; Wherein, step S5 specifically includes: Use the attention mechanism to extract the salient features of the fused features c i , the calculation formula is: Among them, s i-1 represents the fusion feature of the previous moment, h j represents the fusion feature at the current moment, a ij Indicates h j The weight of i Represents the context vector of the input sequence, V a、 W a and U a They represent the weight matrices of the corresponding variables, tanh represents the hyperbolic tangent function used as the activation function, and m represents the length of the input sequence.

Images