An encrypted traffic classification method based on incremental learning

Abstract

The present invention provides an encrypted traffic method based on incremental learning, which relates to the field of network classification traffic, including step one, preprocessing the original traffic, converting and generating a three-channel RGB image data set; step two, building a deep residual network for small-size traffic images; step three, inputting some class samples into the deep residual network training to achieve encrypted traffic classification; step four, updating the network model and parameters, training a loss function based on category balance to minimize it; step five, screening representative old samples in memory, retaining the old sample learning experience; step six, repeating the above steps three, four, and five. The present invention can effectively realize incremental learning from dynamically growing encrypted traffic data, and improve its classification accuracy, while avoiding the model's inclination to new samples.

Description

Technical Field

[0001] The present invention relates to the field of network traffic classification, and in particular to an encrypted traffic classification method based on incremental learning. Background Art

[0002] With the widespread popularization and dissemination of Internet applications, people generate a large amount of network traffic when accessing the Internet. Among them, encrypted traffic is widely used in the network, and its share of the total traffic is increasing year by year. Therefore, the encrypted traffic classification algorithm has gradually received widespread attention from academia and industry, and the correct identification and classification of encrypted traffic has become an important topic in network security and network management. On the one hand, traffic operators can analyze the proportion of traffic occupied by different network applications through encrypted traffic classification, rationally plan the network, improve the level of network management, and enhance the quality of network services. On the other hand, with the increase in the number of unknown encryption protocols, malicious intrusions such as hacker attacks occur frequently, and encrypted traffic classification can provide security for the network.

[0003] In the early stages of the Internet, traffic protocols can be directly identified according to the corresponding port numbers. However, with the development of dynamic port technology and network security technology, traffic classification algorithms based on port numbers are no longer applicable. Subsequently, a method that focuses on the payload in the traffic emerged, which is called deep packet inspection technology. Since different encryption protocols or applications often have their own specific data packet formats, such as fixed strings, this method can explore and summarize the characteristic laws between data from the payloads of different types of traffic. With the popularity of computer technology, more and more researchers choose to use machine learning to solve the problem of encrypted traffic classification, such as classic algorithms such as Bayesian technology, C4.5 decision tree, support vector machine, and K nearest neighbor classification. However, these classic algorithms are highly dependent on the manual feature selection of encrypted traffic. In order to improve the above problems, deep learning technology has been applied to the field of encrypted traffic classification in recent years. These methods use deep neural networks to extract encrypted traffic features and perform classification predictions.

[0004] As encrypted traffic data continues to grow, the time and space complexity of these technologies continue to increase as the amount of data increases. Although the above technologies have achieved good classification results on fixed public data sets, when the amount of data reaches a certain scale, the learning rate of ordinary encrypted traffic classification models often cannot keep up with the rate of data updates and can no longer meet actual needs. Summary of the invention

[0005] In order to solve the technical problems existing in the prior art, the present invention provides an encrypted traffic classification method based on incremental learning. On the basis of meeting the high-accuracy classification of encrypted traffic, the scalability of the method is also achieved, so that the model can accumulate experience and perform effective incremental learning, and avoid the model's inclination towards new samples.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] An encrypted traffic classification method based on incremental learning includes the following steps:

[0008] Step 1: Preprocess the original traffic and convert it into a three-channel RGB image dataset;

[0009] Step 2: Build a deep residual network for small-size traffic images;

[0010] Step 3: Input some class samples into the deep residual network training to achieve encrypted traffic classification;

[0011] Step 4: Update the network model and parameters, and train a loss function based on category balance to minimize it;

[0012] Step 5: Filter representative old samples in the memory and retain the learning experience of old samples;

[0013] Step 6. Repeat steps 3, 4 and 5 above.

[0014] Furthermore, the step 1 pre-processes the original traffic and converts it into a three-channel RGB image data set, specifically including:

[0015] 1) Perform five-tuple clustering and filtering on each flow based on source IP address, destination IP address, source port, destination port, and transport layer protocol;

[0016] 2) Delete the MAC address and IP address in all data packets;

[0017] 3) On the basis of maintaining the original traffic content characteristics, the statistical characteristics of the traffic are emphasized by using a packet splicing method. A certain length of padding bytes is spliced ​​after each packet, and packets in different directions have different padding values.

[0018] 4) Considering that the MTU of Ethernet is 1500 bytes, the traffic is converted into an array of equal length in units of 3072 bytes, and the array elements are normalized;

[0019] 5) Compared with ordinary grayscale images, a multi-channel image with richer texture information is constructed, each array element is regarded as an image pixel, and a three-channel RGB image of size 32×32×3 is produced;

[0020] 6) Create a TFRecords dataset with images and their labels in the form of a string sequence.

[0021] Furthermore, the step 2 specifically includes: input layer, convolution layer, residual layer, pooling layer, fully connected layer, and output layer for the deep residual network of small-size traffic images; the image is transmitted from the input layer to the deep residual network, and deep features are extracted in the convolution layer and the residual layer to construct an identity mapping; then dimensionality reduction is performed, and the features are nonlinearly mapped to form a predicted feature vector logits; finally, the softmax function is used to normalize the feature vector to output the category probability of each traffic type.

[0022] Furthermore, the step three inputs some class samples into the deep residual network training, specifically including: setting incremental tasks, simulating incremental environments, dividing the data set into different training batches, and gradually inputting some encrypted traffic samples for training; wherein, the setting of incremental tasks refers to the common divisor of the total number of categories. When the incremental task is 1, it means that the standard classification task is completed, that is, all training data are input at one time; in order to ensure fairness, some class traffic samples in the input data set are randomly sampled.

[0023] Furthermore, the loss function formula of step 4 based on category balance is:

[0024] Loss = (1-γ) Loss c +γT 2 Loss d

[0025] Among them, γ is a hyperparameter used to adjust the proportion of classification loss function and distillation loss function in the overall loss function; T represents the temperature when softening the softmax function;

[0026] After adding new samples, the weight of the model classification layer changes, making the model more biased towards new types of predictions; therefore, the bias vector V is used to reflect the bias of the model for each type of sample, and the biased prediction of the model is corrected by regularizing it;

[0027] The definition of V and the regularization process are as follows:

[0028] V=∑(1-γ)y i +γT 2 H T (x i )

[0029]

[0030] Among them, x i represents the sample, yi Indicates its label, H T (x i ) represents the probability distribution of old samples corresponding to logits in the previous model after the softmax function;

[0031] Add the balance factor α to Loss c With Loss d middle:

[0032] Loss c =∑C entropy (α·F(x i ),y i )

[0033] Among them, C entropy represents the cross entropy function; F(x i ) represents the probability distribution of new and old samples corresponding to logits in the current model after the softmax function;

[0034] Loss d =C entropy (α·F T (x i ),H T (x i ))

[0035] Similarly, F T (x i ) represents the probability distribution of new and old samples corresponding to logits in the current model after passing through the softmax function.

[0036] Furthermore, the step five selects representative old samples in the memory, and retaining the old sample learning experience includes: storing representative samples in the memory, the memory space size is S, the current number of old sample types is K, and the number of sample types to be added in the future is T, and the old samples and new samples are considered fairly, that is, the number of samples of each type is balanced. Therefore, under the premise of fixed memory space, due to the increase in traffic types, the number of old samples must be reduced from S / K to S / (K+T).

[0037] Furthermore, high-quality old samples are screened: after the training phase of each incremental task, the old samples are stored in an ordered list sorted by confidence, the first S / (K+T) samples are retained, and the remaining unused old samples are deleted to release memory space.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] 1. The present invention proposes a new data packet preprocessing method, which uses padding bytes to achieve splicing between data packets. The padding bytes emphasize the statistical information of the data packets, such as direction, frequency, etc., and better characterize the content characteristics and statistical characteristics of the traffic.

[0040] 2. The present invention proposes a multi-channel encrypted traffic composition method to construct a three-channel RGB traffic image of size 32×32×3, which has the advantage of richer texture features compared to ordinary grayscale traffic images.

[0041] 3. The present invention proposes an incremental learning method to support the continuous updating of the encrypted traffic classification model. When the encrypted traffic of new categories continues to increase, the model will not suffer from catastrophic forgetting under the condition of limited memory space and maintain high accuracy.

[0042] 4. The present invention proposes a loss function based on category balance, which uses a balance factor α to regularize the bias vector V, thereby avoiding the model's inclination towards new samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the method for encrypting traffic based on incremental learning of the present invention.

[0044] Figure 2 This is a flow chart of the original flow preprocessing of the present invention.

[0045] Figure 3 This is a diagram of the deep residual network architecture of the present invention. DETAILED DESCRIPTION

[0046] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0047] like Figure 1 As shown, the present invention provides an encrypted traffic classification method based on incremental learning, comprising the following steps:

[0048] Step 1: Preprocess the original traffic and convert it into a three-channel RGB image dataset;

[0049] Step 2: Build a deep residual network for small-size traffic images;

[0050] Step 3: Input some class samples into the deep residual network training to achieve encrypted traffic classification;

[0051] Step 4: Update the network model and parameters, and train a loss function based on category balance to minimize it;

[0052] Step 5: Filter representative old samples in the memory and retain the learning experience of old samples;

[0053] Repeat steps 3, 4, and 5 above.

[0054] The three-channel RGB composition method proposed in the present invention enriches the texture features of traffic and improves the accuracy of encrypted traffic classification. Furthermore, the incremental learning algorithm is deployed on the encrypted traffic classification model to enhance the practicality and flexibility of the model. The following is a detailed description of each step of this embodiment:

[0055] Step 1: Convert a series of raw traffic pcap files into a three-channel RGB image dataset, such as Figure 2 shown.

[0056] 1) After obtaining the pcap file of the original traffic, perform five-tuple clustering and filtering on each flow according to the source IP address, destination IP address, source port, destination port, and transport layer protocol to eliminate noise interference from non-target user traffic;

[0057] 2) Set the MAC address and IP address in each flow packet to 0 to prevent the classifier from learning fixed features, which will cause the model to overfit;

[0058] 3) A padding byte with a length of 24 is added between every two data packets. Specifically, the padding value of the uplink data packet is 0, and the padding value of the downlink data packet is 60. The padding bytes are used to represent the statistical information (direction, frequency) of the data packet.

[0059] 4) Considering that the MTU of Ethernet is 1500 bytes, the traffic is converted into an array of equal length in units of 3072 bytes, and the array elements are normalized;

[0060] 5) Treat each array element as an image pixel and make a three-channel RGB image of size 32×32×3;

[0061] 6) The images and their labels are made into a TFRecords format dataset saved in the form of string sequences, and the training set and test set are divided into 9:1.

[0062] Step 2: For a small-size traffic image dataset with a side length of 32, a deep residual network is constructed. The present invention refers to the ResNet18 network. Figure 3 The figure shows the deep residual network structure of this embodiment, which includes the following connected in sequence: input layer 101, convolutional layer 102, residual layers 103-106, pooling layer 107, fully connected layer 108, and output layer 109.

[0063] The input layer has a size of 32×32×3 and is used to obtain the three-channel RGB flow image passed in by the dataset.

[0064] The convolution layer, with a size of 3×3, is used to preliminarily extract image features to obtain a feature map.

[0065] The residual layer includes convolution and skip layer parts. The convolution size is 3×3, which further extracts image features. The skip layer structure connects the feature maps of different layers and is the core of the residual layer for constructing the identity mapping.

[0066] The pooling layer uses global average pooling to reduce the dimension of the feature map to 1×1.

[0067] The fully connected layer is used to perform nonlinear mapping on the features to form a prediction feature vector logits.

[0068] The output layer uses a softmax function to normalize the feature vector and outputs the category probability of each traffic type. The number of output nodes is consistent with the number of traffic types.

[0069] Step 3: Set incremental tasks, simulate incremental environments, divide the data set into different training batches, and gradually input some encrypted traffic samples for training. Among them, the setting of incremental tasks refers to the common divisor of the total number of categories. When the incremental task is 1, it means that the standard classification task is completed, that is, all training data is input at one time. In particular, in order to ensure fairness, some class traffic samples in the input data set are randomly sampled. The present invention can achieve a classification accuracy of more than 97% in standard classification tasks.

[0070] Step 4: As mentioned above, the feature extraction part of the deep residual network transmits the features to the classification layer and outputs the predicted feature vector logits. During the training phase, the model uses these logits and calculates the weight gradient information in the neural network through the loss function. In particular, when a new traffic category arrives, the feature extraction part of the network does not change, only a new classification layer is generated. The data used for training consists of two parts: old encrypted traffic samples and new encrypted traffic samples. For the above two categories of samples, a loss function based on category balance is proposed, which is composed of the classification loss function Loss c And the distillation loss function Loss d The goal of the classification loss function is to measure the difference in classification ability of different types of traffic in the model. By suppressing this difference, the classification accuracy of encrypted traffic can be improved; while the goal of the distillation loss function is to measure the difference in classification ability between the previous model and the current model. By suppressing this difference, the learning ability of the current model can be enhanced and catastrophic forgetting can be reduced.

[0071] The loss function formula based on category balance is:

[0072] Loss = (1-γ) Loss c+γT 2 Loss d

[0073] Among them, γ is a hyperparameter used to adjust the proportion of classification loss function and distillation loss function in the overall loss function; T represents the temperature when softening the softmax function.

[0074] After adding new samples, the weight of the model classification layer changes, making the model more biased towards new types of predictions. Therefore, the bias vector V is used to reflect the bias of the model for each type of sample, and the biased prediction of the model is corrected by regularizing it.

[0075] The definition of V and the regularization process are as follows:

[0076] V=∑(1-γ)y i +γT 2 H T (x i )

[0077]

[0078] Among them, x i represents the sample, y i Indicates its label, H T (x i ) represents the probability distribution of old samples corresponding to logits in the previous model after passing through the softmax function.

[0079] Add the balance factor α to Loss c With Loss d middle:

[0080] Loss c =∑C entropy (α·F(x i ),y i )

[0081] Among them, C entropy represents the cross entropy function; F(x i ) represents the probability distribution of new and old samples corresponding to logits in the current model after passing through the softmax function.

[0082] Loss d =C entropy (α·F T (x i ),H T (x i ))

[0083] Similarly, F T (x i) represents the probability distribution of new and old samples corresponding to logits in the current model after passing through the softmax function.

[0084] Step 5: Store representative samples in memory. The size of the memory space is S, the current number of old sample types is K, and the number of sample types to be added in the future is T. We consider old samples and new samples fairly, that is, the number of samples of each type is balanced. Therefore, under the premise of fixed memory space, due to the increase in traffic types, the number of old samples must be reduced from S / K to S / (K+T). We provide a solution for screening high-quality old samples: after the training phase of each incremental task, the old samples are stored in an ordered list sorted by confidence, the first S / (K+T) samples are retained, and the remaining unused old samples are deleted to release memory space.

[0085] Step 6: Repeat steps 3, 4 and 5 above.

[0086] The encrypted traffic method of this embodiment based on incremental learning captures the statistical characteristics and content characteristics of encrypted traffic, constructs a three-channel RGB image with rich texture, deeply extracts encrypted traffic features, and realizes the classification prediction of encrypted traffic types by the model. The scalability of the model is provided. When new types of traffic data are added, the model can be continuously trained based on the experience of the original old samples on the basis of limited memory, while avoiding the model's inclination towards new samples, effectively solving the problem of catastrophic forgetting.

[0087] In order to verify the practicality and effectiveness of the encrypted traffic method based on incremental learning proposed in this paper, we verified it on the ISCXVPN2016 public dataset. Our method not only has an accuracy rate of more than 97% in standard encrypted traffic classification, but also has an accuracy rate of more than 85% in the final classification result in incremental learning with memory space constraints. This example mainly uses three evaluation indicators: accuracy, recall rate and F1 score.

[0088] Table 1: Classification of the method of the present invention under public data sets

[0089] Traffic Type Accuracy Recall F1 Chat 0.90 0.95 0.92 Email 0.77 0.83 0.75 File 0.99 0.98 0.98 P2P 0.99 0.96 0.98 Streaming 1 1 1 VoIP 1 0.99 1 VPN-Chat 0.99 0.98 1 VPN-Email 0.92 0.94 0.93 VPN-File 0.99 0.97 0.98 VPN-P2P 1 1 1 VPN-Streaming 0.99 1 0.99 VPN-Voip 1 1 1

[0090] Table 2: Final classification accuracy of the method of the present invention under different incremental tasks. Taking a 12-category public dataset as an example, the number of incremental types each time is set to 2, and there are 6 incremental tasks in total.

[0091] Incremental tasks Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Accuracy 0.98 0.6 0.94 0.93 0.90 0.86

[0092] Experimental results show that this method not only outperforms many existing technologies in the field of encrypted traffic classification, but also provides a scalable incremental approach that is effective and versatile in real-world applications.

[0093] The embodiments of the present invention are described above, but the present invention is not limited to the above embodiments. Various changes can be made according to the purpose of the invention. Any changes, modifications, substitutions, combinations or simplifications made according to the spirit and principle of the technical solution of the present invention should be equivalent replacement methods. As long as they meet the purpose of the invention and do not deviate from the technical principles and inventive concepts of the present invention, they belong to the protection scope of the present invention.

Claims

1. An encrypted traffic classification method based on incremental learning, characterized in that: The following steps are involved: Step 1: Preprocess the original traffic and convert it into a three-channel RGB image dataset; Step 2: Build a deep residual network for small-size traffic images; Step 3: Input some class samples into the deep residual network training to achieve encrypted traffic classification; Step 4: Update the network model and parameters, and train a loss function based on category balance to minimize it; Step 5: Filter representative old samples in the memory and retain the learning experience of old samples; Step 6: Repeat the above steps 3, 4 and 5; The first step preprocesses the original traffic and converts it to generate a three-channel RGB image data set, specifically including: 1) Perform five-tuple clustering and filtering on each flow based on source IP address, destination IP address, source port, destination port, and transport layer protocol; 2) Delete the MAC address and IP address in all data packets; 3) On the basis of maintaining the original traffic content characteristics, the statistical characteristics of the traffic are emphasized by using a packet splicing method. A certain length of padding bytes is spliced ​​after each packet, and packets in different directions have different padding values. 4) Considering that the MTU of Ethernet is 1500 bytes, the traffic is converted into an array of equal length in units of 3072 bytes, and the array elements are normalized; 5) Compared with ordinary grayscale images, a multi-channel image with richer texture information is constructed, each array element is regarded as an image pixel, and a three-channel RGB image of size 32×32×3 is produced; 6) Create a TFRecords dataset with images and their labels in the form of a string sequence; The loss function formula of step 4 based on category balance is: Loss=(1-γ)Loss c +γT 2 Loss d Among them, γ is a hyperparameter used to adjust the proportion of classification loss function and distillation loss function in the overall loss function; T represents the temperature when softening the softmax function; After adding new samples, the weight of the model classification layer changes, making the model more biased towards new types of predictions; therefore, the bias vector V is used to reflect the degree of bias of the model for each type of sample, and the biased prediction of the model is corrected by regularizing it; The definition of V and the regularization process are as follows: V=∑(1-γ)y i +γT 2 H T (x i ) Among them, x i represents the sample, y i Indicates its label, H T (x i ) represents the probability distribution of old samples corresponding to logits in the previous model after the softmax function; Add the balance factor α to Loss c With Loss d middle: Lossc=∑Centropy(α·F(xi),yi) Among them, C entropy represents the cross entropy function; F(x i ) represents the probability distribution of new and old samples corresponding to logits in the current model after the softmax function; Loss d =C entropy (α·F T (x i ),H T (x i )) Similarly, F T (x i ) represents the probability distribution of new and old samples corresponding to logits in the current model after passing through the softmax function.

2. The encrypted traffic classification method based on incremental learning according to claim 1 is characterized in that: The step 2 specifically includes: input layer, convolution layer, residual layer, pooling layer, fully connected layer, and output layer for the deep residual network of small-size traffic images; the image is transmitted from the input layer to the deep residual network, and deep features are extracted in the convolution layer and the residual layer to construct an identity mapping; then dimensionality reduction is performed, and the features are nonlinearly mapped to form a predicted feature vector logits; finally, the softmax function is used to normalize the feature vector to output the category probability of each traffic type.

3. The encrypted traffic classification method based on incremental learning according to claim 1 is characterized in that: The step three inputs some class samples into the deep residual network training, specifically including: setting incremental tasks, simulating incremental environments, dividing the data set into different training batches, and gradually inputting some encrypted traffic samples for training; wherein, the setting of incremental tasks refers to the common divisor of the total number of categories. When the incremental task is 1, it means that the standard classification task is completed, that is, all training data are input at one time; in order to ensure fairness, some class traffic samples in the input data set are randomly sampled.

4. The encrypted traffic classification method based on incremental learning according to claim 1 is characterized in that: The step five screens representative old samples in the memory, and retains the learning experience of old samples, including: storing representative samples in the memory, the memory space size is S, the current number of old sample types is K, the number of sample types to be added in the future is T, and the old samples and new samples are considered fairly, that is, the number of samples of each type is balanced. Therefore, under the premise of fixed memory space, due to the increase in traffic types, the number of old samples must be reduced from S / K to S / (K+T).

5. The encrypted traffic classification method based on incremental learning according to claim 4 is characterized in that: Screening high-quality old samples: After the training phase of each incremental task, store the old samples in an ordered list sorted by confidence, retain the first S / (K+T) samples, and delete the remaining unused old samples to free up memory space.

Images