A solution to network flow concept drift based on batch update

Abstract

The present invention discloses a method for solving the network flow concept drift based on batch update. The method first classifies the newly arrived traffic to obtain the classification result. In order to detect the network flow concept drift, the present invention proposes an adaptive concept drift detector to monitor the classification result of the traffic feature, judge whether the network flow concept drift occurs according to the number of drift samples, and autonomously trigger the update process of the classification model. In order to complete the update of the classification model in a reasonable time and space, the present invention designs a batch updater to batch update the drift samples and the locally stored clustered historical samples to obtain the updated cluster samples, and then use the updated cluster samples and the supervised machine learning classification algorithm to complete the update of the classification model. The method of the present invention is tested using a public 10G backbone network data set, and therefore, is suitable for updating the classification model of backbone network traffic.

Description

Technical Field

[0001] The invention relates to a method for solving the network flow concept drift problem based on batch updating, and belongs to the technical field of network management. Background Art

[0002] Real-time analysis of backbone network traffic can help service providers understand network status and dynamically optimize network bandwidth configuration and traffic routing strategies, which is of great significance for backbone network management. Network traffic classification plays a vital role in network management. Network operators can further implement network management or network security strategies by monitoring the distribution of different types of traffic in the network.

[0003] Due to the increase in random ports and encrypted traffic, traffic classification methods have changed from traditional port-based methods and payload-based methods to statistical-based methods. The essence of statistical-based traffic classification methods is to construct a mapping relationship from traffic statistical features to classification categories. This mapping relationship is the "concept" of network traffic. In most traffic classification works, a mapping relationship is first established based on historical traffic sample instances, and then the constructed mapping relationship is used to predict the category of newly arrived traffic samples. However, these traffic classification works ignore the concept drift problem of network traffic: as the traffic in the network evolves dynamically, the statistical characteristics of the traffic will change accordingly, resulting in the existing mapping relationship no longer being applicable to the classification of newly arrived traffic samples.

[0004] Two key issues in concept drift research are when to update the classifier and how to update the classifier. In terms of when to update the classifier, periodic incremental update and post-detection update are the two main methods. Among them, the periodic incremental update method will update the model even when no concept drift occurs, wasting time and resources. However, the post-detection update method also has disadvantages. The detection of concept drift depends on parameter settings, and inappropriate parameter settings can lead to false and missed alarms. Most of the current research detects concept drift by reducing classification accuracy. The post-detection update method requires the data to be annotated with classification labels in advance, which requires a lot of time and resources and is not suitable for concept drift detection of high-speed backbone network traffic.

[0005] In addition, even if the concept drift is effectively detected, how to update the classification model in time will affect the accuracy of backbone network traffic classification. There are three main methods for updating the classification model: the first is a one-time update method, which obtains a new classification model by training all historical data and newly arrived data, which is very time-consuming and memory-consuming in backbone network scenarios with massive data. The second is the model retraining method, which is based on the idea of ​​ensemble learning and achieves model update by replacing, adding or selectively updating the basic classifiers in the classifier set. This method is relatively inefficient for the classification of large-scale dynamic backbone network traffic. In addition, the model retraining method has the problem of catastrophic forgetting. The third is an instance-based incremental learning method, which learns and updates the classification model by training the newly arrived data without retraining the historical data. However, this method also has the problem of update overhead and catastrophic forgetting.

[0006] For network traffic classification methods, it is still a challenge to detect the concept drift caused by the dynamic evolution of network traffic and to complete the update of the classification model in a timely manner. The present invention aims to complete the detection of concept drift and update the classification model of massive traffic in the backbone network with reasonable time and memory. Summary of the invention

[0007] In order to solve the problem of concept drift in traffic classification, the present invention first monitors the classification results through a concept drift detector, and then determines whether network flow concept drift occurs according to the number of drift traffic samples, and adaptively triggers the update process of the classification model. Secondly, the present invention performs batch updates based on drift traffic samples and historical traffic samples to obtain updated cluster samples. Finally, the updated cluster samples are trained using a classification algorithm to complete the update of the classification model. The updated classification model can be used to classify subsequent high-speed network traffic. This method can perform adaptive concept drift detection on network traffic and autonomously complete the update of the classification model, which can be used for network traffic analysis and user service quality assurance.

[0008] A method for solving network flow concept drift based on batch update, the method comprising the following steps:

[0009] Step 1 builds a traffic dataset containing drift samples flowing through the backbone network;

[0010] Step 2 uses the traffic dataset constructed in step 1 as input and uses the traffic classification model to predict and classify the newly arrived traffic;

[0011] Step 3: Based on the classification result of step 2, a concept drift detector is used to determine whether the traffic in the data set has concept drift;

[0012] Step 4: If the concept drift occurs in the traffic in the data set, use the batch updater to batch update the drift samples and the locally stored clustered historical samples to obtain the updated cluster samples;

[0013] Step 5: Based on the marked traffic features in the clustered samples updated in step 4, the classification model is updated using a supervised machine learning algorithm;

[0014] Step 6 uses the classification model updated in step 5 to classify the backbone network traffic that arrives subsequently;

[0015] Furthermore, in step 1, the method for constructing a traffic data set containing drift samples flowing through the backbone network is as follows:

[0016] Step 1.1 Use two hosts to build an environment to collect DDoS traffic, which is used to construct drift samples in the real backbone network;

[0017] Step 1.2: Mix the DDoS traffic and the traffic collected on the 10G backbone network link to construct a traffic dataset containing drift samples flowing through the backbone network.

[0018] Furthermore, in step 2, the method for predicting and classifying the newly arrived traffic is as follows:

[0019] 2.1 Due to the different message formats of TCP and UDP protocols, traffic can be divided into four categories: bidirectional TCP flow, unidirectional TCP flow, bidirectional UDP flow and unidirectional UDP flow. Each category of traffic has a corresponding classification model;

[0020] 2.2f i represents the i-th traffic feature vector. For each feature vector f i =(x1, x2, ..., x m ), the classification model will give the predicted category label L i And the probability value p predicted as this category i ;

[0021] Dividing traffic into four categories, and each category has a corresponding classification model, can achieve accurate classification of backbone network traffic. In addition, when network flow concept drift is detected, the classification model corresponding to each type of traffic needs to be updated to maintain the overall accuracy of backbone network traffic classification.

[0022] Furthermore, the method for determining whether concept drift occurs in the traffic in the data set in step 3 is as follows:

[0023] 3.1 First, we need to detect whether the feature sample is a drift sample. Assume that LowThred is the class prediction probability value p i The judgment threshold of . Step 2.2 gives each feature vector fi =(x1,x2,…,x m ) corresponds to the predicted category label L i And the probability value p predicted as this category i If p i is greater than LowThred, indicating that this eigenvector f i is similar to the historical feature vector used to train the classification model, that is, the sample corresponding to the feature vector conforms to the normal pattern of historical data. On the contrary, if p i If it is smaller than LowThred, it means that the sample corresponding to this feature vector is a drift sample.

[0024] 3.2 Consider the triple (transport layer protocol, IP address, port) as a key, and extract one or more feature samples from the data packet with the same key value. Assume that KeyThred is the threshold of the ratio of the number of drift samples in a triple to the total number of samples in the triple, and calculate the ratio of the number of drift samples in a triple to the total number of samples in the triple as P k If P k ≥KeyThred, the triplet is judged as a drift triplet, otherwise the triplet is regarded as a normal triplet.

[0025] 3.3 Assume that UpdateThred is the threshold used to determine whether the classification model update process needs to be triggered, and calculate the ratio of the number of samples in all drift triplets to the total number of all traffic samples P u If P u ≥UpdateThred, which means that there are many drift samples in the backbone network traffic and the traffic in the dataset has concept drift.

[0026] Theoretically, multiple feature samples corresponding to the same triplet are either normal samples or drift samples. However, in the actual classification prediction stage, the classification model predicts the category of each feature sample in turn, which may cause some of the multiple feature samples corresponding to the same triplet to be judged as drift samples, while other feature samples are judged as normal samples. Therefore, first determine whether a triplet is a drift triplet based on the proportion of drift samples in the triplet, and then record all feature samples in the drift triplet as drift samples, which can more objectively count the number of drift samples.

[0027] Furthermore, in step 4, the method for updating the cluster samples is as follows:

[0028] 4.1 Take the drift samples and the locally stored clustered historical samples as input, use the batch updater to complete the batch clustering of all input data, and output the updated cluster samples.

[0029] Furthermore, in step 5, the method for updating the classification model is as follows:

[0030] 5.1 Use a supervised machine learning algorithm to train the labeled traffic features in the updated clustering results to build a new classification model.

[0031] Furthermore, in step 6, the method for classifying the backbone network traffic that arrives subsequently is as follows:

[0032] 6.1 Use the classification model updated in step 5 to classify the backbone network traffic that arrives subsequently.

[0033] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects.

[0034] (1) The present invention is aimed at updating the real-time backbone network traffic classification model and takes into account the concept drift problem that is ignored in most traffic classification work. After explicitly detecting the concept drift, the clustering results containing historical traffic samples and drift traffic samples are updated in batches. While ensuring the classification performance of the classification model, the update efficiency of the classification model is also improved, effectively alleviating the problem of network flow concept drift.

[0035] (2) When updating cluster samples, the present invention not only considers drift samples, but also iterates historical cluster samples, so that historical data information will not be forgotten, alleviating the catastrophic forgetting problem.

[0036] (3) The total time required for the present invention to classify the backbone network traffic data set, detect concept drift, and batch update the classification model is less than the collection time of the backbone network traffic data set. The classification model maintains a high accuracy before and after the update, and the update time is only about half of that of a one-time update. It can accurately detect and promptly resolve the problem of network flow concept drift. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The overall architecture diagram of the network flow concept drift solution based on batch update;

[0038] Figure 2 Flowchart for concept drift detection;

[0039] Figure 3 Flowchart for batch update of classification model;

[0040] Figure 4 Comparison chart of Micro F1 scores of classification models between batch update method and one-time update method;

[0041] Figure 5 This is a comparison chart of clustering result update time between batch update method and one-time update method;

[0042] Figure 6 Comparison of classification model retraining time between batch update method and one-time update method. DETAILED DESCRIPTION

[0043] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0044] Specific embodiment: The batch update-based network flow concept drift solution provided by the present invention can realize the detection of network flow concept drift and the update of classification model in the backbone network scenario. Its overall architecture is as follows: Figure 1 As shown, the following steps are included:

[0045] Step 1 builds a traffic dataset containing drift samples flowing through the backbone network;

[0046] Step 2 uses the traffic dataset constructed in step 1 as input and uses the traffic classification model to predict and classify the newly arrived traffic;

[0047] Step 3: Based on the classification result of step 2, a concept drift detector is used to determine whether the traffic in the data set has concept drift;

[0048] Step 4: If the concept drift occurs in the traffic in the data set, use the batch updater to batch update the drift samples and the locally stored clustered historical samples to obtain the updated cluster samples;

[0049] Step 5: Based on the marked traffic features in the clustered samples updated in step 4, the classification model is updated using a supervised machine learning algorithm;

[0050] Step 6 uses the classification model updated in step 5 to classify the backbone network traffic that arrives subsequently;

[0051] In one embodiment of the present invention, in step (1), constructing a traffic data set containing drift samples flowing through the backbone network specifically includes the following process:

[0052] (1.1) This case uses two hosts to build a DDoS traffic collection environment. One of the two hosts is used as the initiator of the attack, and the other is used as the attacked party. The hping3 tool is used to construct a DDoS flood attack. A total of 141,059,271 packets of DDoS traffic data are collected to construct drift samples in the real backbone network;

[0053] (1.2) Using 453,043,378 data packets collected on the 10G backbone link, 1 / 3 of all data packets are taken out in chronological order as the test set Dtest, which contains a total of 153,043,379 data packets;

[0054] (1.3) Mixed DDoS traffic and traffic collected on the 10G backbone network link are used to construct a traffic dataset containing drift samples flowing through the backbone network.

[0055] (1.3.1) In this case, we first calculate the duration T0 of the traffic in Dtest, which is the time of the last packet in Dtest minus the time of the first packet in Dtest. Similarly, the calculated traffic duration of the DDoS traffic data set is T d If T0>T d , just at (0,T0-T d ) Generate a random initial value T in the numerical interval init On the contrary, construct a random value in the numerical interval (0,σ*T0) as the initial value T init In this case, σ is set to 0.05. init After that, the time of each data packet in the DDoS traffic dataset is different from the initial time T init The difference between the two is considered as the new time for the packet.

[0056] (1.3.2) According to the updated time, the DDoS traffic is mixed into Dtest in chronological order. In this case, the mixed data set of DDoS traffic and test set Dtest is called DataSet2. DataSet2 contains a total of 294,102,650 data packets.

[0057] In one embodiment of the present invention, in step (2), the process of predicting and classifying the newly arrived traffic is as follows:

[0058] (2.1) Due to the different message formats of TCP and UDP protocols, traffic can be divided into four categories: bidirectional TCP flow, unidirectional TCP flow, bidirectional UDP flow and unidirectional UDP flow. Corresponding traffic classification models are designed for TCP and UDP protocols respectively, and each major traffic category has a corresponding classification model.

[0059] (2.2)f i represents the i-th traffic feature vector. For each feature vector f i =(x1,x2,…,x m ), the classification model will give the triple Key corresponding to the feature vector and the predicted category label L i And the probability value p predicted as this category iEtc. In concept drift detection, f is needed i The triple key recorded in and predicted to be the Lth i The probability value of the class p i .

[0060] In one embodiment of the present invention, in step (3), the process of determining whether concept drift occurs in the traffic in the data set is as follows:

[0061] (3.1) Figure 2 First, we need to detect whether the feature sample is a drift sample. In this case, LowThred is set to 0.6. Step (2.2) gives each feature vector f i =(x1,x2,…,x m ) corresponds to the predicted category label L i And the probability value p predicted as this category i If p i is greater than 0.6, indicating that this eigenvector f i is similar to the historical feature vector used to train the classification model, that is, the sample corresponding to the feature vector conforms to the normal pattern of historical data. On the contrary, if p i If it is less than 0.6, it means that the sample corresponding to this feature vector is a drift sample.

[0062] (3.2) In this case, KeyThred is set to 0.9, and the ratio P of the number of drift samples in a triple to the total number of samples in the triple is calculated. k , if P k If ≥0.9, the triplet is judged as a drift triplet, otherwise the triplet is regarded as a normal triplet.

[0063] (3.3) In this case, UpdateThred is set to 0.3, and the ratio of the number of samples in all drift triplets to the total number of all traffic samples is calculated. u , if P u ≥0.3, it means that there are many drift samples in the backbone network traffic and the traffic in the dataset has concept drift.

[0064] In one embodiment of the present invention, in step (4), the process of updating the cluster samples is as follows:

[0065] (4.1) Figure 3 ,This case comprehensively considers drift samples and clustered historical samples saved locally, and takes the test set (drift samples) and the training set (clustered historical samples saved locally) containing DDoS traffic as input, and uses the batch updater to complete batch clustering of all input data, and outputs the updated clustered samples.

[0066] In one embodiment of the present invention, in step (5), the process of updating the classification model is as follows:

[0067] (5.1) Figure 3 ,In this case, the random forest algorithm is used to train the ,labeled traffic features in the updated clustering results to build a new ,classification model.

[0068] In one embodiment of the present invention, in step (6), the method for classifying the subsequently arriving backbone network traffic using the updated classification model is as follows:

[0069] (6.1) Use the classification model updated in step (5) to classify the backbone network traffic that arrives subsequently.

[0070] (6.1.1) In this case, the model is deployed on Xeon(R)Gold 5218CPU@2.30GHZx64 and NVIDIA's TU102, the operating system version used is Ubuntu 20.04.1LTS, and the memory capacity is 25.4GB. In this case, the random forest algorithm is used to train the updated clustering results to obtain a new traffic classification model.

[0071] (6.1.2) Concept drift detection performance. This case defines the accuracy in judging drifted samples (ADS) and uses it as the performance evaluation indicator of concept drift detection. The calculation method of ADS is shown in Formula 1, where n ddos Indicates the number of DDoS feature samples actually mixed into the detected samples, n drift Indicates the number of samples contained in the triplet that is judged to be drifted.

[0072]

[0073] The detection results of concept drift are shown in Table 1: From the overall perspective of the various types of traffic to be detected, since the number of detected drift samples (44294) exceeds 0.3 (the value of UpdateThred) of the total number of detected samples (122421), the concept drift detection will adaptively trigger the update process of the classification model. However, this update process will only update the classification model corresponding to the bidirectional TCP flow and the unidirectional UDP flow. This is because the number of drift samples detected in the bidirectional TCP flow and the unidirectional UDP flow has reached 0.3 of the total number of detected samples, while the number of drift samples detected in the unidirectional TCP flow and the bidirectional UDP flow is relatively small and does not reach 0.3 of the total number of detected samples. In addition, the judgment accuracy of the drift samples of the overall traffic ADS reached 99.3%, indicating that the detected drift samples are basically consistent with the DDoS traffic samples actually mixed into the test set DataSet2, and can accurately detect the drift of network flow concepts.

[0074] Table 1 Detection results of concept drift (* indicates that there is no DDoS traffic mixed in this type of traffic data)

[0075]

[0076] (6.1.3) Batch update performance. This case uses the Update Time of the Clustering Results (UTCR), the Retraining Time of the Classification Model (RTCM), and the Micro F1 score of the Classification Model (MFCM) as performance evaluation indicators for batch update of the classification model. UTCR represents the time from when drift samples and historical clustering samples are input into the batch updater to when the batch updater outputs new clustering results. RTCM represents the time required to train the labeled feature data in the new clustering results using a supervised machine learning algorithm to obtain a new classification model. The Micro F1 score is obtained by averaging the precision and recall of all sample instances. The performance of batch update of the classification model is shown in Table 2. It only takes 104.1 seconds to complete the batch clustering of all traffic data, and only 55.7 seconds to train a new classification model based on the updated clustering samples. In general, the batch update of the classification model can be completed in 159.8 seconds, and the average accuracy of the updated classification model is 96%, which proves that the method proposed in the present invention is available and effective in the actual network environment and can solve the problem of network flow concept drift in a timely manner.

[0077] Table 2 Performance of batch update of classification model

[0078]

[0079] In addition, this paper compares the batch update method with the one-time update method. The one-time update method is a model update method that trains all historical data and newly arrived drift data at one time to obtain a new classification model. The comparison indicators of the two classification model update methods are UTCR, RTCM, and MFCM. In this case, the total time required for the batch update method to complete the model update is the sum of the UTCR and RTCM of all traffic data. Figure 3 As shown in , the accuracy of the classification models obtained by the two updating methods is similar. Figure 4 , 5 As shown in the figure, the total time required for the batch update method in this paper to complete the model update is only 52.5% of that of the one-time update method. This is because compared with the one-time update of historical data and drift data, the batch updater in this paper saves the marked historical clustering results locally based on the idea of ​​trading space for time, omitting the first clustering of the corresponding historical data, so the batch update method saves a lot of time compared to the one-time update method. In addition, since the batch updater not only considers the drift samples, but also iteratively updates the clustered historical samples, it will not forget the information contained in the historical data, thereby alleviating the catastrophic forgetting problem encountered in the one-time update.

[0080] (6.1.4) Real-time performance. This case performs feature extraction on DataSet2 three times. The average time for each feature extraction is 184.6 seconds. The classification model is used to predict the traffic features in DataSet2, and then concept drift detection is performed. The whole process takes a total of 145.4 seconds. As shown in Table 2, the batch update of DataSet2 takes 159.8 seconds. It takes a total of 489.8 seconds from traffic feature extraction to completing the update model, which is shorter than the data acquisition time of 593.079 seconds. In fact, concept drift is not persistent, and there is no need to continuously update the classification model. When the model update is not triggered, it only takes 330 seconds, which is shorter than the data capture time. This case uses a public 10G backbone network dataset for testing, so it can be considered that this method can perform real-time classification on a 10G backbone network. In addition, feature extraction and model prediction can be performed in parallel, indicating that the method proposed in the present invention can realize real-time identification of backbone network traffic and timely update of the classification model, effectively alleviating the problem of network flow concept drift.

[0081] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A batch update-based network flow concept drift solution, characterized in that: The method comprises the following steps: Step 1 builds a traffic dataset containing drift samples flowing through the backbone network; Step 2 uses the traffic dataset constructed in step 1 as input and uses the traffic classification model to predict and classify the newly arrived traffic; Step 3: Based on the classification result of step 2, a concept drift detector is used to determine whether the traffic in the data set has concept drift; Step 4: If the concept drift occurs in the traffic in the data set, use the batch updater to batch update the drift samples and the locally stored clustered historical samples to obtain the updated cluster samples; Step 5: Based on the marked traffic features in the clustered samples updated in step 4, the classification model is updated using a supervised machine learning algorithm; Step 6 uses the classification model updated in step 5 to classify the backbone network traffic that arrives subsequently; In step 1, the method for constructing a traffic data set containing drift samples flowing through the backbone network is as follows: Step 1.1 Use two hosts to build an environment to collect DDoS traffic, which is used to construct drift samples in the real backbone network; Step 1.2: Mix the DDoS traffic and the traffic collected on the 10G backbone network link to construct a traffic dataset containing drift samples flowing through the backbone network. In step 2, the method for predicting and classifying the newly arrived traffic is as follows: Step 2.1 Due to the different message formats of TCP and UDP protocols, the traffic is first divided into four categories: bidirectional TCP flow, unidirectional TCP flow, bidirectional UDP flow and unidirectional UDP flow. Each category of traffic has a corresponding classification model. Step 2.2 f i Indicates i traffic feature vectors, for each feature vector f i =(x 1 , x 2 , …, x m ) , the classification model will give the predicted category label L i and the probability value predicted for this category p i ; In step 3, the method for determining whether the traffic in the data set has concept drift is as follows: Step 3.1 First, we need to detect whether the feature sample is a drift sample. LowThred Predict probability values ​​for the categories p i The judgment threshold, step 2.2 gives each feature vector f i =(x 1 , x 2 , …, x m ) The corresponding predicted category label L i and the probability value predicted for this category p i ,if p i Greater than LowThred , indicating that this feature vector f i Similar to the historical feature vector used to train the classification model, that is, the sample corresponding to the feature vector conforms to the normal pattern of historical data. Conversely, if p i Less than LowThred , indicating that the sample corresponding to this feature vector is a drift sample; Step 3.2 considers the triplet of transport layer protocol, IP address and port as Key , from having the same Key Extract one or more feature samples from the data packet with the value KeyThred The ratio of the number of drift samples in a triple to the total number of feature samples in the triple is calculated as the threshold value of the ratio of the number of drift samples in the triple to the total number of feature samples in the triple. P k ,if P k ≥ KeyThred , the triplet is determined as a drift triplet, otherwise the triplet is regarded as a normal triplet; Step 3.3 UpdateThred To determine whether the update process needs to be triggered, the ratio of the number of samples in all drift triplets to the total number of traffic samples is calculated. P u ,if P u ≥ UpdateThred ,It shows that there are many drift samples in the backbone network traffic and the ,concept drift occurs in the traffic in the dataset.

2. According to the batch update-based network flow concept drift solution method of claim 1, it is characterized in that: In step 4, the method for updating cluster samples is as follows: Step 4.1 takes the drift samples and the locally stored clustered historical samples as input, uses the batch updater to complete batch clustering of all input data, and outputs the updated cluster samples.

3. The method for solving the network flow concept drift problem based on batch update according to claim 1 is characterized in that: In step 5, the method to update the classification model is as follows: Step 5.1 uses a supervised machine learning algorithm to train the labeled traffic features in the updated clustering results to build a new classification model.

Images