Network traffic classification method and device based on machine learning

Abstract

The present invention discloses a network traffic classification method and device based on machine learning, the method comprising: obtaining a training data set containing normal traffic and abnormal traffic; calculating the stiffness IH of the samples based on the nearest neighbors of the samples in the training data set; dividing the training data set into an easy set and a difficult set according to the stiffness IH; dividing the difficult set into a majority class sample set and a minority class sample set according to the number of samples in each class; performing K-means clustering on the majority class sample set, and replacing the samples in each cluster with the cluster center to obtain a sample set S K ; Perform SMOTE oversampling on the majority class sample set to obtain the sample set S Z ; Merge easy set, minority class sample set, sample set S K and sample set S Z , get the training data set S N ; Based on the training data set S K The network traffic classification model is trained, and the trained network traffic classification model is used to predict the traffic to be detected to obtain the classification result of the traffic to be detected. The present invention can improve the accuracy of the network traffic classification model.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary technical field of combining machine learning and information security, and in particular relates to a network traffic classification method and device based on machine learning. Background Art

[0002] In recent years, with the rapid adoption of computer network applications across various fields, network threats have become increasingly severe. Numerous mechanisms, such as firewalls, antivirus software, anti-malware software, and spam filters, are used as tools to protect network security. However, today's network attacks are systematic and long-term, and network traffic data is vast and complex, making it difficult to analyze and detect. Network flow classification is also an effective and powerful network security technology, and machine learning methods are currently widely used in network traffic detection.

[0003] Machine learning can identify abnormal traffic by learning features from large amounts of data. It can be categorized into supervised learning and unsupervised learning. Supervised learning involves studying labeled training data to discover relationships between input and output data for prediction and classification. Methods include deep neural networks, decision trees, support vector machines, K-nearest neighbors, and Gaussian naive Bayes. Unsupervised learning involves learning and summarizing patterns and structures from unlabeled training data for prediction and classification. Methods include principal component analysis and K-means clustering.

[0004] With the increase of network threats in recent years, the demand for network security protection measures has also increased. In order to adapt to the dynamic threat environment, this study uses machine learning to detect network traffic and thus detect malicious traffic mixed in the network flow.

[0005] However, in everyday network environments, normal traffic far outnumbers malicious traffic, resulting in a naturally imbalanced class distribution in network traffic. Most current research fails to consider the class distribution of network traffic, resulting in the final model being biased towards majority class samples and ignoring the more important minority malicious samples, making the model incapable of identifying the smaller amount of malicious traffic. On the other hand, traditional machine learning-based network intrusion detection typically uses a single model to classify traffic, which can lead to numerous problems. First, a single machine learning model has certain limitations, such as being prone to overfitting or underfitting and having difficulty handling multi-classification problems, resulting in low detection rates. Furthermore, a single model has poor generalization capabilities and cannot be applied to a wide range of network traffic. Therefore, if the data class distribution can be processed in advance and the data is trained using a combination of multiple models, the accuracy of intrusion detection can be further improved. Summary of the Invention

[0006] In response to the problems existing in the prior art, the present invention proposes a network traffic classification method and device based on machine learning, which can greatly improve the accuracy of the network traffic classification model through a new balanced data set method (SDsampling) and a new hierarchical ensemble learning framework.

[0007] To achieve the purpose, the present invention adopts the following specific technical solutions:

[0008] A network traffic classification method based on machine learning, the method comprising:

[0009] Obtain a training dataset containing normal traffic and abnormal traffic;

[0010] Calculating the stiffness IH of a sample in a training data set based on its nearest neighbors, and dividing the training data set into an easy set and a difficult set according to the stiffness IH;

[0011] According to the number of samples in each category, the difficult set is divided into a majority class sample set and a minority class sample set;

[0012] Perform K-means clustering on the majority class sample set and replace the samples in each cluster with the cluster center to obtain the sample set S K ;

[0013] Perform SMOTE oversampling on the majority class sample set to obtain the sample set S Z ;

[0014] Merge easy set, minority class sample set, sample set S K and sample set S Z , get the training data set S N ;

[0015] Based on the training dataset S N The network traffic classification model is trained, and the trained network traffic classification model is used to predict the traffic to be detected to obtain the classification result of the traffic to be detected.

[0016] Furthermore, obtaining a training data set includes:

[0017] Obtaining an original training data set; wherein the samples of the original training data set are normal traffic that has not been preprocessed or abnormal traffic that has not been preprocessed;

[0018] Preprocessing the samples of the original training data set to obtain a training data set; wherein the preprocessing includes:

[0019] Delete duplicate samples and samples with missing features in the original training dataset;

[0020] and,

[0021] Convert the symbolic data contained in the original training dataset into digital data using one-hot encoding;

[0022] and,

[0023] Perform numerical data normalization on samples in the original training dataset.

[0024] Furthermore, the stiffness Among them, x neighbor represents the nearest neighbor of sample x, k represents the nearest neighbor x neighbor The number of labels, label represents the label of the sample.

[0025] Furthermore, according to the stiffness IH, the training data set is divided into an easy set and a difficult set, including:

[0026] Get the hardness threshold IH′;

[0027] The samples whose stiffness IH is greater than the hardness threshold IH′ are placed into the difficult set;

[0028] The samples that are smaller than the hardness threshold IH′ and not larger than the hardness threshold IH′ are put into the easy set.

[0029] Furthermore, the network traffic classification model is a two-layer structure of an XGBoost model and a random forest model; wherein, the first layer of the network traffic classification model uses the XGBoost model to predict whether the network traffic is normal or abnormal, and the first layer of the network traffic classification model uses the random forest model to predict the attack type of abnormal network traffic.

[0030] Furthermore, the training data S N Train the network traffic classification model, including:

[0031] The training data set S N Split into training set and test set;

[0032] By duplicating the training set and the test set, a first training set, a first test set, a second training set, and a second test set are obtained, and samples in the first training set and the first test set are marked as normal or abnormal, and samples in the second training set and the second test set are marked with traffic categories;

[0033] Use the XGBoost model to train the first training set to obtain a binary classifier;

[0034] The second training set is trained using a random forest model to obtain a multi-classifier that distinguishes abnormal traffic types;

[0035] Use a binary classifier to classify the first test set and mark the abnormal samples in the first test set as the first data set;

[0036] Select data included in the first data set from the second test set;

[0037] The second test set is classified and predicted using a multi-classifier, the parameters of the XGBoost model and the random forest model are adjusted based on the prediction results, and the first training set is trained using the XGBoost model to obtain a binary classifier.

[0038] Furthermore, the first training set is trained using the XGBoost model to obtain a binary classifier, including:

[0039] For the first training set, the feature selection method based on random forest is used, and the weight of each feature of the sample is evaluated by the Gini index; wherein the Gini index Where D represents the first training set, |C k | represents the number of samples of the Kth class in the first training set;

[0040] The XGBoost model is trained based on several features with the largest weights to obtain a binary classifier.

[0041] A network traffic classification device based on machine learning, the device comprising:

[0042] A data acquisition module is used to obtain a training data set containing normal traffic and abnormal traffic;

[0043] The dataset generation module is used to calculate the stiffness IH of the sample based on the nearest neighbors of the sample in the training dataset, and divide the training dataset into an easy set and a difficult set according to the stiffness IH; divide the difficult set into a majority class sample set and a minority class sample set according to the number of samples in the category; perform K-means clustering on the majority class sample set and replace the samples in each cluster with the cluster center to obtain the sample set S K ; Perform SMOTE oversampling on the majority class sample set to obtain the sample set S Z ; Merge easy set, minority class sample set, sample set S K and sample set S Z , get the training data set S N ;

[0044] Training and prediction module, used to N The network traffic classification model is trained, and the trained network traffic classification model is used to predict the traffic to be detected to obtain the classification result of the traffic to be detected.

[0045] A storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when running.

[0046] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform any of the above methods.

[0047] Compared with the prior art, the present invention has the following positive effects:

[0048] This paper conducted experiments on the CICIDS2017 network traffic dataset, evaluating the model's performance using overall accuracy, precision, recall, F1 score, false positive rate, and false negative rate. The results demonstrate that the proposed model outperforms existing baseline recognition methods, including decision trees, random forests, XGBoost, and deep neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the overall flow chart of the present invention.

[0050] Figure 2 It is a flow chart of the SDsampling algorithm proposed in the present invention.

[0051] Figure 3 It is a flow chart of the hierarchical integration model framework proposed in the present invention.

[0052] Figure 4 This is the distribution of the number of samples in each category of the CICIDS2017 dataset used in this paper.

[0053] Figure 5 This is the case of parameter tuning of the number of features in the feature selection module of the present invention. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention and to make the purposes, features, and advantages of the present invention more readily apparent, the core technology of the present invention is further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention.

[0055] The overall idea of ​​the network traffic classification method of the present invention is to preprocess the network traffic data first, then use the SDsampling algorithm to balance the network traffic, and then use the hierarchical integrated model framework to train the model and detect attack behaviors in the network traffic data.

[0056] Specifically, the SDsampling method divides samples into two categories, easy-to-classify and difficult-to-classify, based on their spatial distribution. The samples in the easy-to-classify set remain unchanged. For the samples in the difficult-to-classify set, the dataset is balanced using a combination of the Synthetic Minority Oversampling Technique (SMOTE) algorithm and random undersampling, based on the number of samples in each category. This method overcomes the overgeneralization of traditional SMOTE while combining the principles of undersampling and oversampling. The balanced samples are then input into a hierarchical ensemble learning framework based on their categories, resulting in more accurate classification of network traffic.

[0057] The overall flow chart of the present invention is as follows: Figure 1 As shown, the specific steps of the method are described in detail as follows:

[0058] (1) Data acquisition module

[0059] The dataset used in the present invention is CICIDS2017. The CICIDS2017 dataset is a widely used dataset collected by the Canadian Institute for Cyber ​​Security in 2017. It contains normal traffic and abnormal traffic, and is simulated and generated in a real network environment, which is closer to the real situation and more reliable. The CICIDS2017 dataset provides the original pcap package, and the present invention extracts statistical features from it as the original dataset for the experiment. This is because most of the transmission content in the dataset is encrypted, so it is difficult to obtain semantic features from the traffic, but the statistical distribution of normal traffic packets and abnormal traffic packets in the session is different, such as the number and length of packets. Therefore, this article uses statistical features instead of original traffic for classification. It should be noted that all packets of a forward flow / reverse flow need to be collected at the end of the session before the statistical features of the flow can be calculated.

[0060] In terms of sample size, the CICIDS2017 dataset contains 2,830,743 records, including 2,273,097 records of normal traffic and 557,646 records of abnormal traffic. This indicates that the dataset is highly unbalanced. In terms of feature size, the CICIDS2017 dataset is high-dimensional, containing 84 feature columns and one label column. To improve the model's generalization, we removed five features: "flow ID," "source IP," "source port," "destination IP," and "timestamp." As a result, the final dataset contains only 79 feature columns and one label column.

[0061] (2) Preprocessing module

[0062] In order to remove redundant data, improve detection efficiency and reduce time consumption, the present invention performs data preprocessing on network traffic data, including removing duplicate and missing values, digitizing symbolic features and normalizing numerical data.

[0063] Remove duplicate and missing values: Delete duplicate samples and samples with missing features in the dataset.

[0064] Symbolic feature digitization: The CICIDS2017 dataset contains symbolic feature data, which is difficult for the model to process directly. Therefore, a one-hot encoder is used in this step to convert the symbolic data into digital data.

[0065] Normalization of numerical data: Data normalization can solve the problem of large dimensional differences between different feature data, and is therefore widely used in data preprocessing steps. In order to ensure the reliability of the detection results, the numerical data in the two data sets need to be normalized. Normalization refers to reducing all feature data to the range of [0, 1]. The present invention proposes to use the Min-Max normalization method to process the data, and its conversion formula is: Where x represents the attribute value of a certain feature, x max Represents the maximum value of this characteristic attribute, x min represents the minimum value of this feature attribute, and x′ represents the result of normalization of x.

[0066] (3) Data sampling module

[0067] In network traffic data, abnormal traffic data is usually far less than normal traffic data, which causes the decision function to be biased towards the majority samples, and low-frequency attack samples are regarded as noise and ignored. Therefore, in order to improve the detection performance of the model, it is necessary to process data with smaller data volumes. There are usually two processing methods, namely solutions at the algorithm level and solutions at the data level. This paper proposes a sampling method at the data level - SDsampling. Specifically, Figure 2 As shown, the process is as follows:

[0068] Step 1: Find k nearest neighbors for each sample in the dataset S, and then calculate the instance hardness IH of each sample.

[0069]

[0070] Where x neighbor ——one of the k nearest neighbors of sample x

[0071] x——input sample

[0072] Step 2: Set an instance hardness threshold IH′ and add samples whose instance hardness is less than or equal to the threshold to the easy set, denoted as S E ; add samples whose instance hardness is greater than the threshold into the difficult set, denoted as S D .

[0073] Step 3: Set a threshold upperbound. For the number of samples in any category, if the value is greater than or equal to upperbound, then add the category to the majority class sample set, denoted as S. maxi ; If it is less than the upperbound, then the category is added to the minority class sample set, recorded as S mini .

[0074] Step 4: From the difficult set S D Find the one that belongs to S maxi The samples of the majority class in the difficult set are called samples of the majority class in the difficult set, denoted by S maj . Maj Perform K-means clustering, and then use the cluster center to replace the samples in each cluster. The final generated sample is recorded as S K .

[0075] Step 5: From the difficult set S D Find the one that belongs to S mini The samples are called the minority samples that are difficult to concentrate, denoted as S Min . Min SMOTE oversampling is performed on the minority class sample categories in the dataset. The oversampling threshold is set to upperbound, and the sampled samples are recorded as S Z .

[0076] Step 6: Combine the processed samples obtained above according to the following formula, and record the combined result as S N

[0077] S N =S E +S K +S Min +S Z Formula (2)

[0078] (4) Feature selection module

[0079] Machine learning algorithms struggle to learn from high-dimensional data. Feature selection is an effective approach to address this problem. By preselecting highly weighted features for training, it improves performance while conserving computing resources. This module uses a feature selection method based on random forests, using the Gini index to evaluate the weight of each feature. A random forest consists of many CART trees, which vote to determine the final classification result.

[0080] The Gini index describes purity. The smaller the value, the higher the purity. Therefore, in the CART tree, the Gini index is used to evaluate the change in node purity before and after the feature is split. The smaller the value, the better the feature. For the sample set D, there are K classes, and the sample size of the Kth class is |C k|, the sample size of D is |D|, then the Gini index expression of sample set D is:

[0081]

[0082] (5) Classification module

[0083] This module focuses on traffic classification. We propose a two-layer architecture that combines XGBoost and random forest. The first layer uses the XGBoost model to distinguish between normal and abnormal samples in the dataset, and the second layer uses the random forest model to distinguish the attack type of each abnormal sample.

[0084] XGBoost is a boosted tree model that combines many tree models to form a powerful ensemble classifier. The idea behind XGBoost is to train K trees, and the final prediction is the sum of the predictions from these K trees. It is an improvement on the gradient boosting algorithm, achieving high accuracy in a very short time. Random forest is an ensemble model that uses multiple decision trees to classify samples and then vote on the classification results. The randomness of random forests is reflected in random data sampling and random feature selection, which results in faster training and higher accuracy.

[0085] Combining XGBoost and random forest's two-layer structure, a hierarchical method is used to perform multi-classification of traffic. Its workflow is as follows Figure 3 As shown, it mainly includes the following three steps:

[0086] Step 1: Dataset Construction. Duplicate the training and test sets into two copies, labeled training set 1, test set 1, training set 2, and test set 2, respectively, and recode the labels. Label all normal samples in training set 1 and test set 1 as 0, and abnormal samples as 1. Label all normal samples in training set 2 and test set 2 as 0, and abnormal samples as 1, 2, 3, ..., m, in sequence. Where m is the number of abnormal categories.

[0087] Step 2: Model training. Use the XGBoost model to train training set 1, obtaining a binary classifier, denoted as Classifier 1. Use the Random Forest model to train training set 2, obtaining a multi-class classifier that can distinguish abnormal traffic types, denoted as Classifier 2.

[0088] Step 3: Classification. Use classifier 1 to classify test set 1. Samples classified as normal are labeled data0, and samples classified as abnormal are labeled data1. Select the data contained in data1 from test set 2. Then, use classifier 2 to classify and predict test set 2. Samples classified as normal are labeled as added to data0, and samples classified as abnormal are labeled data1, data2, ..., datam.

[0089] (6) Comparison of results

[0090] The present invention conducted experiments on the network traffic dataset of CICIDS2017 and evaluated the performance of the model using overall accuracy, precision, recall, F1 value, false positive rate, and false negative rate. To verify the advantages of the proposed sampling method (SDsampling), the present invention conducted experiments on no sampling, SMOTE, and random sampling. On the other hand, to verify the superiority of the proposed hierarchical ensemble model framework, six commonly used machine learning methods were used for comparison: K-nearest neighbor, decision tree, support vector machine, deep neural network, random forest, and XGBoost.

[0091] 1) Performance comparison with common sampling methods

[0092] The performance comparison results of this experiment on the CICIDS2017 dataset are shown in Tables 1, 2, 3, and 4. Table 1 shows the classification performance of the model in the first layer (binary classification of normal and abnormal traffic) and the second layer (multiple classification of specific abnormal traffic types) under no sampling; Table 2 shows the classification performance of the model in the first and second layers under SMOTE sampling; Table 3 shows the classification performance of the model in the first and second layers under random sampling; and Table 4 shows the classification performance of the model in the first and second layers under SDsampling sampling. Comparing the four tables shows that the classification performance of the four sampling modes is roughly the same when it comes to distinguishing normal and abnormal traffic. However, when it comes to distinguishing specific types of abnormal traffic, the model's classification ability for minority samples in the SDsampling dataset is significantly improved, reaching over 99% accuracy for each attack. This is because the SDsampling algorithm oversamples minority samples and clusters majority samples. This reduces the imbalance between majority and minority samples, making it more effective for the model to distinguish abnormal traffic using minority samples.

[0093] Table 1 Classification performance of the original dataset

[0094]

[0095]

[0096] Table 2 Classification performance of the SMOTE algorithm sampling data set

[0097]

[0098] Table 3 Classification performance of randomly sampled datasets

[0099]

[0100] Table 4 Classification performance of SDsampling sampling dataset

[0101]

[0102]

[0103] 2) Performance comparison with machine learning methods

[0104] Table 5 shows the performance comparison results of the seven models in this experiment. This comparison reveals that, under the same sampling pattern, the two-layer ensemble model significantly outperforms the other six models on all six evaluation metrics. The two-layer structure is more effective because the first layer can detect normal traffic and remove it from the dataset, while the second layer only detects traffic that the first layer identifies as anomalous. This increases the proportion of minority samples in the test set, reduces the impact of dataset imbalance, and improves the model's ability to classify minority samples.

[0105] Table 5 Classification performance of seven models under three sampling modes

[0106]

[0107] In summary, the reason why the present invention chooses the SDsampling method to balance the category distribution of traffic data is that it has the following advantages: (1) Compared with the traditional SMOTE algorithm, SDsampling only samples the minority category samples that are determined to be difficult to concentrate, and the number of sampling times is much less than that of the SMOTE algorithm, thereby reducing the space complexity and time complexity. (2) Compared with the traditional SMOTE algorithm, this algorithm not only has the ability to oversample the minority category samples, but also has the ability to undersample the majority category samples to a certain extent. (3) When undersampling, the algorithm clusters the samples that are difficult to classify in the majority category based on whether the samples are easy to classify, which will reduce the impact of these samples and retain the information of these samples to a certain extent. (4) When oversampling, compared with SMOTE, which samples all abnormal category samples indiscriminately, this algorithm further distinguishes the abnormal category samples, only oversamples the minority category in the abnormal category samples, and only performs SMOTE oversampling on the category with difficult-to-classify samples. In this way, the sampled data will not have the problem of distribution marginalization.

[0108] In addition, due to the limited ability of a single model to extract data information, it may lead to low training accuracy. The use of a hierarchical ensemble model can comprehensively consider the results of multiple models, making the final classification effect better. The general process of the hierarchical ensemble model framework is as follows: (1) Dataset construction: The CICIDS2017 dataset is divided into a training set and a test set, and the training set and test set are copied into two copies, and the labels are re-encoded respectively. (2) Model training: The first CICIDS2017 copy training set is trained using the XGBoost model to obtain a binary classifier; the second CICIDS2017 copy training set is trained using the random forest model to obtain a multi-class classifier that can distinguish between abnormal traffic types. (3) Classification: The two trained classifiers are used to perform hierarchical classification on the test set.

[0109] The above-described embodiments merely illustrate the implementation methods of the present invention, and their descriptions are relatively specific, but they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the appended claims.

Claims

1. A network traffic classification method based on machine learning, characterized in that: The method comprises: Obtain a training dataset containing normal traffic and abnormal traffic; Based on the nearest neighbors of a sample in a training data set, a stiffness IH of the sample is calculated, and according to the stiffness IH, the training data set is divided into an easy set and a difficult set; wherein dividing the training data set into the easy set and the difficult set includes: Get the hardness threshold IH′; The samples whose stiffness IH is greater than the hardness threshold IH′ are placed into the difficult set; Put the samples that are less than the hardness threshold IH′ and not greater than the hardness threshold IH′ into the easy set; According to the number of samples in each category, the training data set is divided into the majority class sample set S maxi And the minority class sample set S mini ; Perform K-means clustering on the majority class samples that are difficult to concentrate, and replace the samples in each cluster with the cluster center to obtain the sample set S K ; Find the sample set S belonging to the minority class from the difficult set mini Samples of the minority class are obtained, which are difficult to concentrate. Min ; For the minority class samples S that are difficult to concentrate Min Perform SMOTE oversampling to obtain the sample set S Z ; Merge the minority class samples S that are easy to collect and difficult to collect Min , sample set S K and sample set S Z , get the training data set S N ; Based on the training dataset S N The network traffic classification model is trained, and the trained network traffic classification model is used to predict the traffic to be detected to obtain the classification result of the traffic to be detected.

2. The method according to claim 1, wherein The obtaining of the training data set includes: Obtaining an original training data set; wherein the samples of the original training data set are normal traffic that has not been preprocessed or abnormal traffic that has not been preprocessed; Preprocessing the samples of the original training data set to obtain a training data set; wherein the preprocessing includes: Remove duplicate samples and samples with missing features from the original training dataset; and, Convert the symbolic data contained in the original training dataset into digital data using one-hot encoding; and Perform numerical data normalization on samples in the original training dataset.

3. The method according to claim 1, wherein The stiffness Among them, x neighbor represents the nearest neighbor of sample x, k represents the nearest neighbor x neighbor The number of labels, label represents the label of the sample.

4. The method according to claim 1, wherein The network traffic classification model is a two-layer structure of an XGBoost model and a random forest model; wherein the first layer of the network traffic classification model uses the XGBoost model to predict whether the network traffic is normal or abnormal, and the second layer of the network traffic classification model uses the random forest model to predict the attack type of abnormal network traffic.

5. The method according to claim 1, wherein The training data set S N Train the network traffic classification model, including: The training data set S N Split into training set and test set; By duplicating the training set and the test set, a first training set, a first test set, a second training set, and a second test set are obtained, and samples in the first training set and the first test set are marked as normal or abnormal, and samples in the second training set and the second test set are marked with traffic categories; Use the XGBoost model to train the first training set to obtain a binary classifier; The second training set is trained using a random forest model to obtain a multi-classifier that distinguishes abnormal traffic types; Use a binary classifier to classify the first test set and mark the abnormal samples in the first test set as the first data set; Select data included in the first data set from the second test set; The second test set is classified and predicted using a multi-classifier, the parameters of the XGBoost model and the random forest model are adjusted based on the prediction results, and the first training set is trained using the XGBoost model to obtain a binary classifier.

6. The method according to claim 5, wherein The XGBoost model is used to train the first training set to obtain a binary classifier, including: For the first training set, the feature selection method based on random forest is used, and the weight of each feature of the sample is evaluated by the Gini index; wherein the Gini index Where D represents the first training set, |C k | represents the number of samples of the kth class in the first training set; The XGBoost model is trained based on several features with the largest weights to obtain a binary classifier.

7. A network traffic classification device based on machine learning, characterized in that: The device comprises: A data acquisition module is used to obtain a training data set containing normal traffic and abnormal traffic; The data sampling module is used to calculate the stiffness IH of the sample based on the nearest neighbors of the sample in the training data set, and divide the training data set into easy sets and difficult sets according to the stiffness IH; divide the training data set into majority class sample sets S according to the number of samples in each class maxi And the minority class sample set S mini ; Perform K-means clustering on the majority class samples that are difficult to concentrate, and replace the samples in each cluster with the cluster center to obtain the sample set S K ; Find the sample set S belonging to the minority class from the difficult set mini Samples of the minority class are obtained, which are difficult to concentrate. Min ; For the minority class samples S that are difficult to concentrate Min Perform SMOTE oversampling to obtain the sample set S Z ; Merge the minority class samples S that are easy to collect and difficult to collect Min , sample set S K and sample set S Z , get the training data set S N ; wherein dividing the training data set into an easy set and a difficult set comprises: Get the hardness threshold IH′; The samples whose stiffness IH is greater than the hardness threshold IH′ are placed into the difficult set; Put the samples that are less than the hardness threshold IH′ and not greater than the hardness threshold IH′ into the easy set; Classification module, used to classify N The network traffic classification model is trained, and the trained network traffic classification model is used to predict the traffic to be detected to obtain the classification result of the traffic to be detected.

8. A storage medium storing a computer program, wherein: The computer program is configured to execute the method according to any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

Images