Collaborative defense method for DDOS attacks in software-defined networks based on ensemble learning

Abstract

The present invention relates to the field of intrusion detection technology, and in particular to a software defined network DDOS attack collaborative defense method based on ensemble learning. Firstly, a feature selection algorithm based on bagging ensemble is proposed to avoid the situation that a single feature selection algorithm ignores the connection between features and tends to select redundant features; secondly, a pairwise diversity metric is introduced for selective integration to select a relatively excellent heterogeneous integration model, and a stratified ten-fold cross validation method is adopted to avoid overfitting; finally, a weighted voting mechanism based on bagging ensemble algorithm is adopted to integrate a base classifier model, and the model is embedded in an SDN controller, and a detection time interval is set to realize real-time monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of intrusion detection, and in particular to a software defined network DDOS attack collaborative defense method based on ensemble learning. Background Art

[0002] In recent years, with the rapid development of the Internet, the scale of the network has become increasingly large, and the pressure on traditional networks has increased. Software Defined Networking (SDN) uses a programmable method to achieve the separation of forwarding and control, which can effectively manage network device resources, and embodies the controller-centric management model. It plays an important role in the development of cloud computing networks. However, this model will bring corresponding attack opportunities to various malicious applications. Although OpenFlow can provide some flow-based security detection methods, its underlying assumption is that the application software in the SDN is not maliciously attacked or the northbound interface is not damaged. This will play a certain security protection role for SDN, but it is helpless against DDOS attacks. Based on the fine-grained control of data flows by SDN, the controller can be used to conveniently perform packet filtering, rate limiting and attack tracing in DDOS attacks. However, the traditional classification algorithm uses a relatively single feature dimensionality reduction algorithm for feature dimensionality reduction, without considering the connection between features, and this method only uses a single algorithm to evaluate the model, resulting in weak model stability and redundant features that waste computing resources. Secondly, only a single classification algorithm is used as support in the design of the detection algorithm, or only the accuracy value is used as the weighting indicator of the integrated classifier, which leads to biased classification results. Summary of the invention

[0003] The purpose of the present invention is to provide a software defined network DDOS attack collaborative defense method based on ensemble learning, which avoids the situation that the single feature selection algorithm in the existing defense method ignores the connection between features, and adopts a layered ten-fold cross-validation method to avoid the overfitting problem.

[0004] To achieve the above object, the present invention provides a software defined network DDOS attack collaborative defense method based on ensemble learning, comprising the following steps:

[0005] Start the SDN controller to collect traffic data and store the traffic data in a CSV file;

[0006] Put the collected data into the Double-Bagging detection model for training;

[0007] The integrated model generated by training is embedded in the SDN controller, and the DDOS attack detection module is started.

[0008] Among them, the traffic data includes data of normal traffic and attack traffic, and is composed of the speed of the IP source, the traffic count, the speed of the traffic table item and the traffic comparison value. The values ​​of the speed of the IP source, the traffic count, the speed of the traffic table item and the traffic comparison value are lower in normal traffic than in attack traffic.

[0009] The establishment process of the Double-Bagging detection model includes the following steps:

[0010] Perform data preprocessing and feature dimension reduction, and obtain the optimal feature subset through the feature subset selection voting mechanism based on the bagging ensemble algorithm;

[0011] The optimal feature subset is input into the base classifier for training, and a high-quality heterogeneous ensemble learning base classifier model is selected;

[0012] A weighted voting mechanism based on the Bagging ensemble algorithm is used to select the ensemble of base classifier models.

[0013] Among them, feature dimensionality reduction adopts a dimensionality reduction algorithm that combines filtering method and embedding method, and selects two feature selection algorithms in the filtering method, the chi-square verification and mutual information algorithm, and the extreme random tree algorithm in the embedding method to calculate the ranking of feature contributions.

[0014] The process of obtaining the optimal feature subset is to determine the number of results of each feature selection algorithm by setting a threshold, generate the result feature subset according to the feature contribution ranking, and use a voting strategy to select the feature subset with the largest number of occurrences as the optimal feature subset.

[0015] In the process of inputting the optimal feature subset into the base classifier for training and selecting a high-quality heterogeneous ensemble learning base classifier model, the diversity metric in ensemble learning is introduced to evaluate the combination effect between base classifiers, and the Bayesian optimization parameters are used to complete the first base classifier filtering by calculating the accuracy and area under the curve of the model.

[0016] The weight of the weighted voting mechanism is determined by the accuracy and the area under the curve value, and the area under the curve value is obtained by calculating the receiver operating characteristic curve of the base classifier.

[0017] Among them, after starting the DDOS attack detection module, when normal traffic is generated, the Double-Bagging model algorithm predicts it as normal traffic. When attack traffic is generated, it is immediately detected as a DDOS attack and the port it enters is blocked.

[0018] The present invention provides a software defined network DDOS attack collaborative defense method based on ensemble learning. Firstly, a feature selection algorithm based on bagging ensemble is proposed to avoid the situation that a single feature selection algorithm ignores the connection between features and tends to select redundant features. Secondly, a pairwise diversity metric is introduced for selective integration to select a relatively excellent heterogeneous ensemble model, and a layered ten-fold cross validation method is used to avoid overfitting. Finally, a weighted voting mechanism based on bagging ensemble algorithm is used to integrate base classifier models, and the model is embedded in an SDN controller, and a detection time interval is set to realize real-time monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a software defined network DDOS attack collaborative defense method based on ensemble learning of the present invention.

[0021] Figure 2 It is a specific implementation flow chart of the software-defined network DDOS attack collaborative defense method based on integrated learning of the present invention.

[0022] Figure 3 It is a flow chart of the Double-Bagging algorithm of the present invention;

[0023] Figure 4 This is a flow chart of the feature selection voting mechanism based on Bagging of the present invention. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0025] The following are some nouns and terminology descriptions of the present invention:

[0026] Software Defined Networking (SDN);

[0027] Receiver operating characteristic (ROC);

[0028] Area Under Curve (AUC).

[0029] See also Figure 1 The present invention provides a software-defined network DDOS attack collaborative defense method based on ensemble learning, comprising the following steps:

[0030] S1: Start the SDN controller to collect traffic data and store the traffic data in a CSV file;

[0031] S2: Put the collected data into the Double-Bagging detection model for training;

[0032] S3: Embed the integrated model generated by training into the SDN controller and start the DDOS attack detection module.

[0033] The overall design of the collaborative defense method for DDOS attacks in software-defined networks based on ensemble learning is as follows: Figure 2 As shown in the figure, the design process of the collaborative defense model includes three aspects, namely, the traffic collection module, the algorithm model selection module and the detection and defense mechanism design module. The model is embedded in the Ryu controller, the detection time interval is set, and real-time monitoring is realized. The detailed implementation process is as follows:

[0034] S1: Start the SDN controller to collect traffic data and store the traffic data in a CSV file, wherein normal traffic data is collected first and then attack traffic data is collected for use in Double-Bagging model algorithm training;

[0035] The characteristics and parameters that need to be monitored and collected are as follows:

[0036] (1) Speed ​​of IP sources: This characteristic gives the total number of TP sources entering the network within a specific time interval. Abbreviated as SSIP, it is defined as formula 1:

[0037]

[0038] Where SumIPsrc is the total number of IP sources entering each flow, and T is the sampling time interval. The detection system monitors traffic and collects data every T seconds, and saves the number of source IPs in that time period. The controller needs to have enough normal traffic and attack traffic data for the machine learning algorithm to predict attacks. For normal attacks, SSIP is usually low, while for attacks, the count is usually high.

[0039] (2) Traffic count: Each traffic entering the network has a specific traffic count. Normal traffic is less than DDOS attack traffic.

[0040] (3) Flow table speed: The total number of flow table entries in the network switches within a certain time interval. N . Abbreviated as SFE, defined as formula 2:

[0041]

[0042] This is a very relevant feature for attack traffic detection, since in case of DDOS attack the number of flow table entries in a fixed time interval can increase significantly compared to the rate value of flow table entries for normal traffic.

[0043] (4) Traffic comparison value: refers to the total number of traffic entries flowing into the switch within T time periods. N , which is the number of interactive IPs divided by the total traffic. Abbreviated as RPF, it is defined as formula 3:

[0044]

[0045] Where srcIP is the total number of collaborative IPs in the network flow. In the case of normal traffic, the IP source of the i-th flow is the same as the destination IP of the j-th flow, and the IP source of the j-th flow is the same as the destination IP of the i-th flow. This indicates an interactive flow, which is not the case when it is a DDOS attack traffic. When under attack, the flow table entries with a time T to reach the target host increase rapidly, and the target host cannot respond. Therefore, when the DDOS attack starts, the attack traffic decreases suddenly. Dividing the total number of collaborative flows by the total traffic makes this detection parameter scalable to networks under different operating conditions.

[0046] S2: Put the collected data into the Double-Bagging detection model for training;

[0047] Among them, the steps of establishing the Double-Bagging detection model are as follows: Figure 3 As shown in the figure, a voting mechanism based on bagging is first proposed to select the optimal feature subset. Secondly, a ten-fold cross-validation method is used to detect anomalies in the data set and select the base classifier model. Finally, a weighted voting mechanism based on the bagging ensemble algorithm is used to integrate the base classifier models. For the weighting of the base classifier, it is proposed to use the combination of accuracy and AUC value as the weight to integrate the classifier, which effectively solves the problem of uneven weighting.

[0048] Step S2.1, perform data preprocessing and feature dimensionality reduction. First, perform data standardization, and then perform feature dimensionality reduction. The present invention proposes a dimensionality reduction algorithm that combines filtering and embedding methods, which can avoid feature selection bias as much as possible and make up for the shortcomings of each single feature selection algorithm. The framework diagram is as follows: Figure 4 As shown. First, the feature contribution (weight) is calculated for the two feature selection algorithms, chi-square validation and mutual information algorithm, in the filtering method, and the extreme random tree algorithm in the embedding method.

[0049] The feature contribution calculation algorithm of chi-square verification is shown in formula 4:

[0050]

[0051] Where c represents the degrees of freedom, O represents the observed value, and E represents the expected value.

[0052] The feature contribution calculation algorithm of the mutual information algorithm is shown in Formula 5:

[0053]

[0054] Among them, p(x) represents the probability of X=xi occurring, p(y) represents the probability of Y=yi occurring, and p(x,y) represents the probability of X=xi and Y=yi occurring at the same time, that is, the joint probability.

[0055] The extreme random tree feature contribution calculation algorithm is shown in Formula 6:

[0056]

[0057] k represents k categories, and Pk represents the sample weight of category k.

[0058] The feature contributions (weights) of the three algorithms are then ranked. The larger the weight, the more important the feature. Each feature selection algorithm assigns weights to each feature according to its own algorithm criteria and generates its own feature weight distribution. According to the feature weight ranking of the three feature selection algorithms, the voting mechanism based on the bagging ensemble algorithm is used to select the optimal feature subset. This mechanism determines the number of results of each method through a set threshold, generates multiple result feature subsets for the feature contribution ranking of the three methods, and finally uses the voting strategy to select the feature subset with the most occurrences as the optimal feature subset.

[0059] Step S2.2: Input the optimal feature subset obtained in step S2.1 into the base classifier for training, introduce the diversity metric in ensemble learning to evaluate the combination effect between base classifiers, and select a high-quality heterogeneous ensemble learning base classifier model. For the base classifier model to be selected, Bayesian optimization parameters are used to complete the first base classifier filtering by calculating the accuracy (ACC) and area under the curve (AUC) of the model.

[0060] The calculation formula of model ACC is shown in Equation 7:

[0061]

[0062] In Formula 7, TP represents the number of samples whose predicted values ​​are consistent with the true values ​​and are both positive; FP represents the number of samples whose predicted values ​​are positive and the true values ​​are negative; TN represents the number of samples whose predicted values ​​are consistent with the true values ​​and are both negative; FN represents the number of samples whose predicted values ​​are negative and the true values ​​are positive.

[0063] AUC uses the receiver operating characteristic (ROC) curve, and its calculation method is shown in formula 8. AUC of 1 corresponds to an ideal classifier, and its expression formula is as follows:

[0064]

[0065] In formula 8, M p , M n They represent the number of positive and negative samples respectively, i represents the sorting number of the positive sample, and M*N represents the number of cases where one sample is randomly selected from each of the positive and negative samples.

[0066] In order to obtain the best prediction effect, it is also necessary to select base learners with relatively large differences for the first base classifier filtering. For the selective integration of models, the pairwise diversity metric Dis, Q statistics and double failure metric DF based on ensemble learning are introduced.

[0067] The inconsistency measure Dis focuses on the samples with different classification results of the two classifiers. The more samples with different classification results, the higher the degree of difference between the two classifiers. On the contrary, the less diversity, the value range is [0,1]. The calculation of the inconsistency measure Dis is shown in formula 9. The total number of samples is represented by N.

[0068]

[0069] The Q statistic is related to the diversity between classifiers, and its value range is [-1,1]. If two classifiers are relatively independent, it means that the classification methods are completely unrelated, that is, their Q statistic value is 0. The Q statistic is calculated as follows:

[0070] As shown in formula 10.

[0071]

[0072] The double failure metric DF is mainly aimed at samples that are misclassified by two classifiers, and its value range is [0,1]. The more samples the two classifiers misclassify, the more likely the two classifiers are to make mistakes on the same samples. The calculation of the double failure metric DF is shown in Equation 11. The total number of samples is represented by N.

[0073]

[0074] The present invention performs metric calculations on single classifiers in the target base classifier group in pairs, tending to select a base classifier combination whose inconsistency metric Dis and Q statistics tend to 1, and whose DF value tends to 0, so as to select the most excellent quality ensemble learning base classifier model. In addition, in order to avoid overfitting of the model, a ten-fold cross-validation is used, and the training set is divided into 10 sub-samples. A single sub-sample is retained as data for verifying the model, and the other (10-1) samples are used for training. The cross-validation is repeated 10 times, and each sub-sample is verified once. The results of 10 times are averaged or other combined methods are used to finally obtain a single estimate, thereby the fitting of the model.

[0075] Step S2.3: Use a weighted voting mechanism based on the Bagging ensemble algorithm to select the base classifier model in step S2.2. The system obtains the ROC curve of the base classifier and calculates the AUC value based on the ROC curve, and then uses the accuracy and AUC value as the weights of the weighted voting mechanism. The inter-classifier ensemble weighting function is shown in Formula 12:

[0076]

[0077] In formula 12, U aoc,i represents the area not covered by the ROC curve of the i-th base learner (U aoc,i =1-AUC) and are the maximum and minimum values ​​of the uncovered area of ​​the ROC curve in all base learners, respectively. i , e b , e w They represent the accuracy of the i-th classifier, the accuracy of the classifier with the lowest accuracy in the set, and the accuracy of the classifier with the highest accuracy in the set, respectively.

[0078] S3: Embed the integrated model generated by training into the SDN controller and start the DDOS attack detection module.

[0079] The integrated model generated in step S2.3 is embedded into the SDN controller, and the DDOS attack detection module is started. When normal traffic is generated, the Double-Bagging model algorithm predicts it as normal traffic. When attack traffic is generated, it is immediately detected as a DDOS attack and the port it enters is blocked. If a port is blocked, the controller still allows normal traffic from other ports to pass. The controller unblocks the port after a period of time and restarts detection. If the attack is still active, it will detect and block the port again. The blocking will continue as long as the attack continues.

[0080] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A software-defined network DDOS attack collaborative defense method based on ensemble learning, characterized in that: The following steps are involved: Start the SDN controller to collect traffic data and store the traffic data in a CSV file; The traffic data includes data of normal traffic and attack traffic, and is composed of a speed of an IP source, a traffic count, a speed of a traffic table item, and a traffic comparison value, wherein the values ​​of the speed of the IP source, the traffic count, the speed of the traffic table item, and the traffic comparison value are all lower in normal traffic than in attack traffic; Put the collected data into the Double-Bagging detection model for training; The establishment process of the Double-Bagging detection model includes the following steps: Perform data preprocessing and feature dimension reduction, and obtain the optimal feature subset through the feature subset selection voting mechanism based on the bagging ensemble algorithm; The optimal feature subset is input into the base classifier for training, and a high-quality heterogeneous ensemble learning base classifier model is selected; A weighted voting mechanism based on the Bagging ensemble algorithm is used to select the ensemble of base classifier models; The feature dimensionality reduction adopts a dimensionality reduction algorithm that combines the filtering method and the embedding method, and selects two feature selection algorithms in the filtering method, the chi-square verification and the mutual information algorithm, and the extreme random tree algorithm in the embedding method to calculate the ranking of feature contributions; The feature contribution calculation algorithm formula of chi-square validation is as follows: Where c represents the degree of freedom, O represents the observed value, and E represents the expected value; The feature contribution calculation algorithm formula of the mutual information algorithm is as follows: Where p(x) represents the probability of X=xi, p(y) represents the probability of Y=yi, and p(x,y) represents the probability of X=xi and Y=yi occurring at the same time, i.e., the joint probability. The formula for calculating the extreme random tree feature contribution algorithm is as follows: k represents k categories, and Pk represents the sample weight of category k; The process of obtaining the optimal feature subset is to determine the number of results of each feature selection algorithm by setting a threshold, generate the result feature subset according to the feature contribution ranking, and use the voting strategy to select the feature subset with the most occurrences as the optimal feature subset; Each feature selection algorithm assigns weights to each feature according to its own algorithm criteria and generates its own feature weight distribution; according to the feature weight ranking of the three feature selection algorithms, the Voting mechanism based on the bagging ensemble algorithm is used to select the optimal feature subset; the number of results of each method is determined by the set threshold, and the feature contribution ranking of the three methods is used to generate multiple result feature subsets. Finally, the Voting strategy is used to select the feature subset with the largest number of occurrences as the optimal feature subset; In the process of inputting the optimal feature subset into the base classifier for training and selecting a high-quality heterogeneous ensemble learning base classifier model, the diversity metric in ensemble learning is introduced to evaluate the combination effect between the base classifiers, and the Bayesian optimization parameters are used to complete the first base classifier filtering by calculating the accuracy and area under the curve of the model; The calculation formula of model accuracy ACC is as follows: Among them, TP represents the number of samples whose predicted values ​​are consistent with the true values ​​and are both positive; FP represents the number of samples whose predicted values ​​are positive and the true values ​​are negative; TN represents the number of samples whose predicted values ​​are consistent with the true values ​​and are both negative; FN represents the number of samples whose predicted values ​​are negative and the true values ​​are positive; The area under the curve (AUC) uses the receiver operating characteristic (ROC) curve. An AUC of 1 corresponds to an ideal classifier, and the expression formula is as follows: Among them, M p , M n Respectively represent the number of positive and negative samples, i represents the sorting number of the positive sample, and M*N represents the number of cases where one sample is randomly selected from each of the positive and negative samples; The weight of the weighted voting mechanism is determined by the accuracy and the area under the curve value, and the area under the curve value is obtained by calculating the receiver operating characteristic curve of the base classifier; The ensemble weighting function between classifiers is as follows: Among them, U aoc,i Represents the area not covered by the ROC curve of the i-th base learner, that is, U aoc,i =1-AUC,U aoc,w represents the maximum value of the uncovered area of ​​the ROC curve among all base learners, U aoc,b represents the minimum value of the uncovered area of ​​the ROC curve among all base learners, e i represents the accuracy of the i-th classifier, e b represents the accuracy of the classifier with the lowest accuracy in the set, e w Indicates the accuracy of the classifier with the highest accuracy in the set; The integrated model generated by training is embedded in the SDN controller, and the DDOS attack detection module is started.

2. The software-defined network DDOS attack collaborative defense method based on ensemble learning as claimed in claim 1, characterized in that: After starting the DDOS attack detection module, when normal traffic is generated, the Double-Bagging model algorithm predicts it as normal traffic. When attack traffic is generated, it is immediately detected as a DDOS attack and the port through which it enters is blocked.

Images