Method and system for detecting abnormal traffic based on improved XGBoost algorithm of elite sparrow search

Abstract

The application relates to an abnormal traffic flow detection method based on an elite sparrow search improved XGBoost algorithm, wherein the method steps are as follows: (1) initializing network traffic flow data with an abnormal traffic flow environment collected in real time to obtain a traffic flow data set with an initial feature set; (2) performing feature extraction on the traffic flow data set in step (1) to obtain a traffic flow data set with an extracted feature set; (3) inputting the traffic flow data set with the initial feature set in step (2) into an elite sparrow search improved XGBoost model to output a detection result. The beneficial effect is that XGBoost is used as a base classifier, the elite sparrow search algorithm is used for algorithm performance optimization, and a more accurate abnormal traffic flow classification result is obtained.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of Internet security, in particular to an abnormal traffic detection method and system based on an elite sparrow search improved XGBoost algorithm. BACKGROUND

[0002] With the continuous leap of various technologies in the Internet and the increasingly in-depth research of human beings on computer networks, some lawbreakers can use some principles in the network to conduct attack experiments, such as sending a large amount of SYN request traffic to the attacked party, so that the attacked party cannot respond to ACK in time, and finally the host crashes. In addition, some common malicious traffic software will send traffic attacks to maliciously steal other people's passwords or important secrets of enterprises, and these malicious attacks are threatening our lives at any time. Therefore, real-time and accurate identification of abnormal traffic and isolation and discarding of the abnormal traffic are key factors for improving the network security performance.

[0003] Network intrusion detection aims to identify malicious traffic, so as to protect computers, networks, servers and data from attacks, prevent unauthorized access, modification or destruction, etc. With the surge of new network attacks and the emergence of more and more network dynamic ports, it is difficult to identify the type of traffic by relying on the port or some inherent characteristics in the traffic, therefore, the traditional intrusion detection method relying on pattern matching (such as IP address and port number) and classification is losing effectiveness.

[0004] In recent years, the abnormal traffic detection technology based on artificial intelligence algorithm has made great progress, and there is a record of using support vector machine SVM algorithm for industrial anomaly detection. However, the industrial anomaly detection based on support vector machine is not suitable for network traffic detection, because SVM is better for realizing binary classification, while the anomaly detection of network traffic needs to specifically distinguish the types of anomalies, so as to facilitate the subsequent traffic scheduling work. At present, most network traffic detection is based on the feature selection method of domain knowledge and experience, and the feature selection according to the domain knowledge and experience may contain some subjective factors, so that the result lacks objectivity, and further causes the detection effect to be poor. SUMMARY

[0005] (I) Technical problem to be solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the application provides an abnormal traffic detection method and system based on an elite sparrow search improved XGBoost algorithm, which solves the technical problem that the detection result lacks objectivity in the prior art, resulting in poor detection effect.

[0007] (II) Technical scheme

[0008] In order to achieve the above object, the main technical scheme adopted by the present application comprises:

[0009] In a first aspect, the embodiments of the present application provide an abnormal traffic detection method based on an elite sparrow search improved XGBoost algorithm, characterized in that the method comprises the following steps:

[0010] (1) initializing network traffic data with an abnormal traffic environment collected in real time to obtain a traffic data set with an initial feature set;

[0011] (2) performing feature extraction on the traffic data set in step (1) to obtain a traffic data set with an extracted feature set;

[0012] (3) inputting the traffic data set with the initial feature set in step (2) into an elite sparrow search improved XGBoost model to output a detection result.

[0013] Optionally, in step (1), the network traffic data with the abnormal traffic environment is captured data packets with abnormal traffic, and a traffic processing tool CIC-Flowmeter is used to process the data packets with abnormal traffic into a traffic data set with an initial feature set.

[0014] Optionally, in step (2), the feature extraction is to extract the top 15 features in terms of feature importance from the traffic data set with the initial feature set to obtain a traffic data set with an extracted feature set.

[0015] Optionally, the feature with high importance is a feature with small intra-class distance and large inter-class distance.

[0016] Optionally, in step (3), the elite sparrow search improved XGBoost model is an XGBoost model optimized by an elite sparrow search algorithm.

[0017] Optionally, the method for constructing the XGBoost model comprises:

[0018] (I): inputting a historical network traffic data set processed by feature extraction, the historical network traffic data set being composed of a feature matrix F and a label vector L, wherein the dimension of F is , indicating that there are n samples, and each sample contains m features;

[0019] (II): initializing an XGBoost model as an original model, wherein the XGBoost model uses a decision tree as a base classifier, and the decision tree is initialized to have only one leaf node;

[0020] (Three): Calculate the residual error that occurs each time the prediction: using the initialization XGBoost model in "step (two)" to predict the input in step (one) of the historical traffic set data, the prediction result F(y) is obtained, and the residual error between the prediction result and the actual label is calculated: Where y is the actual label;

[0021] (Four): Train sub-model: using the residual error obtained in step (three) as a label, the data of the historical network traffic dataset as a training set, and train a new decision tree model as a sub-tree of the original model to correct the errors of the original model, and obtain an updated and more accurate new model;

[0022] (Five): Iterative training of the new model: repeat steps (three)-(four) until the preset number of iterations is reached, and save the trained XGBoost model.

[0023] Optionally, in the step of optimizing the constructed XGBoost model using the elite sparrow search algorithm, the network traffic dataset is simulated as a sparrow population, and the specific steps are as follows:

[0024] Step 1: Use tent chaotic mapping to initialize the sparrow population and make it randomly distributed, generate chaotic population Y from initial sparrow population X; at the same time, generate lens imaging reverse population Z from initial sparrow population X through lens imaging reverse learning method;

[0025] Step 2: Sort population Y and population Z, and take the top N population P with fitness value;

[0026] Step 3: Sort population P and initial sparrow population X, and take the top N population X' with fitness value;

[0027] Step 4: Initialize the individual fitness of population X ’ and sort it, and update the individual position;

[0028] Step 5: Compare individual fitness and select the individual with the best fitness;

[0029] Step 6: Mutate the individual selected in step 5 according to the Cauchy-Gauss formula; compare the position changes before and after mutation, and select a better position, which is determined by being close to food and far from danger;

[0030] Step 7: The individual with better position selected in step 6 enters the next iteration;

[0031] Step 8: End of iteration, output model optimal fitness value, optimal solution.

[0032] Optionally, in step 1: randomly initialize the initial sparrow population of N individuals X=x1, x2, …, xN, where x n , x id represents the position of the ith sparrow in the dth dimension; substitute the population X into equation (2) to generate the chaotic population Y, and substitute the population X into equation (3) to generate the lens imaging reverse population Z;

[0033]

[0034]

[0035]

[0036] α is a chaotic coefficient, usually taking 0.5; x i is the ith sparrow, x 1b and x ub correspond to the lower bound and upper bound in the solution space respectively, x j is the jth sparrow, x j * is the jth sparrow after the chaotic initialization and lens imaging reverse, y i corresponds to x n+1 in equation (1), a j and b j represent the minimum value and maximum value in the jth dimension of the current population respectively, and k is the scaling coefficient of the lens.

[0037] Optionally, in step 6: the Cauchy-Gauss mutation strategy mainly selects the individual with the best fitness for mutation, and then compares the positions before and after mutation, and selects the better position after comparison to substitute into the next iteration, the specific formula is as follows:

[0038]

[0039]

[0040] In the formula, U t best represents the position of the optimal individual after mutation; σ 2 represents the standard deviation of the Cauchy-Gauss mutation strategy; Cauchy(0,σ 2 ) represents a random variable satisfying the Cauchy distribution; Gauss(0,σ 2 ) is a random variable satisfying the Gaussian distribution; λ1 and λ2 are dynamic parameters that are adaptively adjusted with the number of iterations; x i is the ith sparrow, x best represents the optimal individual position of the current x; during the optimization process, λ1 gradually decreases and λ2 gradually increases, so that the algorithm can jump out of the current stagnation and coordinate the ability of local development and global exploration.

[0041] In a second aspect, the embodiments of the present application provide a detection system for implementing the above-mentioned abnormal traffic detection method based on elite sparrow search improved XGBoost algorithm; the system comprises an initialization module, a feature extraction module and a processing module;

[0042] The initialization module is used for initializing the network traffic data with abnormal traffic environment collected in real time to obtain a traffic data set with an initial feature set;

[0043] The feature extraction module is used for extracting features from the traffic data set in the initialization module to obtain a traffic data set with an extracted feature set;

[0044] The processing module is used for inputting the traffic data set with the initial feature set in the feature extraction module into the elite sparrow search improved XGBoost model to output a detection result.

[0045] (Three) beneficial effects

[0046] The present application has the beneficial effects that: the abnormal traffic detection method and system based on elite sparrow search improved XGBoost algorithm of the present application optimizes XGBoost parameters by using elite sparrow search algorithm ESSA, in the experiment, a part of traffic is randomly extracted from the public data set CSE-CIC-IDS 2018 for detection and classification, the traffic data used in the experiment is shown in Table 2, and the experimental results are shown in the attached Figure 5 Compared with the traditional machine learning algorithms based on naive Bayes, logistic regression, KNN and XGBoost, the ESSA_XGBoost algorithm has obvious advantages in classification accuracy and training time. In terms of classification accuracy Accuracy, it is improved by 8.93% compared with the unoptimized XGBoost algorithm, by 1.02% compared with the KNN algorithm, by 33.23% compared with the naive Bayes algorithm, and by 53.74% compared with the logistic regression algorithm. In terms of training time, it is 32.43% faster than the unoptimized XGBoost algorithm, 28.57% faster than the logistic regression algorithm, 30.55% faster than the naive Bayes algorithm, and 46.80% faster than the nearest neighbor KNN algorithm. Moreover, the performance of the present application in Precision, Recall and F1-Score values is better than that of other algorithms. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 Fig. 1 is a flowchart of embodiment 1 of the abnormal traffic detection method and system based on elite sparrow search improved XGBoost algorithm of the present application;

[0048] Figure 2The tent chaos mapping is used for initializing the sparrow population graph in the abnormal flow detection method and system based on the elite sparrow search improved XGBoost algorithm of the application, that is, the elite chaos reverse population initialization graph.

[0049] Figure 3 The prediction result schematic diagram in the abnormal flow detection method and system based on the elite sparrow search improved XGBoost algorithm of the application;

[0050] Figure 4 The feature selection graph in the abnormal flow detection method and system based on the elite sparrow search improved XGBoost algorithm of the application;

[0051] Figure 5 The comparison result graph of the abnormal flow detection method and system based on the elite sparrow search improved XGBoost algorithm of the application and other classification algorithms. DETAILED DESCRIPTION

[0052] In order to better explain the application, so as to be understood, the application is described in detail in combination with the specific embodiments and the accompanying drawings.

[0053] The abnormal flow detection method and system based on the elite sparrow search improved XGBoost algorithm of the application solves the technical problem that the detection result lacks objectivity in the prior art, and the detection effect is poor. It includes (1) initializing the network flow data with abnormal flow environment collected in real time to obtain a flow data set with an initial feature set; (2) extracting features from the flow data set in step (1) to obtain a flow data set with an extracted feature set; (3) inputting the flow data set with the initial feature set in step (2) into the elite sparrow search improved XGBoost model to output the detection result. Compared with the traditional machine learning algorithms based on naive Bayes, logistic regression, KNN and XGBoost, the ESSA_XGBoost algorithm has obvious advantages in classification accuracy and training time.

[0054] In order to better understand the above technical solution, the exemplary embodiments of the application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the application are shown in the drawings, it should be understood that the application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the application can be more clearly, thoroughly understood, and the scope of the application can be completely conveyed to those skilled in the art.

[0055] Embodiment 1:

[0056] The application provides an abnormal traffic detection method based on an elite sparrow search improved XGBoost algorithm, and the method steps are as follows:

[0057] (1) The network traffic data with abnormal traffic environment collected in real time is initialized to obtain a traffic data set with an initial feature set.

[0058] Further, in the step (1), the network traffic data with abnormal traffic environment is initialized by capturing data packets with abnormal traffic, and the data packets with abnormal traffic can be processed into a traffic data set with an initial feature set by using a traffic processing tool CIC-Flowmeter.

[0059] (2) The traffic data set in step (1) is subjected to feature extraction to obtain a traffic data set with an extracted feature set. In the step (2), the feature extraction is to extract the top 15 features with important features from the traffic data set with the initial feature set to obtain the traffic data set with the extracted feature set. Further, the feature with important features is a feature with small intra-class distance and large inter-class distance.

[0060] (3) The traffic data set with the initial feature set in step (2) is input into the elite sparrow search improved XGBoost model to output a detection result. In the step (3), the elite sparrow search improved XGBoost model is: an XGBoost model is constructed, and the constructed XGBoost model is optimized by using an elite sparrow search algorithm to obtain the elite sparrow search improved XGBoost model.

[0061] Further, the method for constructing the XGBoost model is as follows:

[0062] (I): The historical network traffic data set subjected to feature extraction processing is input, and the historical network traffic data set is composed of a feature matrix F and a label vector L, wherein the dimension of F is It is indicated that there are n samples, and each sample contains m features.

[0063] (II): The XGBoost model is initialized as an original model: the XGBoost model uses a decision tree as a base classifier, and the decision tree with only one leaf node is initialized.

[0064] (III): The residual error that occurs in each prediction is calculated: the historical traffic set data in step 1 is predicted by using the initialized XGBoost model in step 2 to obtain a prediction result F(y), and the residual error between the prediction result and the actual label is calculated: Where y is the actual label. The actual label refers to the actual category of the traffic, for example, this traffic is actually SYN Flood attack traffic, then its actual label is the label number corresponding to SYN Flood.

[0065] (IV): Train sub-model: use the residual obtained in step (III) As a label, the data of the input historical network traffic dataset is used as a training set to train a new decision tree model as a sub-tree of the original model for correcting the errors of the original model, obtaining an updated and more accurate new model. That is, the residual is also used as the training input of the next decision tree, and the residual is used to correct the next step of learning.

[0066] (V): Iterative training of the new model: repeat steps (III)-(IV) until the preset iteration number is reached, and save the trained XGBoost model.

[0067] Further, in the step of optimizing the constructed XGBoost model using the elite sparrow search algorithm, the network traffic dataset is simulated as a sparrow population, and the specific steps are as follows:

[0068] Step 1: Use tent chaotic mapping to initialize the sparrow population to randomize the distribution, generate chaotic population Y from initial sparrow population X; at the same time, generate lens imaging reverse population Z from initial sparrow population X through lens imaging reverse learning method;

[0069] Step 2: Sort population Y and population Z, take the top N population P in terms of fitness value;

[0070] Step 3: Sort population P and initial sparrow population X, take the top N population X' in terms of fitness value;

[0071] Step 4: Initialize individual fitness of population X' and sort, update individual position;

[0072] Step 5: Compare individual fitness, select the individual with the best fitness;

[0073] Step 6: Mutate the individual selected in step 5 according to the Cauchy-Gauss formula; compare the position changes before and after mutation, select the better position after comparison, the better position is determined by being closer to food and farther away from danger, i.e. the position closer to food and farther away from danger is the better position;

[0074] Step 7: The individual with better position selected in step 6 enters the next iteration;

[0075] Step 8: End of iteration, output model optimal fitness value, optimal solution.

[0076] In summary, the fisher_score feature extraction algorithm is used to sort the initial features by importance and then classify them.

[0077] Embodiment 2: The application also provides a detection system for implementing the above-mentioned abnormal traffic detection method based on the elite sparrow search improved XGBoost algorithm. The system includes an initialization module, a feature extraction module and a processing module. The initialization module is used to initialize the real-time collected network traffic data with abnormal traffic environment to obtain a traffic data set with an initial feature set. The feature extraction module is used to extract features from the traffic data set in the initialization module to obtain a traffic data set with an extracted feature set. The processing module is used to input the traffic data set with the initial feature set in the feature extraction module into the elite sparrow search improved XGBoost model and output the detection result. Using the system can obtain more accurate abnormal traffic classification results.

[0078] The following will be described with reference to Figures 1-5 The implementation, principles and the like of the application will be further sorted out. The description is merely an example of the above-mentioned features to facilitate comprehensive description, but such description is not a limitation of the embodiments of the application, and is described as follows:

[0079] The application provides an abnormal traffic detection method and system based on the elite sparrow search improved XGBoost algorithm. In the feature extraction part, to prevent too many features from causing large data volume, data redundancy and other problems, the fisher_score algorithm is used to sort the features by importance, and the top 15 features with the highest correlation with abnormal flow detection are selected for input into the classifier. In the abnormal traffic detection part, to realize faster and more accurate identification of abnormal traffic, the application uses the XGBoost classifier to detect abnormal traffic. After the model is trained, the saved model can be directly used for real-time abnormal traffic detection. In addition, the application proposes an elite sparrow search algorithm, which improves the initialization population part and the individual position updating part of the classical sparrow search algorithm. The elite sparrow search algorithm is used to optimize the trained XGBoost model, so that it has better ability to search for optimal fitness and optimal solution, and realizes faster and more efficient network abnormal traffic detection.

[0080] Flow detection is a key and important technology in the field of network security. By using specific techniques, it can detect whether malicious program attack behaviors are contained in network communication flow, and provide important information for network overall situation awareness. Therefore, it has been the focus of research in the field of network security at home and abroad. Abnormal flow mainly includes attack flow, malicious software flow, burst flow, etc. Attack flow is divided into Dos, DDoS, network protocol attack, etc. Dos and DDoS are the current mainstream attack methods. Malicious software flow is mainly sent by attackers through malicious software, mainly including Trojan viruses, worm viruses, etc. Burst flow is mainly some burst flow with sudden changes in flow trend. These flows are the main reasons for host crash, host leakage and other security problems. Therefore, the abnormal behavior can be detected quickly to effectively protect the host security. Table 1 shows some common abnormal / attack flow types.

[0081] Table 1 Common abnormal / attack flow

[0082]

[0083] As shown in the accompanying Figure 1 The XGBoost abnormal flow detection method based on elite sparrow search algorithm mainly includes the following parts: initialization of flow data, feature extraction, abnormal flow detection and classification. The initialization of flow data part is to capture data packets with abnormal flow, and use the flow processing tool CIC-Flowmeter to process it into a data set with flow characteristics. The feature extraction part is to extract the top 15 features of feature importance from the initial features. The abnormal flow detection and classification part uses the XGBoost model for abnormal flow detection and classification, that is, the processed data set is used to train the XGBoost model, and its detection performance is verified, and then used as the subsequent detection model. The application also uses the model optimization part, which uses the elite sparrow search algorithm to optimize the trained XGBoost model, and outputs the optimized model parameters, so that the search optimization ability of the model is stronger, and the detection accuracy is higher.

[0084] The application optimizes the XGBoost model based on the elite sparrow search algorithm. Traditional machine learning classification algorithms mainly classify network flow through clustering, decision tree, support vector machine, etc. However, most classification algorithms are not particularly accurate on multiple types of flow data, so using an optimization algorithm can improve the optimization ability of the classification model and optimize the detection time.

[0085] XGBoost algorithm is an additive model based on the boosting ensemble idea. During training, forward distribution algorithm is used for greedy learning. Each iteration learns a CART tree to fit the residual error between the prediction results of the previous t-1 trees and the true values of the training samples. CART regression tree is a binary tree. The feature is continuously split. When the tree node is split based on the jth feature value, the sample with the feature value less than s is divided into the left subtree, and the sample with the feature value greater than s is divided into the right subtree. CART regression tree is essentially a division of the sample space in the feature dimension. In order to solve this space division problem, a heuristic method is used in the decision tree model. In order to solve the optimal split feature j and the optimal split point s, the objective function (6) can be solved:

[0086]

[0087] c1 and c2 are constants. Since this objective function is a general objective function, c1 and c2 can be taken as the actual value of f(xi), y i -c1 or y i -c2 is to find the residual error again. R1 is the sample on the left subtree, and R2 is the sample on the right subtree. x i is the sample value, and y i is the predicted value.

[0088] Therefore, as long as all the split points of all the features are traversed, the optimal split feature and split point can be found. Finally, a regression tree is obtained. The idea of XGBoost algorithm is to continuously add CART regression trees and continuously split features to grow a tree. Each time a tree is added, a new function is learned to fit the residual error of the previous prediction. After training, k trees are obtained. If you want to predict the score of a sample, actually, according to the characteristics of the sample, it will fall into a corresponding leaf node in each tree. Each leaf node corresponds to a score. Finally, only the scores corresponding to each tree are added to obtain the predicted value of the sample.

[0089] Sparrow search algorithm is an optimization algorithm that uses biological characteristics for iterative optimization. Studies have shown that there are two different types of sparrows in captivity: discoverers and joiners. Discoverers are responsible for finding food and providing foraging areas and directions for the entire sparrow population, while joiners use discoverers to obtain food. When there are sparrows in the group that discover predators around them, there will be an alarm in the group, i.e. a sparrow that discovers danger. Once the alarm is raised, the entire population will immediately avoid danger and fly to other safe areas to forage. In the process of simulating the sparrow's search for food and avoidance of danger, this process can be used to optimize the algorithm, so that the regression tree can search for optimization more quickly in the search process, find the best matching parameters, and obtain more accurate prediction values.

[0090] The feature extraction module of the application proposes to use the fisher_score feature ranking algorithm for feature extraction. The fisher_score algorithm is mainly used to distinguish the differences between classes, and the features with strong discrimination performance mainly show small intra-class distance and large inter-class distance. The fisher_score value of each feature is calculated by standard independent calculation, which is used as the score of the importance of the feature. The top 15 features are selected as the training features for subsequent abnormal traffic detection.

[0091] The initialization traffic data and feature extraction stage process of the application comprises the following steps:

[0092] Step 1: Collect network traffic with abnormal traffic environment.

[0093] Step 2: Process the initial traffic using a traffic processing tool to obtain a traffic data set with an initial feature set F start .

[0094] Step 3: Select the top 15 features using the fisher_score feature selection algorithm to generate a traffic data set with an extracted feature set F select .

[0095] The XGBoost model construction method of the application is:

[0096] Step 1: Input the traffic data set with the feature set processed by the feature extraction, which is composed of a feature matrix F and a label vector L, where the dimension of F is , indicating that there are n samples, and each sample contains m features.

[0097] Step 2: Initialize the XGBoost model: the model uses a decision tree as a base classifier, and initializes a decision tree with only one leaf node.

[0098] Step 3: Calculate the residual error that occurs each time the prediction is made, use the initial model to predict the input data, get the prediction result F(y), and calculate the residual error between the prediction result and the actual label:

[0099] Step 4: Train the sub-model: use the residual error obtained in step 3 as the label, use the input data as the training set, and train a new decision tree model as a sub-tree of the original model to correct the errors of the original model, and get an updated and more accurate new model.

[0100] Step 5: Iteratively train the model, repeat steps 3-4 until the preset number of iterations is reached.

[0101] Step 6: Use the trained XGBoost model to make classification prediction on new traffic data, output the abnormal type or normal label of each traffic.

[0102] Step 7: Save the trained XGBoost model to realize real-time detection and classification.

[0103] The model optimization part of the application proposes to use the elite sparrow search algorithm to optimize the already trained XGBoost model, so that the optimal solution can be searched faster. Since normal network traffic has regularity, it can be simulated as producers in sparrow population, and abnormal traffic has suddenness and aggressiveness, which can be simulated as predators in sparrow population, so the network traffic data set can be simulated as sparrow population, and the optimization algorithm can be used to optimize the parameters of the detection model, so that the model can find the optimal parameters faster and detect abnormal flow faster to avoid attacking the host.

[0104] The model optimization stage process of the application includes the following steps:

[0105] Step 1: Use tent chaotic mapping to initialize sparrow population and make it randomly distributed, generate chaotic population Y from initial population X;

[0106] At the same time, step 2: generate lens imaging reverse population Z from initial population X by lens imaging reverse learning method;

[0107] Step 3: Sort population Y and population Z, and take the top N population P in fitness value;

[0108] Step 4: Sort population P and initial sparrow population X, and take the top N population X' in fitness value;

[0109] Step 5: Initialize individual fitness of population X' and sort, and update individual position;

[0110] Step 6: Compare individual fitness and select the individual with the optimal fitness;

[0111] Step 7: Mutate the individual selected in step 6 according to Cauchy-Gauss formula;

[0112] Step 8: Compare the position change before and after mutation, and select the better position, which is determined by being close to food and far away from danger;

[0113] Step 9: The individual with better position selected in step 8 enters the next iteration;

[0114] Step 10: The iteration is ended, and the optimal fitness value and optimal solution of the model are outputted;

[0115] Specifically, this invention mainly optimizes and improves the initial population, individual position update, and optimal solution selection parts of the sparrow search algorithm. An elite chaotic reverse learning strategy is used to optimize the initial sparrow population. The specific process is as follows: randomly initialize an initial sparrow population X = x1, x2, ..., x with N individuals. n x id This represents the position of the i-th sparrow in the d-th dimension; substituting population X into equation (2) generates a chaotic population Y, and simultaneously substituting population X into equation (3) generates a lens-imaged inverse population Z; sorting the sparrow individuals in population Y and population Z according to their fitness values, and selecting the top N better individuals to form an elite chaotic inverse population P; population P and the original sparrow population X are sorted according to their individual fitness values, and the top N individuals are selected to form a new initial sparrow population, X′=x′1,x′2,…,x′ n .

[0116]

[0117]

[0118]

[0119] x n Let x represent the nth sparrow. n+1 Let x be the (n+1)th sparrow. α is a chaos coefficient, typically taken as 0.5. i It refers to the i-th sparrow, x 1b and x ub These correspond to the lower and upper bounds in the solution space, respectively, x j It is the jth sparrow, x j * represents the j-th sparrow after chaotic initialization and nuclear lens imaging inversion, y i x in formula (1) n+1 a j and b j Let represent the minimum and maximum values ​​in the j-th dimension of the current population, respectively, and k be the scaling factor of the lens.

[0120] The Cauchy-Gaussian mutation strategy is used to improve the individual positions of sparrows, selecting the optimal solution for improvement. The Cauchy-Gaussian mutation strategy primarily involves selecting the individual with the best current fitness for mutation, comparing its position before and after mutation, and selecting the better position for the next iteration. The specific formula is as follows:

[0121]

[0122]

[0123] In the formula, U t bestrepresents the position after mutation of the optimal individual; sigma 2 represents the standard deviation of the Cauchy-Gaussian mutation strategy; Cauchy(0, sigma 2 ) represents a random variable satisfying the Cauchy distribution; Gauss(0, sigma 2 ) is a random variable satisfying the Gaussian distribution; and lambda1 and lambda2 are dynamic parameters that are adaptively adjusted with the number of iterations.

[0124] The abnormal flow detection stage process of the application comprises the following steps:

[0125] The categories and introductions of the network flow data sets used in the experiment are shown in Table 2 below:

[0126]

[0127] The features used in the experiment and their introductions are shown in the following table

[0128]

[0129] The pseudo code of the algorithm used in the experiment is as follows:

[0130]

[0131]

[0132]

[0133] Experimental steps: (1) initialize the network flow data in Table 2 above to obtain a flow data set with an initial feature set;

[0134] (2) perform feature extraction on the flow data set described in step (1) to obtain a flow data set with an extracted feature set; feature selection, see Figure 4 .

[0135] (3) input the flow data set with the initial feature set described in step (2) into the improved XGBoost model based on the elite sparrow search to output the detection result.

[0136] The detection result is shown in Figure 3 .

[0137] See Figure 5 , Figure 5The ESSA XGBoost in the table is the elite sparrow search improved XGBoost algorithm of the application, compared with the traditional machine learning algorithms based on naive Bayes, logistic regression, KNN and XGBoost, the ESSA_XGBoost algorithm has obvious advantages in classification accuracy and training time. In terms of classification accuracy, it is improved by 8.93% compared with the unoptimized XGBoost algorithm, by 1.02% compared with the KNN algorithm, by 33.23% compared with the naive Bayes algorithm, and by 53.74% compared with the logistic regression algorithm. In terms of training time, it is 32.43% faster than the unoptimized XGBoost algorithm, 28.57% faster than the logistic regression algorithm, 30.55% faster than the naive Bayes algorithm, and 46.80% faster than the nearest neighbor KNN algorithm.

[0138] In summary, the abnormal traffic detection method and system based on the elite sparrow search improved XGBoost algorithm of the application solve the problems in the prior art, greatly improve the detection effect, and are beneficial to popularization and application.

[0139] In the description of the specification, the description of the terms "embodiment" and the like means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. Furthermore, different embodiments or examples described in the specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0140] Although the embodiments of the application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the application, and those skilled in the art can modify, modify, replace and modify the above embodiments within the scope of the application.

Claims

1. A method for detecting abnormal traffic based on an elite sparrow search improved XGBoost algorithm, characterized in that: The method steps are as follows: (1) The network traffic data with abnormal traffic environment collected in real time is initialized to obtain a traffic data set with an initial feature set; (2) The traffic data set in step (1) is feature extracted to obtain a traffic data set with an extracted feature set; (3) The traffic data set with the initial feature set in step (2) is input into the elite sparrow search improved XGBoost model, and the detection result is output; (3) In step, the elite sparrow search improved XGBoost model is: the elite sparrow search algorithm is used to optimize the constructed XGBoost model to obtain the elite sparrow search improved XGBoost model; The method for constructing the XGBoost model is: (I): input the historical network traffic data set after feature extraction processing, the historical network traffic data set is composed of a feature matrix F and a label vector L, wherein the dimension of F is , indicating that there are n samples, and each sample contains m features; (I): input the historical network traffic data set after feature extraction processing, the historical network traffic data set is composed of a feature matrix F and a label vector L, wherein the dimension of F is , indicating that there are n samples, and each sample contains m features; (II): Initialize the XGBoost model as the original model: the XGBoos model uses a decision tree as the base classifier, and initializes a decision tree with only one leaf node; (Three): Calculate the residual error that occurs each time the prediction: using the initialization XGBoost model in "step (two)" to predict the input in step (one) historical traffic set data, the prediction results , calculate the residual error between the prediction results and the actual label: ; wherein y is the actual label; (Fourth): training sub-model: using the residual obtained in step (three) As a label, the data of the historical network traffic dataset is used as a training set to retrain a new decision tree model as a sub-tree of the original model for correcting errors of the original model, obtaining an updated and more accurate new model; (V): Iterative training of the new model: repeat steps (III)-(IV) until the preset iteration number is reached, and save the trained XGBoost model.

2. The abnormal traffic detection method based on the elite sparrow search improved XGBoost algorithm according to claim 1, characterized in that: (1) In step, the network traffic data of the abnormal traffic environment is captured, and the data packets with abnormal traffic are processed into a traffic data set with an initial feature set using the traffic processing tool CIC-Flowmeter.

3. The method of claim 1, wherein the method is characterized by: (2) In step, the feature extraction is to extract the top 15 features in terms of feature importance from the traffic data set with the initial feature set to obtain a traffic data set with an extracted feature set.

4. The method of claim 3, wherein the method is characterized by: The feature importance is a feature with small intra-class distance and large inter-class distance.

5. The abnormal traffic detection method based on the elite sparrow search improved XGBoost algorithm according to claim 1, characterized in that: In the step of optimizing the constructed XGBoost model using the elite sparrow search algorithm, the network traffic data set is simulated as a sparrow population, and the specific steps are as follows: Step 1: Use tent chaotic mapping to initialize the sparrow population and make it randomly distributed, generate chaotic population Y from initial sparrow population X; at the same time, generate lens imaging reverse population Z from initial sparrow population X through lens imaging reverse learning method; Step 2: Sort population Y and population Z, and take the top N population P in terms of fitness value; Step 3: Sort the population P with the initial sparrow population X, and take the top N population X with fitness ’ ; Step 4: Initialize population X ’ Fitness of individuals of X and sort, update individual positions; Step 5: Compare individual fitness and select the individual with the best fitness; Step 6: Mutate the individual selected in step 5 according to the Cauchy-Gauss formula; compare the position changes before and after mutation, and select the better position, which is determined by being close to food and far from danger; Step 7: The individual with better position selected in step 6 enters the next iteration; Step 8: The iteration is ended, and the optimal fitness value and optimal solution of the model are output.

6. The abnormal traffic detection method based on the elite sparrow search improved XGBoost algorithm according to claim 5, characterized in that: In step 1: Randomly initialize an initial sparrow population of N individuals. , x id Indicates the first i Only sparrows in the first d The location of the population; X Substitute into equation (2) to generate a chaotic population. Y At the same time, the population X Substitute into equation (3) to generate a reverse population for lens imaging. Z； （1） （2） （3）； α This is the chaos coefficient, usually taken as 0.5; x i It refers to the i-th sparrow, x 1b and x ub These correspond to the lower and upper bounds in the solution space, respectively. x j It is the first j A sparrow, x j * For the first time after chaotic initialization nuclear lens imaging reversal j A sparrow, y i In the corresponding formula (1) x n+1 , a j and b j Let these represent the minimum and maximum values ​​in the j-th dimension of the current population, respectively. k This is the scaling factor of the lens.

7. The method of claim 6, wherein the method is based on an elite sparrow search improved XGBoost algorithm. In step 6: the Cauchy-Gaussian mutation strategy is mainly to select the individual with the best fitness for mutation, and then compare the positions before and after mutation, and select the better position after comparison to replace the next iteration, and the specific formula is as follows: (4) (5) In the formula, U t best represents the position of the optimal individual after mutation; represents the standard deviation of the Cauchy-Gaussian mutation strategy; represents a random variable satisfying the Cauchy distribution; is a random variable satisfying the Gaussian distribution; and is a dynamic parameter that is adaptively adjusted with the number of iterations; x i refers to the i-th sparrow, x best the optimal individual position of the current x; In the optimization process, gradually decrease, gradually increase, so that the algorithm can jump out of the current stagnation, and coordinate its local development and global exploration ability.

8. A detection system characterized by: The system is used to implement the method of claim 1-7, and the system comprises an initialization module, a feature extraction module and a processing module. The initialization module is used to initialize the network traffic data with abnormal traffic environment collected in real time to obtain a traffic data set with an initial feature set. The feature extraction module is used to extract features from the traffic data set in the initialization module to obtain a traffic data set with an extracted feature set. The processing module is used to input the traffic data set with the initial feature set in the feature extraction module into the elite sparrow search improved XGBoost model to output a detection result.

Images