A network intrusion detection model and method based on deep learning and decision tree fusion

Abstract

The present application discloses a network intrusion detection model and method integrating deep learning and decision tree, wherein the model includes a first decision tree integrating a simplified convolutional neural network, an improved convolutional neural network and a second decision tree; the first decision tree includes a root node and a leaf node, each node includes a simplified convolutional neural network; the improved convolutional neural network includes a first convolutional layer, a first custom mixed pooling layer, a second convolutional layer, a second custom mixed pooling layer, a third convolutional layer and a flattening layer in sequence; the second decision tree is implemented by a classic decision tree classifier. In order to solve the problem that the training efficiency of the deep neural network model in the prior art is low, the training time is too long, and it is difficult to meet the needs of real-time attack detection, the present application designs a first decision tree structure that can be run offline; at the same time, the present application designs a second decision tree, combines it with the improved convolutional neural network, improves the overall accuracy of the model, and improves the generalization ability of the model.

Description

Technical Field

[0001] The present invention belongs to the field of network security technology, and specifically relates to a network intrusion detection model and method that integrates deep learning and decision tree. Background Art

[0002] With the continuous advancement of data science and machine learning technology, convolutional neural networks (CNNs) and decision trees have become the core technologies for handling classification and feature recognition tasks. However, using these methods alone still faces certain challenges when dealing with complex data problems. Although CNN performs well in feature extraction and processing, when the amount of data is insufficient or there is high noise, the model is prone to overfitting, which affects its generalization ability on new data. On the other hand, decision trees perform well when processing structured data, but when faced with high-dimensional features and complex data, they may encounter problems of overfitting and insufficient feature selection, which affects the classification effect.

[0003] Intelligent diagnosis of information network security is inseparable from accurate detection, especially in specific industry applications such as power information network and wireless information network, where network security detection is of utmost importance. In the field of network security detection, although the existing network intrusion detection system (IDS) based on deep learning and machine learning technology has performed well in terms of performance, there are still some challenges. First, there are many attack categories in the original data set, and the existing models can often only identify broad attack types, ignoring the accurate identification of subdivided categories. Secondly, there are a large number of redundant features in the network traffic data, which makes it difficult for the model to extract effective information, thus affecting the recognition accuracy. Furthermore, the problem of unbalanced number of samples in the data set makes the model tend to learn the features of a large number of categories, ignoring the feature learning of a few categories, resulting in poor classification and recognition results. Therefore, the combination of CNN and decision tree in network security detection can give full play to the advantages of CNN in feature extraction, while using decision tree for accurate classification and anomaly detection, which helps to improve the overall performance and accuracy of network intrusion detection system. Further improving the model structure and training strategy of these combined methods will help solve various problems in the existing network security detection system and improve the security and stability of the system.

[0004] At present, deep learning and machine learning have achieved good performance in the field of network intrusion detection, but there are still many problems:

[0005] (1) For network intrusion detection, a single machine learning method or deep learning method is used. For example, the decision tree method alone usually requires manual feature engineering to extract and select features, which may be difficult and ineffective for complex high-dimensional data. In addition, decision trees are prone to overfitting on training data and may not be able to fully utilize complex feature interactions, especially when the feature space is large or the potential pattern of the data is complex; the convolutional neural network (CNN) method is used alone. CNN models are usually considered "black box" models because their internal working mechanisms are difficult to understand and explain, which may affect the transparency and trust of the model.

[0006] (2) In deep learning algorithms for network intrusion detection, traditional single pooling methods are used. The maximum pooling method focuses too much on local details, and the average pooling method is too smooth. Both lack the ability to adaptively adjust the data. Different data types or tasks may require a balance between local details and global smoothness, and traditional pooling methods cannot meet this demand.

[0007] (3) The hyperparameters in the deep learning methods for network intrusion detection are not optimized, or the optimization methods are prone to fall into local optimality and cannot find hyperparameters with good performance.

[0008] The invention patent with the publication number CN112491796A in the prior art discloses a method for quantitative interpretation of intrusion detection and semantic decision tree based on convolutional neural network, which converts traffic data into traffic grayscale images as input, trains the convolutional neural network with increased interpretability, and uses the detection set to detect the trained model; uses the class activation map to construct the attack detail image and inputs the trained convolutional neural network model to extract clustering features for clustering, calculates the distance between the detected abnormal samples and the centers of various attack clusters, and uses the distance to establish a decision tree; calculates the semantic matching rate to give semantic meaning to the established proxy decision tree, and uses the matching rate to quantitatively evaluate the interpretation effect of the semantic decision tree. The invention establishes a semantic decision tree to interpret the model with the best effect, and designs the semantic matching rate to quantitatively evaluate the interpretation effect.

[0009] Although the above-mentioned prior art combines convolutional neural networks with decision trees, it is mainly used to establish semantic decision trees to explain the model with the best effect; it combines hidden layer analysis with proxy models to establish proxy decision trees to analyze the decision process of the model, thereby enhancing the explanation effect of the proxy decision tree on the decision process of the model. However, big data attack detection is a classification problem of unbalanced data, and a large amount of new data to be detected flows in every day, while the training efficiency of deep neural network models is low. The improved neural network models in the prior art generally require more than one day of training time, which is difficult to meet the needs of real-time attack detection. In actual operation, this solution still cannot solve the problems of easy overfitting and long calculation time. Summary of the invention

[0010] A brief summary of embodiments of the present invention is provided below to provide a basic understanding of certain aspects of the present invention. It should be understood that the following summary is not an exhaustive summary of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to provide certain concepts in a simplified form as a prelude to a more detailed description discussed later.

[0011] According to one aspect of the present application, a network intrusion detection model integrating deep learning and decision tree is provided, which is a decision tree-convolutional neural network-decision tree model (DT-CNN-DT), which is composed of a first decision tree (DT) integrated with a simplified convolutional neural network, an improved convolutional neural network (CNN), and a second decision tree;

[0012] Among them, the first decision tree includes a root node and a leaf node, and each node (root node and leaf node) includes a simplified convolutional neural network; the root node is used to roughly classify the intrusion data to be detected (normal, abnormal, suspected abnormal), and then pass it to the leaf node of the next layer to further identify the roughly classified intrusion data to be detected, and remove the redundant or irrelevant features of the intrusion data to be detected, and finally output a network intrusion detection data set with a label at the leaf node, and the label is the preliminary judgment result; the simplified convolutional neural network includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer. The pooling layer adopts the maximum pooling method, and the others are implemented using the general existing convolutional neural network structure. The first decision tree can be used offline. When in use, a large amount of new data to be detected that flows in every day is first roughly processed by the first decision tree, and a large amount of normal data is filtered, and abnormal and suspected abnormal data are retained. Therefore, the data processing speed of the entire model is greatly accelerated.

[0013] The improved convolutional neural network is a two-dimensional convolutional neural network CNN, which includes a first convolutional layer, a first custom mixed pooling layer, a second convolutional layer, a second custom mixed pooling layer, a third convolutional layer and a flattening layer in sequence, the first convolutional layer is a 3×3 convolutional layer, the second convolutional layer and the third convolutional layer are 2×2 convolutional layers, and the first custom mixed pooling layer and the second custom mixed pooling layer are 2×2 custom mixed pooling layers; after the network intrusion detection dataset is processed by the first decision tree, a network intrusion detection dataset with a label (judgment result) is output, which passes through the first convolutional layer, the first custom mixed pooling layer, the second convolutional layer, the second custom mixed pooling layer, the third convolutional layer and the flattening layer in sequence, and finally outputs a feature vector for classification;

[0014] The second decision tree is implemented by a classic decision tree classifier. The depth of the second decision tree is no more than 10, and the number of samples in each leaf node is no less than 5. At the same time, the pruning parameter range is set to [0.009-0.011] to control the growth of the tree.

[0015] Among them, the first custom mixed pooling layer and the second custom mixed pooling layer have the same structure and are defined as: ;

[0016] in, Represents the output feature map after mixed pooling, which is the result of processing the input x; Represents the feature map after the maximum pooling operation, represents the feature map after the average pooling operation, Represents the weight parameter of hybrid pooling, which controls the weight distribution between maximum pooling and average pooling. The weight parameter value range is [0, 1]. Optimized by roulette particle swarm algorithm.

[0017] Furthermore, the improved convolutional neural network adopts five-fold cross validation, specifically including:

[0018] The original network intrusion detection dataset is randomly divided into five subsets (or folds) of equal size, each containing the same number of data samples;

[0019] Training and testing of the improved convolutional neural network: In each iteration, one of the subsets is selected as the test set, and the remaining four subsets are combined as the training set; the improved convolutional neural network is trained using the training set and evaluated on the test set; the process is repeated five times to ensure that each subset is used as a test set once;

[0020] For each test, use preset performance metrics such as accuracy, precision, recall, F1 score, etc. to evaluate the performance of the improved convolutional neural network on the test set;

[0021] The results of the performance metrics of the five tests are averaged to obtain the final performance evaluation of the improved convolutional neural network.

[0022] This application uses 5-fold cross validation, splits the training set and trains multiple times to improve the convolutional neural network model. In each fold, an early stopping mechanism is used to avoid overfitting, and the model performance of each fold is recorded. The first few layers of the CNN model are used as feature extractors to extract intermediate features, and then the second decision tree classifier is used to classify these features. The coefficients of the mixed pooling The Roulette Wheel Particle Swarm Algorithm (RW-PSO) is used to optimize the model and evaluate each model in each fold, and the accuracy and the current optimal coefficient are calculated respectively. and loss value.

[0023] When in use, the newly added data to be detected first passes through the first decision tree of the decision tree-convolutional neural network-decision tree model to obtain a network intrusion detection data set, and then the network intrusion detection data set is input into the improved convolutional neural network, and then into the second decision tree, and finally the predicted label is output, so as to accurately divide the traffic data in the network intrusion detection data set into normal traffic and attack traffic.

[0024] According to another aspect of the present application, a network intrusion detection method integrating deep learning and decision tree is provided, which is implemented by using the above-mentioned decision tree-convolutional neural network-decision tree model, and includes:

[0025] Step 1: input the data to be detected into the first decision tree to obtain a network intrusion detection data set, which is a data set that filters normal data through the first decision tree and retains abnormal and suspected abnormal data;

[0026] Step 2: Preprocess the network intrusion detection data set obtained in step 1;

[0027] Step 3: Input the preprocessed network intrusion detection data set into the improved convolutional neural network and train the improved convolutional neural network;

[0028] Step 4: Improve the convolutional neural network to extract features and output feature vectors (feature maps);

[0029] Step 5: Input the feature vector to the second decision tree. After making a decision, the second decision tree outputs the classification result to determine whether there is a network intrusion.

[0030] The present application adopts the above scheme to form a decision tree-convolutional neural network-decision tree model (DT-CNN-DT) comprising a first decision tree (DT) with a simplified convolutional neural network, an improved convolutional neural network (CNN), and a second decision tree. The first decision tree can be used offline, and the newly added data to be detected every day can be processed offline by the first decision tree. After the improved convolutional neural network is trained, the newly added data to be detected that has been processed offline by the first decision tree can be detected in real time, and finally the detection result can be output through the second decision tree.

[0031] The preprocessing process of the network intrusion detection data set in step 2 is as follows:

[0032] Step S21: divide the network intrusion detection data set into a training data set and a test data set, wherein the training data set and the test data set respectively store feature and label information; use the pandas component to read the training data set and the test data set, and store the data in a CSV format file;

[0033] Step S22: Separate the features and labels in the training data set and the test data set. and As a feature, and As the corresponding label;

[0034] Step S23: Normalize the features through Normalizer to ensure that the data is within the same numerical range, thereby accelerating the convergence of model training;

[0035] Step S24: convert the labels into binary format to adapt to the subsequent binary classification task;

[0036] Step S25: Convert the normalized features into the three-dimensional format required by the convolutional neural network, that is, reshape the feature data into a pseudo image format with a shape of 7×6×1.

[0037] As an implementation scheme, the specific structure of the improved convolutional neural network in step 3 is shown in Table 1:

[0038] Table 1 CNN structure table

[0039]

[0040] The improved convolutional neural network includes a first convolution layer, a first custom mixed pooling layer, a second convolution layer, a second custom mixed pooling layer, a third convolution layer and a flattening layer in sequence. The first convolution layer is a 3×3 convolution layer, the second convolution layer and the third convolution layer are 2×2 convolution layers, and the first custom mixed pooling layer and the second custom mixed pooling layer are 2×2 custom mixed pooling layers.

[0041] As an implementation scheme, the process of training the improved convolutional neural network in step 3 is as follows: the preprocessed network intrusion detection data set is input into the first convolution layer, the first convolution layer generates 32 feature maps and is sent to the first custom mixed pooling layer for processing, then input into the second convolution layer to generate 64 feature maps, then processed by the second custom mixed pooling layer, then input into the third convolution layer to generate 128 feature maps, and the feature map is flattened into a one-dimensional vector through the flattening layer (flattening layer) to generate a feature vector for classification. This feature vector will be sent as input to the decision tree classifier for classification, and the complexity of the decision tree is controlled.

[0042] Among them, the first custom mixed pooling layer and the second custom mixed pooling layer have the same structure and are defined as: ;

[0043] in, Represents the output feature map after mixed pooling, which is the result of processing the input x; Represents the feature map after the maximum pooling operation, represents the feature map after the average pooling operation, Represents the weight parameter of mixed pooling, which controls the weight distribution between maximum pooling and average pooling. The value range of the weight parameter is [0, 1]. The ReLu activation function is a prior art and will not be described in detail here.

[0044] As an implementation solution, in order to avoid premature convergence and falling into local optimum, the first custom mixed pooling layer and the second custom mixed pooling layer use the Roulette Wheel Selection (RWS) based particle swarm optimization (PSO) to optimize the hyperparameters of the mixed pooling layer. Specifically, the Roulette Wheel Selection based particle swarm algorithm (RWS-PSO) optimizes the weight parameters of the mixed pooling layer. The optimization includes the following steps:

[0045] Step S31: establishing a particle swarm optimization algorithm for performing particle iterative search space S;

[0046] Step S32: The weight coefficients in the first custom mixed pooling layer and the second custom mixed pooling layer are Each value optimization scheme of is taken as a particle to establish a particle swarm;

[0047] Step S33: Calculate the fitness value of each particle at the current time t, that is, the performance index of the model on the validation set (that is, the accuracy and loss value), calculate the probability of each particle being selected by normalizing the fitness value; obtain the optimal hyperparameter value of each particle as its own optimal position , where K is the total number of particles, i is the i-th particle in the particle swarm optimization algorithm, and the hyperparameter value solution with the minimum total loss value in the search space at the current time t is taken as the global optimal value solution, that is, the global optimal position ;

[0048] Step S34: perform iterative search of particles using a particle swarm algorithm based on roulette wheel selection to complete the position update of particles; when the iterative search reaches the set number of iterations, the final optimization result is obtained to achieve the optimization of the value of the hyperparameter; wherein, the particle swarm algorithm based on roulette wheel selection uses the performance indicator (loss value) of the convolutional neural network (CNN) combined with the decision tree model on the validation set as the adaptation value.

[0049] Wherein, the step S33 specifically includes the following steps:

[0050] Step S331: Calculate the probability of each particle being selected; record For particles The cumulative probability interval of :

[0051] ;

[0052] ;

[0053] ……;

[0054] ;

[0055] Each All correspond to particles The probability of being selected, fitness i is the normalized fitness value of the ith particle, Σfitness i It is the sum of the normalized fitness values ​​of all particles. A probability is assigned to each particle according to its fitness value. The higher the fitness of the particle, the greater the probability of being selected.

[0056] Step S332: By random number Roulette selection is performed, and particles are selected in the following intervals:

[0057] ;

[0058] According to random number The size of the Still belongs to ; It represents the upper limit of the probability interval that depends on the individual optimal solution;

[0059] Step S333: Update the position of each particle using the roulette-based particle swarm algorithm, where the position of particle i at time t+1 is Updated to:

[0060] ;

[0061] in, It is a particle In the next iteration location, It is a particle At the current iteration, is the inertia weight, which controls the degree of particle velocity retention and affects the global search capability of the algorithm. It is a particle In the current iteration speed, and are acceleration coefficients, representing the individual learning factor and the social learning factor, respectively, and controlling the degree to which the particle approaches the individual optimum and the global optimum. represents the upper limit of the probability interval that depends on the individual optimal solution, for The i-th particle The individual optimal solution, is the global optimal solution, and is the random number in the speed update formula.

[0062] Step S334: Determine the location Whether it satisfies the individual optimal update condition, if not, it depends on the global optimal update condition for update;

[0063] Step S335: Calculate the performance index (loss value) of the model corresponding to each particle on the validation set at the next time t+1 to obtain the optimal position of each particle. pBest' (The optimal position of particle i is pBest ’ i ), i=1~K, and obtain the global optimal position of the particle swarm gBest' ;Will pBest' The corresponding total loss value of the validation set is The corresponding total loss value of the validation set is compared. pBest' The corresponding total network interference value is smaller, then pBest' The value of ;Will gBest'The corresponding total loss value of the validation set is The corresponding validation set total loss value is compared. gBest' The corresponding validation set total loss value is smaller, then gBest' The value of ;

[0064] Step S336: Determine whether the preset number of iterations has been reached. If so, obtain the pBest of this iteration. i ~pBest x as well as , the global optimal position As the final channel allocation solution, otherwise, return to step S335 and perform the next iteration.

[0065] As an implementation solution, the specific control measures for the complexity of the second decision tree include: the depth of the second decision tree is limited to 10 to prevent overfitting; the minimum number of samples for each leaf node is 5 to ensure that the decision tree is not over-refined; the growth of the tree is controlled by setting the pruning parameter to 0.01 to reduce the risk of overfitting. Finally, the predicted label is output to classify the traffic data in the data set into normal traffic and attack traffic.

[0066] In order to solve the problems of low training efficiency and long training time of deep neural network models in the prior art, which are difficult to meet the needs of real-time attack detection, the present application designs a first decision tree structure that integrates a neural network and can be run offline; at the same time, although the accuracy of the model is high when a single decision tree is used for classification, overfitting may occur when processing complex data, resulting in insufficient generalization ability of the model. To solve this problem, the present application designs a second decision tree and combines it with an improved convolutional neural network, which not only improves the overall accuracy of the model, but also improves the generalization ability of the model, making the classification performance more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The present invention may be better understood by referring to the following description given in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout the drawings to represent the same or similar components. The accompanying drawings, together with the following detailed description, are included in and form a part of this specification and are used to further illustrate the preferred embodiments of the present invention and to explain the principles and advantages of the present invention. In the drawings:

[0068] Figure 1 A block diagram of the network intrusion detection model that integrates deep learning and decision trees for this application;

[0069] Figure 2 This is a flowchart of the network intrusion detection algorithm that integrates deep learning and decision trees in this application. DETAILED DESCRIPTION

[0070] Embodiments of the present invention will be described below with reference to the accompanying drawings. Elements and features described in one drawing or one embodiment of the present invention may be combined with elements and features shown in one or more other drawings or embodiments. It should be noted that for the purpose of clarity, the representation and description of components and processes that are not related to the present invention and are known to those of ordinary skill in the art are omitted in the drawings and descriptions.

[0071] In the description of the present invention, it should be understood that the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0072] The embodiment of the present invention provides a network intrusion detection model that integrates deep learning and decision tree. Figure 1 , the model is a decision tree-convolutional neural network-decision tree model (DT-CNN-DT), which is composed of a first decision tree (DT) fused with a simplified convolutional neural network, an improved convolutional neural network (CNN), and a second decision tree (DT);

[0073] Among them, the first decision tree includes a root node and a leaf node, and each node (root node and leaf node) includes a simplified convolutional neural network; the root node is used to roughly classify the intrusion data to be detected (normal, abnormal, suspected abnormal), and then pass it to the leaf node of the next layer to further identify the roughly classified intrusion data to be detected, and remove the redundant or irrelevant features of the intrusion data to be detected, and finally output the network intrusion detection data set with a label at the leaf node, and the label is the preliminary judgment result; the simplified convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer. The pooling layer adopts the maximum pooling method, and the others are implemented using the general existing convolutional neural network structure. The first decision tree can be used offline.

[0074] The improved convolutional neural network is a two-dimensional convolutional neural network CNN, which includes a first convolutional layer, a first custom mixed pooling layer, a second convolutional layer, a second custom mixed pooling layer, a third convolutional layer and a flattening layer in sequence, the first convolutional layer is a 3×3 convolutional layer, the second convolutional layer and the third convolutional layer are 2×2 convolutional layers, and the first custom mixed pooling layer and the second custom mixed pooling layer are 2×2 custom mixed pooling layers; after the network intrusion detection dataset is processed by the first decision tree, a network intrusion detection dataset with a label (judgment result) is output, which passes through the first convolutional layer, the first custom mixed pooling layer, the second convolutional layer, the second custom mixed pooling layer, the third convolutional layer and the flattening layer in sequence, and finally outputs a feature vector for classification;

[0075] The second decision tree is implemented by the classic decision tree classifier. The depth of the second decision tree is not greater than 10, and the number of samples in each leaf node is not less than 5. At the same time, the pruning parameter range is set to [0.009-0.011] to control the growth of the tree. As a feasible solution, the depth of the decision tree is 10 to prevent overfitting; the minimum number of samples in each leaf node is 5 to ensure that the decision tree is not over-refined; the growth of the tree is controlled by setting the pruning parameter to 0.01, thereby reducing the risk of overfitting. The classic decision tree classifier is a prior art and has not been improved here, so it will not be described in detail.

[0076] As another embodiment, the present application provides a network intrusion detection method implemented by using the network intrusion detection model fused with the above-mentioned deep learning and decision tree, which includes:

[0077] Step 1: input the data to be detected into the first decision tree to obtain a network intrusion detection data set, which is a data set that filters normal data through the first decision tree and retains abnormal and suspected abnormal data;

[0078] Step 2: Preprocess the network intrusion detection data set obtained in step 1;

[0079] Step 3: Input the preprocessed network intrusion detection data set into the improved convolutional neural network and train the improved convolutional neural network;

[0080] Step 4: Improve the convolutional neural network to extract features and output feature vectors (feature maps);

[0081] Step 5: Input the feature vector to the second decision tree. After making a decision, the second decision tree outputs the classification result to determine whether there is a network intrusion.

[0082] The architecture of the present application adopts a network structure consisting of a first decision tree for offline use, an improved convolutional neural network (CNN), and a second decision tree for real-time judgment; CNN adopts a mixed pooling layer; and the proposed Roulette Wheel Selection (RWS)-based Particle Swarm Optimization (PSO) (RWS-PSO) is used to optimize the hyperparameters of the mixed pooling layer.

[0083] When in use, a large amount of new data to be detected that flows in every day is first roughly processed by the first decision tree, and a large amount of normal data is filtered, and abnormal and suspected abnormal data are retained as network intrusion detection data sets, thereby greatly speeding up the data processing speed of the entire model. At the same time, in order to avoid a large amount of data to be detected causing a large change in the structure of the tree, the depth of the first decision tree is set to 3 or 4.

[0084] Then, the network intrusion detection data set is input into the designed improved convolutional neural network (CNN). After the network performs feature extraction, the feature map is input into the second decision tree. After the second decision tree makes a decision, it outputs the classification result to determine whether there is a network intrusion. See the detailed algorithm flow chart for details. Figure 2 . Firstly, the network intrusion detection dataset is preprocessed; then, for the network intrusion detection dataset used, a convolutional neural network assisted decision tree model applied to network intrusion detection is proposed, which is realized by fusing the improved convolutional neural network and the second decision tree.

[0085] The KDD CUP99 dataset can be used as a dataset for improving convolutional neural network training. The specific description of the dataset is as follows:

[0086] (1) Dataset content: This dataset contains network connection records, which are marked as normal or under attack. The attack types are diverse, including DoS attacks, U2R attacks, R2L attacks, and probing attacks.

[0087] (2) Dataset characteristics: a. Large scale: The original dataset contains about 5 million records. b. Diversity: Contains a variety of attack modes and normal network behaviors.

[0088] (3) Dataset format: The dataset is usually provided in CSV format, containing 41 feature attributes and 1 class identifier. The class identifier contains two categories, namely "invasion" and "non-invasion".

[0089] (4) Feature type: The features in the dataset include continuous and discrete types, involving various aspects of network connections, such as duration, protocol type, traffic size, etc.

[0090] Preprocess the network intrusion detection dataset, including:

[0091] Step S21: Use the pandas component to read the training and test data sets, and the data is stored in a CSV format file. The training data set and the test data set store feature and label information respectively.

[0092] Step S22: Separate the features and labels in the training and test data sets. and As a feature, and as the corresponding label.

[0093] Step S23: Normalize the feature data through Normalizer to ensure that the data is within the same numerical range, thereby accelerating the convergence of model training.

[0094] Step S24: The labels are converted into binary format to suit the subsequent binary classification task.

[0095] Step S25: In order to match the input of the convolutional neural network, the data is padded so that the number of features increases from 41 to 42. Then, the feature data is reshaped into a pseudo image format with a shape of 7x6x1.

[0096] CNN is a two-dimensional convolutional neural network, including 1 3×3 convolutional layer, 2 2×2 convolutional layers, 2 2×2 custom mixed pooling layers and 1 flattening layer. The specific structure is shown in Table 1. The process of training the convolutional neural network CNN is as follows: the preprocessed and reshaped data is input into the first convolutional layer, the first convolutional layer generates 32 feature maps and is sent to the first custom mixed pooling layer for processing, then input into the second convolutional layer to generate 64 feature maps, then processed by the second custom mixed pooling layer, and then input into the third convolutional layer to generate 128 feature maps, and the feature map is flattened into a one-dimensional vector through the flattening layer to generate a feature vector for classification. This feature vector will be sent as input to the decision tree classifier for classification, and the complexity of the decision tree is controlled.

[0097] The specific control measures for the complexity of the decision tree include: the depth of the decision tree is limited to 10 to prevent overfitting; the minimum number of samples for each leaf node is 5 to ensure that the decision tree is not over-refined; the growth of the tree is controlled by setting the pruning parameter to 0.01 to reduce the risk of overfitting. Finally, the predicted label is output to classify the traffic data in the data set into normal traffic and attack traffic.

[0098] The custom mixed pooling layer is defined as: ;in, Represents the output feature map after mixed pooling, which is the result of processing the input x. Represents the feature map after the maximum pooling operation, represents the feature map after the average pooling operation, Represents the weight parameter of mixed pooling, which controls the weight distribution between maximum pooling and average pooling. This parameter is usually in the range of [0, 1].

[0099] The proposed roulette wheel selection based particle swarm algorithm RWS-PSO is used to perform the weight parameters of the mixed pooling layer. Optimization, including the following steps:

[0100] Step S31: establishing a particle swarm optimization algorithm for performing particle iterative search space S;

[0101] Step S32: Customize the weight coefficients in the mixed pooling layer Each value optimization scheme of is taken as a particle to establish a particle swarm;

[0102] Step S33: Calculate the fitness value of each particle at the current time t, that is, the performance index of the model on the validation set (that is, the accuracy and loss value). By normalizing the fitness value, the probability of each particle being selected can be calculated. Obtain the optimal hyperparameter value of each particle as its own optimal position , where K is the total number of particles, i is the i-th particle in the particle swarm optimization algorithm, and the hyperparameter value solution with the minimum total loss value in the search space at the current time t is taken as the global optimal value solution, that is, the global optimal position ;

[0103] Step S34: perform iterative search of particles using the RWS-PSO algorithm to complete the position update of the particles; when the iterative search reaches the set number of iterations, the final optimization result is obtained, and the value optimization of the hyperparameter is achieved; wherein the roulette particle swarm algorithm RWS-PSO uses the performance index (loss value) of the convolutional neural network (CNN) combined with the decision tree model on the validation set as the fitness value;

[0104] Wherein, the step S33 specifically includes the following steps:

[0105] Step S331: Calculate the probability of each particle being selected. is the cumulative probability interval of the particle:

[0106] ;

[0107] ;

[0108] ……;

[0109] ;

[0110] Each All correspond to particles Selected interval. A probability is assigned to each particle according to its fitness value. The higher the fitness of the particle, the greater the probability of being selected.

[0111] Step S332: Using random numbers Roulette selection is performed, and particles are selected in the following intervals:

[0112] ;

[0113] According to random number The size of the Still belongs to ; Represents the upper limit of the probability interval that depends on the individual optimal solution;

[0114] Step S333: Update the position of each particle using RWS-PSO, where the position of particle i at time t+1 is Updated to:

[0115] ;

[0116] in, It is a particle In the next iteration location, It is a particle At the current iteration, is the inertia weight, which controls the degree of particle velocity retention and affects the global search capability of the algorithm. It is a particle In the current iteration speed, and are acceleration coefficients, representing the individual learning factor and the social learning factor, respectively, and controlling the degree to which the particle approaches the individual optimum and the global optimum. represents the upper limit of the probability interval that depends on the individual optimal solution, is the individual optimal solution of the ith particle, is the global optimal solution, is the random number used in the velocity update formula.

[0117] Step S334: Determine position X i (t+1) Whether it satisfies the individual optimal update condition, if not, it depends on the global optimal update condition for update;

[0118] Step S335: Calculate the performance index (loss value) of the model corresponding to each particle on the validation set at the next time t+1 to obtain the optimal position of each particle. pBest' (The optimal position of particle i is pBest' i ), i=1~K, and obtain the global optimal position of the particle swarm gBest' ;Will pBest' The corresponding total loss value of the validation set is The corresponding total loss value of the validation set is compared. pBest' The corresponding total network interference value is smaller, then pBest' The value of ;Will gBest' The corresponding total loss value of the validation set is The corresponding validation set total loss value is compared. gBest' The corresponding validation set total loss value is smaller, then gBest' The value of ;

[0119] Step S336: Determine whether the preset number of iterations has been reached. If so, obtain the value of this iteration. pBest i ~pBest x as well as , the global optimal position As the final channel allocation solution, otherwise, return to step S335 and perform the next iteration.

[0120] In addition, this application uses 5-fold cross validation, splits the training set and trains the model multiple times. In each fold, an early stopping mechanism is used to avoid overfitting, and the model performance of each fold is recorded. The first few layers of the CNN model are used as feature extractors to extract intermediate features, and then the decision tree classifier is used to classify these features. The coefficients of the mixed pooling The Roulette Wheel Particle Swarm Algorithm (RW-PSO) is used to optimize the model and evaluate each model in each fold, and the accuracy and the current optimal coefficient are calculated respectively. and loss value.

[0121] At the same time, this application proposes a hybrid pooling layer that combines the advantages of maximum pooling and average pooling and uses adjustable parameters Controls the weights of the two pooling operations. Max pooling is used to extract the most significant features, while average pooling is used to smooth features. By adjusting the parameters of this layer, the model can show better feature extraction effects on different data sets, thereby improving the overall classification performance.

[0122] Adjustable parameters of the hybrid pooling layer The use of the roulette wheel-based particle swarm algorithm RWS-PSO for hyperparameter optimization can effectively avoid falling into the local optimal solution. Standard PSO relies on two important positions: the individual optimal position and the global optimal position. The roulette wheel selection introduces probability to particle updates, increases the randomness of the algorithm, and makes the particles not always move towards the same goal, thus having more exploration capabilities in preventing local optimal solutions.

[0123] The method of integrating the deep learning method and the decision tree proposed in this application is denoted as DT-CNN-DT. The comparative examples are the method of using only convolutional neural network and decision tree, the method of first using decision tree and then using convolutional neural network, denoted as DT-CNN; the method of first using convolutional neural network and then using decision tree, denoted as CNN-DT.

[0124] Table 2 shows the performance improvement of the DT-CNN-DT algorithm by using the hyperparameters optimized by the RWS-PSO algorithm. The table shows that after using the hyperparameters, the proposed DT-CNN-DT algorithm has achieved significant improvements in accuracy, precision, recall, and F1-score. In addition, the table also shows that the DT-CNN-DT algorithm has better performance than the CNN-DT algorithm and DT-CNN.

[0125] Table 2 Comparison of hyperparameter optimization results of DT-CNN-DT model using RWS-PSO

[0126]

[0127] Table 3 compares the results of DT, CNN, DT-CNN, CNN-DT, and DT-CNN-DT algorithms. It can be seen from the table that the performance of the DT-CNN-DT algorithm with optimized hyperparameters is the best among the algorithms.

[0128] Table 3 Comparison of the results of DT, CNN, DT-CNN, CNN-DT, and DT-CNN-DT algorithms

[0129]

[0130] When using decision trees for classification, although the model has high accuracy, overfitting may occur when processing complex data, resulting in insufficient model generalization ability. This application processes the complex data in advance through an offline first decision tree model, and then combines the convolutional neural network with the second decision tree, which not only improves the overall accuracy of the model, but also improves the generalization ability of the model, making the classification performance more stable.

[0131] The method of the present invention is not limited to being executed in the time sequence described in the specification, and may also be executed in other time sequences, in parallel or independently. Therefore, the execution order of the method described in this specification does not limit the technical scope of the present invention.

[0132] Although the present invention has been disclosed above by describing specific embodiments of the present invention, it should be understood that all the above embodiments and examples are exemplary rather than restrictive. Those skilled in the art may design various modifications, improvements or equivalents of the present invention within the spirit and scope of the appended claims. These modifications, improvements or equivalents should also be considered to be included in the protection scope of the present invention.

Claims

1. A network intrusion detection model integrating deep learning and decision tree, characterized by: The model is a decision tree-convolutional neural network-decision tree model, which is composed of a first decision tree fused with a simplified convolutional neural network, an improved convolutional neural network, and a second decision tree; Among them, the first decision tree includes a root node and a leaf node, each node includes a simplified convolutional neural network; the root node is used to roughly classify the input network intrusion data to be detected into normal, abnormal and suspected abnormal data; and then it is transmitted to the leaf nodes of the next layer to further identify the roughly classified network intrusion data to be detected, and at the same time remove redundant or irrelevant features of the network intrusion data to be detected, and finally output a network intrusion detection data set with a label at the leaf node, and the label is a preliminary judgment result; the simplified convolutional neural network includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer, and the pooling layer adopts a maximum pooling method; The improved convolutional neural network is a two-dimensional convolutional neural network CNN, which includes a first convolutional layer, a first custom mixed pooling layer, a second convolutional layer, a second custom mixed pooling layer, a third convolutional layer and a flattening layer in sequence, the first convolutional layer is a 3×3 convolutional layer, the second convolutional layer and the third convolutional layer are 2×2 convolutional layers, and the first custom mixed pooling layer and the second custom mixed pooling layer are 2×2 custom mixed pooling layers; after the network intrusion detection dataset is processed by the first decision tree, a network intrusion detection dataset with a label is output, which passes through the first convolutional layer, the first custom mixed pooling layer, the second convolutional layer, the second custom mixed pooling layer, the third convolutional layer and the flattening layer in sequence, and finally outputs a feature vector for classification; The second decision tree is implemented by the classic decision tree classifier. The depth of the second decision tree is no more than 10, and the number of samples in each leaf node is no less than 5. At the same time, the pruning parameter range is set to [0.009-0.011] to control the growth of the tree. The first custom mixed pooling layer and the second custom mixed pooling layer have the same structure and are defined as: ; in, Represents the output feature map after mixed pooling, which is the result of processing the input x; Represents the feature map after the maximum pooling operation, represents the feature map after the average pooling operation, Represents the weight parameter of mixed pooling, with a value range of [0, 1].

2. The network intrusion detection model according to claim 1, characterized in that: The depth of the first decision tree is set to 3 or 4.

3. The network intrusion detection model according to claim 1, characterized in that: The weight parameters of the mixed pooling Optimized by roulette particle swarm algorithm.

4. The network intrusion detection model according to claim 1, characterized in that: The improved convolutional neural network adopts five-fold cross validation, which specifically includes: The original network intrusion detection dataset is randomly divided into five subsets of equal size, each subset contains the same number of data samples; Training and testing of the improved convolutional neural network: In each iteration, one of the subsets is selected as the test set, and the remaining four subsets are combined as the training set; the improved convolutional neural network is trained using the training set and evaluated on the test set; the process is repeated five times to ensure that each subset is used as a test set once; For each test, use the preset performance metrics to evaluate the performance of the improved convolutional neural network on the test set; The results of the performance metrics of the five tests are averaged to obtain the final performance evaluation of the improved convolutional neural network.

5. A network intrusion detection method integrating deep learning and decision tree, characterized in that: The network intrusion detection model comprising any one of claims 1 to 4, wherein the detection method comprises: Step 1: input the data to be detected into the first decision tree to obtain a network intrusion detection data set, which is a data set that filters normal data through the first decision tree and retains abnormal and suspected abnormal data; Step 2: Preprocess the network intrusion detection data set obtained in step 1; Step 3: Input the preprocessed network intrusion detection data set into the improved convolutional neural network and train the improved convolutional neural network; Step 4: Improve the convolutional neural network to extract features and output feature vectors; Step 5: Input the feature vector to the second decision tree. After making a decision, the second decision tree outputs the classification result to determine whether there is network intrusion.

6. The network intrusion detection method according to claim 5, characterized in that: The preprocessing process of the network intrusion detection data set in step 2 is as follows: Step S21: dividing the network intrusion detection data set into a training data set and a test data set, wherein the training data set and the test data set respectively store feature and label information; Step S22: Separate the features and labels in the training data set and the test data set. and As a feature, and As the corresponding label; Step S23: normalize the features to ensure that the data are within the same numerical range; Step S24: convert the labels into binary format to adapt to the subsequent binary classification task; Step S25: Convert the normalized features into the three-dimensional format required by the convolutional neural network.

7. The network intrusion detection method according to claim 5, characterized in that: The improved convolutional neural network in step 3 includes the first convolution layer, the first custom mixed pooling layer, the second convolution layer, the second custom mixed pooling layer, the third convolution layer and the flattening layer in sequence. The first convolution layer is a 3×3 convolution layer, the second convolution layer and the third convolution layer are 2×2 convolution layers, and the first custom mixed pooling layer and the second custom mixed pooling layer are 2×2 custom mixed pooling layers.

8. The network intrusion detection method according to claim 7, characterized in that: The first custom mixed pooling layer and the second custom mixed pooling layer have the same structure and are defined as: ; in, Represents the output feature map after mixed pooling; Represents the feature map after the maximum pooling operation, represents the feature map after the average pooling operation, Represents the weight parameter of mixed pooling, with a value range of [0, 1]; The weight parameters of the mixed pooling The optimization is done through the roulette particle swarm algorithm, which includes the following steps: Step S31: establishing a particle swarm optimization algorithm for performing particle iterative search space S; Step S32: The weight coefficients in the first custom mixed pooling layer and the second custom mixed pooling layer are Each value optimization scheme of is taken as a particle to establish a particle swarm; Step S33: Calculate the fitness value of each particle at the current time t, calculate the probability of each particle being selected by normalizing the fitness value; obtain the optimal hyperparameter value of each particle as its optimal position , where K is the total number of particles, i is the i-th particle in the particle swarm optimization algorithm, and the hyperparameter value solution with the minimum total loss value in the search space at the current time t is taken as the global optimal value solution, that is, the global optimal position ; Step S34: perform iterative search of particles using a particle swarm algorithm based on roulette wheel selection to complete the position update of particles; when the iterative search reaches the set number of iterations, the final optimization result is obtained to achieve the optimization of the value of the hyperparameter.

9. The network intrusion detection method according to claim 8, characterized in that: The step S33 specifically includes the following steps: Step S331: Calculate the probability of each particle being selected; record For particles The cumulative probability interval of : ； ； ……； ； Each All correspond to particles The probability of being selected, fitness i is the normalized fitness value of the i-th particle, i ranges from 1 to n; Σfitness i It is the sum of the normalized fitness values ​​of all particles. A probability is assigned to each particle according to its fitness value. The higher the fitness of the particle, the greater the probability of being selected. Step S332: By random number Roulette selection is performed, and particles are selected in the following intervals: ； According to random number The size of the Still belongs to ; Represents the upper limit of the probability interval that depends on the individual optimal solution; Step S333: Update the position of each particle using the roulette-based particle swarm algorithm, where the position of particle i at time t+1 is Updated to: ； in, It is a particle In the next iteration location, It is a particle At the current iteration, is the inertia weight, which controls the degree of particle velocity retention and affects the global search capability of the algorithm. It is a particle In the current iteration speed, and are acceleration coefficients, representing the individual learning factor and the social learning factor, respectively, and controlling the degree to which the particle approaches the individual optimum and the global optimum. represents the upper limit of the probability interval that depends on the individual optimal solution, is the individual optimal solution of the ith particle, is the global optimal solution, is the random number in the speed update formula; Step S334: Determine the location Whether it satisfies the individual optimal update condition, if not, it depends on the global optimal update condition for update; Step S335: Calculate the performance index of the model corresponding to each particle on the validation set at the next time t+1 to obtain the optimal position of each particle. pBest' , the optimal position of particle i is pBest' i , i=1~K, and obtain the global optimal position of the particle swarm gBest' ;Will pBest' The corresponding total loss value of the validation set is The corresponding total loss value of the validation set is compared. pBest' The corresponding total network interference value is smaller, then pBest' The value of ;Will gBest' The corresponding total loss value of the validation set is The corresponding validation set total loss value is compared. gBest' The corresponding validation set total loss value is smaller, then gBest' The value of ; Step S336: Determine whether the preset number of iterations has been reached. If so, obtain the pBest of this iteration. i ~pBest x as well as , pBest x is the individual optimal solution of the xth particle, x=i~K, and the global optimal position As the final channel allocation solution, otherwise, return to step S335 and perform the next iteration.

Images