Network intrusion detection method based on joint random forest

Abstract

The application provides a network intrusion detection method based on a joint random forest; each participant first prepares a local network traffic data set, pre-processes the data, trains a local first random forest model, and uploads the model to a central server; the central server integrates the models of the participants into an integrated random forest model and distributes the model to the participants; each participant locally constructs a new data set for training of a logistic regression model; the central server and the participants cooperatively complete training of the logistic regression model; after adjustment of the trained joint random forest model, the integrated random forest model and the logistic regression model are deployed locally and perform an intrusion detection task. The method can realize construction of an efficient, robust, interpretable and expandable intrusion detection model by the participants under the premise of full protection of data privacy.

Description

Technical Field

[0001] The present invention relates to a network intrusion detection method, in particular to a network intrusion detection method based on joint random forest. Background Art

[0002] In today's digital age, the rapid development of internet technology has profoundly impacted every aspect of society. With the continuous advancement of informatization, the internet has become an integral part of daily life and work. However, this advancement in information technology is also accompanied by growing cybersecurity challenges. With the explosive growth in the number of internet users and the widespread adoption of various smart devices, such as computers, mobile devices, and IoT devices, the frequency and sophistication of cyberattacks have increased significantly. Cyberattacks not only cause direct economic losses to individual users and businesses, but can also threaten national security and even trigger geopolitical conflicts.

[0003] Against this backdrop, network intrusion detection technology has become a crucial tool for ensuring network security. Traditional intrusion detection methods primarily rely on rule and signature matching. While these methods offer high accuracy in detecting known threats, their effectiveness significantly decreases when dealing with new and unknown threats. Specifically, rule-based intrusion detection systems require continuous updating of their rule bases, which becomes increasingly difficult to maintain and update given the complex and ever-changing nature of attack patterns. Another drawback of traditional methods is their inadequacy in combating advanced persistent threats (APTs), which are often highly cryptic and cannot be effectively detected through simple signature matching or rule-based analysis.

[0004] To address this challenge, a growing number of researchers are exploring the use of machine learning techniques to improve the intelligence and accuracy of intrusion detection. Machine learning, particularly supervised learning, has been widely used in the field of network intrusion detection. Trained on samples of normal and abnormal traffic in a dataset, machine learning models can automatically identify potential network attacks. Among them, random forest, as an ensemble learning method, demonstrates strong application potential in network intrusion detection due to its efficiency, robustness, interpretability, and scalability. Random forest constructs multiple decision trees and combines the predictions of each tree during classification, thereby improving the accuracy and stability of the overall model's predictions. However, the effectiveness of traditional machine learning models such as random forests relies on the sufficiency and diversity of data. With the increasing prominence of data privacy and security issues, traditional centralized machine learning methods face numerous limitations when collecting and processing large-scale user data. In particular, characteristic data of network attacks is more sensitive than general data, and centralized storage and processing of this data poses serious security risks.

[0005] Federated learning is an emerging distributed machine learning technology. By training models locally on different participants and transmitting only the model information to a central server for aggregation, federated learning protects user data privacy while fully leveraging distributed data resources for model training. However, existing federated learning frameworks still face challenges in processing heterogeneous data and improving model generalization capabilities. This is particularly true when facing complex and diverse cyberattack patterns, often leading to unstable training processes, slow model convergence, or even non-convergence. Summary of the Invention

[0006] This paper addresses the shortcomings of existing technologies by proposing a network intrusion detection method that combines ensemble learning and federated learning. This method fully utilizes local data from all parties while protecting data privacy, building an efficient, robust, interpretable, and highly generalizable intrusion detection model. This method effectively addresses the shortcomings of existing technologies in data privacy protection, model generalization, training convergence difficulties, and complex network attack detection.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] The present invention provides a network intrusion detection method based on joint random forest, which comprises the following steps:

[0009] (1) Each participant prepares a local network traffic raw data set, divides the network traffic samples in the data set into normal samples and abnormal samples, and divides and preprocesses the training set and test set;

[0010] (2) Each participant uses the local original data set to train the first random forest model and tune the parameters to minimize the AUC of the first random forest model; the trained first random forest model is uploaded to the central server, and the central server integrates the first random forest models from each participant to obtain an integrated random forest model and distributes it to each participant;

[0011] (3) Each participant divides the local original training set into k subsets and performs k groups of training and prediction tasks; each group selects a different subset as the subset to be predicted, and trains a new second random forest model on the remaining k-1 training subsets, and uses the trained second random forest model to predict on the subset to be predicted and the original test set; the prediction results of the prediction subsets obtained by the k groups of tasks are spliced ​​together as the features of the new training set; the average of the prediction results of the k original test sets obtained by the k groups of tasks is taken as the feature of the new test set;

[0012] (4) After receiving the integrated random forest model from the central server, each participant uses the first random forest model from other participants in the received integrated random forest model to predict its own local original data set, and uses the prediction results as the features of the new training set and test set;

[0013] (5) Each participant uses the features obtained in steps (3) and (4) to construct a new local data set; the central server initializes the logistic regression model and distributes it to each participant;

[0014] (6) Each participant uses the FedAvg algorithm to train the logistic regression model based on the new training set and test set, and stops training after reaching the number of training epochs or the required accuracy;

[0015] (7) Each participant will integrate the random forest model and the trained logistic regression model to form a joint random forest model, deploy the joint random forest locally, and perform intrusion detection tasks.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] (1) The method proposed in the present invention is efficient, robust, interpretable, and scalable. The efficiency and robustness of the combined random forest enable it to efficiently perform intrusion detection tasks even on edge devices with low computing power. Since the split of each node in the random forest has its own characteristic basis, the prediction results of the model can be fully explained, and the results of the model classification are traceable. In addition, since the combined random forest is an integrated learning method, it allows manual addition of artificial prior knowledge such as rule trees, which greatly improves the generalization and detection capabilities of the model, while facilitating updates and iterations.

[0018] (2) The method proposed in the present invention uses an integrated random forest as the basic model and logistic regression as the stacking method of the meta-model, and trains the logistic regression model through a federated learning algorithm. During the construction of the entire intrusion detection model, the training data of each participant does not leave its own local domain, so that the data resources of each participant can be fully utilized for model training under the premise of ensuring data privacy. The entire model has strong interpretability, and the parameters of the logistic regression can be understood as the weight of each random forest prediction value in the integrated random forest model in the final prediction result. The optimal combination of weights can be achieved through training. This algorithm achieves the efficiency, robustness, interpretability and scalability of the intrusion detection model under the premise of protecting the privacy of each participant. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of the method of the present invention;

[0020] Figure 2It is the model structure diagram of the joint random forest;

[0021] Figure 3 This is a schematic diagram of using the KFold method to split the data set and make predictions;

[0022] Figure 4 This is a schematic diagram of the construction of new dataset features for logistic regression model training.

[0023] Figure 5 This is a schematic diagram of federated learning completing joint modeling tasks. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] The network intrusion detection method based on joint random forest proposed in this paper can be referred to Figure 1 In order to describe the complementarity of this method in more detail, this embodiment describes the overall method in 10 sub-steps. It should be noted that, without violating the concept of the present invention, two or more sub-steps can be combined into one sub-step as needed, or the sub-steps can be processed in parallel or the order of processing can be adjusted as needed. Specifically, the method illustrated in this embodiment mainly includes the following steps:

[0026] Step 1: Each participant (C1~C n ) Collect traffic data from network monitoring devices, intrusion detection system (IDS) logs, etc., and clean the data to remove redundant, invalid or erroneous traffic data;

[0027] Extract basic features (such as source IP, destination IP, protocol type, port number, packet length, etc.) and statistical features (such as transmission rate, connection duration, etc.) from traffic data, and label the data (normal samples are labeled as 0, abnormal samples are labeled as 1). Normal and abnormal samples can be automatically labeled based on predefined rules or manually reviewed and labeled.

[0028] Divide the dataset D into training set D according to a certain ratio train and the test set D test .

[0029] Step 2: Each participant reaches a consensus on the unified specifications and requirements for the original features and feature engineering of the data set, ensures the consistency of feature selection and encoding methods, follows the unified specifications, and strictly follows the data privacy protection regulations.

[0030] Each participant detects and processes the missing values in their local data set, and deletes or fills the missing values. If necessary, the data feature values are standardized and normalized.

[0031] The formula for standardization is as follows:

[0032]

[0033] where z is the standardized value, x is the original data value, μ is the mean of the original data, and σ is the standard deviation of the original data.

[0034] The formula for normalization is as follows:

[0035]

[0036] where x' is the normalized value, x is the original data value, min(x) is the minimum value in the data set, and max(x) is the maximum value in the data set.

[0037] Step 3: Each participant initializes their local first random forest model (RF1 ~ RF n ), sets the initial parameters such as the number of trees and the maximum depth, etc. Using the full pre-processed training set data locally, the first random forest model is trained, and the model is parameterized (e.g., using grid search, random search, Bayesian optimization, etc. to systematically adjust the parameter combinations), and the parameter configuration that minimizes the model AUC is found. The AUC of each parameter combination is calculated on the test set to evaluate the classification performance of the model. The larger the AUC (closer to 1), the better the classification performance of the model. The formula for calculating AUC is as follows:

[0038]

[0039] where TPR(t) is the true positive rate at a given threshold t, and FPR(t) is the false positive rate. The calculation formula is as follows:

[0040]

[0041] TP (True Positives): True positives, that is, the number of samples correctly predicted as positive. FN (False Negatives): False negatives, that is, the number of samples that are actually positive but predicted as negative. FP (False Positives): False positives, that is, the number of samples that are actually negative but predicted as positive. TN (True Negatives): True negatives, that is, the number of samples correctly predicted as negative.

[0042] Step 4: Each participant uploads the first random forest model trained locally to the central server S. The central server receives the uploaded model file, loads the model file of each participant into an available random forest object, and adds a source attribute to the random forest object of each participant, indicating which participant the random forest object comes from. A new random forest model is created, and all the random forest objects of the participants are added to the new random forest model, and the source attribute of the random forest object is retained. This new random forest model is named the integrated random forest model (such as Figure 2 ), the central server distributes the integrated random forest model to each participant. Step 4 of the present invention integrates the random forests of each party into an integrated random forest model and uses this model as the base model. This method can efficiently perform intrusion detection tasks on low-computing edge devices. In addition, because the split of each node in the random forest has its own characteristic basis, the model's prediction results can be fully explained. The model classification results are well-founded, greatly improving the efficiency, robustness, and interpretability of the intrusion detection model.

[0043] Step 5: Figure 3 , each participant divides the local training set into n subsets (in this embodiment, n is 5, and each subset is represented by D1 to D5), and ensures that each subset has a similar distribution. A cross-validation method (such as KFold) can be used for division. Each participant performs k groups of training and prediction tasks on a new random forest model (called the second random forest model to distinguish it from the first random forest model in step 3 of the embodiment); wherein each group of tasks selects 1 different subset as the subset to be predicted, and trains the second random forest model on the remaining k-1 training subsets, and uses the trained second random forest model to make predictions on the subset to be predicted and the original test set. This process is Figure 3 It is represented as steps S1 to S5. The predicted value F of the prediction subset of k groups of tasks is 1p ~F 5p Spliced ​​together as the feature F of the new training set new ; Take the average of the prediction results predict1 to predict5 of the k original test sets obtained by the k groups of tasks as the feature F of the new test set test_pStep 5 of the present invention adopts the KFold split prediction method, which can avoid the data leakage problem in the prediction process, thereby making the entire method more rigorous and the model more generalizable.

[0044] Step 6: Each participant receives the ensemble random forest model from the central server and ensures the integrity of the model file. Use the first random forest model from other participants in the ensemble random forest model to predict the local original training set and test set. The prediction results of each model are used as the features of the new training set and test set ( Figure 4 The new training set features F in new2 ~F newn , and the new test set features F test2 ~F testn ), the prediction results of each model correspond to a column of features (such as Figure 4 ). Combine the generated new features with the new features obtained in step 5 (the training set features F obtained in step 4 new exist Figure 5 Expressed as F new1 , the obtained test set features F test_p exist Figure 4 Expressed as F test1 ) are merged to form the features of the new training and test sets for the meta-model.

[0045] Step 7: Get the features of the new training set for the meta-model in step 6 ( Figure 4 F in new1 ~F newn ) and the features of the new test set ( Figure 4 F in test1 ~F testn ), the labels of the original training set and test set in step 1 are used to combine them into new training set and test set for the meta-model. The central server initializes the logistic regression model as the meta-model and distributes it to each participant (such as Figure 5 In step ①), the number of features input to the logistic regression model is consistent with the number of participants. The expression of the logistic regression model is as follows:

[0046] Linear part:

[0047] z=β0+β1x1+β2x2+…+β n x n , where x1, x2, …, x n Represents the input features, β0, β1, β2, ..., β n is the parameter to be learned, and z is the output of the linear part.

[0048] Logical functions:

[0049] Where p is the final output of the logistic regression model.

[0050] Step 7 of the present invention selects a logistic regression model as a meta-model. The model has strong interpretability and a simple model structure that is not easy to overfit. The parameters of the logistic regression can be understood as the weights of the prediction values ​​of each random forest in the integrated random forest model in the final prediction result, thereby achieving the efficiency, robustness, interpretability and scalability of the intrusion detection model.

[0051] Step 8: Each participant uses the new training set obtained in step 7 to perform several rounds of logistic regression model training locally to complete the local training (such as Figure 5 Each participant encrypts the model parameters after local training to ensure the security of data transmission, and uploads the encrypted model parameters to the central server through a secure network (such as Figure 5 The central server receives and decrypts the model parameters uploaded by each participant, and uses the FedAvg algorithm to aggregate the decrypted model parameters (such as weighted average) to generate new global model parameters (such as Figure 5 The central server encrypts the aggregated global model parameters to ensure the security of data transmission and distributes the encrypted global model parameters to each participant (such as Figure 5 1). Each participant receives and decrypts the global model parameters, and uses the decrypted global model parameters to update the local logistic regression model. Each participant continues to train locally using the updated logistic regression model parameters to complete a new training round. Repeat the above steps, perform multiple rounds of federated learning iterations, and gradually optimize the performance of the logistic regression model until the predetermined convergence conditions are met or the predetermined training rounds are reached. Step 8 of the present invention adopts a federated learning model training method. During the entire intrusion detection model construction process, the training data of each participant does not leave its own local domain, thereby fully utilizing the data resources of each participant for model training while ensuring data privacy.

[0052] Step 9: Refer to the rule base of an existing rule-based intrusion detection system. These rules typically include, but are not limited to, specific port scans, unusual traffic patterns, protocol violations, and access from specific malicious IP addresses. Convert these rules into decision trees and add them to the ensemble random forest model to improve the accuracy and responsiveness of the intrusion detection model. Based on newly discovered cyberattack cases, manually construct a decision tree that conforms to the rules and use it as part of the ensemble random forest, ensuring that the structure and characteristics of the rule tree are consistent with those of the existing ensemble random forest model. After inserting the constructed rule tree into the ensemble random forest model, add the input dimensions of the logistic regression model obtained in Step 8) and assign weights to the features of the newly added dimensions. Evaluate the new ensemble random forest model, analyzing the performance and contribution of each forest. Adjust the weight of the logistic regression model based on prediction accuracy, increasing the weight of forests with high accuracy and decreasing the weight of forests with low accuracy. Finally, the adjusted ensemble random forest model and the logistic regression model form a joint random forest model. Step 9 of the present invention adopts a method of integrating artificial prior knowledge in the field of intrusion detection into an integrated random forest model. This method can make full use of artificial data analysis and expert knowledge to enhance the model's ability to identify known and unknown attacks, thereby greatly improving the model's generalization and detection capabilities, while facilitating updates and iterations.

[0053] Step 10: Deploy the joint random forest model locally and load it into the intrusion detection system. Figure 2 As shown in the figure, data is collected from network traffic in real time, and preprocessed and feature extracted to obtain a data set D. The data is first input into the integrated random forest model, and each random forest gives a corresponding prediction value (R1~R n ), and then input these predicted values ​​as features into the logistic regression model to obtain the final predicted value R. R is a value between 0 and 1, indicating the probability that the model will judge the network traffic to be abnormal. By comparing it with a set threshold (for example, 0.5), if R is greater than or equal to the threshold, it is judged as abnormal traffic; otherwise, it is judged as normal traffic. Based on the final predicted value R and the threshold, the predicted result is determined to be normal traffic or abnormal traffic (in this implementation, abnormal traffic is considered to be intrusion behavior), and the corresponding response measures are triggered based on the result.

[0054] The above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and many variations are possible. All variations that can be directly derived or imagined by a person skilled in the art from the disclosure of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A network intrusion detection method based on joint random forest, characterized in that: The following steps are involved: (1) Each participant prepares a local network traffic raw data set, divides the network traffic samples in the data set into normal samples and abnormal samples, and divides and preprocesses the training set and test set; (2) Each participant uses the local original data set to train the first random forest model and tune the parameters to minimize the AUC of the first random forest model; the trained first random forest model is uploaded to the central server, and the central server integrates the first random forest models from each participant to obtain an integrated random forest model and distributes it to each participant; (3) Each participant divides the local original training set into k subsets and performs k groups of training and prediction tasks; each group selects a different subset as the subset to be predicted, and trains a new second random forest model on the remaining k-1 training subsets, and uses the trained second random forest model to predict on the subset to be predicted and the original test set; the prediction results of the prediction subsets obtained by the k groups of tasks are spliced ​​together as the features of the new training set; the average of the prediction results of the k original test sets obtained by the k groups of tasks is taken as the feature of the new test set; (4) After receiving the integrated random forest model from the central server, each participant uses the first random forest model from other participants in the received integrated random forest model to predict its own local original data set, and uses the prediction results as the features of the new training set and test set; (5) Each participant uses the features obtained in steps (3) and (4) to construct a new local data set; the central server initializes the logistic regression model and distributes it to each participant; (6) Each participant uses the FedAvg algorithm to train the logistic regression model based on the new training set and test set, and stops training after reaching the number of training epochs or the required accuracy; (7) Each participant will integrate the random forest model and the trained logistic regression model to form a joint random forest model, deploy the joint random forest locally, and perform intrusion detection tasks.

2. The network intrusion detection method based on joint random forest according to claim 1 is characterized in that: The step (1) is specifically as follows: (1.1) Each participant collects traffic data from network monitoring devices and intrusion detection system (IDS) logs; (1.2) Clean the data to remove redundant, invalid or erroneous traffic data; (1.3) Feature extraction: extracting basic features and statistical features from traffic data; (1.4) Label the data as normal or abnormal; (1.5) Divide the dataset into training set and test set according to a certain ratio.

3. The network intrusion detection method based on joint random forest according to claim 1 is characterized in that: The pretreatment in step (1) is as follows: Detect missing values ​​in the data set and handle missing values ​​by deletion or filling; Establish unified specifications and requirements for the original features and feature engineering of the dataset to ensure that feature selection and encoding methods are consistent and follow unified specifications.

4. The network intrusion detection method based on joint random forest according to claim 1 is characterized in that: The step (2) is specifically as follows: (2.1) Each participant initializes their first random forest model and sets initial parameters; (2.2) Use the full preprocessed training set data locally to train the first random forest model; (2.3) Perform appropriate parameter tuning on the model, systematically adjust the parameter combination, and find the parameter configuration that minimizes the model AUC; (2.4) Each participant uploads the trained first random forest model to the central server. The central server integrates the first random forest models from each participant to obtain an integrated random forest model and distributes it to each participant.

5. The network intrusion detection method based on joint random forest according to claim 4 is characterized in that: The step (2.4) is specifically: (2.4.1) Each participant uploads the first random forest model trained locally to the central server; (2.4.2) The central server receives the uploaded model files and loads each participant’s model files into a usable random forest object; (2.4.3) Add an attribute to each participant’s random forest object to indicate which participant the random forest object originated from; (2.4.4) Create a new random forest model, add all the random forest objects of the participants to the new random forest model, and retain the source attributes of the random forest objects. This new random forest model is named the ensemble random forest model; (2.4.5) The central server distributes the integrated random forest model to each participant.

6. The network intrusion detection method based on joint random forest according to claim 1 is characterized in that: The step (4) is specifically as follows: (4.1) Each participant receives the integrated random forest model from the central server and ensures the integrity of the model file; (4.2) using the first random forest model from other participants in the received ensemble random forest model to make predictions on the local original training set and test set; The prediction results of each model are used as features of the new training set and test set, and the prediction results of each model correspond to a column of features; (4.3) The generated new features are merged with the new features obtained in step (5) to form the features of the new training set and test set for the meta-model.

7. The network intrusion detection method based on joint random forest according to claim 6 is characterized in that: The step (5) is specifically as follows: (5.1) After obtaining the features of the new training set and test set for the meta-model in step (4.3), keep the labels of the original test set and validation set unchanged to form new training set and test set; (5.2) The central server initializes the logistic regression model as a meta-model and distributes it to each participant. The number of features input to the logistic regression model is consistent with the number of participants. The expression of the logistic regression model is as follows: Linear part: z=β0+β1x1+β2x2+…+β n x n , where x1, x2, …, x n Represents the input features, β0, β1, β2, ..., β n is the parameter to be learned, z is the output of the linear part; Logical functions: Where p is the final output of the logistic regression model.

8. The network intrusion detection method based on joint random forest according to claim 7 is characterized in that: The step (6) is specifically as follows: (6.1) Each participant uses the new training set to perform several rounds of logistic regression model training locally to complete local training; (6.2) Each participant encrypts the locally trained model parameters to ensure the security of data transmission, and uploads the encrypted model parameters to the central server via a secure network; (6.3) The central server receives and decrypts the model parameters uploaded by each participant, and uses the FedAvg algorithm to perform weighted averaging on the decrypted model parameters to generate new global model parameters; (6.4) The central server encrypts the aggregated global model parameters to ensure data transmission security and distributes the encrypted global model parameters to each participant. Each participant receives and decrypts the global model parameters and uses the decrypted global model parameters to update the local logistic regression model. (6.5) Each participant continues training locally using the updated global model parameters to complete a new training round; (6.6) Repeat the above steps (6.1) to (6.5) for multiple rounds of federated learning iterations to gradually optimize the model performance until the predetermined convergence conditions are met or the predetermined number of training rounds is reached.

9. The network intrusion detection method based on joint random forest according to claim 1 is characterized in that: The step (7) is specifically as follows: (7.1) Integrate the random forest model and the logistic regression model obtained in step 6) to form a joint random forest model. Each participant loads the joint random forest model into the intrusion detection system to ensure that the model can be called in real time; (7.2) Collect data from network traffic in real time, perform preprocessing and feature extraction, and input the preprocessed real-time data into the deployed ensemble random forest model for prediction; each random forest model generates a corresponding prediction result; (7.3) These prediction results are input as features into the logistic regression model to obtain the final prediction results; based on the model prediction results, abnormal traffic is identified and detected in real time.

10. The network intrusion detection method based on joint random forest according to claim 1, characterized in that: Before forming the joint random forest model, step (7) further includes adding rule trees to the integrated random forest model based on the existing rule base and the newly discovered network attack cases, and optimizing the weights of each forest. The specific steps are: (a) Referring to the rule base of existing rule-based intrusion detection systems, these rules are converted into decision trees and added to the ensemble random forest model to improve the accuracy and response capability of the intrusion detection model; (b) Based on newly discovered cyberattack cases, manually construct a rule-based decision tree as part of an ensemble random forest, ensuring that the structure and characteristics of the rule tree are consistent with the existing ensemble random forest model; (c) After inserting the constructed rule tree into the ensemble random forest model, the input dimension of the logistic regression model is increased and weights are assigned to the features of the newly added dimension; (d) Evaluate the new ensemble random forest model, analyze the performance and contribution of each forest, and adjust the weight of the logistic regression model based on the prediction accuracy, increasing the weight of forests with high accuracy and reducing the weight of forests with low accuracy.

Images