A layered network intrusion detection method

Abstract

This invention discloses a layered network intrusion detection method. It includes: deploying an anomaly detector at the first layer to distinguish between benign and malicious traffic; deploying a malicious traffic discriminator at the second layer to distinguish between visible and undetected malicious traffic detected by the anomaly detector; deploying a visible malicious traffic classifier at the third layer to identify the category of visible malicious traffic detected by the malicious traffic discriminator; and deploying a custom model at the third layer to analyze undetected malicious traffic detected by the malicious traffic discriminator. The network intrusion detection method designed based on this invention achieves better performance in terms of inference speed, prediction accuracy, and layered deployment, thereby improving the security of computer systems.

Description

Technical Field

[0001] This invention relates to the field of network traffic intrusion detection, and more specifically, to a layered network intrusion detection method. Background Technology

[0002] Network intrusion detection systems (NICs) are widely used to detect malicious traffic and issue timely alerts. However, due to the increasing types and quantities of malicious network traffic, accurately detecting it using a single machine learning model is becoming increasingly difficult. Integrating multiple machine learning models is considered one of the development directions for NICs, namely, building a hybrid network intrusion detection system. The core idea of ​​a hybrid network intrusion detection system is to cascade multiple machine learning models to detect visible malicious traffic (i.e., malicious traffic categories included in the training set) and unseen malicious network traffic (i.e., malicious traffic categories not seen by the model during training) with high accuracy, and to identify the specific category of the visible malicious traffic.

[0003] Different models in a hybrid network intrusion detection system can be deployed at different levels (e.g., edge, fog, and cloud layers) to fully utilize the computing and storage capabilities of each level to improve performance metrics such as inference speed. However, existing hybrid network intrusion detection systems generally treat system design as a stack of multiple machine learning models. This complex model stacking neglects one or more performance metrics such as functional completeness (detecting unseen malicious traffic, distinguishing between visible and unseen malicious traffic, and identifying the specific category of visible malicious traffic), inference speed, layered deployment, and prediction accuracy. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a hierarchical network intrusion detection method that improves inference speed and prediction accuracy while supporting the deployment of multiple machine learning models at different levels.

[0005] The technical content of this invention includes:

[0006] A hierarchical network intrusion detection method, characterized in that the method includes the following steps:

[0007] S1. Deploy an anomaly detector at the first level to distinguish between benign and malicious traffic. Given by... A sample of benign traffic and A sample set consisting of malicious traffic samples The anomaly detector outputs an anomaly score for each sample; an anomaly score exceeding a certain threshold is considered an anomaly. The samples were identified as malicious traffic and forwarded to subsequent model analysis, which was lower than [a certain percentage]. The samples were identified as benign traffic;

[0008] S2. Deploy a malicious traffic discriminator at the second level to distinguish between visible and invisible malicious traffic detected by the anomaly detector. Given a sample set in the anomaly detector... Detected in One visible sample of malicious traffic and A set of malicious traffic samples was formed by identifying no malicious traffic samples. The malicious traffic discriminator outputs the unseen score for each sample; if the unseen score exceeds a certain threshold... Samples with no malicious traffic were considered as having no malicious traffic, while those without were considered as having visible malicious traffic. Both visible and non-visible malicious traffic were forwarded to subsequent model analysis.

[0009] S3. Deploy a visible malicious traffic classifier at the third level to identify the categories of visible malicious traffic detected by the malicious traffic discriminator, ensuring the malicious traffic discriminator accurately distinguishes between them. In One visible sample of malicious traffic and No malicious traffic samples were found. The malicious traffic classifier is visible. predict The category of each sample, where Represents the visible category label space;

[0010] S4. Deploy a custom model at the third level to analyze undetected malicious traffic identified by the malicious traffic discriminator. The functionality of the custom model is designed and implemented according to actual needs.

[0011] The beneficial effects of this invention are:

[0012] Anomaly detectors deployed at the first level can quickly distinguish between benign and malicious traffic, avoiding security risks caused by undetected malicious traffic. The majority of network traffic, which is benign, is analyzed solely by the anomaly detectors, while the malicious traffic discriminator, visible malicious traffic classifier, and custom model only need to analyze a small amount of malicious traffic, thus improving the inference speed of the hybrid network intrusion detection system.

[0013] The malicious traffic discriminator deployed at the second tier only performs the function of distinguishing between visible and invisible malicious traffic. Compared to the approach that relies on a classifier to simultaneously perform (1) distinguishing between visible and invisible malicious traffic and (2) identifying the category of visible malicious traffic, it can achieve higher prediction accuracy. At the same time, potentially threatening invisible malicious traffic can be detected at the second tier. Administrators can quickly assess invisible malicious traffic and take defensive measures to reduce the risks posed by invisible malicious traffic.

[0014] The visible malicious traffic classifier deployed at the third tier can accurately identify the categories of visible malicious traffic. Network administrators can judge the categories of visible malicious traffic based on model predictions rather than their own analysis, which improves the speed at which network administrators can take defensive measures and reduces the computational load on the first and second tiers.

[0015] Custom models deployed at the third tier support the introduction of additional models for further analysis of malicious traffic, improving the functional integrity of the hybrid network intrusion detection system and reducing the computational load on the first and second tiers. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a layered network intrusion detection method.

[0017] Figure 2 This is a deployment diagram of a layered network intrusion detection method model; Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely specific embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] Figure 1 A schematic flowchart of a layered network intrusion detection method provided by an embodiment of the present invention is shown in the figure. The method includes:

[0020] S1. Deploy an anomaly detector at the first level to distinguish between benign and malicious traffic. Given by... A sample of benign traffic and A sample set consisting of malicious traffic samples The anomaly detector outputs an anomaly score for each sample; an anomaly score exceeding a certain threshold is considered an anomaly. The samples were identified as malicious traffic and forwarded to subsequent model analysis, which was lower than [a certain percentage]. The samples were identified as benign traffic;

[0021] In this embodiment of the invention, the Extreme Gradient Boosting (XGBoost) model is trained as an anomaly detector to distinguish between benign and malicious traffic;

[0022] S2. Deploy a malicious traffic discriminator at the second level to distinguish between visible and invisible malicious traffic detected by the anomaly detector, given a sample set in the anomaly detector. Detected in One visible sample of malicious traffic and A set of malicious traffic samples was formed by identifying no malicious traffic samples. The malicious traffic discriminator outputs the unseen score for each sample. A score exceeding [a certain threshold] indicates a malicious traffic violation. Samples with no malicious traffic are considered as not showing malicious traffic, while those without are considered as showing malicious traffic. Both visible and non-visible malicious traffic are forwarded to subsequent model analysis.

[0023] In this embodiment of the invention, an Isolation Forest model is trained as a malicious traffic discriminator to distinguish between visible and invisible malicious traffic.

[0024] S3. Deploy a visible malicious traffic classifier at the third level to identify the categories of visible malicious traffic detected by the malicious traffic discriminator. Given a malicious traffic discriminator, accurately distinguish... In One visible sample of malicious traffic and No malicious traffic samples were found. The malicious traffic classifier is visible. predict The category of each sample, where Represents the visible category label space.

[0025] In this embodiment of the invention, the Extreme Gradient Boosting (XGBoost) model is trained as a visible malicious traffic classifier to identify visible malicious traffic categories.

[0026] S4. Deploy a custom model at the third level to analyze undetected malicious traffic identified by the malicious traffic discriminator. The functionality of the custom model is designed and implemented according to actual needs.

[0027] In this embodiment of the invention, a zero-shot learning classifier is introduced as a custom model to identify unseen malicious traffic categories. The trained unseen malicious traffic classifier... Able to predict The category of each sample, where This indicates a category tag space where no malicious traffic was detected.

[0028] Figure 2This diagram illustrates the deployment of a layered network intrusion detection method. As shown, distributed nodes are located at the edge layer (first layer). Each node deploys an anomaly detector to distinguish between benign and malicious traffic. Malicious traffic detection has high real-time requirements; deploying detectors at the edge layer meets the real-time needs of the network intrusion detection system. Detected malicious traffic is then transmitted to the fog layer (second layer), which has relatively strong computing and storage capabilities. The malicious traffic discriminator deployed in the fog layer distinguishes between visible and invisible malicious traffic and further transmits the malicious traffic to the cloud layer (third layer). The classifier and custom model deployed in the cloud layer identify the categories of visible malicious traffic and further analyze it. The cloud layer's strong computing and storage capabilities support in-depth analysis of malicious traffic by the classifier and custom model, reducing the computational load on the anomaly detectors in the edge layer.

[0029] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A hierarchical network intrusion detection method, characterized in that, The method includes the following steps: S1. Deploy an anomaly detector at the first level to distinguish between benign and malicious traffic, given by... A sample of benign traffic and A sample set consisting of malicious traffic samples The anomaly detector outputs an anomaly score for each sample; an anomaly score exceeding a certain threshold is considered an anomaly. The samples were identified as malicious traffic and forwarded to subsequent model analysis, which was lower than [a certain percentage]. The samples were identified as benign traffic; S2. Deploy a malicious traffic discriminator at the second level to distinguish between visible and invisible malicious traffic detected by the anomaly detector, given a sample set in the anomaly detector. Detected in One visible sample of malicious traffic and A set of malicious traffic samples was formed by identifying no malicious traffic samples. The malicious traffic discriminator outputs the unseen score for each sample; if the unseen score exceeds a certain threshold... Samples with no malicious traffic are considered as having no malicious traffic, while those without are considered as having visible malicious traffic. Both visible and no malicious traffic are forwarded to subsequent model analysis. S3. Deploy a visible malicious traffic classifier at the third level to identify the categories of visible malicious traffic detected by the malicious traffic discriminator, ensuring the malicious traffic discriminator accurately distinguishes between them. In One visible sample of malicious traffic and No malicious traffic samples were found. The malicious traffic classifier is visible. predict The category of each sample, where Represents the visible category label space; S4. Deploy a custom model at the third level to analyze undetected malicious traffic detected by the malicious traffic discriminator. The functions of the custom model are designed and implemented according to actual needs.

Images