Distance-aware in-network anomaly detection scheme

Abstract

The application discloses a kind of in-network anomaly detection schemes based on distance perception, belong to computer network security field.The scheme includes: first, clustering analysis is executed DBSCAN clustering to network traffic data, the local structure characteristics of network traffic is represented, and different traffic mode is distinguished.Then, the distance between samples is converted into matching operation in hyperrectangle neighborhood by rule generation, and the scale of rule is decoupled from the number of samples by using granular computing, to realize the regular expression of detection logic.Subsequently, rule merging takes the hyperrectangle region identified as normal as initial object, and adopts iterative method to preferentially select the hyperrectangle pair with highest score for merging.Finally, the data plane component is composed of feature extractor and model table, to execute in-network anomaly detection.The in-network anomaly detection scheme provided by the application can realize fast and accurate detection of unknown attack traffic while reducing the demand for data plane operation and storage resource consumption.

Description

Technical Field

[0001] This invention belongs to the field of computer network security, and specifically relates to a distance-aware network anomaly detection scheme. Background Technology

[0002] With the rapid development of information technology and the continuous expansion of the internet, network application scenarios are becoming increasingly complex, and network traffic is exhibiting characteristics such as high dimensionality, diversity, and dynamic evolution. Traditional intrusion detection methods based on feature database matching typically rely on known attack patterns to construct detection rules. When faced with variant attacks and unknown attacks, the lack of effective prior knowledge makes timely identification and response difficult, thus limiting network security protection capabilities. Anomaly detection technology learns the normal behavior patterns of network systems, establishes corresponding behavioral baselines, and identifies anomalies based on the degree of deviation between real-time network traffic and normal patterns. This enables the discovery of potential threats even in the absence of known attack samples, providing an effective approach for detecting unknown attacks.

[0003] Programmable data planes offer a new approach to network architecture evolution. Developers can leverage programming languages ​​like P4 (Programming Protocol-Independent Packet Processors) to flexibly define packet parsing, matching, state maintenance, and statistical counting functions within programmable switches, enabling dynamic expansion of the network data plane processing logic. Intelligent data planes combine the line-speed data processing capabilities of programmable switches with the intelligent analysis capabilities of machine learning models, giving network devices a certain level of data awareness, analysis, and decision-making ability. This provides new technical avenues for security applications such as intrusion detection and abnormal traffic identification. Compared to traditional network security detection methods that rely on centralized analysis by controllers or in the cloud, these methods reduce the centralized transmission of large amounts of raw traffic, lower network bandwidth and computing resource overhead, and shorten response times for security incidents.

[0004] Currently, mainstream research on intelligent data planes largely focuses on offloading various tree-based models to the data plane. Some studies have also expanded the model types, enabling the deployment and offloading of various machine learning models such as k-means clustering, support vector machines, and Naive Bayes to the data plane. However, current mainstream work only achieves in-network offloading of supervised learning models, lacks anomaly detection capabilities, and most methods do not pay sufficient attention to optimizing data plane resource overhead, lacking systematic resource-saving mechanisms.

[0005] Due to limitations in computational operations and storage resources of the programmable data plane, anomaly detection within the network faces three core challenges: limited computational operations, storage resource constraints, and insufficient detection accuracy.

[0006] The computational limitations of the programmable data plane present a mismatch between its computational capabilities and the complex computational demands of existing anomaly detection algorithms. Existing anomaly detection models typically involve numerous complex mathematical operations; for example, calculating local outlier factors involves distance calculations, division, and accumulation, while single-class support vector machines require kernel function calculations and decision function solutions. However, due to the limitations of the programmable switch chip's data plane architecture and the types of computational operations natively supported by the P4 language, it is generally only suitable for basic processing such as simple integer arithmetic, logical operations, and bit manipulations. This computational limitation makes it difficult to directly deploy anomaly detection models involving complex mathematical operations in the data plane, thus restricting the application of intelligent data planes in network anomaly detection scenarios.

[0007] Storage resource constraints exist, creating a conflict between the storage resources of the programmable data plane and the scale of detection rules generated based on machine learning models. Taking the Tofino 1 switching chip as an example, its single pipeline provides limited SRAM (Static Random-Access Memory) and TCAM (Ternary Content-Addressable Memory) resources, approximately 120MB and 6.2MB respectively. Meanwhile, anomaly detection models typically require storing large amounts of feature vectors, model parameters, and intermediate computation results during inference. As the network feature dimension increases or the detection scenario becomes more complex, a large number of matching rules are generated, leading to rapid consumption of storage resources.

[0008] Insufficient detection accuracy and difficulties in improving the performance of in-network anomaly detection in unsupervised environments are significant challenges. Due to the lack of labeled data, unsupervised anomaly detection models typically need to learn normal traffic characteristics to identify anomalies, resulting in lower detection performance compared to supervised learning-based methods and a higher false positive and false negative rate. Furthermore, in resource-constrained programmable data planes, limitations in computing power and storage space make it difficult to directly deploy complex anomaly detection models to improve accuracy. Therefore, achieving high-accuracy in-network anomaly detection under conditions that support only limited computational operations and consume minimal storage resources is a pressing issue that current intelligent data plane security detection technologies need to address.

[0009] This invention proposes a distance-aware network anomaly detection scheme. The scheme consists of four parts: cluster analysis, rule generation, rule merging, and data plane deployment. First, cluster analysis performs DBSCAN clustering on network traffic data to obtain several traffic clusters. These clusters characterize the local structural characteristics of network traffic and distinguish different traffic patterns. Then, rule generation transforms distance calculation between samples into matching operations within hyperrectangular neighborhoods, and uses granular computation to decouple rule size from sample size, achieving a rule-based expression of the detection logic. Subsequently, rule merging uses hyperrectangular regions marked with +1 as initial objects, calculates a merging score for each pair of mergingable hyperrectangles, and iteratively selects the pair with the highest score for merging. Finally, the data plane component consists of a feature extractor and a model table, performing anomaly detection within the data plane. Summary of the Invention

[0010] To address the shortcomings of current intelligent data plane solutions, a distance-aware intra-network anomaly detection scheme is proposed. This scheme ensures high detection accuracy while minimizing data plane computational requirements and storage resource consumption. Therefore, it is universally applicable to real-time detection of unknown attacks.

[0011] The technical solution adopted by this invention to achieve the above objectives is as follows: the distance-aware intra-network anomaly detection scheme mainly includes four steps: cluster analysis, rule generation, rule merging, and data plane deployment.

[0012] 1. Cluster analysis: To characterize the intrinsic structure and local characteristics of network traffic, DBSCAN clustering is used to aggregate network traffic data into several clusters and filter out outliers. This has two advantages: first, it identifies abnormal traffic from a local perspective, reducing the discrimination bias that may be caused by global anomaly detection; second, it can identify traffic data with different components, facilitating separate analysis of different traffic patterns and avoiding mutual interference.

[0013] 2. Rule generation: This process transforms distance calculation between samples into matching operations within a hyperrectangular neighborhood. Granular computing is used to decouple rule size from the number of samples, enabling a rule-based expression of the detection logic. This includes the following four steps: (1) Particle set aggregation, using k-means clustering, to group the clusters The samples within are aggregated as A set of particles is defined, and the centroid of each set is taken as the position of the corresponding particle. The centroid can be represented as... , in, It is a sample in the particle set. It is the number of samples in the particle set. Hyperparameters It is used to characterize the granular resolution, control the number and scale of the granular set, and adjust the granularity of the superrectangular neighborhood in the feature space, thereby controlling the scale of the generated rules. (2) Construction of hyperrectangular neighborhood: With each particle as the center, construct axis-aligned hyperrectangular neighborhoods to obtain the resulting neighborhoods. A hyperrectangular region defined by an axis-aligned hyperplane, wherein Representing the feature space The number of dimensions. For a sample If it falls into the sample Within the hyperrectangular neighborhood, the sample is considered to be Neighboring samples , represented as Otherwise, the sample is considered Far from the sample , represented as If the sample Neighboring samples Then it is called a sample It is a sample The nearest neighbor of the sample; if the sample Far from the sample Then it is called a sample It is a sample The point farthest from the origin can be represented as in, Indicates sample The hyperrectangular neighborhood is defined as follows: A grain, as the representative element of its corresponding grain set, has a consistent neighborhood relationship with each element in the grain set within the feature space. The hyperrectangular neighborhood not only represents the neighborhood relationship of the grain's location but also implicitly contains the neighborhood relationship of the entire grain set. For any sample falling within the hyperrectangular neighborhood, its neighbors are both the grain and all elements contained in the grain set. (3) Feature space partitioning: Considering the overlap of axis-aligned hyperplanes obtained in the construction of the hyperrectangular neighborhood, duplicate hyperplanes are removed. Then, using these deduplicated axis-aligned hyperplanes, the feature space is partitioned into... A hyperrectangular region, denoted as... . Samples falling within the same hyperrectangular region have the same outlier score. Therefore, the hyperrectangular region can be regarded as a basic unit with consistent label information and directly identified, thus saving the operation of calculating the outlier score for each sample in the region one by one, and avoiding the complex calculation and resource consumption caused by point-by-point scoring; (4) Rule generation, from Find all superrectangular regions with a label of +1 and convert them into range matching rules. For superrectangular regions If a sample falling within its region is marked as +1, then its corresponding label is set to +1. The indicator function for the sample label can be expressed as: in, Cluster medium sample The proportion of the number of points far away is the abnormal score defined in the distance perception anomaly definition. The larger the value, the better the sample. The higher the degree of abnormality.

[0014] 3. Rule merging: Each hyperrectangle region marked with +1 is used as the initial object. A merging score is calculated for each pair of mergingable hyperrectangles, and an iterative approach is used to prioritize merging the hyperrectangle pairs with the highest scores until no more merging pairs are available or the maximum number of iterations is reached. This merging strategy can effectively compress the rule size and reduce the resource overhead of the data plane without affecting the anomaly detection results. In feature space In the context of feature space partitioning, the set of hyperrectangular regions can be represented as follows: For two hyperrectangles and The necessary and sufficient condition for them to be merged is that there exists a certain feature dimension. , making and They are adjacent in this dimension, that is, they satisfy... and Equal or and Equal, and in other dimensions superior, and The values ​​of are equal, that is, they satisfy . and Equal and and Equal; in other words, except for the first Weiwai, and The intervals are the same, and in the first... Wei Shang and Adjacent; under this condition, and The merged superrectangle remains axis-aligned and can be represented as follows: For mergeable hyperrectangles and Their merge score is defined as the area of ​​the shared hyperplane; in other words, the merge score is expressed as the area of ​​the hyperplane after the first fold. Outside of dimension 1, the cumulative product of all other dimensional intervals can be expressed as: , If super rectangle and If they cannot be merged, their merge score is 0.

[0015] 4. Data plane deployment: The data plane component consists of two parts: a feature extractor and a model table. The feature extractor is deployed on the packet processing path of the programmable switch to extract network traffic features online. The model table is implemented in the form of a match-action table to store whitelist rules for range matching. If the network traffic features match the key values ​​in the model table, it is determined to be benign traffic; otherwise, it is determined to be malicious traffic. Beneficial effects

[0016] Distance-aware network anomaly detection schemes ensure high detection accuracy while minimizing data plane computation requirements and storage resource consumption. They offer good effectiveness, flexibility, scalability, and real-time performance, enabling accurate and rapid detection of malicious traffic from unknown attacks within the network. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of superrectangular neighborhood partitioning based on granular computation in two-dimensional space.

[0018] Figure 2 A framework diagram for distance-aware rule generation.

[0019] Figure 3 This is an example of merging super rectangles.

[0020] Figure 4 This is a framework diagram of a distance-aware intra-network anomaly detection scheme. Detailed Implementation

[0021] The invention will be further described below with reference to the accompanying drawings.

[0022] like Figure 4 As shown, the distance-aware intra-network anomaly detection scheme includes four steps: cluster analysis, rule generation, rule merging, and data plane deployment.

[0023] Cluster analysis performs DBSCAN clustering on network traffic data to obtain several traffic clusters, and filters out outlier noise points. Traffic clusters are used to characterize the local structural characteristics of network traffic, so as to distinguish different traffic patterns, reduce the discrimination bias of global anomaly detection, and avoid mutual interference between different traffic patterns.

[0024] Rule generation transforms distance calculation between samples into matching operations within a hyperrectangular neighborhood, and uses granular computing to decouple rule size from sample quantity, thereby realizing the rule-based expression of detection logic.

[0025] The rule merging method takes each super rectangle region marked with +1 as the initial object, calculates a merging score for each pair of merging super rectangles, and uses an iterative approach to prioritize merging the pair of super rectangles with the highest score until there are no more pairs of merging super rectangles or the maximum number of iterations is reached. This merging strategy can effectively compress the rule size and reduce the resource overhead of the data plane without affecting the anomaly detection judgment results.

[0026] The data plane deployment includes a feature extractor and a model table, which are deployed on the programmable switch's data plane. The feature extractor is deployed on the packet processing path of the programmable switch to extract network traffic features online. The model table is implemented as a match-action table to store range matching rules. If the network traffic features match the key values ​​in the model table, it is determined to be benign traffic; otherwise, it is determined to be malicious traffic.

[0027] Figure 1 This is a schematic diagram of hyperrectangular neighborhood partitioning based on granular computation in two-dimensional space. First, the samples are aggregated into 3 granule sets (i.e. , and Then, for each particle, its axis-aligned hyperrectangular neighborhood is constructed, resulting in a set of corresponding boundary hyperplanes. These hyperplanes divide the feature space into several axis-aligned hyperrectangular regions, and seven representative hyperrectangular regions are marked in the figure. Region 1 contains only one hyperrectangle, and for all samples falling into region 1... , and They are all its neighboring points. For ease of description, here we will... , and These are called the neighboring points of region 1, and similar designations will be used thereafter. Regions 2 and 3 each contain one hyperrectangle, where... and It is a neighboring point of region 2. and It is a neighboring point of region 3. Region 4 contains 3 hyperrectangles, and and These are the neighboring points of region 4. Region 5 contains 6 hyperrectangles. These are the neighboring points of region 4. Regions 6 and 7 both contain 4 hyperrectangles, among which... It is a neighboring point of region 6. These are the neighboring points of region 7. For the unlabeled regions in the figure, no particles are their neighbors.

[0028] Figure 2 The framework diagram for distance-aware rule generation includes four steps: particle set aggregation, super-rectangular neighborhood construction, feature space partitioning, and rule generation.

[0029] Particle sets are clustered using k-means clustering. The samples within are aggregated as A set of particles, with the cluster center location as the particle location. Hyperparameters This can be viewed as the resolution of granulation, determining the fineness of the granulation. By changing... The value of this hyperparameter can be adjusted to change the number of granules and the scale of the granule set, thereby controlling the fineness of the feature space partitioning by the hyperrectangular neighborhood, and thus controlling the scale of rule generation. In the resource-constrained deployment environment of a programmable switch, adjusting the hyperparameter... To achieve a balance between resource constraints and detection performance.

[0030] Hyperrectangular neighborhood construction: For each grain, a hyperrectangular neighborhood is constructed to obtain... Each axis-aligned hyperplane. A grain, as the representative element of the entire grain set, shares a consistent neighborhood relationship with all elements within the grain set in the feature space. The hyperrectangular neighborhood not only represents the neighborhood relationship of a grain's location but also implicitly contains the neighborhood relationship of the entire grain set. For a sample falling into the hyperrectangular neighborhood, it indicates that the grain's location is its neighbor, and also that all elements within the entire grain set are its neighbors.

[0031] For feature space partitioning, considering the overlap of axis-aligned hyperplanes obtained in the previous step, duplicate hyperplanes are removed. Then, using these deduplicated axis-aligned hyperplanes, the feature space is divided into several hyperrectangular regions, denoted as... Based on the label consistency attribute, samples falling within the same hyperrectangular region have the same outlier score, meaning samples within the same hyperrectangular region have consistent labels. Therefore, the hyperrectangular region can be regarded as a basic unit with consistent label information and directly identified, thus eliminating the need to calculate the outlier score for each sample within the region individually, avoiding the complex calculations and resource consumption caused by point-by-point scoring.

[0032] Rule generation, from Find all superrectangular regions with a label of +1 and convert them into range matching rules.

[0033] Figure 3 This example illustrates the merging of super rectangles. The boxes indicate super rectangles marked as normal. Initially, there are 7 normal super rectangles, corresponding to 7 range matching rules. After 5 merges, only 2 super rectangles remain, corresponding to 2 range matching rules. That is, the number of rules is reduced by approximately 71% after the merging operation.

Claims

1. A distance-aware intra-network anomaly detection scheme, characterized in that, The anomaly detection scheme deployed within the switch is called the intranet anomaly detection scheme, which includes the following four steps: Step 1: Cluster analysis. Perform DBSCAN clustering on the network traffic data to obtain... A traffic cluster, denoted as It filters out outlier noise points, and traffic clusters are used to characterize the local structural characteristics of network traffic in order to distinguish different traffic patterns, reduce the discrimination bias of global anomaly detection, and avoid mutual interference between different traffic patterns. Step 2, rule generation, transforms the calculation of distance between samples into a matching operation within a hyperrectangular neighborhood, and uses granular computing to decouple the rule size from the number of samples, realizing the rule-based expression of the detection logic. The rule generation includes the following four steps: Step 2.1: Particle set aggregation, using k-means clustering to group clusters. The samples within are aggregated as A set of particles is defined, and the centroid of each set is taken as the position of the corresponding particle. The centroid can be represented as... ， in, It is a sample in the particle set. It is the number of samples in the granular set; the hyperparameters Setting it to 6 controls the number of granular sets, adjusts the granularity of the hyperrectangular neighborhood in partitioning the feature space, and thus controls the scale of the generated rules. Step 2.2: Constructing a hyperrectangular neighborhood. Using each particle as the center, construct an axis-aligned hyperrectangular neighborhood, resulting in a neighborhood composed of… A hyperrectangular region defined by an axis-aligned hyperplane. Representing the feature space The number of dimensions; the particle, as the representative element of its corresponding particle set, has a consistent neighborhood relationship with each element contained in the particle set in the feature space. The hyperrectangular neighborhood not only represents the neighborhood relationship of the particle's location, but also implies the neighborhood relationship of the entire particle set. For any sample falling into the hyperrectangular neighborhood, it takes the particle as its neighbor and all the elements contained in the particle set as its neighbors. Step 2.3: Feature Space Partitioning. Considering the overlap of the axis-aligned hyperplanes obtained in Step 2.2, duplicate hyperplanes are removed, and then the feature space is partitioned using the deduplicated hyperplanes. A hyperrectangular region, denoted as... For samples falling within the same hyperrectangular region, they have the same outlier score, which can be expressed as: ， in, Cluster medium sample The proportion of the number of points far away from the target. The larger the value, the better the sample. The higher the degree of abnormality, the more it can be treated as a basic unit with consistent label information and directly identified, saving the operation of calculating the abnormal score for each sample in the region one by one, and avoiding the complex calculation and resource consumption caused by point-by-point scoring. Step 2.4: Rule generation for super-rectangular regions. If a sample falling within its region is marked as +1, then its corresponding label is set to +1. Find all superrectangular regions with a label of +1 and convert them into range matching rules, where the indicator function of the sample label can be expressed as: threshold Set to 0.5 when the anomaly score of a sample is greater than the threshold. If the condition is abnormal, mark it as -1; otherwise, mark it as +1, indicating that the sample is normal. Step 3: Rule merging. Each super rectangle region marked with +1 is used as the initial object. A merging score is calculated for each pair of merging super rectangles. An iterative approach is used to prioritize merging the pair of super rectangles with the highest scores until there are no more pairs of merging super rectangles or the maximum number of iterations is reached. This merging strategy can effectively compress the rule size and reduce the resource overhead of the data plane without affecting the anomaly detection judgment results. Step 4: Data plane deployment. The data plane component consists of two parts: a feature extractor and a model table. The feature extractor is deployed on the packet processing path of the programmable switch to extract network traffic features online. The model table is implemented in the form of a match-action table to store range matching rules. If the network traffic features match the key values ​​in the model table, it is determined to be benign traffic; otherwise, it is determined to be malicious traffic.

2. The detection scheme according to claim 1, characterized in that, In step 2, for a sample If it falls into the sample Within the hyperrectangular neighborhood, the sample is considered to be Neighboring samples , represented as Otherwise, the sample is considered Far from the sample , represented as If the sample Neighboring samples Then it is called a sample It is a sample The nearest neighbor of the sample; if the sample Far from the sample Then it is called a sample It is a sample The point farthest from the origin can be represented as in, Indicates sample The superrectangular neighborhood.

3. The detection scheme according to claim 1, characterized in that, In step 3, in the feature space In the context of feature space partitioning, the set of hyperrectangular regions can be represented as follows: For two hyperrectangles and The necessary and sufficient condition for them to be merged is that there exists a certain feature dimension. , making and They are adjacent in this dimension, that is, they satisfy... and Equal or and Equal, and in other dimensions superior, and The values ​​of are equal, that is, they satisfy . and Equal and and Equal; in other words, except for the first Weiwai, and The intervals are the same, and in the first... Wei Shang and Adjacent; under this condition, and The merged superrectangle remains axis-aligned and can be represented as follows: For mergeable hyperrectangles and Their merge score is defined as the area of ​​the shared hyperplane; in other words, the merge score is expressed as the area of ​​the hyperplane after the first fold. Outside of dimension 1, the cumulative product of all other dimensional intervals can be expressed as: ， If super rectangle and If they cannot be merged, their merge score is 0.

Images