A Dynamic Data Packet Classification Method and System Based on Trie and Multi-Level Decision Tree

Abstract

This invention belongs to the field of computer networks and discloses a dynamic packet classification method and system based on trie and multi-level decision tree. The method includes: S1, constructing a trie for the IP prefixes of all rules in the preprocessing stage of packet classification; S2, obtaining prefix drastic change points based on the trie; S3, dividing the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; S4, constructing a main decision tree for the precise rule subsets; and inputting the complex rule subsets into a Box for processing; S5, constructing a shadow tree or shadow Box copy for rule modification operations and performing pointer replacement through CAS atomic instructions. This invention adopts "structure layering + behavior-driven + field decoupling" as its core design concept, comprehensively integrating rule pre-partitioning, path structure optimization, update behavior awareness, and efficient matching mechanisms, and breaking through the problems of rule duplication redundancy, low path efficiency, and delayed dynamic update response in packet classification.

Description

Technical Field

[0001] This invention relates to the field of computer networks, and in particular to a dynamic data packet classification method and system based on trie and multi-level decision tree. Background Technology

[0002] Packet classification is a crucial fundamental function in modern computer networks, widely used in network devices such as routers, firewalls, intrusion detection systems (IDS), Quality of Service (QoS) assurance, and load balancers. With the explosive growth of internet traffic, the increasing complexity of network services, and the continuous expansion of rule sets, network devices need to perform real-time, accurate, and efficient classification of massive amounts of data packets. To meet this performance requirement, researchers have proposed various packet classification algorithms. Among them, decision tree-based classification algorithms have received widespread attention and research due to their ability to handle multi-dimensional fields, support complex rules, and theoretically achieve efficient lookups. Decision trees construct a multi-way or binary search tree by using multiple fields of the data packet (such as source / destination IP address, source / destination port number, protocol type, etc.) as decision attributes. This progressively divides the search space into smaller subspaces, ultimately guiding the data packet to match the corresponding rule.

[0003] However, despite the theoretical advantage of high search efficiency, traditional decision tree algorithms still face significant challenges when dealing with large-scale, highly complex, and dynamically changing rule sets. These challenges mainly manifest in the following ways: 1) Rule replication: Especially when the rule set contains a large number of "large rules" with short IP prefixes or many wildcards, these large rules often need to be repeatedly copied across multiple branches of the decision tree to correctly handle overlaps and priorities, leading to a rapid increase in tree size and consuming a large amount of memory. 2) Degraded search performance: Rule replication and increased tree depth lengthen the average search path for data packets, significantly reducing classification search efficiency and making it difficult to meet line-speed processing requirements. 3) Low rule update efficiency: Structural updates to the decision tree (such as adding, deleting, or modifying rules) may require time-consuming reconstruction operations on the local or even global parts of the tree, resulting in long update times and difficulty adapting to frequent adjustments in network policies, thus affecting the normal operation of services.

[0004] To overcome the rule duplication and performance bottlenecks encountered by traditional decision tree algorithms when handling large-scale rule sets, researchers have proposed various optimization schemes. Among them, the CutSplit decision tree bag classification algorithm based on hybrid cutting technology is one attempt to solve these problems. The CutSplit algorithm aims to select the optimal cutting dimension (such as bit cutting or field cutting) during the construction of the decision tree through a hybrid cutting strategy, in order to reduce rule duplication and optimize the tree structure to a certain extent. It attempts to explore the root causes of the poor scalability of decision tree algorithms through quantitative evaluation and analysis of existing decision tree algorithms, and on this basis, designs a corresponding algorithm framework, hoping to improve classification efficiency and scalability.

[0005] Although existing decision tree optimization algorithms such as CutSplit have attempted to alleviate the rule duplication problem of traditional decision trees and improve search efficiency to some extent, they still have significant limitations when dealing with extremely large-scale, highly dynamic, and complex and diverse network rule sets. Specifically, their drawbacks include:

[0006] 1) The rule duplication problem remains unresolved: CutSplit's hybrid splitting strategy still struggles to completely avoid multi-path duplication of rules within the tree when dealing with "large rules" in extreme scenarios (such as rules with very short IP prefixes or containing a large number of wildcards). This local optimization strategy is ill-suited to global complexity, resulting in severe rule duplication at certain depths or branches. This makes it difficult to effectively control the size of the decision tree, thus impacting search performance and memory consumption. The root cause is that traditional decision tree splitting strategies (whether bit-based or field-based) must maintain the integrity and correctness of the tree through duplication when dealing with large rules with high overlap and inclusiveness, in order to ensure that all packets correctly match the highest priority rule. This inherent duplication mechanism, even with hybrid splitting, only alleviates rather than eradicates.

[0007] 2) Insufficient Dynamic Update Efficiency: Existing optimization algorithms, including CutSplit, often require time-consuming local or global adjustments to the tree structure when the rule set is frequently updated (e.g., adding, deleting, or modifying rules). This adjustment process may involve splitting, merging, or even rebuilding subtrees, and its computational complexity increases sharply with the number of rules. When the update frequency is high, this adjustment can lead to system performance fluctuations and even brief service interruptions. The root cause is that the optimization strategies of these algorithms are usually completed in the static construction phase of the tree, while insufficient consideration is given to the dynamic maintenance capabilities of the tree. Once rule changes affect the core structure, a significant computational cost is required to restore the tree's balance and search efficiency, lacking a truly incremental and uninterrupted update mechanism.

[0008] 3) Limited Scalability: The aforementioned shortcomings in rule replication and update efficiency mean that current technologies struggle to linearly scale search performance and update response time when facing continuous network expansion and rule sets reaching millions or even tens of millions of rules. As the number of rules increases, even with optimization, the depth and width of the tree may still exceed the effective range of hardware caching, triggering numerous memory accesses and further reducing actual performance. The fundamental reason is that these algorithms fail to achieve more thorough hierarchical decoupling and rule feature preprocessing at the data structure level, making it difficult to effectively balance the combined needs of search efficiency, memory usage, and dynamic updates in single-level or finite-level decision tree structures. Summary of the Invention

[0009] The purpose of this invention is to disclose a dynamic data packet classification method and system based on trie and multi-level decision tree, and to solve the technical problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] In a first aspect, the present invention provides a dynamic data packet classification method based on a trie and a multi-level decision tree, including:

[0012] S1, In the preprocessing stage of packet classification, a trie is built for the IP prefixes of all rules;

[0013] S2, obtain prefix drastic change points based on the trie;

[0014] S3, based on the prefix drastic change point, divides the rule subsets to obtain precise rule subsets and complex rule subsets;

[0015] S4: Construct the main decision tree for the precise subset of rules; input the complex subset of rules into Box for processing;

[0016] S5 constructs a shadow tree or shadow Box copy of the rule modification operation and performs pointer replacement using CAS atomic instructions.

[0017] Preferably, constructing a trie includes:

[0018] Each node represents a bit, and the prefix path is constructed sequentially from the root downwards.

[0019] Preferably, S2 includes:

[0020] S30, for each depth d, count the number of unique IP prefixes N(d) from the root node to depth d;

[0021] S31, Calculate the probability distribution of all rules at depth d in the IP address bits. where i is a bit;

[0022] S32, calculate the relative growth acceleration rate A(d);

[0023] S33, based on Calculate the KL divergence;

[0024] S34, calculate the Z-score of A(d), and obtain ;

[0025] S35, based on A(d), KL divergence and Calculate the mutation significance score S(d);

[0026] S36. Among all depths with mutation significance scores greater than a preset mutation significance score threshold, the depth with the highest mutation significance score is taken as the prefix catastrophic change point.

[0027] Preferably, S3 includes:

[0028] If the IP prefix length is greater than the mutation significance score, the port range length is less than the preset length and does not contain wildcards, and the protocol type is not ANY, then the rule is divided into the exact rule subset; otherwise, the rule is divided into the complex rule subset.

[0029] Preferably, constructing a master decision tree for a subset of precise rules includes:

[0030] During the construction process, a bit balance strategy is used to select split bits;

[0031] During the construction process, if the number of rules in a leaf node exceeds a preset threshold, the rules will be input into the Box for processing.

[0032] Preferably, the Box's processing of rules includes:

[0033] For the port field in the rule, if the port field is an exact field, then a hierarchical hash structure is used for management;

[0034] If the port field is a range value, then Y-FastTrie is used for management;

[0035] If the port field is a wildcard, then the DAWG structure is used to implement fuzzy matching.

[0036] Preferably, if the rules are dynamically updated, the leaf nodes are re-splitned based on the structural adaptive mechanism of runtime behavior feedback.

[0037] Preferably, the leaf nodes are re-splitting based on the structural adaptive mechanism of runtime behavior feedback, including: calculating the update surge rate and local path redundancy of the leaf nodes;

[0038] Determine whether the update surge rate and local path redundancy meet the re-splitting conditions. If so, re-split the leaf nodes.

[0039] Preferably, the re-division conditions include:

[0040] The update surge rate is greater than the preset update surge rate threshold and the path redundancy is greater than the preset path redundancy threshold.

[0041] Secondly, the present invention provides a dynamic data packet classification system based on a trie and a multi-level decision tree, including a construction module, an acquisition module, a partitioning module, a processing module, and a replacement module;

[0042] The building module is used to construct a trie for the IP prefixes of all rules during the preprocessing stage of packet classification;

[0043] The acquisition module is used to retrieve prefix drastic change points based on the trie;

[0044] The partitioning module is used to partition rule subsets based on prefix drastic change points, obtaining precise rule subsets and complex rule subsets;

[0045] The processing module is used to construct the main decision tree from a precise subset of rules; and to input a complex subset of rules into the Box for processing.

[0046] The replacement module is used to construct a shadow tree or shadow box copy of the rule modification operation and perform pointer replacement via CAS atomic instructions.

[0047] Beneficial effects:

[0048] This invention adopts the core design concept of "structure layering + behavior-driven + field decoupling", and fully integrates rule pre-division, path structure optimization, update behavior awareness and efficient matching mechanism. It breaks through the problems of rule duplication redundancy, low path efficiency and dynamic update response lag in data packet classification. It has good performance scalability and engineering practicality, and is suitable for high-performance scenarios such as network security and traffic scheduling for millions of rule sets. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the dynamic data packet classification method based on a trie and a multi-level decision tree according to the present invention.

[0051] Figure 2This is a schematic diagram illustrating the relationship between the box and the main decision tree in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Existing packet classification algorithms, especially those based on decision trees (such as CutSplit), generally suffer from severe structural redundancy, low path efficiency, and high cost in rule update processes when handling large-scale, highly dynamic, and complex rule sets containing wildcard rules. This invention aims to fundamentally solve these bottlenecks. It achieves intelligent partitioning of rule sets through trie preprocessing combined with a dynamic prefix drastic change point identification mechanism; it introduces an innovative data structure, Box, to uniformly manage wildcard rules and leaf node rule overflows during tree construction; furthermore, addressing the static mapping defects and low memory utilization issues of existing hash table storage of port rules, this invention proposes a hierarchical hash structure, achieving synergistic optimization of storage efficiency and query speed through dynamic partitioning and density-adaptive hashing strategies. Furthermore, it dynamically drives leaf node re-splitting through update surge rate (USR) and local path redundancy (LPR) metrics, thereby constructing a classification system with lightweight structure, high search efficiency, and adaptive update capabilities, comprehensively improving the system's scalability and practical performance.

[0054] like Figure 1 and Figure 2 As shown, this invention provides a dynamic data packet classification method based on a trie and a multi-level decision tree, including:

[0055] S1, in the preprocessing stage of packet classification, constructs a trie for the IP prefixes of all rules.

[0056] Trie trees can achieve storage compression and fast location by sharing prefix paths.

[0057] Preferably, constructing a trie includes:

[0058] Each node represents a bit, and the prefix path is constructed sequentially from the root downwards.

[0059] Prefix paths can provide a foundation for subsequent rule partitioning and tree structure optimization.

[0060] S2, obtain the prefix drastic change point based on the trie.

[0061] Preferably, S2 includes:

[0062] S30, for each depth d, count the number of unique IP prefixes N(d) from the root node to depth d.

[0063] d∈[0,D], where D is the maximum value of the depth.

[0064] S31, Calculate the probability distribution of all rules at depth d in the IP address bits. , where i is a bit:

[0065] = The number of times bit i appears at depth d / the total number of bits at depth d, where i∈{0,1}.

[0066] S32, Calculate the relative growth acceleration rate A(d):

[0067] A(d) =

[0068] Use extremely small constants (such as 0.0001) to prevent division by zero.

[0069] A(d) undergoes a sudden increase in quantity through the second-order difference quantization rule.

[0070] S33, based on Calculate the KL divergence:

[0071] KL divergence is used to evaluate the differences in bit distribution between adjacent depths, and the calculation formula is:

[0072]

[0073] Let represent the probability of bit i occurring at depth d.

[0074] Represents a smooth reference distribution for adjacent depths. .

[0075] S34, calculate the Z-score of A(d), and obtain :

[0076] Set the window size to 5 depth units:

[0077] , This represents the mean of A(d) within the window. This represents the standard deviation of A(d) within the window.

[0078] S35, based on A(d), KL divergence and Calculate the mutation significance score S(d):

[0079] A comprehensive evaluation of the mutation significance score at each depth:

[0080] S(d) =

[0081] , Harmony Weighting coefficients.

[0082] In practical implementation, the importance of the three types of indicators can be expressed through weighting coefficients. , and Adjustments are made. To account for both changes in the number and structure of rules, a typical setting is as follows:

[0083] =0.4 (for A(d));

[0084] =0.4 (used for) );

[0085] =0.2 (for Z(d));

[0086] The above settings are applicable to most rule sets. If the system is more sensitive to structural abrupt changes, the settings can be appropriately increased. Reduced to 0.5 and .

[0087] S36. Among all depths with mutation significance scores greater than a preset mutation significance score threshold, the depth with the highest mutation significance score is taken as the prefix catastrophic change point.

[0088] Specifically, the preset threshold for mutation significance score includes 0.7.

[0089] S3 divides the rule subsets based on prefix drastic change points to obtain precise rule subsets and complex rule subsets.

[0090] Preferably, S3 includes:

[0091] If the IP prefix length is greater than the mutation significance score, the port range length is less than the preset length and does not contain wildcards, and the protocol type is not ANY, then the rule is divided into the exact rule subset; otherwise, the rule is divided into the complex rule subset.

[0092] Specifically, the preset length includes 1000.

[0093] Dynamic adjustment process:

[0094] When the LPR of a leaf node exceeds the limit, the KL divergence of the rules under that leaf node is recalculated. If the distribution changes abruptly, subset repartitioning is triggered.

[0095] In the specific implementation, the threshold for judging Local Path Redundancy (LPR) is set to 0.7. When the LPR(v) of a leaf node v > 0.7, it indicates that its rules are highly concentrated within the field value range, and there are potential problems such as path compression and query conflicts, which can be regarded as excessive structural redundancy. This threshold is set through experience testing multiple sets of real rule sets, which can effectively identify local structural bottleneck areas while ensuring classification accuracy.

[0096] When performing structural adaptation in the local region of a leaf node, it is necessary to determine whether the bit distribution of the current node's rule set has undergone a sudden change. To address this, a local KL divergence, DKL(local), is introduced.

[0097] in:

[0098] P(i): The probability of the i-th bit appearing in the rule set of the current leaf node;

[0099] Q(i): The smooth distribution (e.g., average) of bits in the rule set of the parent node and its adjacent sibling nodes;

[0100] If DKL(local) > 0.2 (empirical threshold), it is considered that the node has a significant difference in distribution from the surrounding structure, i.e., a "distribution mutation", which triggers subset repartitioning.

[0101] If a rule within a Box becomes a more precise rule due to an update (e.g., a narrower port range), it is automatically migrated to the main decision tree. Note: Although the main decision tree only uses the IP field for path partitioning, the system still performs a joint evaluation of the port and protocol fields during the rule allocation phase to ensure that rules entering the main tree have structural regularity and search stability. This filtering is only used for admission control logic and does not affect the singularity of the main tree's construction fields.

[0102] S4: Construct the main decision tree for the precise rule subset; input the complex rule subset into Box for processing.

[0103] Preferably, constructing a master decision tree for a subset of precise rules includes:

[0104] During the construction process, a bit balance strategy (such as minimizing the 0 / 1 distribution difference) is used to select split bits to ensure that the tree structure is as balanced as possible, thereby improving search efficiency and path compactness.

[0105] In the construction process, this invention proposes a bit balance strategy to select the optimal splitting bit. This strategy aims to balance the distribution of rules in the left and right subspaces to the greatest extent possible, thereby reducing rule duplication rate and improving the query efficiency of the classification tree.

[0106] In practice, for the set of rules to be partitioned, the distribution of high / low bits in each bit of each dimension is statistically analyzed, and the balance of the partitioning result when each bit is used as a partitioning point is calculated. The balance can be calculated using the following formula:

[0107] Balance(b)=|N{left}(b)-N{right}(b)|

[0108] Where (N{left}(b)) and (N{right}(b)) represent the number of rules falling into the left and right subspaces when the b-th bit is used for partitioning. Finally, the bit that minimizes Balance(b) is selected as the splitting bit.

[0109] This strategy effectively avoids the skew problem present in traditional heuristic partitioning, improving the overall structural uniformity of the rule tree and query performance.

[0110] During the construction process, if the number of rules in a leaf node exceeds a preset threshold, the rules will be input into the Box for processing.

[0111] During the construction process, the number of rules in the leaf nodes exceeds a preset threshold. Then the rules within that node are input into the Box structure for further processing. This quantity threshold. It can be calculated based on the following methods:

[0112]

[0113] in: Indicates the maximum number of rules allowed by the system (e.g., 128); M represents the memory space that can be allocated to a single Box; avg_rule_size represents the average space occupied by each rule (in bytes or bits).

[0114] This method allows for dynamic adjustment of rule grouping granularity based on runtime resources, thereby improving Box utilization and system scalability.

[0115] Preferably, the Box's processing of rules includes:

[0116] For the port field in the rule, if the port field is an exact field, then a hierarchical hash structure is used for management;

[0117] For example, in the port field, the hierarchical hash design for port storage is as follows: For the exact port field, a two-level hash structure is adopted: The first level of partitioning: According to the IANA standard, three major intervals are pre-divided: system ports (0-1023), registered ports (1024-49151), and dynamic ports (49152-65535), and the rule density of each interval is monitored in real time. When the density of a certain interval exceeds a threshold (e.g., >1000), dynamic sub-partitioning is performed based on Shannon entropy calculation (e.g., splitting 1024-49151 according to the 25% / 50% / 75% percentile).

[0118] Second-level hashing: Each sub-partition constructs its own hash table, with the number of buckets calculated according to the formula. Dynamically adjusted, where n is the current number of sub-partition rules. This is the compression factor that is automatically adjusted based on query latency.

[0119] If the port field is a range value, then Y-FastTrie is used for management, which has efficient lookup capabilities from O(1) to O(loglogU);

[0120] loglogU means taking the logarithm of U first, and then taking the logarithm of the result again.

[0121] U stands for universesize, which refers to the range of key-value pairs that can be supported; it can also be understood as the maximum upper limit of the field's value range.

[0122] For example: for a 16-bit integer field, U= =65536;

[0123] If the port field is a wildcard, then the DAWG structure is used to implement fuzzy matching.

[0124] If a field in a rule has a single exact value, it is considered an exact field, for example, destination port = 80;

[0125] If the field value is a continuous interval, such as source port ∈ [1000, 2000], then it is considered an interval field;

[0126] If a field value is a wildcard (*), it is considered a wildcard field.

[0127] Box not only receives wildcard rules encountered during tree construction, but also receives overflow rules when the number of rules in a leaf node exceeds a preset threshold.

[0128] By using structured storage and path isolation, the problems of multi-path rule replication and structural expansion in traditional decision trees are effectively solved.

[0129] The Box of this invention achieves efficient rule management through a three-level structured classification:

[0130] Precise port rules: Dynamic hierarchical hashing (first-level IANA partitioning + second-level density adaptive hashing);

[0131] Interval port rule: Y-FastTrie (O(loglogU) complexity);

[0132] Wildcard rule: DAWG (compression rate ≥ 3x). Technical effect: Real-world testing shows that Box improves the matching speed of wildcard rules by 2.1 times (compared to traditional linear search) and reduces memory usage by 45% (compared to uncategorized storage).

[0133] The main decision tree only carries a well-structured subset of rules (satisfying long IP prefixes, precise port fields, and explicit protocols). Through prefix trie preprocessing and dynamic drastic change point determination mechanisms, wildcards and large-scale rules are effectively avoided from entering the tree construction process, thereby greatly reducing the number of nodes and path redundancy and significantly optimizing the access efficiency of the main tree.

[0134] Secondly, Box, as an auxiliary classification structure, does not follow the unified decision tree model. Instead, it selects a special structure for decoupling based on the features of the rule fields: the precise port field uses hierarchical hashing (O(1) lookup), the interval field uses Y-FastTrie (O(loglogU) complexity), and the wildcard field uses DAWG (high compression, fuzzy matching), which respectively improves the lookup efficiency and avoids structural expansion and path duplication.

[0135] Furthermore, the packet classification process employs a path isolation mechanism. The system prioritizes matching within the main tree, only venturing into the Box if a match is missed or if rules overflow (exceeding the threshold) in the matched path. Therefore, the classification path for a Box is not a continuation or redundancy of the main tree, but rather an independent optimization process for areas that the main tree cannot effectively cover.

[0136] More importantly, during system runtime, most data packets are matched in the main tree, with the Box only handling some complex rules and its access ratio being much lower than that of the main path. Therefore, the overall system's average search latency is dominated by the main tree, with the Box serving as a supplement, working together to achieve the optimal balance between classification throughput and structural complexity.

[0137] In summary, the Box mechanism does not transfer complexity, but rather achieves a more efficient and controllable packet classification strategy than the traditional unified structure through structural decoupling, field isolation, and path sharing.

[0138] S5 constructs a shadow tree or shadow Box copy of the rule modification operation and performs pointer replacement using CAS atomic instructions.

[0139] The backend constructs a shadow tree or shadow Box copy and performs pointer replacement using CAS (Compare and Swap) atomic instructions to achieve uninterrupted rule switching. For the field structure within the Box, each supports localized incremental updates and independent replacements, ensuring that classification performance is unaffected under high-concurrency rule update environments.

[0140] Preferably, if the rules are dynamically updated, the leaf nodes are re-splitned based on the structural adaptive mechanism of runtime behavior feedback.

[0141] To address the issue of local overload caused by dynamic rule updates, this invention introduces a structural adaptive mechanism based on runtime behavior feedback. This mechanism uses two quantitative indicators to collaboratively determine whether a leaf node has entered a potential structural bottleneck state: an "UpdateSurgeRate (USR)" and a "LocalPathRedundancy (LPR)" dual-indicator judgment mechanism.

[0142] Preferably, the leaf nodes are re-splitting based on the structural adaptive mechanism of runtime behavior feedback, including: calculating the update surge rate and local path redundancy of the leaf nodes.

[0143] Local path redundancy formula definition:

[0144]

[0145] v: Current leaf node;

[0146] F: Set of rule fields (such as source IP, destination port, etc.);

[0147] The set of rules in leaf node v;

[0148] : The value of rule r in field f (discrete values ​​need to be encoded as integers);

[0149] FieldRangef: The theoretical range of values ​​for field f (e.g., port range = 65535);

[0150] Judgment criterion: When LPR(v)>0.7, it indicates that the rules of the node are highly homogeneous.

[0151] The formula for the U.S. Update Rate (USR) is defined as follows:

[0152]

[0153] Time window The number of new rules added to the inner leaf node v;

[0154] AvgChildSize: The average rule capacity of the subtree of this leaf node;

[0155] Determine whether the update surge rate and local path redundancy meet the re-splitting conditions. If so, re-split the leaf nodes.

[0156] Preferably, the re-division conditions include:

[0157] The update surge rate is greater than the preset update surge rate threshold and the path redundancy is greater than the preset path redundancy threshold.

[0158] Specifically, the update surge rate threshold is 5, and the path redundancy threshold is 0.7.

[0159] When USR(v) > 5.0, it indicates that the node's rule growth far exceeds its branch processing capacity. When LPR(v) > 0.7, it indicates that the node's rules are highly homogeneous.

[0160] The system determines that a leaf node is in a high-load / low-differentiation state if and only if LPR(v) > 0.7 and USR(v) > 5.0, posing a risk of decreased path-finding efficiency and structural accumulation. In this case, the system will automatically trigger path re-split as a runtime contingency measure.

[0161] Select the field with the lowest redundancy to build a new subtree to alleviate local concentration;

[0162] If the newly split rules meet the Box admission criteria (such as excessively large port range or wildcard), they will be automatically transferred to Box management.

[0163] The essence of this mechanism is not the adjustment or partitioning optimization of the rule set, but rather the adaptive adjustment of the system structure to "abnormal behavior signals" during runtime. This strategy ensures that the system can maintain the stability of the search path and the classification throughput performance even with frequent rule evolution, avoiding the interruptions and performance losses caused by full tree reconstruction.

[0164] Experimental results show that this mechanism only adjusts the local structure and avoids overall cost reconstruction. Under a rule set of tens of thousands, it can significantly reduce the update response latency from 200μs to 15μs, verifying its runtime scalability and stability.

[0165] This invention also provides a dynamic data packet classification system based on a trie and a multi-level decision tree, including a construction module, an acquisition module, a partitioning module, a processing module, and a replacement module;

[0166] The building module is used to construct a trie for the IP prefixes of all rules during the preprocessing stage of packet classification;

[0167] The acquisition module is used to retrieve prefix drastic change points based on the trie;

[0168] The partitioning module is used to partition rule subsets based on prefix drastic change points, obtaining precise rule subsets and complex rule subsets;

[0169] The processing module is used to construct the main decision tree from a precise subset of rules; and to input a complex subset of rules into the Box for processing.

[0170] The replacement module is used to construct a shadow tree or shadow box copy of the rule modification operation and perform pointer replacement via CAS atomic instructions.

[0171] To address the problems of severe rule duplication, structural redundancy, lengthy search paths, and low dynamic update efficiency in existing packet classification algorithms (such as CutSplit) when dealing with large-scale, highly dynamic rule sets, this invention proposes an efficient dynamic packet classification method based on trie preprocessing, dynamic prefix drastic change point identification mechanism, and combined with the innovative data structure "Box" and an adaptive update optimization mechanism, collectively referred to as the "TrieBox algorithm".

[0172] This invention effectively overcomes the structural bottlenecks and updating challenges of existing technologies through the following technological innovations:

[0173] 1) This invention proposes an innovative data structure, Box, to uniformly manage rule overflow and wildcard rules, significantly reducing structural redundancy and rule duplication: In traditional decision tree construction, wildcard rules and wide-range rules often lead to rule duplication across paths, causing structural bloat and memory waste. The Box data structure designed in this invention can be triggered in two scenarios: first, when a wildcard field is encountered during tree construction, the rule is uniformly distributed to Box; second, when the number of rules in a leaf node exceeds a threshold, overflowing rules are entered into Box. Internally, Box is structured based on the characteristics of the rule fields (e.g., hierarchical hash structure for exact port matching, Y-FastTrie for interval port matching, and DAWG for wildcard matching), achieving unified management and efficient searching, fundamentally eliminating the rule duplication problem.

[0174] 2) Introducing a dynamic prefix drastic change point identification mechanism to achieve intelligent pre-partitioning and path hierarchical optimization of the rule set: This invention analyzes the changes in rule distribution at different depths in the IP prefix trie, and combines growth acceleration, KL divergence, and Z-score anomaly detection to dynamically identify the locations where significant structural changes occur in the rule distribution, as prefix "drastic change points." This mechanism can intelligently divide the rule set into subsets that are more suitable for entering the main tree (precise rules) and subsets that are more suitable for being managed by Box (wildcards and large rules), reducing subsequent path conflicts and redundant copying, and effectively improving search efficiency.

[0175] 3) Design a leaf node update-driven structural re-splitting mechanism to enhance the system's dynamic adaptability: To address the structural local overload problem caused by continuous rule updates, this invention introduces a "leaf node re-splitting mechanism" driven by two innovative indicators: one is the Update Surge Rate (USR), which reflects the rule growth pressure per unit time in real time; the other is the Local Path Redundancy (LPR), which assesses the overlap of current leaf node rules in field positions. When a leaf node exhibits high USR and high LPR during updates, the system determines that the node has entered a structural bottleneck state, automatically triggering further path splitting to refine the decision structure and ensure that classification search performance does not degrade with updates.

[0176] 4) Achieve localized and structured incremental updates of the rule set to ensure service continuity and high system availability: This invention supports shadow structure updates and atomic switching mechanisms for both the main decision tree and the Box structure. The addition, deletion and modification of rules can be completed independently in the background and take effect through instant switching. This avoids the service interruption problem that requires rebuilding the entire tree during the update process in traditional algorithms, thereby ensuring that the system has continuous and efficient operation capabilities when facing dynamic rule environments.

[0177] This invention does not perform global rearrangement of the rule set or solve for the optimal subset structure, nor does it rely on any greedy strategy, overlapping graph construction, or integer linear programming solution; instead, it dynamically senses the update pressure and field redundancy of each leaf node through a real-time monitoring mechanism during system operation.

[0178] When a leaf node experiences rapid rule growth (USR indicator) and highly overlapping rule fields (LPR indicator) within a unit of time, the system will trigger local path re-splitting in a behavior-driven manner. By prioritizing the least redundant fields, the structural bottleneck will be locally reconstructed into a more compact, distributed structure.

[0179] This mechanism does not "adjust" or "optimize" the rule set itself, but rather evolves and adjusts the structural path itself. The purpose is to ensure the search efficiency and update stability of the system in the long run, rather than compressing the rule set structure all at once.

[0180] Compared to existing packet classification techniques (such as decision tree optimization algorithms like CutSplit), the algorithm proposed in this invention brings the following significant technical effects through a series of structural and behavioral innovations:

[0181] (1) It fundamentally solves the problems of rule replication and path redundancy, and significantly optimizes memory structure and space efficiency:

[0182] Traditional methods, when constructing decision trees, often have to copy the same rule to multiple path nodes when faced with wildcard rules or short prefix rules, resulting in structural bloat and memory waste. This invention addresses this by designing an innovative data structure called "Box," which uniformly receives wildcard rules and leaf node rule overflows encountered during the tree construction phase, physically isolating the impact of wildcards and avoiding redundant copying across multiple paths. Within Box, specialized structures such as hierarchical hashing (for precise port matching), Y-FastTrie (for port range matching), and DAWG (for fuzzy wildcards) are used to manage the rules based on their field characteristics, achieving structural hierarchy compression and search space optimization.

[0183] (2) By using intelligent rule partitioning mechanism and field-aware structure diversion, search efficiency and system scalability are significantly improved:

[0184] Compared to traditional fixed-threshold partitioning methods, the dynamic prefix drastic change point identification mechanism of this invention uses formulas to quantify and analyze the growth acceleration, bit variability, and statistical significance of rule distribution, making the rule partitioning boundary adaptive and mathematically interpretable. Theoretical derivation shows that, under typical enterprise-level rule sets (generated by ClassBench), this method can reduce the mispartition rate to below 5% (compared to 15%-20% for static methods). The resulting master decision tree structure is simpler and the rule coverage is more explicit, avoiding the bottleneck of structural adaptability inherent in traditional static partitioning methods. Simultaneously, the structured splitting of rules in Box further accelerates field matching, enabling the entire system to maintain near-linear search efficiency even when facing millions of rule sets, demonstrating good horizontal scalability.

[0185] (3) Supports structured, adaptive dynamic updates to ensure the stability and continuity of the system's long-term operation:

[0186] This invention, in addition to supporting switching of atomization rules, further introduces a dynamic splitting mechanism based on quantization indicators:

[0187] Update Burst Rate (USR): Real-time monitoring of the rate at which leaf node rules are added. When USR > 5.0, it indicates that the burst load exceeds the subtree's processing capacity.

[0188] Local Path Redundancy (LPR): By calculating the redundancy of fields, when LPR>0.7, it is determined that the rules are severely homogenized;

[0189] Collaborative triggering: Local subtree splitting is triggered only when both USR > 5.0 and LPR > 0.7 are simultaneously satisfied, compared to the global reconstruction method.

[0190] Update latency reduced by 92% (from 200μs to 15μs, tested with tens of thousands of rules);

[0191] Improved splitting accuracy: false trigger rate <3% (validated on ClassBench-Enterprise dataset).

[0192] (4) Achieve efficient storage and real-time response through the collaborative design of hierarchical hashing and dynamic splitting:

[0193] Memory efficiency: Box's three-level storage structure (hash table / Y-Fast Trie / DAWG) reduces memory waste by 30%-50%;

[0194] Query acceleration: Partition pruning reduces hash collision rate to less than 15%, and the LPR-triggered splitting strategy reduces average search path length by 17%;

[0195] Dynamic adaptability: The USR mechanism ensures that 95% of query throughput is maintained even under burst traffic (a 40% decrease compared to traditional methods).

[0196] In summary, the solution of this invention systematically solves the fundamental problems of traditional data packet classification technology in terms of rule replication, search performance, and update stability through the end-to-end design concept of "structural decoupling, dynamic reconstruction, and field awareness". It has excellent performance with high search throughput, high memory efficiency and strong dynamic adaptability, and is suitable for the core needs of real-time classification capability and large-scale rule set management in modern network environment.

[0197] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A dynamic data packet classification method based on trie and multi-level decision tree, characterized in that, include: S1, In the preprocessing stage of packet classification, a trie is built for the IP prefixes of all rules; S2, obtain prefix transition points based on the trie; S3, based on the prefix drastic change point, divides the rule subsets to obtain precise rule subsets and complex rule subsets; S4: Construct the main decision tree for the precise rule subset; input the complex rule subset into the data structure Box for processing; S5, construct a shadow tree or shadow data structure Box copy for rule modification operations, and perform pointer replacement using CAS atomic instructions; S2 includes: S30, for each depth d, count the number of unique IP prefixes N(d) from the root node to depth d; S31, Calculate the probability distribution of all rules at depth d in the IP address bits. where i is a bit; S32, calculate the relative growth acceleration rate A(d); S33, based on Calculate the KL divergence; S34, calculate the Z-score of A(d), and obtain ; S35, based on A(d), KL divergence and Calculate the mutation significance score S(d); S36. Among all depths with mutation significance scores greater than a preset mutation significance score threshold, the depth with the highest mutation significance score is taken as the prefix catastrophic change point.

2. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 1, characterized in that, Constructing a trie includes: Each node represents a bit, and the prefix path is constructed sequentially from the root downwards.

3. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 1, characterized in that, S3 include: If the IP prefix length is greater than the mutation significance score, the port range length is less than the preset length and does not contain wildcards, and the protocol type is not ANY, then the rule is divided into the exact rule subset; otherwise, the rule is divided into the complex rule subset.

4. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 1, characterized in that, Construct the main decision tree from the exact subset of rules, including: During the construction process, a bit balance strategy is used to select split bits; During the construction process, if the number of rules in a leaf node exceeds a preset threshold, the rules will be input into the data structure Box for processing.

5. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 1, characterized in that, The Box data structure handles rules in the following ways: For the port field in the rule, if the port field is an exact field, then a hierarchical hash structure is used for management; If the port field is a range value, then Y-FastTrie is used for management; If the port field is a wildcard, then the DAWG structure is used to implement fuzzy matching.

6. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 1, characterized in that, If the rules are dynamically updated, the leaf nodes are re-splitned based on the structural adaptive mechanism of runtime behavior feedback.

7. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 6, characterized in that, The leaf nodes are re-splitting based on the structural adaptive mechanism of runtime behavior feedback, including: calculating the update surge rate and local path redundancy of the leaf nodes; Determine whether the update surge rate and local path redundancy meet the re-splitting conditions. If so, re-split the leaf nodes.

8. The dynamic data packet classification method based on a trie and a multi-level decision tree according to claim 7, characterized in that, The conditions for further splitting include: The update surge rate is greater than the preset update surge rate threshold and the local path redundancy is greater than the preset path redundancy threshold.

9. A dynamic data packet classification system based on trie and multi-level decision tree, characterized in that, This includes a building module, an acquisition module, a partitioning module, a processing module, and a replacement module; The building module is used to construct a trie for the IP prefixes of all rules during the preprocessing stage of packet classification; The acquisition module is used to retrieve prefix drastic change points based on the trie; The partitioning module is used to partition rule subsets based on prefix drastic change points, obtaining precise rule subsets and complex rule subsets; The processing module is used to construct the main decision tree from the precise subset of rules; and to input the complex subset of rules into the data structure Box for processing. The replacement module is used to construct a shadow tree or shadow data structure Box copy for rule modification operations and to perform pointer replacement via CAS atomic instructions; Obtaining prefix transition points based on a trie includes: S30, for each depth d, count the number of unique IP prefixes N(d) from the root node to depth d; S31, Calculate the probability distribution of all rules at depth d in the IP address bits. where i is a bit; S32, calculate the relative growth acceleration rate A(d); S33, based on Calculate the KL divergence; S34, calculate the Z-score of A(d), and obtain ; S35, based on A(d), KL divergence and Calculate the mutation significance score S(d); S36. Among all depths with mutation significance scores greater than a preset mutation significance score threshold, the depth with the highest mutation significance score is taken as the prefix catastrophic change point.

Images