Dynamic data packet classification method and system based on dictionary tree and multilevel decision tree

Abstract

The invention belongs to the field of computer networks, and discloses a dynamic data packet classification method and system based on a dictionary tree and a multilevel decision tree, and the method comprises the steps: S1, constructing the dictionary tree for IP prefixes of all rules in a preprocessing stage of data packet classification; s2, obtaining prefix huge change points based on the dictionary tree; s3, dividing rule subsets based on the prefix huge change points to obtain accurate rule subsets and complex rule subsets; s4, constructing a main decision tree for the accurate rule subset; inputting the complex rule subset into Box for processing; and S5, constructing a shadow tree or a shadow Box copy of the rule modification operation, and performing pointer replacement through a CAS atomic instruction. According to the method, structural layering, behavior driving and field decoupling are taken as a core design concept, rule pre-division, path structure optimization, update behavior perception and an efficient matching mechanism are comprehensively integrated, and the problems of rule replication redundancy, low path efficiency, dynamic update response lag and the like in data packet classification are solved in a breakthrough manner.

Description

Technical Field

[0001] The present invention relates to the field of computer networks, and in particular to a dynamic data packet classification method and system based on a dictionary tree and a multi-level decision tree. Background Art

[0002] Packet classification is a crucial, fundamental function in modern computer networks, widely used in network devices such as routers, firewalls, intrusion detection systems (IDS), quality of service (QoS) assurance, and load balancing. With the explosive growth of internet traffic, the increasing complexity of network services, and the ever-expanding set of rules, network devices must classify and process massive amounts of data packets in real time, accurately, and efficiently. To meet this performance requirement, researchers have proposed a variety of packet classification algorithms. Among them, decision tree-based classification algorithms have attracted widespread attention and research due to their ability to handle multidimensional fields, support complex rules, and theoretically achieve efficient search. Decision trees use multiple packet fields (such as source / destination IP addresses, source / destination port numbers, and protocol type) as decision attributes to construct a multi-way or binary search tree, thereby gradually partitioning the search space into smaller subspaces and ultimately guiding packets to matching corresponding rules.

[0003] However, despite the theoretical advantage of high search efficiency, traditional decision tree algorithms still face severe challenges when faced with large-scale, highly complex, and dynamically changing rule sets. These challenges are mainly reflected in the following aspects: 1) Rule replication: Especially when there are a large number of "big rules" with short IP prefixes or many wildcards in the rule set, in order to correctly handle the overlap and priority between rules, these big rules often need to be copied repeatedly on multiple branches of the decision tree, causing the tree to expand rapidly and occupy a large amount of memory. 2) Degraded search performance: Rule replication and increased tree depth make the average search path of data packets longer, thereby significantly reducing the classification search efficiency and making it difficult to meet the requirements of line-speed processing. 3) Low rule update efficiency: Structural updates of the decision tree (such as adding, deleting, or modifying rules) may require time-consuming reconstruction operations locally or even globally on the tree, resulting in long update times and difficulty adapting to frequent adjustments to network policies, thus affecting the normal operation of services.

[0004] To overcome the rule duplication and performance bottlenecks encountered by traditional decision tree algorithms when processing large rule sets, researchers have proposed various optimization schemes. CutSplit, a decision tree bag classification algorithm based on hybrid cutting technology, is one such attempt to address these issues. The CutSplit algorithm uses a hybrid cutting strategy to select optimal cutting dimensions (such as bit cutting or field cutting) during decision tree construction, thereby reducing rule duplication and optimizing the tree structure. Through quantitative evaluation and analysis of existing decision tree algorithms, it attempts to identify the root causes of their poor scalability and, based on this, designs a corresponding algorithmic framework, hoping to improve classification efficiency and scalability.

[0005] Although existing decision tree optimization algorithms such as CutSplit have attempted to alleviate the rule duplication problem of traditional decision trees and improve search efficiency to a certain extent, they still have significant limitations when faced with extremely large-scale, highly dynamic, and complex and diverse network rule sets. Specific shortcomings are as follows: 1) The problem of rule duplication has not been fundamentally resolved: CutSplit's hybrid splitting strategy still struggles to completely avoid multi-path duplication of rules within the tree when handling "large rules" in some extreme scenarios (such as rules with very short IP prefixes or containing a large number of wildcards). This local optimization strategy struggles to cope with global complexity, resulting in significant rule duplication at certain depths or branches. This makes it difficult to effectively control the size of the decision tree, which in turn affects search performance and memory consumption. The fundamental reason is that traditional decision tree splitting strategies (whether bitwise or fieldwise) are forced to replicate to maintain the integrity and correctness of the tree when handling large, highly overlapping and inclusive rules, ensuring that all packets correctly match the highest-priority rule. This inherent duplication mechanism, even with hybrid splitting, can only be mitigated, not eliminated.

[0006] 2) Inefficient dynamic updates: Existing optimization algorithms, including CutSplit, often require time-consuming local or global adjustments to the tree structure when the rule set is frequently updated (such as when rules are added, deleted, or modified). This adjustment process may involve splitting, merging, or even rebuilding nodes, and its computational complexity increases dramatically with the number of rules. When the update frequency is high, such adjustments can cause system performance fluctuations or even brief service interruptions. The fundamental reason is that the optimization strategies of these algorithms are usually completed during the static construction phase of the tree, and insufficient consideration is given to the dynamic maintenance capabilities of the tree. Once the rule changes affect the core structure, a large computational cost is required to restore the tree's balance and search efficiency, lacking a truly incremental, non-interruption update mechanism.

[0007] 3) Limited Scalability: Combined with the aforementioned shortcomings of inefficient rule replication and updates, existing technologies face difficulties in scaling search performance and update response times linearly as network size continues to expand and the number of rule sets reaches millions or even tens of millions. As the number of rules grows, even after optimization, the depth and width of the tree may exceed the effective range of the hardware cache, triggering a large number of memory accesses and further reducing actual performance. The fundamental reason is that these algorithms fail to achieve more thorough hierarchical decoupling and rule feature preprocessing at the data structure level. This makes it difficult to effectively balance the combined requirements of search efficiency, memory usage, and dynamic updates within a single or limited-level decision tree structure. Summary of the Invention

[0008] The purpose of the present invention is to disclose a dynamic data packet classification method and system based on a dictionary tree and a multi-level decision tree, so as to solve the technical problems raised in the background technology.

[0009] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a dynamic data packet classification method based on a dictionary tree and a multi-level decision tree, comprising: S1, in the pre-processing stage of packet classification, a dictionary tree is constructed for the IP prefixes of all rules; S2, obtain the prefix big change point based on the dictionary tree; S3, divide the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; S4, builds the main decision tree for the precise rule subset; inputs the complex rule subset into Box for processing; S5, builds a shadow tree or shadow Box copy of the rule modification operation and replaces the pointer through the CAS atomic instruction.

[0010] Preferably, constructing a dictionary tree includes: Each node represents a bit, and the prefix path is constructed from the root downward.

[0011] Preferably, S2 includes: S30, for each depth d, count the number N(d) of unique IP prefixes from the root node to the depth d; S31, calculate the probability distribution of all rules at depth d on the IP address bits , i is the bit; S32, calculate the relative growth acceleration rate A(d); S33, based on Calculate KL divergence; S34, calculate the Z-score value of A(d) and get ; S35, based on A(d), KL divergence and Calculate the mutation significance score S(d); S36: Among all depths where the mutation significance scores are greater than a preset mutation significance score threshold, the depth with the largest mutation significance score is taken as the prefix catastrophic change point.

[0012] Preferably, S3 includes: If the IP prefix length is greater than the mutation significance score and the port range length is less than the preset length and does not contain wildcards and the protocol type is not ANY, the rule is classified into the precise rule subset; otherwise, the rule is classified into the complex rule subset.

[0013] Preferably, a main decision tree is constructed for the precise rule subset, including: During the construction process, the split bits are selected using a bit balance strategy; During the construction process, if the number of rules of a leaf node is greater than the preset threshold, the rules are input into the Box for processing.

[0014] Preferably, Box's processing of rules includes: For the port field in the rule, if the port field is an exact field, it is managed using a hierarchical hash structure; If the port field is an interval value, Y-FastTrie is used for management; If the port field is a wildcard, the DAWG structure is used to implement fuzzy matching.

[0015] Preferably, if the rules are dynamically updated, the leaf nodes are re-split based on a structural adaptive mechanism based on operational behavior feedback.

[0016] Preferably, the structural adaptive mechanism based on the operational behavior feedback re-splits the leaf nodes, including: calculating the update surge rate and local path redundancy of the leaf nodes; Determine whether the update surge rate and local path redundancy meet the re-split conditions. If so, re-split the leaf node.

[0017] Preferably, the resplitting conditions include: The update surge rate is greater than a preset update surge rate threshold and the path redundancy is greater than a preset path redundancy threshold.

[0018] In a second aspect, the present invention provides a dynamic data packet classification system based on a dictionary tree and a multi-level decision tree, comprising a construction module, an acquisition module, a division module, a processing module, and a replacement module; The construction module is used to construct a dictionary tree for all IP prefixes of rules in the preprocessing stage of packet classification; The acquisition module is used to obtain the prefix big change point based on the dictionary tree; The partitioning module is used to partition the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; The processing module is used to construct the main decision tree for the precise rule subset; the complex rule subset is input into the Box for processing; The replacement module is used to build a shadow tree or shadow box copy of the rule modification operation and perform pointer replacement through the CAS atomic instruction.

[0019] Beneficial effects: This invention takes "structural layering + behavior-driven + field decoupling" as its core design concept, comprehensively integrating rule pre-division, path structure optimization, update behavior perception and efficient matching mechanism, and breakthroughly solves the problems of rule duplication redundancy, path inefficiency and dynamic update response lag in data packet classification. It has good performance scalability and engineering practicality, and is suitable for high-performance scenarios such as network security and traffic scheduling for million-level rule sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 Schematic diagram of the dynamic data packet classification method based on dictionary tree and multi-level decision tree of the present invention.

[0022] Figure 2 Schematic diagram of the relationship between the box and the main decision tree of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0024] Existing packet classification algorithms, particularly those based on decision trees (such as CutSplit), commonly suffer from severe structural redundancy, low path efficiency, and costly rule updates when processing large-scale, highly dynamic, and complex rule sets containing wildcard rules. This invention aims to fundamentally address these bottlenecks. It achieves intelligent partitioning of rule sets through dictionary tree preprocessing combined with a dynamic prefix tipping point recognition mechanism. It also introduces an innovative data structure, Box, to uniformly manage wildcard rule and leaf node rule overflow during tree construction. Furthermore, to address the static mapping flaws and low memory utilization inherent in hash tables used to store port rules in existing technologies, this invention proposes a hierarchical hash structure. Through dynamic partitioning and a density-adaptive hashing strategy, it achieves a coordinated optimization of storage efficiency and query speed. Furthermore, leaf node re-splitting is dynamically driven by the Update Surge Rate (USR) and Local Path Redundancy (LPR) metrics, resulting in a classification system with a lightweight structure, high search efficiency, and update adaptability, comprehensively improving system scalability and practical performance.

[0025] like Figure 1 and Figure 2 As shown, the present invention provides a dynamic data packet classification method based on a dictionary tree and a multi-level decision tree, comprising: S1, in the preprocessing stage of packet classification, a dictionary tree is constructed for the IP prefixes of all rules.

[0026] The dictionary tree can achieve storage compression and fast positioning through shared prefix paths.

[0027] Preferably, constructing a dictionary tree includes: Each node represents a bit, and the prefix path is constructed from the root downward.

[0028] The prefix path can provide a basis for subsequent rule division and tree structure optimization.

[0029] S2, obtain the prefix big change point based on the dictionary tree.

[0030] Preferably, S2 includes: S30 , for each depth d, counting the number N(d) of unique IP prefixes from the root node to the depth d.

[0031] d∈[0,D], where D is the maximum depth.

[0032] S31, calculate the probability distribution of all rules at depth d on the IP address bits , i is the bit: = the number of occurrences of bit i at depth d / the total number of bits at depth d, i∈{0,1}.

[0033] S32, calculate the relative growth acceleration rate A(d): A(d)=

[0034] Prevent division by zero for very small constants (such as 0.0001).

[0035] A(d) quantifies the number of growth mutations through second-order difference quantization rules.

[0036] S33, based on Calculate KL divergence: KL divergence is used to evaluate the difference in the distribution of adjacent depth bits. The calculation formula is:

[0037] represents the probability of occurrence of bit i at depth d, represents a smooth reference distribution of adjacent depths, .

[0038] S34, calculate the Z-score value of A(d) and get : The window size is set to 5 depth units: , represents the mean value of A(d) in the window, represents the standard deviation of A(d) within the window.

[0039] S35, based on A(d), KL divergence and Calculate the mutation significance score S(d): Comprehensively evaluate the mutation significance scores at each depth: S(d)=

[0040] 、 Hewei Weighting coefficient.

[0041] In the specific implementation, the importance of the three types of indicators can be calculated by weighting coefficients 、 and To take into account both the changes in the number of rules and the changes in the structure, the typical settings are as follows: = 0.4 (for A(d)); =0.4 (for ); = 0.2 (for Z(d)); The above settings are applicable to most rule sets. If the system is more sensitive to structural mutations, the settings can be appropriately increased. to 0.5, reduce and .

[0042] S36: Among all depths where the mutation significance scores are greater than a preset mutation significance score threshold, the depth with the largest mutation significance score is taken as the prefix catastrophic change point.

[0043] Specifically, the preset mutation significance score threshold includes 0.7.

[0044] S3, divide the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets.

[0045] Preferably, S3 includes: If the IP prefix length is greater than the mutation significance score and the port range length is less than the preset length and does not contain wildcards and the protocol type is not ANY, the rule is classified into the precise rule subset; otherwise, the rule is classified into the complex rule subset.

[0046] Specifically, the preset length includes 1000.

[0047] Dynamic adjustment process: When the LPR of a leaf node exceeds the standard, the KL divergence of the rule under the leaf node is recalculated. If the distribution changes suddenly, the subset repartition is triggered.

[0048] In the implementation, the local path redundancy (LPR) threshold is set at 0.7. When LPR(v) for a leaf node v exceeds 0.7, it indicates that its rules are highly concentrated within the field value range, potentially leading to path compression and query conflicts, and is considered excessive structural redundancy. This threshold was empirically determined through testing multiple real-world rule sets, effectively identifying local structural bottlenecks while ensuring classification accuracy.

[0049] When performing structural adaptation in the local area of ​​a leaf node, it is necessary to determine whether the bit distribution of the current node rule set has undergone a sudden change. To this end, the local KL divergence DKL (local) =

[0050] in: P(i): the probability of occurrence of the i-th bit in the current leaf node rule set; Q(i): the smooth distribution (e.g., average) of bits in the rule set of the parent node and its adjacent sibling nodes; If DKL (local) > 0.2 (empirical threshold), it is considered that there is a significant difference in distribution between the node and the surrounding structure, that is, a "distribution mutation", which triggers subset re-division.

[0051] If rules within the Box become more precise due to updates (e.g., a narrowed port range), they are automatically migrated to the main decision tree. Note: Although the main decision tree only uses the IP field for path division, the system still jointly evaluates the port field and protocol field during the rule allocation phase to ensure that the rules entering the main tree are structurally regular and searchable. This screening only serves as access control logic and does not affect the single field nature of the main tree's construction.

[0052] S4, builds the main decision tree for the precise rule subset; inputs the complex rule subset into Box for processing.

[0053] Preferably, a main decision tree is constructed for the precise rule subset, including: During the construction process, a bit balance strategy (such as minimizing the 0 / 1 distribution difference) is used to select split bits to ensure that the tree structure is as balanced as possible, thereby improving search efficiency and path compactness.

[0054] During the construction process, the present invention proposes a bit balancing strategy for selecting the optimal splitting bit. This strategy aims to maximize the balance of the distribution of rules in the left and right subspaces, thereby reducing the rule duplication rate and improving the query efficiency of the classification tree.

[0055] In specific implementation, for the rule set to be divided, the distribution of high / low bits in each bit of each dimension is counted, and the balance of the division result when each bit is used as the division point is calculated. The balance can be calculated using the following formula: Balance(b)=|N{left}(b)-N{right}(b)| Where (N{left}(b)) and (N{right}(b)) represent the number of rules that fall into the left and right subspaces, respectively, when splitting at the bth bit. The bit that minimizes Balance(b) is ultimately selected as the splitting bit.

[0056] This strategy can effectively avoid the skew problem existing in traditional heuristic partitioning and improve the overall structural uniformity and query performance of the rule tree.

[0057] During the construction process, if the number of rules of a leaf node is greater than the preset threshold, the rules are input into the Box for processing.

[0058] During the construction process, if the number of rules in the leaf node exceeds the preset threshold , then the rules in the node are input into the Box structure for further processing. It can be calculated based on the following method:

[0059] in: Indicates the maximum number of rules allowed by the system (e.g., 128); M indicates the memory space that can be allocated to a single Box; avg_rule_size indicates the average space occupied by each rule (in bytes or bits); In this way, the rule grouping granularity can be dynamically adjusted according to runtime resources to improve Box utilization and system scalability.

[0060] Preferably, Box's processing of rules includes: For the port field in the rule, if the port field is an exact field, it is managed using a hierarchical hash structure; For example, in the port field, the hierarchical hash design for port storage uses a two-level hash structure for precise port fields. First-level partitioning: System ports (0-1023), registered ports (1024-49151), and dynamic ports (49152-65535) are pre-divided according to IANA standards. The rule density of each interval is monitored in real time. When the density of a particular interval exceeds a threshold (e.g., >1000 entries), dynamic sub-partitioning is performed based on Shannon entropy calculations (e.g., slicing 1024-49151 at the 25th / 50th / 75th percentiles).

[0061] Second-level hashing: Each sub-partition independently builds a hash table, and the number of buckets is calculated according to the formula Dynamic adjustment, where n is the number of current sub-partitioning rules, The compression factor is automatically adjusted based on query latency.

[0062] If the port field is an interval value, it is managed using Y-FastTrie, which has an efficient search capability of O(1) to O(loglogU); loglogU means taking the logarithm of U first and then taking the logarithm of the result.

[0063] U stands for universesize, which refers to the supported range of key values ​​and can also be understood as the maximum upper limit of the field value range.

[0064] For example: For a 16-bit integer field, U= =65536; If the port field is a wildcard, the DAWG structure is used to implement fuzzy matching.

[0065] If the value corresponding to a field in the rule is a single precise value, it is determined to be a precise field, for example, destination port = 80; If the field value is a continuous interval, such as source port∈[1000,2000], it is considered an interval field; If the field value is a wildcard character (*), it is considered a wildcard field.

[0066] Box not only receives wildcard rules encountered during tree construction, but also receives overflow rules when the number of leaf node rules exceeds a preset threshold.

[0067] Through structured storage and path isolation, the problems of multi-path rule duplication and structural expansion in traditional decision trees are effectively solved.

[0068] The Box of the present invention achieves efficient management of rules through three-level structured classification: Precise port rules: dynamic hierarchical hashing (first-level IANA zoning + second-level density-adaptive hashing); Interval port rule: Y-FastTrie (O(loglogU) complexity); Wildcard rules: DAWG (compression ratio ≥ 3x). Technical results: Tests show that Box's wildcard rule matching speed is 2.1 times faster (compared to traditional linear search) and memory usage is reduced by 45% (compared to unclassified storage).

[0069] The main decision tree only carries a subset of rules with a regular structure (satisfying the requirements of long IP prefixes, accurate port fields, and clear protocols). Through prefix dictionary tree preprocessing and dynamic catastrophic point determination mechanism, it effectively prevents wildcards and large-scale rules from entering the tree construction process, thereby greatly reducing the number of nodes and path redundancy, and significantly optimizing the access efficiency of the main tree.

[0070] Secondly, as an auxiliary classification structure, Box does not follow the unified decision tree model. Instead, it selects a special structure for decoupling processing based on the characteristics of the rule fields: the precise port field uses hierarchical hashing (O(1) search), the interval field uses Y-FastTrie (O(loglogU) complexity), and the wildcard field uses DAWG (high compression, fuzzy matching), which specifically improves the search efficiency and avoids structural expansion and path duplication.

[0071] Furthermore, the packet classification process utilizes a path isolation mechanism. The system prioritizes matching on the main tree. Only when a match is missed or the rule overflow (exceeds the threshold) does the packet enter the Box. Therefore, the Box's classification path is not a continuation or redundancy of the main tree, but rather an independent optimization process for areas that the main tree cannot effectively cover.

[0072] More importantly, during system operation, most packets are matched against the primary tree, with the Box handling only a portion of the complex rule processing, resulting in a much lower access ratio than the primary path. As a result, the overall system average lookup latency is dominated by the primary tree, with the Box providing a supplementary component, collaboratively achieving an optimal balance between classification throughput and structural complexity.

[0073] In summary, the Box mechanism does not transfer complexity, but achieves a more efficient and controllable packet classification strategy than the traditional unified structure through structural decoupling, field isolation and path sharing.

[0074] S5, builds a shadow tree or shadow Box copy of the rule modification operation and replaces the pointer through the CAS atomic instruction.

[0075] The backend constructs a shadow tree or shadow box copy and replaces pointers using atomic CAS (compare and swap) instructions, enabling seamless rule switching. Field structures within the box support localized incremental updates and independent replacement, ensuring unaffected classification performance in highly concurrent rule update environments.

[0076] Preferably, if the rules are dynamically updated, the leaf nodes are re-split based on a structural adaptive mechanism based on operational behavior feedback.

[0077] To address the local overload problem caused by dynamic rule updates, this paper introduces a structural adaptive mechanism based on operational behavior feedback. This mechanism uses two quantitative indicators to collaboratively determine whether a leaf node has reached a potential structural bottleneck: the "Update Surge Rate" (USR) and the "Local Path Redundancy" (LPR).

[0078] Preferably, the structural adaptive mechanism based on operational behavior feedback re-splits the leaf nodes, including: calculating the update surge rate and local path redundancy of the leaf nodes: The local path redundancy formula is defined as:

[0079] v: current leaf node; F: rule field set (such as source IP, destination port, etc.); : the set of rules in the leaf node v; : The value of rule r on field f (discrete values ​​must be encoded as integers); FieldRangef: The theoretical value range of field f (e.g. port range = 65535); Judgment criteria: When LPR(v)>0.7, it indicates that the node rules are highly homogeneous.

[0080] Update Surge Rate (USR) formula definition:

[0081] : Time window The number of new rules added to the inner leaf node v; AvgChildSize: the average rule capacity of the leaf node subtree; Determine whether the update surge rate and local path redundancy meet the re-split conditions. If so, re-split the leaf node.

[0082] Preferably, the resplitting conditions include: The update surge rate is greater than a preset update surge rate threshold and the path redundancy is greater than a preset path redundancy threshold.

[0083] Specifically, the update surge rate threshold is 5, and the path redundancy threshold is 0.7.

[0084] When USR(v)>5.0, it indicates that the node rule growth far exceeds its branch processing capacity. When LPR(v)>0.7, it indicates that the node rule is highly homogenized.

[0085] If and only if LPR(v)>0.7 and USR(v)>5.0, the system determines that the leaf node is in a high-load / low-differentiation state, and there is a risk of decreased path search efficiency and structure accumulation. In this case, the system will automatically trigger path re-splitting as a runtime contingency measure: Select the field with the lowest redundancy to build a new subtree to alleviate local concentration; If the newly split rule meets the Box access conditions (such as the port range is too large or wildcard), it will be automatically transferred to Box management; The essence of this mechanism isn't about adjusting or optimizing the rule set, but rather about adaptively adjusting the system structure to "behavioral anomalies" during runtime. This strategy ensures that the system maintains search path stability and classification throughput performance despite frequent rule evolution, avoiding the interruptions and performance losses associated with full tree rebuilds.

[0086] The measured results show that this mechanism only adjusts the local structure and avoids overall cost reconstruction. It can significantly reduce the update response delay from 200μs to 15μs under a 10,000-level rule set, verifying its runtime scalability and stability.

[0087] The present invention also provides a dynamic data packet classification system based on a dictionary tree and a multi-level decision tree, comprising a construction module, an acquisition module, a division module, a processing module and a replacement module; The construction module is used to construct a dictionary tree for all IP prefixes of rules in the preprocessing stage of packet classification; The acquisition module is used to obtain the prefix big change point based on the dictionary tree; The partitioning module is used to partition the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; The processing module is used to construct the main decision tree for the precise rule subset; the complex rule subset is input into the Box for processing; The replacement module is used to build a shadow tree or shadow box copy of the rule modification operation and perform pointer replacement through the CAS atomic instruction.

[0088] In response to the problems of existing packet classification algorithms (such as CutSplit) when processing large-scale, highly dynamic rule sets, such as serious rule duplication, structural redundancy, lengthy search paths, and low dynamic update efficiency, the present invention proposes an efficient dynamic packet classification method based on dictionary tree preprocessing, dynamic prefix drastic change point recognition mechanism, and combined with the innovative data structure "Box" and update adaptive optimization mechanism, collectively referred to as the "TrieBox" algorithm.

[0089] The present invention effectively breaks through the structural bottlenecks and update problems of the existing technology through the following technical innovations: 1) Proposing an innovative data structure, Box, to uniformly manage rule overflow and wildcard rules, significantly reducing structural redundancy and rule duplication: In traditional decision tree construction, wildcard rules and large-scale rules often lead to cross-path rule duplication, causing structural bloat and memory waste. The Box data structure designed in this invention can be triggered in two scenarios: first, when a wildcard field is encountered during tree construction, the rules are uniformly diverted to the Box; second, when the number of leaf node rules exceeds a threshold, overflow rules are transferred to the Box. The Box internally implements structured division based on the characteristics of the rule fields (e.g., a hierarchical hash structure is used for exact port matching, a Y-FastTrie is used for range port matching, and a DAWG is used for wildcard matching), enabling unified management and efficient search, fundamentally eliminating the problem of rule duplication.

[0090] 2) Introducing a dynamic prefix breakpoint identification mechanism to enable intelligent pre-partitioning of rule sets and hierarchical path optimization: This method analyzes the distribution of rules at different depths within the IP prefix dictionary tree. Combining growth acceleration, KL divergence, and Z-score anomaly detection, it dynamically identifies locations where significant structural changes in the rule distribution occur, defining them as prefix breakpoints. This mechanism intelligently partitions the rule set into subsets more suitable for inclusion in the main tree (precise rules) and those more suitable for Box management (wildcard and large rules). This reduces subsequent path conflicts and redundant replication, effectively improving search efficiency.

[0091] 3) Designing a leaf node update-driven structural resplitting mechanism to enhance the system's dynamic adaptability: To address the localized structural overload caused by continuous rule updates, this paper introduces a leaf node resplitting mechanism driven by two innovative metrics: the Update Surge Rate (USR), which reflects the pressure of rule growth per unit time in real time; and the Local Path Redundancy (LPR), which assesses the degree of overlap in the field positions of the current leaf node rules. When a leaf node exhibits both a high USR and a high LPR during an update, the system determines that the node has reached a structural bottleneck and automatically triggers further path splitting to refine the decision structure and ensure that classification search performance does not degrade with updates.

[0092] 4) Implement localized, structured, incremental updates of rule sets to ensure service continuity and high system availability: This invention supports shadow structure updates and atomic switching mechanisms for both the main decision tree and the Box structure. The addition, deletion, and modification of rules can be completed independently in the background and take effect through instantaneous switching, avoiding the service interruption problem of rebuilding the entire tree during the update process in traditional algorithms, thereby ensuring that the system has the ability to operate continuously and efficiently when facing a dynamic rule environment.

[0093] The present invention does not globally rearrange the rule set or solve the optimal subset structure, nor does it rely on any greedy strategy, overlapping graph construction or integer linear programming. Instead, it dynamically perceives the update pressure and field redundancy of each leaf node through a real-time monitoring mechanism during the system operation phase. When a leaf node experiences rapid rule growth per unit time (USR indicator) and a high degree of rule field overlap (LPR indicator), the system triggers local path re-splitting in a behavior-driven manner. By prioritizing the least redundant fields, the system restructures the bottleneck into a more compact, diversion-oriented structure. This mechanism does not "adjust" or "optimize" the rule set itself, but rather makes evolutionary adjustments to the structural path itself. The purpose is to ensure the search efficiency and update stability in the long-term operation of the system, rather than compressing the rule set structure at one time.

[0094] Compared with existing packet classification technologies (such as decision tree optimization algorithms like CutSplit), the algorithm proposed in this paper achieves the following significant technical effects through a series of structural and behavioral innovations: (1) It fundamentally solves the problems of rule duplication and path redundancy, and significantly optimizes memory structure and space efficiency: When building decision trees, traditional methods often have to copy the same rule to multiple path nodes when encountering wildcard rules or short prefix rules, resulting in structural bloat and memory waste. This invention utilizes an innovative data structure called "Box" to uniformly receive wildcard rules and leaf node rule overflows encountered during the tree building phase, physically isolating the impact of wildcards and avoiding redundant replication across multiple paths. The Box then manages rule fields using specialized structures such as a hierarchical hash structure (for precise port matching), a Y-FastTrie (for port range matching), and a DAWG (for fuzzy wildcards), based on their characteristics. This achieves structural hierarchy compression and optimized search space.

[0095] (2) Through intelligent rule division mechanism and field-aware structure diversion, search efficiency and system scalability are significantly improved: Compared with the traditional fixed threshold division method, the dynamic prefix big change point identification mechanism of the present invention uses formulas to quantitatively analyze the growth acceleration, bit variability and statistical significance of the rule distribution, making the rule division boundary adaptive and mathematically interpretable. According to theoretical derivation, under a typical enterprise-level rule set (generated by ClassBench), this method can reduce the misdivision rate to less than 5% (compared to 15%-20% of the static method). The main decision tree structure constructed in this way is simpler and the rule coverage is clearer, avoiding the bottleneck of structural adaptability of the traditional static division method. At the same time, the structured diversion of rules in Box further accelerates the field matching speed, so that the entire system can still maintain approximately linear search efficiency when facing a million-level rule set, and has good horizontal scalability.

[0096] (3) Support structured and adaptive dynamic updates to ensure the stability and continuity of the system's long-term operation: On the basis of supporting atomic rule switching, this invention further introduces a dynamic splitting mechanism based on quantitative indicators: Update surge rate (USR): monitors the rate at which new leaf node rules are added in real time. When USR > 5.0, it indicates that the burst load exceeds the subtree's processing capacity. Local Path Redundancy (LPR): By calculating the field redundancy, when LPR>0.7, it is determined that the rules are highly homogenized; Collaborative triggering: triggers local subtree splitting only when both USR>5.0 and LPR>0.7 are satisfied. Compared with the global reconstruction method: Update latency reduced by 92% (from 200μs to 15μs, measured with 10,000-level rules); Improved splitting accuracy: false trigger rate <3% (verified by the ClassBench-Enterprise dataset).

[0097] (4) Through the collaborative design of hierarchical hashing and dynamic splitting, efficient storage and real-time response are achieved: Memory efficiency: Box's three-level storage structure (hash table / Y-Fast Trie / DAWG) reduces memory waste by 30%-50%; Query acceleration: Partition pruning reduces the hash collision rate to less than 15%, and the LPR-triggered split strategy reduces the average search path length by 17%; Dynamic adaptability: The USR mechanism ensures 95% query throughput under burst traffic (a 40% decrease compared to traditional methods).

[0098] In summary, the solution of the present invention systematically solves the fundamental problems of traditional packet classification technology in rule replication, search performance, update stability, etc. through the full-link design concept of "structural decoupling, dynamic reconstruction, and field awareness". It has excellent performance of high search throughput, high memory efficiency and strong dynamic adaptability, and is suitable for the core needs of real-time classification capabilities and large-scale rule set management in modern network environments.

[0099] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A dynamic data packet classification method based on a dictionary tree and a multi-level decision tree, characterized in that: include: S1, in the pre-processing stage of packet classification, a dictionary tree is constructed for the IP prefixes of all rules; S2, obtain the prefix big change point based on the dictionary tree; S3, divide the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; S4, builds the main decision tree for the precise rule subset; inputs the complex rule subset into Box for processing; S5, builds a shadow tree or shadow Box copy of the rule modification operation and replaces the pointer through the CAS atomic instruction.

2. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: Build a dictionary tree, including: Each node represents a bit, and the prefix path is constructed from the root downward.

3. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that S2 include: S30, for each depth d, count the number N(d) of unique IP prefixes from the root node to the depth d; S31, calculate the probability distribution of all rules at depth d on the IP address bits , i is the bit; S32, calculate the relative growth acceleration rate A(d); S33, based on Calculate KL divergence; S34, calculate the Z-score value of A(d) and get ; S35, based on A(d), KL divergence and Calculate the mutation significance score S(d); S36: Among all depths where the mutation significance scores are greater than a preset mutation significance score threshold, the depth with the largest mutation significance score is taken as the prefix catastrophic change point.

4. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1, characterized in that S3 include: If the IP prefix length is greater than the mutation significance score and the port range length is less than the preset length and does not contain wildcards and the protocol type is not ANY, the rule is classified into the precise rule subset; otherwise, the rule is classified into the complex rule subset.

5. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: Construct the main decision tree for the exact rule subset, including: During the construction process, the split bits are selected using a bit balance strategy; During the construction process, if the number of rules of a leaf node is greater than the preset threshold, the rules are input into the Box for processing.

6. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: Box's processing of rules includes: For the port field in the rule, if the port field is an exact field, it is managed using a hierarchical hash structure; If the port field is an interval value, Y-FastTrie is used for management; If the port field is a wildcard, the DAWG structure is used to implement fuzzy matching.

7. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: If the rules are dynamically updated, the leaf nodes will be split again based on the structural adaptive mechanism based on the feedback of the running behavior.

8. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: The structural adaptive mechanism based on operational behavior feedback re-splits the leaf nodes, including: calculating the update surge rate and local path redundancy of the leaf nodes; Determine whether the update surge rate and local path redundancy meet the re-split conditions. If so, re-split the leaf node.

9. The dynamic data packet classification method based on dictionary tree and multi-level decision tree according to claim 1 is characterized in that: Refraction conditions include: The update surge rate is greater than a preset update surge rate threshold and the path redundancy is greater than a preset path redundancy threshold.

10. A dynamic data packet classification system based on a dictionary tree and a multi-level decision tree, characterized in that: It includes building module, acquiring module, dividing module, processing module and replacing module; The construction module is used to construct a dictionary tree for all IP prefixes of rules in the preprocessing stage of packet classification; The acquisition module is used to obtain the prefix big change point based on the dictionary tree; The partitioning module is used to partition the rule subsets based on the prefix drastic change points to obtain precise rule subsets and complex rule subsets; The processing module is used to construct the main decision tree for the precise rule subset; the complex rule subset is input into the Box for processing; The replacement module is used to build a shadow tree or shadow box copy of the rule modification operation and perform pointer replacement through the CAS atomic instruction.

Images