A network packet classification method based on field distribution entropy and coverage difference

Abstract

The present application relates to the field of computer network security technology. The present application provides a network packet classification method based on field distribution entropy and coverage difference, including: obtaining a rule set for a network data packet; dividing the rule set into multiple levels, selecting a rule set with a high optimization level as a candidate rule set; traversing multiple bit segments in the candidate rule set and performing a comprehensive score; selecting the best partitioned bit segment based on the comprehensive score and constructing a tuple hash bucket; establishing a hash table based on the actual value length of the best partitioned bit segment, and distributing the rules to the corresponding tuple hash bucket; traversing each tuple hash bucket for judgment and recursive processing; when the rules of all tuple hash buckets are effectively divided, the tuple hash subtree of the current node is constructed, and the process of rolling back to the previous level or completing the construction of the entire tree is completed. The present application addresses the deficiencies of packet classification technology in terms of search efficiency, structural balance, space utilization, and rule update support.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer network security, and in particular relates to a network packet classification method based on field distribution entropy and coverage difference. Background Art

[0002] Packet classification is a core function of network devices such as firewalls, routers, SDN switches, and traffic monitoring systems. Its mission is to quickly find the highest-priority matching rule within a set of tens of millions of rules. With the growth of network traffic and increasing security requirements, classification algorithms are facing increasing demands for higher search performance, memory efficiency, and rule update speed.

[0003] In the early days, the most straightforward approach was linear search, which prioritizes rules and matches them one by one until the first matching rule is found. While this approach is simple to implement, its search time is proportional to the number of rules, making it difficult to meet high-performance requirements. Subsequently, the Trie multi-layer structure algorithm emerged, which performs a hierarchical search based on field dimensions (for example, matching source IP, destination IP, port, and protocol in sequence). This approach improves search speed, but for rule sets containing a large number of wildcards, it suffers from frequent backtracking, high storage consumption, and complex updates. Other research has proposed decision-cutting algorithms (such as HiCuts and HyperCuts), which partition the multidimensional space into multiple subregions, reducing the number of rules in each region and thus accelerating searches. These algorithms generally provide faster searches but require a large amount of memory to store the indexes of the subspaces after the cuts. Another approach is tuple space and hash queries (such as TSS and PTSS). These algorithms distribute rules across multiple tuples and hash tables for parallel search, supporting precise matches and fast updates. However, the presence of many wildcards can cause hash conflicts and uneven bucketing, affecting efficiency.

[0004] In addition, some studies have attempted clustering-assisted Trie construction (such as HTEMC, distributed cutting, EM clustering, single-link clustering, etc.), which constructs a more compact Trie with path compression by statistically dividing the rules, thereby reducing backtracking and improving update capabilities.

[0005] Overall, current mainstream methods each have their own advantages, but also share common bottlenecks: poor handling of wildcards or unbalanced field spaces; a corresponding trade-off between lookup time and memory consumption; and limited support for online update mechanisms (especially Trie-like structures). Summary of the Invention

[0006] In view of the above-mentioned shortcomings of the existing technology, the present invention proposes a network packet classification method based on field distribution entropy and coverage difference, which is used to address the shortcomings of packet classification technology in search efficiency, structural balance, space utilization and rule update support, as well as the problems of matching anomalies and structural instability when processing wildcard fields. The technical solution designed by the present invention includes the following steps:

[0007] S10: Obtaining a rule set for network data packets;

[0008] S20: Divide the rule set into multiple levels, and select the rule set with a high optimization level as the candidate rule set;

[0009] S30: traverse multiple bit segments in the candidate rule set and perform comprehensive scoring;

[0010] S40: Select the best partition segment based on the comprehensive score and construct a tuple hash bucket;

[0011] S50: Building a hash table based on the actual value length of the optimally divided bit segment, and distributing the rules to the corresponding tuple hash buckets;

[0012] S60: traverse each tuple hash bucket to perform judgment and recursive processing;

[0013] S70: When the rules of all tuple hash buckets are effectively divided, the tuple hash subtree of the current node is constructed, and the construction process of the previous layer or the entire tree is completed.

[0014] Preferably, the S10 includes:

[0015] The firewall system first receives or intercepts the network data packet to be classified, extracts the five-tuple field of the network data packet, including source IP, destination IP, source port, destination port and protocol type, and provides a feature basis for subsequent rule matching.

[0016] Preferably, the step S20 includes the following steps:

[0017] S2001: Each level of the rule set independently constructs the corresponding tuple space hash tree structure;

[0018] S2002: Extract the partitioning strategy from the current node. Each non-leaf node records two partitioning field segments.

[0019] S2003: extracting the actual prefix matching length of the segment of the division field of the record from the data packet, and constructing a hash key for locating the bucket of the hash table of the current layer;

[0020] S2004: Search the bucket and determine the node type. If the key is not matched, skip the current level and proceed to the next priority rule set. If the key matched is a leaf node, traverse the rules in the bucket and perform a match based on the five-tuple field, supporting wildcards. If the key matched is a hash node, segment the bits according to the next level of the record, return to step S2002, and recursively continue matching in the sub-hash table.

[0021] S2005: Each time a matching rule is found, it is added to the candidate rule set.

[0022] Preferably, the comprehensive scoring in S30 includes:

[0023] Calculate the distribution entropy and coverage difference, and output a comprehensive score based on the distribution entropy and coverage difference.

[0024] Preferably, the distribution entropy is expressed as follows:

[0025]

[0026] Where, is the distribution entropy, For all possible values ​​of the bit segment s, is the proportion of the rule set that is mapped to the value v.

[0027] Preferably, the coverage difference is expressed as follows:

[0028]

[0029] Where, To cover the difference, is all possible values ​​of the bit segment s.

[0030] Preferably, the comprehensive scoring formula is as follows:

[0031]

[0032] Where, is a comprehensive scoring function for each fine-grained segment s, is the entropy weight, is the distribution entropy, is the penalty weight of the conflict, To cover the difference.

[0033] Preferably, the S60 includes:

[0034] Traverse each tuple hash bucket. If the number of rules in the bucket is less than or equal to binth, build the bucket as a leaf node. If the number of rules is still greater than binth, recursively call the partitioning process to further subdivide the rules in the bucket.

[0035] Beneficial effects:

[0036] This application proposes a network packet classification method based on field distribution entropy and coverage difference. Through the joint scoring of entropy + difference, the tuple partitioning strategy of double-field bit segment combination, the priority hierarchical mechanism, the local recursive construction and the dynamically maintainable data structure, it is superior to the existing technology in terms of search speed, structural compactness, wildcard compatibility and scalability, and has good engineering practicality and deployment prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of a preferred embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of field bit segment selection according to a preferred embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of hash structure splitting and subtree recursive construction in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are described in detail below. The following embodiments are implemented based on the technical solutions of the present invention, and provide detailed implementation methods and specific operating procedures. However, the protection scope of the present invention is not limited to the following embodiments.

[0041] The present invention designs a network packet classification method based on field distribution entropy and coverage difference. The technical solution includes the following steps, specifically including:

[0042] S10: Obtaining a rule set for network data packets;

[0043] S20: Divide the rule set into multiple levels, and select the rule set with a high optimization level as the candidate rule set;

[0044] S30: traverse multiple bit segments in the candidate rule set and perform comprehensive scoring;

[0045] S40: Select the best partition segment based on the comprehensive score and construct a tuple hash bucket;

[0046] S50: Building a hash table based on the actual value length of the optimally divided bit segment, and distributing the rules to the corresponding tuple hash buckets;

[0047] S60: traverse each tuple hash bucket to perform judgment and recursive processing;

[0048] S70: When the rules of all tuple hash buckets are effectively divided, the tuple hash subtree of the current node is constructed, and the construction process of the previous layer or the entire tree is completed.

[0049] Preferably, S10 includes:

[0050] The firewall system first receives or intercepts the network data packet to be classified, extracts the five-tuple field of the network data packet, including source IP, destination IP, source port, destination port and protocol type, and provides a feature basis for subsequent rule matching.

[0051] Specifically, a rule set is a set of rules that defines how to process network packets. The rule set is the core input of the packet classification algorithm, and the algorithm decides how to process each packet based on the rule set.

[0052] The design or acquisition of a rule set requires consideration of factors such as network policy, security requirements, and traffic management objectives. For example, a firewall rule set might contain rules to allow or disallow specific types of traffic, while a traffic management rule set might prioritize certain types of packets to ensure quality of service.

[0053] Preferably, S20 includes the following steps:

[0054] S2001: Each level of the rule set independently constructs the corresponding tuple space hash tree structure;

[0055] S2002: Extract the partitioning strategy from the current node. Each non-leaf node records two partitioning field segments.

[0056] S2003: extracting the actual prefix matching length of the segment of the division field of the record from the data packet, and constructing a hash key for locating the bucket of the hash table of the current layer;

[0057] S2004: Search the bucket and determine the node type. If the key is not matched, skip the current level and proceed to the next priority rule set. If the key matched is a leaf node, traverse the rules in the bucket and perform a match based on the five-tuple field, supporting wildcards. If the key matched is a hash node, segment the bits according to the next level of the record, return to step S2002, and recursively continue matching in the sub-hash table.

[0058] S2005: Each time a matching rule is found, it is added to the candidate rule set.

[0059] Specifically, for S20, based on a rule priority hierarchy mechanism, the rule set is divided into multiple priority levels according to system configuration or policy. The classification process begins with the highest-priority rule level and proceeds downwards. A recursive tuple hash tree is independently constructed for each priority level to efficiently match the rules at that level. Rule level 1 matches critical rules with priorities between 0 and 500; rule level 2 matches medium-priority rules (500 to 1000); and rule level 3 matches low-priority rules (1000+). For example, the system first matches the hash tuple tree structure for rules with priorities between 0 and 500. The current node records 4-bit segments from two different fields (e.g., segment 3 X3 of the source IP address and segment 2 Y2 of the destination IP address). The system extracts the prefix lengths of the corresponding segments from the packet, constructs a tuple key (lenX, lenY) based on these two prefix lengths, and searches the current hash table for the corresponding tuple bucket. If a match is not found, the node is skipped. All successfully matched rules are added to the "candidate rule set" in descending priority order (a maximum of 5 rules are stored). Each layer of the structure adopts a joint scoring mechanism of "field distribution entropy + coverage difference", selects two field bit segments to construct a two-dimensional tuple hash table, and uses the prefix length pair of the rule on the bit segment as the key to achieve path-unique recursive classification.

[0060] Preferably, the comprehensive scoring in S30 includes:

[0061] Calculate the distribution entropy and coverage difference, and output a comprehensive score based on the distribution entropy and coverage difference.

[0062] Preferably, the distribution entropy is given by the following formula:

[0063]

[0064] Where, is the distribution entropy, For all possible values ​​of the bit segment s, is the proportion of the rule set that is mapped to the value v.

[0065] Preferably, the coverage difference is expressed as follows:

[0066]

[0067] Where, To cover the difference, is all possible values ​​of the bit segment s.

[0068] Preferably, the comprehensive scoring formula is as follows:

[0069]

[0070] Where, is a comprehensive scoring function for each fine-grained segment s, is the entropy weight, is the distribution entropy, is the penalty weight of the conflict, To cover the difference.

[0071] Specifically, for S30, the rule set is input to the field segment extractor for implementation, such as Figure 2 The figure shows the scoring process for multiple fields and the multiple bit segments within them. For each bit segment, distribution entropy and coverage difference are calculated to produce a comprehensive score. All bit segments are then entered into a scoring pool for comparison, and the highest-scoring bit segment (which can be set to two or more, with two being used as an example) is selected as the basis for classification at that level. Multiple candidate bit segments within the rule field (such as bits 5-8 of the source IP address) are traversed, and each bit segment is evaluated. Distribution entropy measures the uniformity of value distribution, indicating whether the rules within that bit segment are evenly distributed. Coverage difference assesses the impact of wildcards or range values ​​on classification conflicts, measuring whether there is significant "overlap" (wildcarding or redundant) in the rules contained within each value. Ideally, more rules should "exactly match" the segment value, rather than * (wildcards). If all rules are wildcards, this bit is not suitable for hashing, as all rules are in the same bucket.

[0072] Preferably, S60 includes:

[0073] Traverse each tuple hash bucket. If the number of rules in the bucket is less than or equal to binth, build the bucket as a leaf node. If the number of rules is still greater than binth, recursively call the partitioning process to further subdivide the rules in the bucket.

[0074] Specifically, such as Figure 3 The following figure illustrates the multi-layer tuple hash partitioning process based on field segments in the rule classification structure. The system first extracts field segments (such as X3 and Y2) from high-priority rules (e.g., those with a priority of 0–500) using a partitioning selection algorithm based on entropy and coverage differences to construct a two-dimensional tuple hash table. If the number of rules in a bucket (e.g., Tuple(4,4)) exceeds the preset threshold binth, the bit segment discrimination of the rules in that bucket is re-evaluated (e.g., selecting P2 and D3) to construct the next sub-hash table. If there are still buckets (e.g., Tuple(4,3)) that exceed binth, recursive partitioning continues (e.g., using X5 and the protocol field Proto1). Ultimately, all rules are distributed to leaf buckets with a rule count ≤ binth, achieving an on-demand, structurally balanced packet indexing system.

[0075] The above describes in detail the preferred embodiments of the present invention. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible by those skilled in the art without inventive effort. Therefore, any technical solution that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A network packet classification method based on field distribution entropy and coverage difference, characterized in that: include: S10: Obtaining a rule set for network data packets; S20: Divide the rule set into multiple levels, and select the rule set with a high optimization level as the candidate rule set; S30: traverse multiple bit segments in the candidate rule set and perform comprehensive scoring; S40: Select the best partition segment based on the comprehensive score and construct a tuple hash bucket; S50: Building a hash table based on the actual value length of the optimally divided bit segment, and distributing the rules to the corresponding tuple hash buckets; S60: traverse each tuple hash bucket to perform judgment and recursive processing; S70: When the rules of all tuple hash buckets are effectively divided, the tuple hash subtree of the current node is constructed, and the construction process of the previous layer or the entire tree is completed.

2. A network packet classification method based on field distribution entropy and coverage difference according to claim 1, characterized in that: The S10 includes: The firewall system first receives or intercepts the network data packet to be classified, extracts the five-tuple field of the network data packet, including source IP, destination IP, source port, destination port and protocol type, and provides a feature basis for subsequent rule matching.

3. The network packet classification method based on field distribution entropy and coverage difference according to claim 1 is characterized in that: The S20 includes the following steps: S2001: Each level of the rule set independently constructs the corresponding tuple space hash tree structure; S2002: Extract the partitioning strategy from the current node. Each non-leaf node records two partitioning field segments. S2003: extracting the actual prefix matching length of the segment of the division field of the record from the data packet, and constructing a hash key for locating the bucket of the hash table of the current layer; S2004: Search the bucket and determine the node type. If the key is not matched, skip the current level and proceed to the next priority rule set. If the key matched is a leaf node, traverse the rules in the bucket and perform a match based on the five-tuple field, supporting wildcards. If the key matched is a hash node, segment the bits according to the next level of the record, return to step S2002, and recursively continue matching in the sub-hash table. S2005: Each time a matching rule is found, it is added to the candidate rule set.

4. The network packet classification method based on field distribution entropy and coverage difference according to claim 1 is characterized in that: The comprehensive score in S30 includes: Calculate the distribution entropy and coverage difference, and output a comprehensive score based on the distribution entropy and coverage difference.

5. The network packet classification method based on field distribution entropy and coverage difference according to claim 4 is characterized in that: The distribution entropy is formulated as follows: Where, is the distribution entropy, For all possible values ​​of the bit segment s, is the proportion of the rule set that is mapped to the value v.

6. The network packet classification method based on field distribution entropy and coverage difference according to claim 4 is characterized in that: The coverage difference is expressed as follows: Where, To cover the difference, is all possible values ​​of the bit segment s.

7. The network packet classification method based on field distribution entropy and coverage difference according to claim 4 is characterized in that: The comprehensive score is calculated as follows: Where, is a comprehensive scoring function for each fine-grained segment s, is the entropy weight, is the distribution entropy, is the penalty weight of the conflict, To cover the difference.

8. The network packet classification method based on field distribution entropy and coverage difference according to claim 1 is characterized in that: The S60 includes: Traverse each tuple hash bucket. If the number of rules in the bucket is less than or equal to binth, build the bucket as a leaf node. If the number of rules is still greater than binth, recursively call the partitioning process to further subdivide the rules in the bucket.

Images