A packet classification method based on multiple decision trees
Through the packet classification method of multiple decision trees, by dividing and merging sub-rule sets, the problem of too long creation time and excessive storage resource consumption is solved, and high-speed packet classification and resource optimization are achieved.
Patent Information
- Application Number
- CN202111075302.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-14
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-09-14
AI Technical Summary
The problem of the long creation time of existing decision trees and the excessive consumption of storage resources is mainly due to the ineffective division of multi-level tree nodes caused by rule replication.
Using a packet classification method based on multiple decision trees, the sub-rule sets are divided by the number of wild-gathered users and the sub-rule sets are merged according to the method of creating multiple decision trees, and the sub-rule sets are merged to create multiple decision trees. And perform parallel searches in FPGA, optimize rule replication and storage resource usage.
It significantly reduces the decision tree creation time, reduces RAM usage, reduces logical resources and processing delays, and effectively reduces the number of rule replication and storage consumption when there are many rules.
Smart Images

Figure CN113810311B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of communication technology, and in particular relates to a data packet classification method based on multiple decision trees. Background Art
[0002] The packet classification algorithm is a commonly used algorithm in the network. Its purpose is to classify packets into different types according to the header information carried by the packet, which is generally the five domains of DIP, SIP, SP, DP, and Protocol. In SDN, there are more domains, such as 11, so that they can be processed differently. Common applications include: access control lists, firewalls, and flow-based billing statistics. There are different applications in core network devices and edge network devices. Decision tree is a commonly used algorithm among many packet classification algorithms.
[0003] Figure 1 For example, the following fields represent the source IP address value / source IP mask, destination IP address value / destination IP mask, source port number range, destination port number range, and protocol number value / protocol number mask value. Each rule represents a range, and the rule with a smaller mask represents a larger range.
[0004] In a common decision tree implementation, since each rule represents a larger or smaller scope, when dividing a rule from a parent node to a child node, the same rule may be divided into multiple child nodes; or the scope represented by the rule itself is very small, but multiple rules overlap with each other, resulting in multiple rules not being able to be effectively divided into different nodes after the multi-level tree node division; these two situations are collectively referred to as rule duplication in this patent. Rule duplication can lead to problems such as long decision tree creation time and excessive consumption of storage resources. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] The technical problem to be solved by the present invention is how to provide a data packet classification method based on multiple decision trees to solve the problems of long decision tree creation time and excessive consumption of storage resources.
[0007] (II) Technical solution
[0008] In order to solve the above technical problems, the present invention proposes a data packet classification method based on multiple decision trees, which comprises the following steps:
[0009] Step 1: Divide the user sub-rule set by the number of dimension wildcards; according to the number of dimension wildcards, the corresponding user sub-rule set includes sub-rule set 1, sub-rule set 2, sub-rule set 3, sub-rule set 4 and sub-rule set 5.
[0010] Step 2: Merge the sub-rule sets according to the merging method when creating multiple decision trees. The merging method is: use sub-rule set 1 as the rule set for creating the first decision tree, name the sub-rule set tree2_rules_set1, and name the rule set after merging sub-rule sets 2 and 3 tree2_rules_set2, which is used to create the second decision tree.
[0011] Step 3: Create decision trees according to the rule sets merged in step 2. If a decision tree fails to be created, execute step 6 to release the created decision tree, and then execute step 2 to combine a new rule set using the secondary or alternative solutions. Otherwise, execute step 4.
[0012] Step 4: After multiple decision trees are created successfully, calculate the sum of the RAM occupied by each tree and compare whether it exceeds the threshold; if it exceeds, execute step 6 to release the created decision tree, and then execute step 2 to use the secondary or alternative solutions to combine a new rule set; otherwise, execute step 5;
[0013] Step 5: Send the information of multiple decision trees to FPGA, and the processing of multiple trees is completed.
[0014] Further, rule subset 1 (rules_subset1): contains rules whose number of wildcard dimensions in the user input rule set is 0 or 1;
[0015] Sub-rule set 2 (rules_subset2): contains the rules with 2 wildcard dimensions in the user-input rule set;
[0016] Sub-rule set 3 (rules_subset3): contains the rules with 3 wildcard dimensions in the user-input rule set;
[0017] Sub-rule set 4 (rules_subset4): contains the rules with 4 wildcard dimensions in the user-input rule set;
[0018] Sub-rule set 5 (rules_subset5): contains the rules with 5 wildcard dimensions in the user-input rule set.
[0019] Furthermore, the rules in sub-rule sets 4 and 5 are searched in parallel by the FPGA and do not participate in the creation of the decision tree.
[0020] Furthermore, when the number of decision trees is 2, the merging method in step 2 includes:
[0021] Merge method 1: Use sub-rule set 1 as the rule set for creating the first decision tree, and name the sub-rule set tree2_rules_set1; merge sub-rule sets 2 and 3 into a rule set tree2_rules_set2, which is used to create the second decision tree;
[0022] Merge method 2: Name the rule set formed by merging sub-rule sets 1 and 2 tree2_rules_set1 as the rule set for creating the first decision tree; use sub-rule set 3 to create the second decision tree and name it tree2_rules_set2.
[0023] Furthermore, when the number of decision trees is 3, sub-rule set 1, sub-rule set 2, and sub-rule set 3 are used as input rule sets to create 3 decision trees respectively.
[0024] Furthermore, the secondary or alternative solutions in step 3 and step 4 are three decision trees.
[0025] Furthermore, the method for determining dimension wildcarding is as follows: for the five-tuple rule set: source IP / mask, destination IP / mask, source port number range, destination port number range and protocol number / mask, for the two dimensions of source IP or destination IP, a mask value of 0 is determined as a wildcard; for the two dimensions of source port or destination port, a port range value of 0 to 65535 is determined as a wildcard; for the protocol number, a mask of 0x00 is determined as a wildcard.
[0026] Furthermore, step 5 also includes a message search process.
[0027] Furthermore, for the search process, when querying which rule a data packet matches in the FPGA, parallel searches are performed through multiple decision trees and parallel search modules.
[0028] Furthermore, if multiple rules match the data packet, they need to go through a priority arbitration module and output the rule with the highest priority as the final matching rule.
[0029] (III) Beneficial effects
[0030] The present invention proposes a data packet classification method based on multiple decision trees. By implementing a high-speed packet classification algorithm through multiple decision trees, the following obvious advantages can be obtained:
[0031] 1) By using multiple trees, the time to create a decision tree is greatly reduced, by 3 to 20 times.
[0032] 2) For For rules <10,000, multiple trees will slightly increase the FPGA's on-chip RAM usage.For a rule number of 10,000, multiple trees can significantly reduce RAM usage by 2 / 3. This is because the more rules there are, the more trees there are, which can effectively reduce the number of rule duplications.
[0033] 3) Using the multiple-tree approach, the number of large rules that need to be processed separately is less than 284, which is within the range that the FPGA can handle.
[0034] 4) Using multiple trees, the tree height is slightly reduced, which can reduce the corresponding logic resources and processing delay. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 For example of rules;
[0036] Figure 2 is a flow chart of the present invention;
[0037] Figure 3 is an example of a rule set. DETAILED DESCRIPTION
[0038] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0039] Through in-depth research on the principles of decision trees and statistical analysis of a large amount of test data, it is found that by dividing the user's original rule set into multiple sub-rule sets according to the number of wildcard dimensions, and then using multiple sub-rule sets to create multiple decision trees, the total tree building time and storage resource consumption are significantly improved.
[0040] The principle is: during the decision tree creation process, each parent node obtains the specified dimension as the division dimension based on the rule characteristics it contains; there will be multiple wildcard rules in the original rule set on the obtained dimension. Based on this, when dividing the rules from the parent node to the child node, it will cause serious rule duplication.
[0041] In the rule set with practical significance, the rules with small number of wildcard dimensions account for a large proportion; the rules with large number of wildcard dimensions account for a small proportion. The principle of this solution is to separate the rules with small number of wildcard dimensions from the rules with large number of wildcard dimensions, reduce the number of copies of the rules with large number of wildcard dimensions during the decision tree creation process, and thus reduce the total decision tree creation time and storage resource consumption.
[0042] Method for determining dimension wildcards: For the five-tuple rule set (source IP / mask, destination IP / mask, source port number range, destination port number range, protocol number / mask), for the two dimensions of source IP or destination IP, if it is a mask mode, a mask value of 0 is considered wildcard. For the two dimensions of source port or destination port, a port range value of 0 to 65535 is considered wildcard. For the protocol number, if it is a mask mode, a mask of 0x00 is considered wildcard. First, divide the rules in the rule set entered by the user into the following sub-rule sets according to the number of wildcard dimensions:
[0043] 1) Sub-rule set 1 (rules_subset1): Contains the rules in the user-input rule set whose number of wildcard dimensions is 0 or 1.
[0044] 2) Sub-rule set 2 (rules_subset2): Contains the rules with 2 wildcard dimensions in the user-input rule set.
[0045] 3) Sub-rule set 3 (rules_subset3): contains the rules with 3 wildcard dimensions in the user-input rule set.
[0046] 4) Sub-rule set 4 (rules_subset4): contains the rules with 4 wildcard dimensions in the user-input rule set.
[0047] 5) Sub-rule set 5 (rules_subset5): contains the rules with 5 wildcard dimensions in the user-input rule set.
[0048] Without considering the FPGA logic resources and bandwidth, the above five sub-rule sets rules_subset1 to rules_subset5 can be used to create five decision trees respectively. However, after creating multiple decision trees in the CPU, the hardware needs to search multiple trees in parallel when searching for data packets based on the decision tree information. As the number of decision trees increases, the total scale of the search logic will increase, and the bandwidth demand will increase exponentially. From the perspective of bandwidth requirements and possible performance improvements, the number of decision trees is not the more the better. In this embodiment, according to the actual test process, based on the tree building time and FPGA logic resource occupancy data of multiple decision trees of different numbers, the number of multiple decision trees is limited to 2 or 3. Therefore, the five small rule sets rules_subset1 to rules_subset5 divided according to the number of dimensional wildcards need to be merged. Finally, a variety of combinations of multiple trees are sorted out, as shown in Table 1:
[0049] Table 1 Combination scheme of two or three trees
[0050]
[0051] Since the number of rules in the sub-rule sets rules_subset4 and rules_subset5 is very small, and the rules in them all represent a large wildcard range, which is not conducive to creating a decision tree, the rules in these two sub-rule sets are searched in parallel by FPGA and do not participate in the creation of the decision tree.
[0052] When there are two decision trees, the following merging methods can be used:
[0053] 1) Merge method 1: Use the sub-rule set rules_subset1 as the rule set for creating the first decision tree. For ease of description, the sub-rule set is named tree2_rules_set1. The rule set formed by merging the sub-rule sets rules_subset2 and rules_subset3 is named tree2_rules_set2, which is used to create the second decision tree.
[0054] 2) Merge method 2: The rule set after merging sub-rule sets rules_subset1 and rules_subset2 is named tree2_rules_set1, which is used as the rule set for creating the first decision tree. The sub-rule set rules_subset3 is used to create the second decision tree. For the convenience of description, the sub-rule set is named tree2_rules_set2.
[0055] By comprehensively comparing the tree building time and the size of FPGA logic resources occupied, when the number of decision trees is 2, merging method 1 is better than merging method 2.
[0056] When there are three decision trees, use rules_subset1, rules_subset2 and rules_subset3 as input rule sets to create three decision trees.
[0057] After comprehensively comparing the tree building time and the size of FPGA logic resources occupied, two decision trees are selected as the first choice of multiple tree combination, and three decision trees are selected as the second choice of multiple tree combination.
[0058] The overall processing flow of multiple decision trees is as follows: Figure 2 As shown:
[0059] The following is Figure 2 The processing flow and main modules in the following are explained step by step:
[0060] Step 1: Divide the rule set entered by the user into multiple user sub-rule sets according to the number of dimension wildcards. Figure 3 The example rule set in .
[0061] Figure 1 The meaning of each field in the rule has been explained, so I will not repeat it here. In the above figure, only the source port number is wildcarded in rules 1, 4, 5, and 6, so rule 1 is divided into rs1; the five dimensions of rules 2 and 3 are not wildcarded, so they are divided into rs1; the source IP address and source port number of rule 7 are wildcarded, so it is divided into rs2; the source IP address, destination IP address, and source port number of rule 8 are wildcarded, so it is divided into rs3; the source IP address, destination IP address, source port number, and destination port number of rule 9 are wildcarded, so it is divided into rs4; the five dimensions of rule 10 are all wildcarded, so it is divided into rs5.
[0062] The above rule division results are sorted out to obtain the following table:
[0063] Table 2 Rule division results
[0064]
[0065]
[0066] For a rule set with a larger capacity, such as one containing tens of thousands or even hundreds of thousands of rules, the same method is used to divide the original rule set into five sub-rule sets.
[0067] Step 2: According to the above-mentioned method 1 for merging sub-rule sets when creating multiple decision trees, merge the sub-rule sets.
[0068] Specifically, the number of trees of the multiple trees is 2. The first tree uses the sub-rule set rules_subset1 as the rule set when creating the decision tree. The second tree uses the rule set obtained by merging the sub-rule sets rules_subset2 and rules_subset3 as the rule set for creating the decision tree.
[0069] The process of creating a decision tree using a sub-rule set is the same as the process of creating a decision tree using the user's original rule set. Since this patent mainly describes the reasons for using multiple trees and the construction method of the sub-rule set required to create multiple trees, how to use the sub-rule set to create a decision tree is not described in detail.
[0070] Step 3: Create decision trees according to the rule sets merged in step 2. If a decision tree fails to be created, execute step 6 to release the created decision tree, and then execute step 2 to use the secondary or alternative solutions to combine a new rule set; otherwise, execute step 4.
[0071] Step 4: After multiple decision trees are created successfully, calculate the sum of RAM occupied by each tree and compare whether it exceeds the threshold. If it exceeds, execute step 6 to release the created decision tree, and then execute step 2 to use the secondary or alternative solutions to combine a new rule set; otherwise, execute step 5.
[0072] Step 5: Send multiple decision tree information to FPGA. At this point, the processing of multiple trees is completed.
[0073] For the search process, when querying which rule a data packet matches in the FPGA, multiple decision trees and parallel search modules are used for parallel search. If multiple rules match the data packet, it needs to go through the priority arbitration module, and the rule with the highest priority is output as the final matching rule. This patent gives an example process in the FPGA, which only explains how to form a corresponding relationship with the search module in the hardware after the multiple decision trees are created, and no detailed description is given.
[0074] Implementing a high-speed packet classification algorithm using multiple decision trees can provide the following significant advantages:
[0075] 1) By using multiple trees, the time to create a decision tree is greatly reduced, by 3 to 20 times.
[0076] 2) For For rules <10,000, multiple trees will slightly increase the FPGA's on-chip RAM usage. For a rule number of 10,000, multiple trees can significantly reduce RAM usage by 2 / 3. This is because the more rules there are, the more trees there are, which can effectively reduce the number of rule duplications.
[0077] 3) Using the multiple-tree approach, the number of large rules that need to be processed separately is less than 284, which is within the range that the FPGA can handle.
[0078] 4) Using multiple trees, the tree height is slightly reduced, which can reduce the corresponding logic resources and processing delay.
[0079] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A data packet classification method based on multiple decision trees, characterized in that: The method comprises the following steps: Step 1: Divide the rule set input by the user into multiple sub-rule sets according to the number of dimension wildcards; according to the number of dimension wildcards, the corresponding user sub-rule sets include sub-rule set 1, sub-rule set 2, sub-rule set 3, sub-rule set 4 and sub-rule set 5. Step 2: Merge the sub-rule sets according to the merging method when creating multiple decision trees. The merging method is: use sub-rule set 1 as the rule set for creating the first decision tree, name the sub-rule set tree2_rules_set1, and name the rule set after merging sub-rule sets 2 and 3 tree2_rules_set2, which is used to create the second decision tree. Step 3: Create decision trees according to the rule sets merged in step 2. If a decision tree fails to be created, execute step 6 to release the created decision tree, and then execute step 2 to combine a new rule set using the secondary or alternative solutions. Otherwise, execute step 4. Step 4: After multiple decision trees are created successfully, calculate the sum of RAM occupied by each tree and check whether it exceeds the threshold; If it exceeds, execute step 6 to release the created decision tree, and then execute step 2 to combine a new rule set using the secondary or alternative solutions; otherwise, execute step 5; Step 5: Send the information of multiple decision trees to FPGA, and the processing of multiple trees is completed; in, Sub-rule set 1 (rules_subset1): contains the rules with 0 or 1 wildcard dimensions in the user-input rule set; Sub-rule set 2 (rules_subset2): contains the rules with 2 wildcard dimensions in the user-input rule set; Sub-rule set 3 (rules_subset3): contains the rules with 3 wildcard dimensions in the user-input rule set; Sub-rule set 4 (rules_subset4): contains the rules with 4 wildcard dimensions in the user-input rule set; Sub-rule set 5 (rules_subset5): contains the rules with 5 wildcard dimensions in the user-input rule set; The rules in sub-rule sets 4 and 5 are searched in parallel by the FPGA and do not participate in the creation of the decision tree; When there are three decision trees, three decision trees are created using sub-rule set 1, sub-rule set 2, and sub-rule set 3 as input rule sets respectively; The secondary or alternative solutions in step 3 and step 4 are three decision trees.
2. The data packet classification method based on multiple decision trees as claimed in claim 1, characterized in that: When the number of decision trees is 2, the merging methods in step 2 include: Merge method 1: Use sub-rule set 1 as the rule set for creating the first decision tree, and name the sub-rule set tree2_rules_set1; merge sub-rule sets 2 and 3 into a rule set tree2_rules_set2, which is used to create the second decision tree; Merge method 2: Name the rule set formed by merging sub-rule sets 1 and 2 tree2_rules_set1 as the rule set for creating the first decision tree; use sub-rule set 3 to create the second decision tree and name it tree2_rules_set2.
3. The data packet classification method based on multiple decision trees according to any one of claims 1 to 2, characterized in that: Method for determining dimension wildcards: For the five-tuple rule set: source IP / mask, destination IP / mask, source port number range, destination port number range and protocol number / mask, for the two dimensions of source IP or destination IP, a mask value of 0 is considered wildcarded; for the two dimensions of source port or destination port, a port range value of 0 to 65535 is considered wildcarded; for the protocol number, a mask of 0x00 is considered wildcarded.
4. The data packet classification method based on multiple decision trees as claimed in claim 3, characterized in that: The step 5 also includes a message search process.
5. The data packet classification method based on multiple decision trees as claimed in claim 4, characterized in that: For the search process, when querying which rule a data packet matches in the FPGA, parallel searches are performed through multiple decision trees and parallel search modules.
6. The data packet classification method based on multiple decision trees as claimed in claim 5, characterized in that: If multiple rules match the data packet, they need to go through the priority arbitration module and output the rule with the highest priority as the final matching rule.
