A fast strategy matching method based on VPP
The method addresses inefficient policy matching by combining HASH and binary search to achieve O(1) time complexity, enhancing VPP's data flow forwarding efficiency through policy priority uniqueness and 64-bit CPU optimization.
Patent Information
- Application Number
- CN202211683895.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-27
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-12-27
AI Technical Summary
Conventional policy matching methods in firewalls, such as ACL, QOS, NAT, and user authentication, are inefficient when the number of policies exceeds a certain threshold, leading to slowed down data forwarding processes due to insufficient policy matching speed.
A fast strategy matching method using a combination of HASH and binary search algorithms, with policy priority uniqueness and optimization for 64-bit CPUs, reduces time complexity from O(log2N) to O(1) by generating HASH matching tables and sorting hit counts, ensuring efficient data flow forwarding.
The method significantly improves the forwarding efficiency of VPP by reducing search time complexity and optimizing policy matching, especially for high-speed data streams, by utilizing a 64-bit CPU to process 64 bits at a time and employing HASH matching for high-speed flows.
Smart Images

Figure CN116192745B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer networks, and in particular relates to a fast strategy matching method based on VPP. Background Art
[0002] Nowadays, firewalls use policy matching in multiple functions, such as ACL, QOS, NAT, user authentication, blacklists, and other functions. At the same time, policy matching is user configuration, and the priorities or policies between various functional modules are different. In this way, the same data needs to match the policy multiple times in a row. If the policy matching function is not fast enough, it will affect the forwarding efficiency of the entire system. Conventional priority policy matching functions match policies in the order of high and low priorities from top to bottom. When the number is small, it is not obvious. When the number of policies exceeds a certain threshold, it will cause the entire data forwarding process to slow down. Therefore, a more efficient policy matching method is needed to meet this demand.
[0003] VPP itself is an efficient flow forwarding framework, and we do not want these added matching strategies to significantly reduce its forwarding performance. As we all know, the fastest algorithm can be measured based on time complexity. O(1) means that the HASH algorithm is the fastest, but we know that HASH cannot be used for range search. Binary search is a simple and fast method in the search method, with a time complexity of O(log2N). Secondly, a 32-bit processor can only process 32 bits, or 4 bytes of data at a time; while a 64-bit processor can process 64 bits, or 8 bytes of data at a time. Except for the IPV6 address, multiple fields such as IPV4 address, protocol, port, etc. can be represented by 32-bit integers. Through conversion, when comparing, 2 32-bit integers can be converted to 1 64-bit integer. By reducing the number of comparisons, the search time can be further reduced. Summary of the invention
[0004] The purpose of the present invention is to provide a fast strategy matching method based on VPP, aiming to solve the above problems.
[0005] The present invention is mainly achieved through the following technical solutions:
[0006] A fast policy matching method based on VPP processes input multi-tuple policy data, uniqueizes policy priorities and corresponding policies, and performs policy search. First, a HASH algorithm is used to search. If the search is successful, the priority is returned. Otherwise, a binary search method is used to search. If the search is successful, the N-tuple hit counts are counted to generate a HASH matching table, and M HASH matching tables with higher hit counts are generated according to the hit count sorting to determine the data flow of the high-speed flow. The hit count table is updated according to the set time, and finally the priority is returned. When the next data flow comes, the HASH algorithm is used to search.
[0007] In order to better implement the present invention, further, in the first search, if there is no HASH value or statistical value, then a binary search is performed and statistics are performed. After the T time is reached, a HASH table is generated, and the subsequent data streams can be HASH matched. If the HASH match fails, it is a new data stream and a binary strategy match is performed.
[0008] In order to better implement the present invention, further, the process of actually generating the HASH table in VPP is not in the forwarding thread, but in the main thread. This is to prevent the forwarding from being affected by the process of generating the HASH table. The generated HASH table has a main table and a backup table. The forwarding thread uses the main table. When the HASH table is regenerated, the main thread switches the flag bit, and the forwarding thread switches the main table and the backup table.
[0009] In order to better implement the present invention, further, based on the binary search method, a 64-bit CPU is used to optimize the number of comparisons, and the number of statistical data flow matches is counted and sorted from high to low according to the statistical number of times, and the HASH algorithm is used when the next data flow comes.
[0010] In order to better implement the present invention, further, when a configuration policy is added, it is determined whether a configuration policy with the same priority already exists. If not, it is further searched by binary search to determine whether the same configuration policy already exists. If not, the configuration policy priority table and the configuration policy table are saved, and the configuration policy priority table and the configuration policy table are sorted. The policy with the same priority or the same configuration data is only added once; when a configuration policy is deleted, a matching policy is searched by binary search to determine whether there is a corresponding policy. If so, the configuration policy priority is deleted, the configuration policy is deleted, and the configuration policy priority table and the configuration policy table are re-sorted; when a configuration policy is modified, it is determined whether the deletion of the configuration policy is successful. If successful, the addition of the configuration policy is executed.
[0011] In order to better implement the present invention, further, when the search strategy is generated, the configuration strategy is first sorted according to the priority to determine whether it is the first configuration strategy. If so, it is directly inserted and the search configuration strategy is generated; otherwise, the remaining configuration strategies need to be compared with all the search strategies. If the search strategy completely includes the configuration strategy, it means that the high-priority strategy includes the low-priority strategy. This policy should be matched at a high priority, so this policy is not added to the search strategy; if the search strategy does not include the configuration strategy at all, this strategy can be directly added to the search strategy; if the search strategy and the configuration strategy are partially included, it is necessary to remove the duplicate data segments and then insert the search strategy.
[0012] In order to better implement the present invention, further, in policy uniqueness, the high priority policy is always used as the basis. If the low priority policy and the high priority policy are repeated, the low priority policy will be deleted from the policy table; if the high priority policy contains the low priority policy, the low priority policy will be deleted from the policy table; if the high priority policy contains part of the low priority policy, the included low priority policy part will be deleted from the policy table; if the low priority policy contains the high priority policy, the low priority policy will be truncated into two parts by the high priority policy; if the low priority policy contains part of the high priority policy, the low priority policy will be partially truncated.
[0013] The present invention reduces the time complexity from O(log2N) to O(1), thereby achieving fast matching of priority strategies in a multi-dimensional range, effectively improving the forwarding efficiency of VPP. Compared with the time complexity O(n) of sequential strategy matching, the time complexity is reduced to O(log2n) by ordering and unique multi-dimensional priority strategies, and the search time is further reduced by using a 64-bit CPU to process 64 bits at a time. Through a large number of high-speed streams for HASH matching, the time complexity is reduced to O(1), thereby speeding up the speed of strategy matching and significantly improving the system's high-speed data flow forwarding when multiple strategies are applied. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 Added flowchart for configuration strategy;
[0015] Figure 2 Flowchart for configuration policy deletion;
[0016] Figure 3 Flowchart for configuration policy modification;
[0017] Figure 4 Flowcharts generated for the search strategy;
[0018] Figure 5 Flowchart for strategy search;
[0019] Figure 6 Flowchart updated for hit count table. DETAILED DESCRIPTION
[0020] Embodiment 1:
[0021] A fast policy matching method based on VPP processes input multi-tuple policy data, uniqueizes policy priorities and corresponding policies, and performs policy search. First, a HASH algorithm is used to search. If the search is successful, the priority is returned. Otherwise, a binary search method is used to search. If the search is successful, the N-tuple hit counts are counted to generate a HASH matching table, and M HASH matching tables with higher hit counts are generated according to the hit count sorting to determine the data flow of the high-speed flow. The hit count table is updated according to the set time, and finally the priority is returned. When the next data flow comes, the HASH algorithm is used to search.
[0022] Preferably, in the first search, if there is no HASH value or statistical value, then a binary search is performed and statistics are performed. After the T time is reached, a HASH table is generated, and the subsequent data streams can be HASH matched. If the HASH match fails, it is a new data stream and a binary strategy match is performed.
[0023] Preferably, the actual process of generating the HASH table in VPP is not in the forwarding thread, but in the main thread. This is to prevent the forwarding from being affected by the process of generating the HASH table. The generated HASH table has a main table and a backup table. The forwarding thread uses the main table. When the HASH table is regenerated, the main thread switches the flag bit, and the forwarding thread switches the main table and the backup table.
[0024] Preferably, based on the binary search method, a 64-bit CPU is used to optimize the comparison times, and the matching times of statistical data flow are counted and sorted from high to low according to the statistical times, and the HASH algorithm is used when the data flow comes next time.
[0025] Preferably, when adding a configuration policy, determine whether a configuration policy with the same priority already exists. If not, further search by binary search to determine whether the same configuration policy already exists. If not, save the configuration policy priority table, save the configuration policy table, and sort the configuration policy priority table and configuration policy table. Policies with the same priority or the same configuration data are only added once. When deleting a configuration policy, find a matching policy by binary search to determine whether there is a corresponding policy. If so, delete the configuration policy priority, delete the configuration policy, and re-sort the configuration policy priority table and configuration policy table. When modifying a configuration policy, determine whether the deletion of the configuration policy is successful. If successful, add the configuration policy.
[0026] Preferably, when generating a search strategy, the configuration strategy is first sorted according to priority to determine whether it is the first configuration strategy. If so, it is directly inserted and a search configuration strategy is generated; otherwise, the remaining configuration strategies need to be compared with all search strategies. If the search strategy completely includes the configuration strategy, it means that the high-priority strategy includes the low-priority strategy. This strategy should be matched at a high priority, so this strategy is not added to the search strategy; if the search strategy does not include the configuration strategy at all, this strategy can be directly added to the search strategy; if the search strategy and the configuration strategy are partially included, it is necessary to remove the duplicate data segments before inserting the search strategy.
[0027] Preferably, in policy uniqueness, the high-priority policy is always used as the basis. If a low-priority policy and a high-priority policy are repeated, the low-priority policy will be deleted from the policy table; if a high-priority policy contains a low-priority policy, the low-priority policy will be deleted from the policy table; if a high-priority policy contains part of the low-priority policy, the included low-priority policy part will be deleted from the policy table; if a low-priority policy contains a high-priority policy, the low-priority policy will be truncated into two parts by the high-priority policy; if a low-priority policy contains part of the high-priority policy, the low-priority policy will be partially truncated.
[0028] The present invention reduces the time complexity from O(log2N) to O(1), thereby achieving fast matching of priority strategies in a multi-dimensional range, effectively improving the forwarding efficiency of VPP. Compared with the time complexity O(n) of sequential strategy matching, the time complexity is reduced to O(log2n) by ordering and unique multi-dimensional priority strategies, and the search time is further reduced by using a 64-bit CPU to process 64 bits at a time. Through a large number of high-speed streams for HASH matching, the time complexity is reduced to O(1), thereby speeding up the speed of strategy matching and significantly improving the system's high-speed data flow forwarding when multiple strategies are applied.
[0029] Embodiment 2:
[0030] A fast strategy matching method based on VPP, by processing the input multi-group strategy data, uniquely making the strategy priority and the corresponding strategy through an algorithm, and optimizing the comparison times by using a binary search method and a 64-bit CPU, on this basis, by counting the number of data flow matches, sorting them from high to low according to the statistical number, and using a HASH algorithm when the next data flow comes, the time complexity is reduced from O(log2N) to O(1), thereby achieving fast matching of multi-group and range priority strategies and improving the forwarding efficiency of VPP. The present invention improves the query efficiency of the strategy by combining binary search and HASH. In the matched flow, the value of the N-tuple is determined, so the HASH algorithm can be used to improve the efficiency. For example, there are 1W flows, if we have 20% of high-speed flows that can use the HASH algorithm to hit the strategy, according to the 80 / 20 rule, there are nearly 80% of the matches, and the flow matching can be performed through O(1), and the remaining 20% of the flows are slow flows, and then binary search is used, which has little impact on forwarding, thereby greatly improving the VPP flow forwarding efficiency.
[0031] The following steps are involved:
[0032] 1. Configuration strategy addition process
[0033] like Figure 1 As shown in the figure, when adding a configuration policy, the priority table and configuration data table are used to ensure that the policy with the same priority or the same configuration data is only added once. After adding the configuration policy, the priority sorting table and configuration policy table are saved to prevent duplication. The purpose of sorting is to use the binary search method to speed up the search next time.
[0034] 2. Configuration policy deletion process
[0035] like Figure 2 As shown, the process of deleting a configuration policy is to find the policy, then delete the corresponding configuration policy and configuration policy priority, and reorder the priority table and configuration policy table.
[0036] 3. Configuration policy modification process
[0037] like Figure 3 As shown in the figure, modifying the configuration strategy reuses the functions of deleting and adding configuration strategy modules. After the deletion is successful, the module can be added again.
[0038] 4. Search strategy generation process
[0039] like Figure 4As shown in the figure, the main purpose of generating search strategies is to make the strategies of each priority unique. To generate search strategies, the configuration strategies are sorted according to priority. If it is the first strategy, which is the highest priority strategy, this configuration strategy is directly inserted into the search strategy. The subsequent strategies need to be compared with all the search strategies. The results are as follows:
[0040] The search policy completely includes the configuration policy. This means that the high-priority policy includes the low-priority policy. This policy should be matched when it has a high priority, so this policy is not added to the search policy.
[0041] If the search policy does not contain any configuration policy at all, this policy can be added directly to the search policy.
[0042] When the search policy and configuration policy are partially included, you need to remove the duplicate data segments and then insert the search policy.
[0043] The number of search strategies generated is as follows:
[0044] The number of search strategies is equal to the number of configuration strategies: each strategy is uniquely configured, that is, mutually exclusive and non-inclusive.
[0045] The number of search policies is less than the number of configured policies: When a high priority policy contains a low priority policy, the number of search policies may be less than the number of configured policies.
[0046] The number of search policies is greater than the number of configuration policies: When the high priority is in the middle of the low priority, that is, the high priority is included at both ends of the low priority, the low priority policy will be split into two sections.
[0047] 5. Strategy search process and acceleration
[0048] like Figure 5 As shown in the figure, the initial search process is a common binary search. In order to speed up the search and make full use of the 64-bit CPU capabilities, the data can be split into data ranges corresponding to the 64-bit CPU. For example, on a 64-bit CPU, 64 bits of data length are compared at a time.
[0049] For example, the source IP and the destination IP enter the comparator at the same time. Because the data range constructed at this time is startSip is 32 bits, startDip = 32 bits. endSip is 32 bits, and endDip is 32 bits. If the input data ip is within the range of startSip (high bit) startDip (low bit) and endSip (high bit) endDip (low bit), then this IP matches the search strategy range. Other dimensions can be converted using the same method, which can reduce the number of comparisons for each search. Usually, for 16-dimensional u32 data on 64 bits, all comparisons (worst case) can be completed in 8 comparisons.
[0050] The purpose of counting the hit counts of the N-tuple strategy is to generate a HASH matching table, and sort by the hit counts to generate M HASH matching tables with higher hit counts, such as Figure 6 As shown, the hit count table is updated according to the set time.
[0051] ① The present invention uniquely implements the multi-dimensional range strategy:
[0052] Each data stream will only hit one policy in each policy matching process. When the data in multiple policies are repeated or contain each other, fast search cannot be achieved because the policy that hits at this time is not necessarily the highest priority policy. Therefore, linear search from high to low is required according to the priority, and binary search cannot be used. The purpose of fast matching cannot be achieved.
[0053] It should be noted here that configuration data and policy data are two different data tables. Configuration data can be used for verification and viewing, and policy data is generated based on configuration data. Every time the configuration data is changed, the policy data table needs to be regenerated. The generation process includes deduplication and deinclusion. For example, using IP in one field:
[0054] Table 1
[0055]
[0056] As can be seen from Table 1: the high priority policy is always the basis. If the low priority and high priority are repeated, the low priority policy will be deleted from the policy table; if the high priority contains the low priority, the low priority policy will be deleted from the policy table; if the high priority contains part of the low priority, the included low priority part will be deleted from the policy table. If the low priority contains the high priority, the low priority is truncated into two parts by the high priority; if the low priority contains part of the high priority, the low priority policy is partially truncated.
[0057] Other data segments or more data are processed cyclically, and finally each strategy data is unique, although one strategy may be split into multiple strategies.
[0058] ②Use 64-bit CPU to accelerate:
[0059] Reduce the number of comparisons and speed up result output.
[0060] Note: This part of the function is implemented in binary search, and interacts with binary search to speed up the search process.
[0061] Let's take an example: when a policy data such as source IP: 192.168.1.1 (0xC0A80101), destination IP: 192.168.1.10 (0xC0A8010A); under normal circumstances, the source IP is compared with the source IP of the flow, and the destination IP is compared with the destination IP of the flow. In order to speed up the matching speed and reduce the number of matches (the policy has been uniqued before), the structured data can be compared in sequence according to U64. In this way, only one time is needed, that is, 0xC0A80101C0A8010A is compared with the flow data, and the next data is pipelined in the same way, thereby reducing the number of comparisons and speeding up the result output.
[0062] ③ Addition of statistics and HASH methods:
[0063] Statistics are used to test which data flows are high-speed flows. After binary search or HASH search matches, the number of matches is counted. According to the configured statistical time T, after the statistical time is reached, it is updated according to the number of matches (when the HASH table is large, the statistical data with fewer matches can be discarded), and a HASH table is generated based on the statistical value.
[0064] HASH search access point: In the first search, if there is no HASH value or statistical value, then binary search is performed and statistics are performed. After the T time is reached, a HASH table is generated. The subsequent data flow can be matched by HASH. If the HASH match fails, it is a new data flow and binary strategy matching is performed. The process of generating the HASH table is the process of HASH table aging (Note: In VPP, the actual process of generating the HASH table is not in the forwarding thread, but in the main thread. This is to prevent the forwarding from being affected by the process of generating the HASH table. Obviously, the generated HASH table has a main table and a backup table. The forwarding thread uses the main table. When the HASH table is regenerated, the main thread switches the flag bit and the forwarding thread switches to the main and backup tables).
[0065] It shows that binary search is a range search, and HASH search is an exact search. The addition of the HASH table can further improve the matching rate of the strategy.
[0066] Compared with the time complexity O(n) of sequential strategy matching, the present invention reduces the time complexity to O(log2n) by ordering and uniquely implementing multi-dimensional priority strategies. It also uses a 64-bit CPU to process 64 bits at a time, further reducing the search time. It uses a large number of high-speed streams for HASH matching to reduce the time complexity to O(1), thereby speeding up strategy matching and significantly improving the system's high-speed data stream forwarding when multiple strategies are applied.
[0067] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.
Claims
1. A fast strategy matching method based on VPP, It is characterized in that Process the input multi-tuple policy data, uniquely identify the policy priority and the corresponding policy, and perform policy search. First, search through the HASH algorithm. If the search is successful, return the priority. Otherwise, use the binary search method to search. If the search is successful, count the hit counts of the N-tuples to generate a HASH matching table, and generate M HASH matching tables with higher hit counts according to the hit count sorting to determine the data flow of the high-speed flow. The hit count table is updated according to the set time, and finally returns the priority. When the next data flow comes, use the HASH algorithm to search; When generating a search strategy, first sort the configuration strategy by priority to determine whether it is the first configuration strategy. If so, insert it directly and generate a search configuration strategy. Otherwise, the remaining configuration strategies need to be compared with all the search strategies. If the search strategy completely includes the configuration strategy, it means that the high-priority strategy includes the low-priority strategy. This strategy should be matched at a high priority, so this strategy is not added to the search strategy. If the search strategy does not include the configuration strategy at all, this strategy can be directly added to the search strategy. If the search strategy and the configuration strategy are partially included, it is necessary to remove the duplicate data segments and then insert the search strategy. In policy uniqueness, the high-priority policy is always used as the basis. If the low-priority policy and the high-priority policy are repeated, the low-priority policy will be deleted from the policy table; if the high-priority policy contains the low-priority policy, the low-priority policy will be deleted from the policy table; if the high-priority policy contains part of the low-priority policy, the included low-priority policy part will be deleted from the policy table; if the low-priority policy contains the high-priority policy, the low-priority policy will be truncated into two parts by the high-priority policy; If a low-priority policy contains part of a high-priority policy, the low-priority policy is partially truncated.
2. According to the VPP-based fast strategy matching method of claim 1, It is characterized in that In the first search, there is no HASH value or statistical value, so a binary search is performed and statistics are performed. After the T time is reached, a HASH table is generated, and the subsequent data stream is matched with a HASH. If the HASH match fails, the subsequent data stream is regarded as a new data stream and a binary strategy match is performed.
3. According to the VPP-based fast strategy matching method of claim 2, It is characterized in that In VPP, the actual process of generating the HASH table is not in the forwarding thread, but in the main thread. This is to prevent the forwarding from being affected by the process of generating the HASH table. The generated HASH table has a main table and a backup table. The forwarding thread uses the main table. When the HASH table is regenerated, the main thread switches the flag bit, and the forwarding thread switches the main table and the backup table.
4. According to claim 2, a VPP-based fast strategy matching method, It is characterized in that Based on the binary search method, the 64-bit CPU is used to optimize the number of comparisons. The number of matches of the statistical data flow is counted and sorted from high to low according to the statistical number. The HASH algorithm is used when the next data flow comes.
5. A VPP-based fast strategy matching method according to any one of claims 1 to 4, It is characterized in that When adding a configuration policy, determine whether a configuration policy with the same priority already exists. If not, further search by binary search to determine whether the same configuration policy already exists. If not, save the configuration policy priority table, save the configuration policy table, and sort the configuration policy priority table and the configuration policy table. The policy with the same priority or the same configuration data is only added once. When deleting a configuration policy, a matching policy is searched through a binary search method to determine whether there is a corresponding policy. If so, the configuration policy priority is deleted, the configuration policy is deleted, and the configuration policy priority table and configuration policy table are re-sorted. When modifying a configuration policy, it is determined whether the configuration policy is deleted successfully. If successful, the configuration policy is added.
