Method for quickly matching IP address range

Abstract

The invention belongs to the technical field of IP address matching, and provides a method for quickly matching an IP address range. The IP address range can be flexibly and efficiently matched, and large-scale IP matching is processed. According to the method, an IP range is quickly stored and retrieved by adopting an interval tree, and metadata is stored by using a hash table. And developing an optimization search algorithm, rapidly positioning a target IP node in the interval tree, and determining a search direction by comparing intervals. And a parallel processing technology is introduced to improve the processing speed. And a dynamic updating mechanism is designed to ensure data consistency. And optimizing the performance of the hash table by adopting a conflict resolution strategy, and setting an automatic conflict resolution strategy to process the overlapping IP range.

Description

Technical Field

[0001] The invention belongs to the technical field of IP address matching, and in particular relates to a method for quickly matching an IP address range. Background Art

[0002] With the rapid development of the Internet, the management of IP addresses has become increasingly complex. In large-scale networks, quickly and accurately matching and processing IP address ranges is critical to network security and operation and maintenance. Existing IP address matching methods are often inefficient when processing large amounts of data and are difficult to meet real-time requirements.

[0003] The existing technology often uses a one-to-one comparison method when matching IP addresses, which is inefficient in large-scale IP address ranges and difficult to cope with dynamically changing network environments. In addition, the existing method also has shortcomings when dealing with overlapping IP address ranges.

[0004] 1. Low efficiency: IP addresses are compared one by one, the processing speed is slow and cannot meet the needs of large-scale networks.

[0005] 2. Lack of flexibility: It is difficult to adapt to changes in network structure and dynamic adjustment of IP addresses.

[0006] 3. High error rate: It is easy to make mistakes when dealing with complex or overlapping IP address ranges.

[0007] 4. Poor scalability: Existing algorithms are difficult to expand to larger-scale network environments. Summary of the invention

[0008] In order to solve the technical problem that the existing IP address matching method is inefficient and difficult to process large-scale IP address ranges, the present invention proposes a method for quickly matching IP address ranges, providing a more flexible and efficient IP address range matching method that can handle large-scale IP address matching.

[0009] The specific plan is as follows:

[0010] A quick way to match IP address ranges.

[0011] Step 1: Construct a data structure: The data structure includes an interval tree and a hash table; the interval tree is used to store a set of IP address ranges; the hash table is used to store metadata of the IP address range, i.e., index information, the metadata including the start IP, the end IP, and the network mask;

[0012] Step 2: IP address preprocessing: standardize the input IP address and convert it into a unified format;

[0013] Step 3: Quickly search and locate the target IP node: Set the time complexity of the interval search algorithm to O(log n), use the interval search algorithm to locate the target IP node in the interval tree, find the node overlapping with the given interval in the interval tree, and determine which interval the IP address to be matched falls in; handle overlapping IP address ranges based on the automated conflict resolution strategy;

[0014] Step 4: Introduce a parallel processing mechanism in the fast search process: perform a matching operation on at least one IP address at the same time; the parallel processing mechanism is: use a bitmap to represent the IP address space, use the divide-and-conquer strategy for parallel processing, decompose a large problem into small problems, and solve them on different processing units respectively; and use multi-threading technology to create a thread pool for dynamic multi-threading management;

[0015] Step 5: Match verification: Use the hash table to locate the specific IP address range and perform match verification;

[0016] Step 6: Result output: Output the matching results, including the network segment to which the IP address belongs and related attribute information;

[0017] Step 7: Introduce a dynamic update mechanism: When the IP address range changes, update the interval tree based on the Lai marking technology and update the hash table based on the average time complexity.

[0018] Preferably, the storage method of the interval tree in step 1 is: the network administrator uses the interval tree to maintain a group of IP address segments, and each IP address segment is stored as an interval on a node of the tree; when it is necessary to quickly locate the subnet to which an IP address belongs or to calculate the number of available IP addresses in a subnet, a query is performed through the interval tree; each node of the interval tree includes: data, interval attributes, and additional information.

[0019] Preferably, the format of the IP address after standardization in step 2 is: comprising four groups of decimal numbers, each group of numbers ranging from 0 to 255, and each group of numbers is separated by a period (.).

[0020] Preferably, the method for quickly searching and locating the target IP node in step 3 is: determining the direction of the search by comparing the interval where the given target IP node is located with the left endpoint of the node interval and the maximum endpoint in the right subtree; if the left endpoint of the target interval is less than or equal to the left endpoint of the current node, and less than or equal to the maximum endpoint in the right subtree, searching the left subtree; otherwise, searching the right subtree; this process continues until a node overlapping with the target interval is found or the end of the tree is reached, and finally determining in which interval the IP address to be matched falls.

[0021] Preferably, in step 4, the parallel processing technology steps are:

[0022] S41: Bitmap representation of IP address space: define and create bitmap; convert IP address to bitmap index; IP address conversion;

[0023] S42: Divide and conquer strategy implementation:

[0024] S421 Task decomposition: Divide the IP address range, decompose the large range of IP addresses into small ranges, and obtain subtasks;

[0025] S422 Processing subtask: Perform a matching operation within a sub-range to check whether the IP address exists;

[0026] S423: Create a thread pool and perform parallel matching: use Python's concurrent.futures module to manage the thread pool; set the maximum number of threads, submit tasks, and collect results; run parallel processing: call the matching function: set the IP address to be matched based on the initialization bitmap; perform parallel matching;

[0027] S44: Performance optimization: Dynamically adjust the thread pool size: Dynamically adjust the number of threads according to the system load and task complexity; ensure that each thread processes the same number of tasks to avoid overloading a thread.

[0028] Preferably, the method of locating the specific IP address range using the hash table and performing matching verification in step 5 is:

[0029] S51: Construct a hash table: convert the IP address and subnet mask into binary form; apply a logical AND operation to the binary representations of the IP address and subnet mask; determine the length of the network prefix by counting the number of non-zero bits in the subnet mask; construct a hash key by combining the network prefix length and the result of the AND operation;

[0030] S52: IP address conversion: converting the IP address to be queried into the form of a network address and a subnet mask for comparison with the hash key;

[0031] S53: Match verification: Use a hash function to calculate the hash value of the IP address to be queried, and search for the corresponding key in the hash table; if a matching key is found, verify whether the IP address does belong to the range, and return the associated value information.

[0032] Preferably, in step 7, the dynamic update mechanism is implemented by:

[0033] S71: The interval tree implements interval update through the lazy marking technology of nodes; the lazy marking technology is: when an interval needs to be updated, the update operation is passed to the corresponding node. If the child nodes of the node have not been updated, the update operation to be performed is recorded in the node mark. When a subsequent query or update request reaches the child node, the update is completed using the recorded node mark;

[0034] S72: Update hash table: When the IP address range changes, use the hash table to insert, delete, and modify key-value pairs to ensure that the average time complexity of the operation is O(1); regularly check the load factor of the hash table to make the average time complexity of the operation O(1), use open addressing or chain address method as a conflict resolution strategy, and re-hash when it degenerates to O(n).

[0035] Preferably, the automated conflict resolution strategy in step 3 is: perform conflict detection, use a search algorithm to locate whether the IP is repeated with an existing IP in the server-side MySQL database, and if there are duplicates, remove the duplicate and overlapping IPs, store the remaining IPs in the MySQL database, and prompt through the interface that there is a duplicate IP: XXX.

[0036] Beneficial effects:

[0037] The present invention provides a method for quickly matching an IP address range. The method provides a more flexible and efficient IP address range matching solution, which can effectively meet the needs of large-scale IP address matching. First, an interval tree is used as the main data structure, and the tree structure can quickly store and retrieve the IP address range. At the same time, a hash table is used to store metadata of the IP address range, such as the start IP, the end IP, the network mask, etc., to achieve fast access. When it is necessary to quickly locate the subnet to which an IP address belongs or to calculate the number of available IP addresses in a subnet, an efficient query can be performed through the interval tree. Third, the present invention develops an optimized search algorithm, which can quickly locate the node containing the target IP in the interval tree, thereby quickly returning the matching IP address range. Specifically, the search direction is determined by comparing the interval where a given target IP node is located with the left endpoint of the node interval and the maximum endpoint in the right subtree. If the left endpoint of the target interval is less than or equal to the left endpoint of the current node, and less than or equal to the maximum endpoint in the right subtree, the left subtree is searched; otherwise, the right subtree is searched. This process continues until a node overlapping with the target interval is found or the end of the tree is reached. The present invention can effectively locate the target IP node through the interval tree and interval search algorithm, and provide fast search performance. Fourth, the present invention introduces parallel processing technology, allowing matching operations on multiple IP addresses at the same time, greatly improving processing speed and efficiency. Fifth, the present invention designs a dynamic update mechanism, which can quickly update the interval tree and hash table when the IP address range changes, ensuring data consistency and accuracy. Sixth, in order to maintain the performance of the hash table, conflict resolution strategies such as open addressing or chain address method are adopted, and rehashing is performed when necessary to optimize hash distribution. Seventh, the present invention sets an automated conflict resolution strategy: when dealing with overlapping IP address ranges, an automated conflict resolution strategy is developed to ensure the correctness and reliability of the matching results.

[0038] This method achieves rapid matching of IP addresses through optimized data structure and search algorithm to meet the needs of real-time processing. By combining interval tree and hash table, the error rate in the matching process is reduced and the matching accuracy is improved. This method has high flexibility and adaptability, can adapt to network environments of different scales, and can cope with dynamic changes in network structure and IP address range. It has strong scalability, supports IP address matching in large-scale network environments, and can be horizontally expanded as the network scale expands. During the design process, security has been fully considered to ensure the security and privacy of IP address data. In addition, this method also supports complex query operations, including range overlap detection, subnet division, etc. Through parallel processing and algorithm optimization, the efficiency of computing resources is improved and the energy consumption of the system is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of a method for quickly matching IP address ranges. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the embodiments and drawings.

[0041] like Figure 1 As shown, a method for quickly matching IP address ranges.

[0042] Step 1: Construct a data structure: The data structure includes an interval tree and a hash table; the interval tree is used to store a set of IP address ranges; the hash table is used to store metadata of the IP address range, i.e., index information, the metadata including the start IP, the end IP, and the network mask;

[0043] Storage method: In IP address management, interval trees can be used to store a set of IP address ranges. For example, a network administrator can use interval trees to maintain a set of IP address segments, each of which is stored as an interval on a node in the tree. When you need to quickly locate the subnet to which an IP address belongs or need to calculate the number of available IP addresses in a subnet, you can use interval trees for efficient queries.

[0044] Step 2: IP address preprocessing: standardize the input IP address and convert it into a unified format;

[0045] Step 3: Perform a quick search: Set the time complexity of the interval search algorithm to O(log n), use the interval search algorithm to locate the target IP node in the interval tree, find the node overlapping with the given interval in the interval tree, and determine in which interval the IP address to be matched falls;

[0046] Interval search algorithm: The search algorithm for locating the target IP node in the interval tree usually adopts the interval search algorithm (Interval-Search). The interval search algorithm is an algorithm for finding nodes that overlap with a given interval in the interval tree. The interval tree is a data structure extended on the basis of the red-black tree. Each node contains not only data, but also an interval attribute, as well as additional information, such as the maximum value of the endpoints of all intervals in the subtree.

[0047] The time complexity of the interval search algorithm is O(log n), because each iteration can eliminate at least half of the search space, thus ensuring logarithmic time efficiency. In applications such as network routing and IP address lookup, interval trees and interval search algorithms can effectively locate target IP nodes and provide fast search performance.

[0048] Step 4: Performance optimization: Introduce a parallel processing mechanism in the fast search process to perform matching operations on multiple IP addresses at the same time; the parallel processing mechanism includes: using bitmaps to represent the IP address space, because IP addresses are numbers of fixed length, suitable for fast matching using bit operations. In addition, the divide-and-conquer strategy can be used for parallel processing, breaking down large problems into small problems and solving them on different processing units; and using multi-threading technology to create a thread pool for dynamic multi-threading management.

[0049] Step 5: Match verification: Use the hash table to locate the specific IP address range and perform match verification;

[0050] Step 6: Result output: Output the matching results, including the network segment to which the IP address belongs and related attribute information;

[0051] Step 7: Introduce a dynamic update mechanism: When the IP address range changes, update the interval tree and hash table in real time;

[0052] The interval tree uses lazy propagation technology to achieve efficient interval updates. When an interval needs to be updated, the update operation is passed to the corresponding node and passed down to the child nodes when necessary. This method can reduce unnecessary node updates and thus improve the performance of update operations. When the IP address range changes, the interval tree is used to quickly locate the affected nodes and apply the update operation.

[0053] Hash table updates usually involve inserting, deleting, or modifying key-value pairs. When the IP address range changes, new IP addresses are added to the hash table or old IP addresses are deleted. The design of the hash table ensures that the average time complexity of these operations is O(1), but it may degenerate to O(n) in the worst case. In order to maintain the performance of the hash table, conflict resolution strategies such as open addressing or chain addressing are used, and rehashing is performed when necessary to optimize the hash distribution.

[0054] Preferably, the storage method of storing a group of IP address ranges in the interval tree in step 1 is: the network administrator uses the interval tree to maintain a group of IP address segments, and each IP address segment is stored as an interval on a node of the tree; when it is necessary to quickly locate the subnet to which an IP address belongs or to calculate the number of available IP addresses in a subnet, a query is performed through the interval tree; each node of the interval tree includes: data, interval attributes, and additional information.

[0055] Preferably, the method for performing fast search in step 3 is: determine the search direction by comparing the interval where the given target IP node is located with the left endpoint of the node interval and the maximum endpoint in the right subtree; if the left endpoint of the target interval is less than or equal to the left endpoint of the current node and less than or equal to the maximum endpoint in the right subtree, search the left subtree; otherwise, search the right subtree. This process continues until a node overlapping with the target interval is found or the end of the tree is reached.

[0056] Preferably, in step 4, the parallel processing technology steps are:

[0057] S41: Bitmap representation of IP address space:

[0058] 1.1: Bitmap initialization: define and create a bitmap;

[0059] 1.2: Set and clear IP address: Convert IP address to bitmap index;

[0060] 1.3: IP address conversion: IP to index;

[0061] S42: Divide and conquer strategy implementation:

[0062] 2.1 Task decomposition: Divide the IP address range, decompose the large range of IP addresses into small ranges, and obtain subtasks;

[0063] 2.2 Processing subtask: Perform a matching operation within a sub-range to check whether the IP address exists;

[0064] S43: Multithreading technology and thread pool:

[0065] 3.1 Create a thread pool: Use Python's concurrent.futures module to manage the thread pool; set the maximum number of threads, submit tasks, and collect results;

[0066] 3.2 Run parallel processing: call the matching function: set the IP address to be matched based on the initialization bitmap; perform parallel matching;

[0067] S44: Performance Optimization:

[0068] 1.3 Dynamically adjust the thread pool size: dynamically adjust the number of threads according to the system load and task complexity;

[0069] 1.4 Task allocation strategy: Ensure that each thread processes the same number of tasks to avoid overloading a thread.

[0070] Preferably, in step 7, the dynamic update mechanism is implemented by:

[0071] S71: The interval tree uses lazy marking of nodes to implement interval updates; when an interval needs to be updated, the update operation is passed to the corresponding node and passed down to the child nodes when necessary;

[0072] S72: When the IP address range changes, use the hash table to insert, delete, and modify key-value pairs to ensure that the average time complexity of the operation is O(1); regularly check the load factor of the hash table to make the average time complexity of the operation O(1), use open addressing or chain address method as a conflict resolution strategy, and re-hash when it degenerates to O(n).

[0073] Automated conflict resolution strategy: When dealing with overlapping IP address ranges, develop an automated conflict resolution strategy to ensure the correctness and reliability of matching results.

[0074] Preferably, the automated conflict resolution strategy is: perform conflict detection, use a search algorithm to locate whether the IP is repeated with an existing IP in the library, remove the repeated and overlapping IPs if there are duplicates, store the remaining IPs in the library, and prompt the interface that there is a duplicate IP: XXX.

[0075] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.

Claims

1. A method for quickly matching an IP address range, characterized in that: Step 1: Construct a data structure: The data structure includes an interval tree and a hash table; the interval tree is used to store a set of IP address ranges; the hash table is used to store metadata of the IP address range, i.e., index information, the metadata including the start IP, the end IP, and the network mask; Step 2: IP address preprocessing: standardize the input IP address and convert it into a unified format; Step 3: Quickly search and locate the target IP node: Set the time complexity of the interval search algorithm to O(log n), use the interval search algorithm to locate the target IP node in the interval tree, find the node overlapping with the given interval in the interval tree, and determine which interval the IP address to be matched falls in; handle overlapping IP address ranges based on the automated conflict resolution strategy; Step 4: Introduce a parallel processing mechanism in the fast search process: perform a matching operation on at least one IP address at the same time; the parallel processing mechanism is: use a bitmap to represent the IP address space, use the divide-and-conquer strategy for parallel processing, decompose a large problem into small problems, and solve them on different processing units respectively; and use multi-threading technology to create a thread pool for dynamic multi-threading management; Step 5: Match verification: Use the hash table to locate the specific IP address range and perform match verification; Step 6: Result output: Output the matching results, including the network segment to which the IP address belongs and related attribute information; Step 7: Dynamically update the interval tree and hash table: When the IP address range changes, update the interval tree based on the Lai marking technology and update the hash table based on the average time complexity.

2. A method for quickly matching IP address ranges according to claim 1, characterized in that: The storage method of the interval tree in step 1 is as follows: the network administrator uses the interval tree to maintain a group of IP address segments, and each IP address segment is stored as an interval on a node of the tree; when it is necessary to quickly locate the subnet to which an IP address belongs or to calculate the number of available IP addresses in a subnet, query through the interval tree; Each node of the interval tree includes: data, interval attributes, and additional information.

3. A method for quickly matching IP address ranges according to claim 1, characterized in that: The format of the IP address after standardization in step 2 is: containing four groups of decimal numbers, each group of numbers ranges from 0 to 255, and each group of numbers is separated by a period (.).

4. A method for quickly matching IP address ranges according to claim 1, characterized in that: The method for quickly searching and locating the target IP node in step 3 is: determining the search direction by comparing the interval where the given target IP node is located with the left endpoint of the node interval and the maximum endpoint in the right subtree; If the left endpoint of the target interval is less than or equal to the left endpoint of the current node, and less than or equal to the maximum endpoint in the right subtree, search the left subtree; Otherwise, search the right subtree; This process continues until a node overlapping the target interval is found or the end of the tree is reached, and finally it is determined in which interval the IP address to be matched falls.

5. A method for quickly matching IP address ranges according to claim 1, characterized in that: In step 4, the parallel processing technology steps are: S41: Bitmap representation of IP address space: define and create bitmap; convert IP address to bitmap index; IP address conversion; S42: Divide and conquer strategy implementation: S421 Task decomposition: Divide the IP address range, decompose the large range of IP addresses into small ranges, and obtain subtasks; S422 Processing subtask: Perform a matching operation within a sub-range to check whether the IP address exists; S423: Create a thread pool and perform parallel matching: Use Python's concurrent.futures module to manage the thread pool; Set the maximum number of threads, submit tasks, and collect results; run parallel processing: call the matching function: set the IP address to be matched based on the initialization bitmap; perform parallel matching; S44: Performance optimization: Dynamically adjust the thread pool size: Dynamically adjust the number of threads according to the system load and task complexity; ensure that each thread processes the same number of tasks to avoid overloading a thread.

6. A method for quickly matching IP address ranges according to claim 1, characterized in that: Step 5 uses a hash table to locate a specific IP address range and perform matching verification; S51: Construct a hash table: convert the IP address and subnet mask into binary form; apply a logical AND operation to the binary representations of the IP address and subnet mask; determine the length of the network prefix by counting the number of non-zero bits in the subnet mask; construct a hash key by combining the network prefix length and the result of the AND operation; S52: IP address conversion: converting the IP address to be queried into the form of a network address and a subnet mask for comparison with the hash key; S53: Match verification: Use a hash function to calculate the hash value of the IP address to be queried, and search for the corresponding key in the hash table; if a matching key is found, verify whether the IP address does belong to the range, and return the associated value information.

7. A method for quickly matching IP address ranges according to claim 1, characterized in that: In step 7, the dynamic update mechanism is implemented as follows: S71: The interval tree implements interval update through the lazy marking technology of nodes; the lazy marking technology is: when an interval needs to be updated, the update operation is passed to the corresponding node. If the child nodes of the node have not been updated, the update operation to be performed is recorded in the node mark. When a subsequent query or update request reaches the child node, the update is completed using the recorded node mark; S72: Update hash table: When the IP address range changes, use the hash table to insert, delete, and modify key-value pairs to ensure that the average time complexity of the operation is O(1); regularly check the load factor of the hash table to make the average time complexity of the operation O(1), use open addressing or chain address method as a conflict resolution strategy, and re-hash when it degenerates to O(n).

8. A method for quickly matching IP address ranges according to claim 1, characterized in that: The automated conflict resolution strategy described in step 3 is: perform conflict detection, use a search algorithm to locate whether the IP is repeated with an existing IP in the server-side MySQL database, and if there are duplicates, remove the duplicate and overlapping IPs, store the remaining IPs in the MySQL database, and prompt the interface that there is a duplicate IP: XXX.

Images