Hash-based multi-level cache index-based IP address offline data matching method

Abstract

The application provides an IP address offline data matching method and system based on a hash multi-level cache index, and comprises the following steps: obtaining IP address information offline data; preheating IP information of each record of the IP address information offline data to a target IP information index pool; wherein, the IP information is segmented and split, and is stored in different levels of the target IP information index pool according to different IP segments; according to IP information that needs to be matched in real-time transactions, an IP search engine is notified in real time to index IP attribution information to the target IP information index pool that has been preheated; wherein, the IP search engine splits the IP information that needs to be matched and searches the corresponding target IP information index pool. The application combines modules organically, and combines a fast index search method to realize millisecond-level IP attribution matching capability and complete real-time IP attribution identification functions in the field of online real-time transactions.

Description

Technical Field

[0001] This invention relates to the field of data matching, and more specifically, to a method and system for offline IP address data matching based on a hash-based multi-level cache index. Background Technology

[0002] In the context of the Internet of Things, IP information is experiencing explosive growth. IPv4 is exhausted, and IPv6 is being widely adopted. Theoretically, IPv6 can allocate approximately 340 trillion IP addresses. In the field of online real-time transactions, it is often necessary to use the location corresponding to the customer's transaction network IP to complete business scenarios such as real-time transaction risk identification, post-transaction transaction analysis, and report statistics.

[0003] In high-concurrency, high-response real-time transactions, the location of an IP address is usually obtained by asynchronously converting the network IP to its corresponding location using a third-party API (referred to as "IP location"). This method has slow response time and low concurrency capability, making it difficult to meet the needs of scenarios that require real-time acquisition of IP location for real-time business analysis, such as real-time transaction risk identification.

[0004] Patent document CN106953806B discloses a method for matching IP addresses based on suffix indexes, including the following steps: S1. Constructing a suffix index for IP routing data; S2. Dividing IPv6 addresses into k equal segments, and then constructing different hash tables for each segment of IPv6 addresses, with each hash table storing the interval values ​​of different suffixes within the segment in the suffix index; S3. For a given destination IP address, dividing it into k equal segments, and then using a binary search method to search for matching suffix index intervals in the k hash tables for the k segments of destination IP addresses, and recording the segment number corresponding to the hash table containing the matching suffix index interval; S4. Performing an intra-segment backtracking search based on the recorded segment number, and outputting the routing information of the destination IP.

[0005] However, the index information of patent document CN106953806B is only distributed in a limited hash table, and the query efficiency is limited by the amount of data in IPv6. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for offline IP address data matching based on hash-based multi-level cache indexes.

[0007] The present invention provides a method for offline IP address data matching based on hash-based multi-level cache index, comprising:

[0008] Step SA: Obtain offline data for IP address information;

[0009] Step SB: Cache and preheat the IP information of each record of the offline IP address information data to the hash target IP information index pool; wherein, the IP information is segmented and stored in different levels of target IP information index pools according to different IP segments;

[0010] Step SC: Based on the IP information that needs to be matched in the real-time transaction, notify the IP retrieval engine in real time to index the IP attribution information of the target IP information index pool that has been preheated; wherein, the IP retrieval engine splits the IP information that needs to be matched and retrieves it from the corresponding target IP information index pool.

[0011] Preferably, step SB includes the following steps:

[0012] Step SB.1: Distinguish between IPv4 and IPv6 for the offline data of the IP address information;

[0013] Step SB.2: For IPv4, organize the first two segments of the starting IP and ending IP into a first-level IP information index and store them in the first-level IP information index pool; in the first-level IP information index pool, the KEY of the index is the integer value corresponding to the first two segments of the IP, the VALUE of the index is the integer value corresponding to the starting IP of the IP segment $ the integer value corresponding to the ending IP of the IP segment, and the sorting number of the index is the KEY;

[0014] Complete IP information is stored in a secondary IP information index pool. The name of the secondary IP information index pool includes the integer value corresponding to the start IP and the integer value corresponding to the end IP. In the secondary IP information index pool, the KEY of the index is the integer value corresponding to the complete start IP and the end IP, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0015] If the starting IP and ending IP span multiple segments, then the starting IP and ending IP are stored in the cross-segment IP information index pool. In the cross-segment IP information index pool, the index keys are the complete integer values ​​corresponding to the starting IP and ending IP, respectively, and the index VALUE values ​​are the secondary index KEY|starting IP corresponding integer value$ending IP corresponding integer value, and the secondary index KEY#starting IP corresponding integer value$ending IP corresponding integer value. The sorting number of the index is KEY. The symbols |, #, and $ all represent information separators within the code index.

[0016] Step SB.3: For IPv6, organize the first integer values ​​of the start IP and end IP into a first-level IP information index and store it in the first-level IP information index pool; in the first-level IP information index pool, the KEY of the index is the first integer value of the start IP and end IP, the VALUE of the index is the first integer value of the start IP and the first integer value of the end IP, and the sorting number of the index is the KEY.

[0017] The starting IP, ending IP, and the integer values ​​of the middle segment are organized into a secondary IP information index and stored in the corresponding secondary IP information index pool. In the secondary IP information index pool, the name of the secondary IP information index pool contains the starting IP first segment integer value and the starting IP middle segment integer value. The keys of the index are the starting IP middle segment integer value and the ending IP middle segment integer value, respectively. The value of the index is secondary index key|starting IP middle segment integer value$ending IP middle segment integer value. The sorting number of the index is key.

[0018] The starting IP and the ending IP tail integer values ​​are organized into a three-level IP information index and stored in the corresponding three-level IP information index pool. In the three-level IP information index pool, the name of the three-level IP information index pool contains the starting IP middle segment integer value and the starting IP tail segment integer value information. The KEY of the index is the starting IP tail segment integer value, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0019] Preferably, step SD includes the following steps:

[0020] Step SD.1: Automatically identify IPv4 and IPv6 for the IP information that needs to be matched;

[0021] Step SD.2: For IPv4, convert the first three segments of the IP address into integer values, and retrieve the second-level IP information index whose sort number is less than or equal to the integer value from the first-level IP information index pool;

[0022] If the secondary IP information index is not found, the cross-segment IP information index is searched in the secondary IP information index pool to obtain the secondary IP information index value; the secondary IP information index is then used to search the corresponding secondary IP information index pool to obtain the attribution information.

[0023] Step SD.3: For IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information;

[0024] Step SD.4: Convert the full IPv6 information to obtain the first segment integer value, middle segment integer value, and last segment integer value of IPv6; use the first segment integer value to retrieve the first second-level IP information index with a sort number less than or equal to the first segment integer value from the first-level IP information index pool; use the middle segment integer value to retrieve the third-level IP information index value from the corresponding second-level IP information index pool; use the last segment integer value to retrieve the attribution information from the corresponding third-level IP information index pool.

[0025] Preferably, in step SA, the offline data of IP address information is obtained by fragmentation using a multi-concurrency method;

[0026] During storage, the cache indexes at all levels are synchronously sorted according to the sorting index from smallest to largest;

[0027] In step SB, a hash-based storage cache is used according to the index storage model, wherein, in the index storage model, the VALUE value of each level of index stores all the starting and ending indexes of the next level of index for that segment.

[0028] An offline IP address data matching system based on a hash-based multi-level cache index, provided by the present invention, includes:

[0029] Module MA: Obtains offline data including IP address information;

[0030] Module MB: Caches and preheats the IP information of each record of the offline IP address information data to the hash target IP information index pool; wherein, the IP information is segmented and stored in different levels of target IP information index pools according to different IP segments;

[0031] Module MC: Based on the IP information that needs to be matched in real-time transactions, it notifies the IP retrieval engine in real time to index the IP attribution information of the target IP information index pool that has been preheated; wherein, the IP retrieval engine splits the IP information that needs to be matched and retrieves it from the corresponding target IP information index pool.

[0032] Preferably, the module MB includes the following modules:

[0033] Module MB.1: Distinguish between IPv4 and IPv6 for the offline data of the IP address information;

[0034] Module MB.2: For IPv4, the first two segments of the starting IP and ending IP are organized into a first-level IP information index and stored in the first-level IP information index pool; in the first-level IP information index pool, the KEY of the index is the integer value corresponding to the first two segments of the IP, the VALUE of the index is the integer value corresponding to the starting IP of the IP segment $the integer value corresponding to the ending IP of the IP segment, and the sorting number of the index is the KEY;

[0035] Complete IP information is stored in a secondary IP information index pool. The name of the secondary IP information index pool includes the integer value corresponding to the start IP and the integer value corresponding to the end IP. In the secondary IP information index pool, the KEY of the index is the integer value corresponding to the complete start IP and the end IP, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0036] If the starting IP and ending IP span multiple segments, then the starting IP and ending IP are stored in the cross-segment IP information index pool. In the cross-segment IP information index pool, the index keys are the complete integer values ​​corresponding to the starting IP and ending IP, respectively, and the index VALUE values ​​are the secondary index KEY|starting IP corresponding integer value$ending IP corresponding integer value, and the secondary index KEY#starting IP corresponding integer value$ending IP corresponding integer value. The sorting number of the index is KEY. The symbols |, #, and $ all represent information separators within the code index.

[0037] Module MB.3: For IPv6, the first integer values ​​of the start IP and end IP are organized into a first-level IP information index and stored in the first-level IP information index pool. In the first-level IP information index pool, the KEY of the index is the first integer value of the start IP and end IP, the VALUE of the index is the first integer value of the start IP and the first integer value of the end IP, and the sorting number of the index is the KEY.

[0038] The starting IP, ending IP, and the integer values ​​of the middle segment are organized into a secondary IP information index and stored in the corresponding secondary IP information index pool. In the secondary IP information index pool, the name of the secondary IP information index pool contains the starting IP first segment integer value and the starting IP middle segment integer value. The keys of the index are the starting IP middle segment integer value and the ending IP middle segment integer value, respectively. The value of the index is secondary index key|starting IP middle segment integer value$ending IP middle segment integer value. The sorting number of the index is key.

[0039] The starting IP and the ending IP tail integer values ​​are organized into a three-level IP information index and stored in the corresponding three-level IP information index pool. In the three-level IP information index pool, the name of the three-level IP information index pool contains the starting IP middle segment integer value and the starting IP tail segment integer value information. The KEY of the index is the starting IP tail segment integer value, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0040] Preferably, the module MD includes the following modules:

[0041] Module MD.1: Automatically identifies IPv4 and IPv6 for the IP information that needs to be matched;

[0042] Module MD.2: For IPv4, convert the first three segments of the IP address into integer values, and retrieve the first second-level IP information index in the primary IP information index pool whose sort number is less than or equal to the integer value;

[0043] If the secondary IP information index is not found, the cross-segment IP information index is searched in the secondary IP information index pool to obtain the secondary IP information index value; the secondary IP information index is then used to search the corresponding secondary IP information index pool to obtain the attribution information.

[0044] Module MD.3: For IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information;

[0045] Module MD.4: Converts the full IPv6 information to obtain the first segment integer value, middle segment integer value, and last segment integer value of IPv6; uses the first segment integer value to retrieve the first second-level IP information index with a sort number less than or equal to the first segment integer value from the first-level IP information index pool; uses the middle segment integer value to retrieve the third-level IP information index value from the corresponding second-level IP information index pool; uses the last segment integer value to retrieve the attribution information from the corresponding third-level IP information index pool.

[0046] Preferably, in the module MA, offline data of IP address information is obtained in fragments using a multi-concurrency method;

[0047] During storage, the cache indexes at all levels are synchronously sorted according to the sorting index from smallest to largest;

[0048] In the MB module, a hash-based storage cache is used according to the index storage model, wherein the VALUE value of each level index stores all the starting and ending indexes of the next level index of that segment.

[0049] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the IP address offline data matching method based on multi-level cache index are implemented.

[0050] An electronic device according to the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the IP address offline data matching method based on multi-level cache index.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. In order to meet the needs of high-concurrency and high-response real-time transaction scenarios, this invention proposes a solution for fast matching of offline data of IP address information based on hash-based multi-level caching index.

[0053] 2. This invention designs functional modules such as a hash index pool, a multi-level index pool cache, an index pool storage, an IP retrieval engine, and index pool failure prevention, which are organically combined.

[0054] 3. This invention combines a fast index retrieval method to achieve millisecond-level IP attribution matching capability, thereby completing the real-time IP attribution identification function in the field of online real-time transactions. Attached Figure Description

[0055] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 This is a schematic diagram illustrating the working principle of the present invention.

[0057] Figure 2 This is a schematic diagram of the IPv4 index structure.

[0058] Figure 3 This is a schematic diagram of the IPv6 index structure. Detailed Implementation

[0059] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0060] This invention provides an offline IP address data matching scheme based on a hash-based multi-level caching index. The invention employs functional modules such as a hash-based index pool, a multi-level index pool cache, index pool storage, an IP retrieval engine, and index pool failure prevention. These modules are organically combined with a fast index retrieval method to achieve millisecond-level IP attribution matching capabilities, enabling real-time IP attribution identification in the online real-time transaction field. The multi-level caching index pool implements a multi-level caching index.

[0061] This invention provides a comprehensive solution encompassing storage, matching, preheating, hashing, and multi-level indexing, addressing issues such as access hotspots, supporting massive horizontal scaling, excessive lookups affecting timeliness, and IPv6 update accuracy. Details are as follows:

[0062] By designing a hash-based index cache, the data distribution within the index pool is more even, resulting in a smaller amount of matching data and faster, more stable lookup efficiency. Furthermore, in scenarios with massive IPv6 data volumes, the index pool has inherent horizontal scalability; this avoids the situation where index information is distributed only in a limited hash table, preventing query efficiency from being constrained by the amount of IPv6 data.

[0063] With only a three-level index designed for IPv6 and combined with hash caching, the overall matching only requires three small-scale data matching operations to find the corresponding information; under the case of large data volumes, the overall search efficiency meets the requirements of real-time transaction pair response time.

[0064] In real-time transaction high-concurrency scenarios, by quickly locating the hash cache index pool through the target IPv6 information, the problem of data access hotspots can be effectively solved, and it has a good access heat dissipation capability.

[0065] By using a cache preheating solution, when updating massive amounts of IPv6 data, only the corresponding index pool needs to be preheated and updated, and seamlessly switched, ensuring the accuracy of information during real-time transaction matching. When dealing with massive IPv6 updates, it avoids the risk that the correctness of information for real-time transaction matching may be affected during the update process due to an excessively large hash table.

[0066] Based on existing IP specifications, it supports matching of IPv4 and IPv6, effectively solving the matching needs of various IPs in online real-time transaction scenarios.

[0067] Hash-based index pool: This design employs a hash-based index pool caching system to store massive amounts of offline IP data according to specific rules. This hashing effectively reduces the data volume required for IP attribution retrieval, improves data dissipation under high concurrency, and prevents the loss of baseline data due to cache invalidation. Specifically, IPs are segmented and stored in different levels of hash-based cache index pools for different IP segments. A three-level IP information index pool supports the storage of index information for different 128-bit IPv6 IP segments, while the second-level IP information index pool is compatible with IPv4.

[0068] Multi-level cache index: A multi-level cache index is used to store IP data in an ordered manner to address the issue of IPv6's 128-bit storage limitation, thus improving IP matching and retrieval efficiency. Upward compatibility with second- and third-level caches enables matching of IP addresses from different versions. Specifically, IPv6 segments 1-2, 3-5, and 5-8 are stored using first-, second-, and third-level cache indexes, respectively; for IPv4, the first three segments form the first-level index, and the entire IP address forms the second-level cache index. The data is pre-sorted during the multi-level cache index storage process.

[0069] Index pool storage: Multiple concurrent sharding is used to quickly build and store indexes on massive amounts of offline data. Specifically, multiple concurrent sharding is used to retrieve offline IP data, and the start and end IPs of the IP data are cached and indexed according to a designed storage model.

[0070] Index pool failure prevention: A hashed index pool preheating and seamless hot-swapping approach is adopted to ensure search capability during periods without index failure. This is achieved by combining index pool storage preheating with notifications to the search engine to retrieve the target index pool.

[0071] IP retrieval engine: The retrieved IP is split according to the index pool rules, and the fast retrieval method is used to quickly retrieve the IP from the multi-level cache according to the agreed rules.

[0072] The following describes an offline IP address data matching method based on a multi-level cache index provided by the present invention, including:

[0073] Step S1: Design a multi-level cache index for offline data to form an index storage model. Step S1 includes the following steps:

[0074] Step S1.1: Design a two-level cache index for IPv4, such as... Figure 2 As shown. The first two segments of IPv4 serve as the primary index name, with KEY values ​​being long integers corresponding to the complete start and end IPs, respectively, and VALUE values ​​storing all long integers corresponding to the start and end IPs of that segment. The secondary index KEY value is the primary index VALUE value, and VALUE values ​​are the necessary information required for attribution. If the start and end IPs span different segments, a cross-segment IP information index table is created.

[0075] Step S1.2: Design a three-level cache index for IPv6, such as... Figure 3 As shown, the IPv6 three-level index is designed as follows: segments 1-2 corresponding to long integers (referred to as "first segment integer values") form the first-level index; segments 3-5 corresponding to long integers (referred to as "middle segment integer values") form the second-level index; and segments 6-8 corresponding to long integers (referred to as "last segment integer values") form the third-level index. The VALUE value of each level of index stores the starting and ending indices of all the next-level indexes for that segment. The VALUE value of the third-level index is the necessary information for attribution.

[0076] Step S2: Use a multi-concurrency approach to fragment and obtain offline IP data information, i.e., offline IP address information.

[0077] Step S3: Store the start IP and end IP of each record of offline IP data information in a hashed manner according to the index storage model, and preheat the cache to the target IP information index pool;

[0078] Step S3 includes the following steps:

[0079] Step S3.1: Distinguish between IPv4 and IPv6 in the offline IP data information.

[0080] Step S3.2: When using IPv4, organize the first two segments of the starting IP and ending IP into a first-level index and store them in the first-level IP information index pool. The KEY is the integer value corresponding to the first two segments of the IP, and the VALUE is "the integer value corresponding to the starting IP of this IP segment $ the integer value corresponding to the ending IP of this IP segment", with the sorting number being KEY. Here, the symbol $ represents the information separator within the code index. Store the complete IP information in the second-level IP information index pool. The second-level IP information index pool is named with the information "the integer value corresponding to the starting IP and the integer value corresponding to the ending IP", the KEY is the complete integer value corresponding to the starting IP and the ending IP respectively, the VALUE is the complete attribution information, and the sorting number is KEY.

[0081] If the starting IP and ending IP span different segments, then the starting IP and ending IP are stored in the segment IP information index pool. The keys are the complete integer values ​​corresponding to the starting IP and ending IP, respectively, and the values ​​are "secondary index key | starting IP integer value $ ending IP integer value" and "secondary index key # starting IP integer value $ ending IP integer value", respectively. The sorting index is KEY. During storage, the cache indexes at each level are synchronously sorted according to the sorting index from smallest to largest. The symbols |, #, and $ all represent information separators within the code index.

[0082] Step S3.3: In IPv6, organize the first segment integer values ​​of the start IP and end IP into a first-level index and store it in the first-level IP information index pool. The keys are the first segment integer values ​​of the start IP and end IP respectively, the value is "start IP first segment integer value $ end IP first segment integer value", and the sorting number is KEY. Organize the middle segment integer values ​​of the start IP and end IP into a second-level index and store it in the corresponding second-level IP information index pool. The name of the second-level IP information index pool contains "start IP first segment integer value, start IP middle segment integer value information", the keys are the start IP middle segment integer value and end IP middle segment integer value respectively, the value is "second-level index KEY | start IP middle segment integer value $ end IP middle segment integer value", and the sorting number is KEY. The start IP and end IP tail integer values ​​are organized into a three-level index and stored in the corresponding three-level IP information index pool. The name of the three-level IP information index pool contains the information "start IP middle IP integer value" and "start IP tail integer value". The keys are the start IP tail integer value and the end IP tail integer value, respectively. The VALUE values ​​are the complete attribution information, and the sorting index is the key. During storage, the cache indexes at each level are synchronously sorted in ascending order according to the sorting index.

[0083] Step S4: Based on the IP information in the real-time transaction, in the IP information index pool, notify the IP retrieval engine in real time to retrieve the target IP information index pool using a seamless switching method, targeting the already preheated IP information index pool. During this process, the IP retrieval engine sends attribution information to the internet real-time transaction client.

[0084] Step S4 includes the following steps:

[0085] Step S4.1: Automatically identify IPv4 and IPv6 for the IPs that need to be matched.

[0086] Step S4.2: When using IPv4, convert the first three segments of the IP address to integer values. Use a binary search algorithm in the primary IP information index pool to retrieve the second-level index whose sort number is less than or equal to the integer value. If no second-level index is found, search the second-level IP information index pool using the same method to find the cross-segment index and obtain the second-level index value. Use the second-level index to retrieve detailed attribution information from the corresponding second-level IP information index pool.

[0087] Step S4.3: When using IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information.

[0088] Step S4.4: Convert the IPv6 information to obtain the first segment integer value, middle segment integer value, and last segment integer value of IPv6; use the first segment integer value to retrieve the first secondary index with a sort number less than or equal to the first segment integer value from the primary IP information index pool using a binary search method; use the middle segment integer value to retrieve the tertiary index value from the corresponding secondary IP information index pool; use the last segment integer value to retrieve the attribution information from the corresponding tertiary index.

[0089] This invention also provides an offline IP address data matching system based on a multi-level cache index. Those skilled in the art can implement the offline IP address data matching system based on a multi-level cache index by following the process steps of the method. In other words, the method of offline IP address data matching based on a multi-level cache index can be understood as a preferred embodiment of the offline IP address data matching system based on a multi-level cache index.

[0090] An offline IP address data matching system based on a hash-based multi-level cache index, provided by the present invention, includes:

[0091] Module MA: Obtains offline data including IP address information;

[0092] Module MB: Caches and preheats the IP information of each record of the offline IP address information data to the hash target IP information index pool; wherein, the IP information is segmented and stored in different levels of target IP information index pools according to different IP segments;

[0093] Module MC: Based on the IP information that needs to be matched in real-time transactions, it notifies the IP retrieval engine in real time to index the IP attribution information of the target IP information index pool that has been preheated; wherein, the IP retrieval engine splits the IP information that needs to be matched and retrieves it from the corresponding target IP information index pool.

[0094] Preferably, the module MB includes the following modules:

[0095] Module MB.1: Distinguish between IPv4 and IPv6 for the offline data of the IP address information;

[0096] Module MB.2: For IPv4, the first two segments of the starting IP and ending IP are organized into a first-level IP information index and stored in the first-level IP information index pool; in the first-level IP information index pool, the KEY of the index is the integer value corresponding to the first two segments of the IP, the VALUE of the index is the integer value corresponding to the starting IP of the IP segment $the integer value corresponding to the ending IP of the IP segment, and the sorting number of the index is the KEY;

[0097] Complete IP information is stored in a secondary IP information index pool. The name of the secondary IP information index pool includes the integer value corresponding to the start IP and the integer value corresponding to the end IP. In the secondary IP information index pool, the KEY of the index is the integer value corresponding to the complete start IP and the end IP, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0098] If the starting IP and ending IP span multiple segments, then the starting IP and ending IP are stored in the cross-segment IP information index pool. In the cross-segment IP information index pool, the index keys are the complete integer values ​​corresponding to the starting IP and ending IP, respectively, and the index VALUE values ​​are the secondary index KEY|starting IP corresponding integer value$ending IP corresponding integer value, and the secondary index KEY#starting IP corresponding integer value$ending IP corresponding integer value. The sorting number of the index is KEY. The symbols |, #, and $ all represent information separators within the code index.

[0099] Module MB.3: For IPv6, the first integer values ​​of the start IP and end IP are organized into a first-level IP information index and stored in the first-level IP information index pool. In the first-level IP information index pool, the KEY of the index is the first integer value of the start IP and end IP, the VALUE of the index is the first integer value of the start IP and the first integer value of the end IP, and the sorting number of the index is the KEY.

[0100] The starting IP, ending IP, and the integer values ​​of the middle segment are organized into a secondary IP information index and stored in the corresponding secondary IP information index pool. In the secondary IP information index pool, the name of the secondary IP information index pool contains the starting IP first segment integer value and the starting IP middle segment integer value. The keys of the index are the starting IP middle segment integer value and the ending IP middle segment integer value, respectively. The value of the index is secondary index key|starting IP middle segment integer value$ending IP middle segment integer value. The sorting number of the index is key.

[0101] The starting IP and the ending IP tail integer values ​​are organized into a three-level IP information index and stored in the corresponding three-level IP information index pool. In the three-level IP information index pool, the name of the three-level IP information index pool contains the starting IP middle segment integer value and the starting IP tail segment integer value information. The KEY of the index is the starting IP tail segment integer value, the VALUE of the index is the complete attribution information, and the sorting number of the index is the KEY.

[0102] Preferably, the module MD includes the following modules:

[0103] Module MD.1: Automatically identifies IPv4 and IPv6 for the IP information that needs to be matched;

[0104] Module MD.2: For IPv4, convert the first three segments of the IP address into integer values, and retrieve the first second-level IP information index in the primary IP information index pool whose sort number is less than or equal to the integer value;

[0105] If the secondary IP information index is not found, the cross-segment IP information index is searched in the secondary IP information index pool to obtain the secondary IP information index value; the secondary IP information index is then used to search the corresponding secondary IP information index pool to obtain the attribution information.

[0106] Module MD.3: For IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information;

[0107] Module MD.4: Converts the full IPv6 information to obtain the first segment integer value, middle segment integer value, and last segment integer value of IPv6; uses the first segment integer value to retrieve the first second-level IP information index with a sort number less than or equal to the first segment integer value from the first-level IP information index pool; uses the middle segment integer value to retrieve the third-level IP information index value from the corresponding second-level IP information index pool; uses the last segment integer value to retrieve the attribution information from the corresponding third-level IP information index pool.

[0108] Preferably, in the module MA, offline data of IP address information is obtained in fragments using a multi-concurrency method;

[0109] During storage, the cache indexes at all levels are synchronously sorted according to the sorting index from smallest to largest;

[0110] In the MB module, a hash-based storage cache is used according to the index storage model, wherein the VALUE value of each level index stores all the starting and ending indexes of the next level index of that segment.

[0111] According to the present invention, a computer-readable storage medium storing a computer program is provided, wherein when the computer program is executed by a processor, the steps of the IP address offline data matching method based on multi-level cache index are implemented.

[0112] An electronic device according to the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the IP address offline data matching method based on multi-level cache index.

[0113] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0114] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An IP address offline data matching method based on a hash multi-level cache index, characterized in that, include: Step SA: Obtain offline data for IP address information; Step SB: Cache and preheat the IP information of each record of the offline IP address information data to the hash target IP information index pool; wherein, the IP information is segmented and stored in different levels of target IP information index pools according to different IP segments; Step SC: Based on the IP information that needs to be matched in the real-time transaction, notify the IP retrieval engine in real time to index the IP attribution information of the target IP information index pool that has been preheated; wherein, the IP retrieval engine splits the IP information that needs to be matched and retrieves it from the corresponding target IP information index pool. Step SB includes the following steps: Step SB.1: Distinguish between IPv4 and IPv6 for the offline data of the IP address information; Step SB.2: For IPv4, organize the first two segments of the start IP and end IP into a first-level IP information index and store them in the first-level IP information index pool; specifically: in the first-level IP information index pool, the KEY of the index is the integer value corresponding to the first two segments of the IP, and the VALUE of the index is the integer value corresponding to the start IP and the integer value corresponding to the end IP. Complete IP information is stored in a secondary IP information index pool. The name of the secondary IP information index pool includes the integer value corresponding to the start IP and the integer value corresponding to the end IP. In the secondary IP information index pool, the KEY of the index is the integer value corresponding to the start IP and the integer value corresponding to the end IP, and the VALUE of the index is the complete attribution information. If the starting IP and ending IP span multiple segments, then the starting IP and ending IP are stored in the cross-segment IP information index pool. In the cross-segment IP information index pool, the index keys are the integer values ​​corresponding to the starting IP and the ending IP, respectively, and the index VALUE values ​​are the secondary index KEY|starting IP integer value &ending IP integer value and the secondary index KEY#starting IP integer value &ending IP integer value, respectively. The sorting number of the index is KEY. The symbols |, #, and & all represent information separators within the code index, and the sorting number of the index is KEY. Step SB.3: For IPv6, organize the integer values ​​corresponding to the first segment of the start IP and end IP into a first-level IP information index and store them in the first-level IP information index pool; specifically: in the first-level IP information index pool, the index keys are the integer values ​​corresponding to the first segment of the start IP and the integer values ​​corresponding to the first segment of the end IP, and the index VALUE is the integer values ​​corresponding to the first segment of the start IP & the integer values ​​corresponding to the first segment of the end IP. The integer values ​​corresponding to the start IP, end IP, and middle segment are organized into a secondary IP information index and stored in the corresponding secondary IP information index pool. Specifically, in the secondary IP information index pool, the name of the secondary IP information index pool contains the integer value of the start IP first segment and the integer value of the start IP middle segment. The KEY of the index is the integer value corresponding to the start IP middle segment and the integer value corresponding to the end IP middle segment, respectively. The VALUE of the index is secondary index KEY|integer value corresponding to the start IP middle segment & integer value corresponding to the end IP middle segment. The sorting number of the index is KEY. The integer values ​​corresponding to the start IP and end IP tail segments are organized into a three-level IP information index and stored in the corresponding three-level IP information index pool. Specifically, in the three-level IP information index pool, the name of the three-level IP information index pool contains the integer values ​​of the start IP middle segment and the start IP tail segment. The KEY of the index is the integer value corresponding to the start IP tail segment and the integer value corresponding to the end IP tail segment. The VALUE of the index is the complete attribution information. The sorting number of the index is the KEY.

2. The hash-based multi-level cache index based IP address offline data matching method according to claim 1, characterized in that, It also includes step SD, which includes the following steps: Step SD.1: Automatically identify IPv4 and IPv6 for the IP information that needs to be matched; Step SD.2: For IPv4, convert the first three segments of the IP address into integer values, and retrieve the second-level IP information index whose sort number is less than or equal to the integer value from the first-level IP information index pool; If the secondary IP information index is not found, the cross-segment IP information index is searched in the cross-segment IP information index pool to obtain the secondary IP information index value; the secondary IP information index value is then used to retrieve the attribution information from the corresponding secondary IP information index pool. Step SD.3: For IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information; Step SD.4: Convert the full IPv6 information to obtain the integer values ​​corresponding to the first segment, the middle segment, and the last segment of IPv6; Use the integer value corresponding to the first segment to retrieve the second-level IP information index whose sort number is less than or equal to the integer value corresponding to the first segment from the first-level IP information index pool; use the integer value corresponding to the middle segment to retrieve the third-level IP information index value from the corresponding second-level IP information index pool; use the integer value corresponding to the last segment to retrieve the attribution information from the corresponding third-level IP information index pool.

3. The hash-based multi-level cache index based IP address offline data matching method according to claim 2, characterized in that, In step SA, the IP address information offline data is obtained by fragmentation using a multi-concurrency method.

4. An IP address offline data matching system based on hash multi-level cache index, characterized in that, include: Module MA: Obtains offline data including IP address information; Module MB: Caches and preheats the IP information of each record of the offline IP address information data to the hash target IP information index pool; wherein, the IP information is segmented and stored in different levels of target IP information index pools according to different IP segments; Module MC: Based on the IP information that needs to be matched in real-time transactions, it notifies the IP retrieval engine in real time to index the IP attribution information of the target IP information index pool that has been preheated; wherein, the IP retrieval engine splits the IP information that needs to be matched and retrieves it from the corresponding target IP information index pool. The module MB includes the following modules: Module MB.1: Distinguish between IPv4 and IPv6 for the offline data of the IP address information; Module MB.2: For IPv4, the first two segments of the starting IP and the ending IP are organized into a first-level IP information index and stored in the first-level IP information index pool; specifically: in the first-level IP information index pool, the KEY of the index is the integer value corresponding to the first two segments of the IP, and the VALUE of the index is the integer value corresponding to the starting IP and the integer value corresponding to the ending IP. Complete IP information is stored in a secondary IP information index pool. The name of the secondary IP information index pool includes the integer value corresponding to the start IP and the integer value corresponding to the end IP. In the secondary IP information index pool, the KEY of the index is the integer value corresponding to the start IP and the integer value corresponding to the end IP, and the VALUE of the index is the complete attribution information. If the starting IP and ending IP span multiple segments, then the starting IP and ending IP are stored in the cross-segment IP information index pool. In the cross-segment IP information index pool, the index keys are the integer values ​​corresponding to the starting IP and the ending IP, respectively, and the index VALUE values ​​are the secondary index KEY|starting IP integer value &ending IP integer value and the secondary index KEY#starting IP integer value &ending IP integer value, respectively. The sorting number of the index is KEY. The symbols |, #, and & all represent information separators within the code index, and the sorting number of the index is KEY. Module MB.3: For IPv6, the integer values ​​corresponding to the first segment of the start IP and end IP are organized into a first-level IP information index and stored in the first-level IP information index pool; specifically: in the first-level IP information index pool, the index keys are the integer values ​​corresponding to the first segment of the start IP and the integer values ​​corresponding to the first segment of the end IP, and the index VALUE is the integer values ​​corresponding to the first segment of the start IP & the integer values ​​corresponding to the first segment of the end IP. The integer values ​​corresponding to the start IP, end IP, and middle segment are organized into a secondary IP information index and stored in the corresponding secondary IP information index pool. Specifically, in the secondary IP information index pool, the name of the secondary IP information index pool contains the integer value of the start IP first segment and the integer value of the start IP middle segment. The KEY of the index is the integer value corresponding to the start IP middle segment and the integer value corresponding to the end IP middle segment, respectively. The VALUE of the index is secondary index KEY|integer value corresponding to the start IP middle segment & integer value corresponding to the end IP middle segment. The sorting number of the index is KEY. The integer values ​​corresponding to the start IP and end IP tail segments are organized into a three-level IP information index and stored in the corresponding three-level IP information index pool. Specifically, in the three-level IP information index pool, the name of the three-level IP information index pool contains the integer values ​​of the start IP middle segment and the start IP tail segment. The KEY of the index is the integer value corresponding to the start IP tail segment and the integer value corresponding to the end IP tail segment. The VALUE of the index is the complete attribution information. The sorting number of the index is the KEY.

5. The IP address offline data matching system based on hash-based multi-level cache index according to claim 4, characterized in that, It also includes module MD, which includes the following modules: Module MD.1: Automatically identifies IPv4 and IPv6 for the IP information that needs to be matched; Module MD.2: For IPv4, convert the first three segments of the IP address into integer values, and retrieve the first second-level IP information index in the primary IP information index pool whose sort number is less than or equal to the integer value; If the secondary IP information index is not found, then search the cross-segment IP information index pool to obtain the secondary IP information index value; Use the secondary IP information index value to retrieve the corresponding secondary IP information index pool and obtain the attribution information; Module MD.3: For IPv6, perform full IP completion on the IPv6 to form the complete IPv6 information; Module MD.4: Converts the full IPv6 information to obtain the integer values ​​corresponding to the first segment, the middle segment, and the last segment of IPv6; Use the integer value corresponding to the first segment to retrieve the second-level IP information index whose sort number is less than or equal to the integer value corresponding to the first segment from the first-level IP information index pool; use the integer value corresponding to the middle segment to retrieve the third-level IP information index value from the corresponding second-level IP information index pool; use the integer value corresponding to the last segment to retrieve the attribution information from the corresponding third-level IP information index pool.

6. The IP address offline data matching system based on hash-based multi-level cache index according to claim 5, characterized in that, In the MA module, offline data of IP address information is obtained in fragments using a multi-concurrency approach.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the offline IP address data matching method based on hash-based multi-level cache index as described in any one of claims 1 to 3.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the offline IP address data matching method based on hash-based multi-level cache index as described in any one of claims 1 to 3.

Images