IP address library loading and positioning method and system based on second-level cache mechanism, computer readable storage medium and electronic equipment

Abstract

The invention belongs to the technical field of computer information technology and network data processing, and discloses an IP address library loading and positioning method and system based on a second-level cache mechanism, a computer readable storage medium and electronic equipment. The method comprises the following steps of: partitioning a complete IP address library according to a first section numerical value of an IP, and storing the partitioned IP address library in an ordered set of an external cache Redis; establishing a local cache for storing hotspot data in the application; during query, the internal cache is accessed firstly, if the internal cache is not hit, a Redis partition key is determined according to an IP first segment, an external cache is queried by using a ZRANGEBYSCORE instruction, and a result is backfilled into the internal cache; and managing the internal cache through regular expiration and an LRU strategy, and synchronously updating external cache data. The system comprises an external cache, an internal cache, an IP matching module and a cache management module. According to the method, memory occupation and database pressure are effectively reduced, query efficiency and system expansibility are remarkably improved, and the method is suitable for a high-concurrency IP positioning scene.

Description

Technical Field

[0001] This invention belongs to the field of computer information technology and network data processing, and particularly relates to a technology for storing, querying and locating Internet Protocol (IP) address databases. Specifically, it relates to a method and system for loading and locating IP address databases based on a two-level caching mechanism, a computer-readable storage medium, and an electronic device. Background Technology

[0002] IP address location technology refers to determining the corresponding geographical location information (such as administrative divisions, latitude and longitude) of an IP address. This technology has wide applications in many fields such as network security, traffic analysis, user behavior statistics, access control, and advertising. With the continuous expansion of the Internet and the rapid growth of data volume, the size of the IP address database has reached tens of millions. How to efficiently and accurately achieve IP address matching and querying has become a key technical challenge.

[0003] Currently, the common methods for locating IP addresses mainly include the following three: Linked list structure and binary search: The IP address database is sorted according to certain rules and loaded into memory, stored in the form of a linked list. A binary search algorithm is used for matching during queries. Although this method is simple to implement, its memory usage increases significantly as the IP address database grows, and each query requires traversing a large amount of data, resulting in low efficiency.

[0004] Binary Tree Storage and Query: The IP address database is stored in memory as a binary tree structure, leveraging the characteristics of binary trees to improve query efficiency. Although this is an improvement over linked list structures, the entire IP address database still needs to be loaded into memory, resulting in high memory consumption. Furthermore, in extreme cases, a binary tree may degenerate into a linked list, leading to a decrease in query performance.

[0005] Direct database retrieval: This method stores the IP address database in a database and directly accesses the database for matching during each query. While this method saves memory, in high-concurrency query scenarios, frequent database access can lead to a surge in database pressure and prolonged response time, making it difficult to meet the real-time requirements of applications.

[0006] Furthermore, although some existing caching technologies (such as local caching and distributed caching) can improve query efficiency to some extent, traditional caching strategies cannot effectively utilize data access patterns due to the large data volume and special access patterns of IP address databases (such as frequent access by hot IPs and very few access by cold IPs). This makes it difficult to significantly reduce the access pressure on the underlying storage system, thus limiting the overall performance of the system.

[0007] Therefore, there is an urgent need for an IP address library loading and location method that can balance memory efficiency, query performance and system scalability to meet the IP address matching needs in large-scale, high-concurrency scenarios. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for loading and locating IP address databases based on a two-level caching mechanism, so as to solve the problems of large memory consumption, low query efficiency, high database pressure and poor scalability in existing IP address location technologies.

[0009] This invention provides a method for loading and locating an IP address database based on a two-level caching mechanism. The method performs at least the following steps: Step 1: Initialize the external cache: Store the complete IP address database into an ordered set in Redis, using the first segment of the IP address as the partition key, and store different IP addresses in different partitions in Redis according to the value of their first segment; Step 2: Build an internal cache: Create a cache space within the system to store recently hit and missed hot data from the external cache; Step 3: Matching and querying IP addresses: The application receives the query request and parses the target IP address; first, it performs a match in the internal cache, and if a match is found, it returns the result; if no match is found, it extracts the first segment of the IP address, constructs a partition key, and then determines the target sorted set key name in Redis; using this partition key, it queries the corresponding sorted set partition in Redis to obtain information related to the target IP address.

[0010] The IP address database loading and locating method based on the second-level caching mechanism described above further includes a cache update and management step, specifically including: For internal caching, manage it by setting a cache expiration policy; For external caching, periodically check for updates to the IP address database, and update the database when new IP addresses are added. When data needs to be added or existing data needs to be modified, the data in the corresponding sorted set partition in Redis should be updated promptly.

[0011] The IP address database loading and location method based on the second-level caching mechanism described above, wherein the internal cache is managed by setting a cache expiration policy, specifically includes: Set an internal cache data expiration policy of one hour and use the least recently used algorithm; when the internal cache... When the space is full, delete the least recently used data to make room for newly matched data.

[0012] As described above, the method for loading and locating the IP address database based on a second-level caching mechanism involves periodically checking the update status of the IP address database for the external cache. When new IP address data needs to be added or existing data needs to be modified, the data in the corresponding ordered set partition in Redis is updated promptly. Specifically, this includes: When new IP address data needs to be added, the IP address database is fully loaded into the external cache. When existing data needs to be modified, incrementally add IP data to the external cache.

[0013] As described above, the IP address library loading and locating method based on a two-level caching mechanism uses a hash table data structure for the internal cache to facilitate fast data lookup and insertion.

[0014] As described above, the method for loading and locating IP address libraries based on a two-level caching mechanism includes, in step 2, the following steps: Within the application, using the Guava Cache or Caffeine caching library, two cache instances are created: CACHE_HIT, used to cache recently successfully hit IP addresses and their complete information; and CACHE_NOT_HIT, used to cache recently confirmed non-existent IP addresses.

[0015] As described above, the method for loading and locating the IP address database based on the second-level cache mechanism includes, in step 3, the following steps: When performing internal cache matching, first match CACHE_HIT. If a match is found, directly return the relevant information of the target IP address; if CACHE_HIT is not found, query CACHE_NOT_HIT. If a match is found, directly return empty; if CACHE_NOT_HIT is not found, query Redis. If a result is found in Redis, it is deserialized into an object, and the IP address is stored in CACHE_HIT. If no result is found in Redis, the IP address is stored in CACHE_NOT_HIT. At the same time, when querying in Redis, the IP address is converted into an integer value, and the ZRANGEBYSCORE instruction is used to perform a range search in the corresponding sorted set to match the range containing the IP address.

[0016] This invention also provides an IP address database loading and location system based on a two-level caching mechanism. This system executes the IP address database loading and location method based on a two-level caching mechanism as described in any one of claims 1-6. The system includes: an external caching module, an internal caching module, an IP address matching module, and a cache management module; wherein... The external cache module is used to store the complete IP address database data. It uses the first segment of the IP address as the partition key to store the IP address database data in different ordered set partitions and provides data query, insertion and update functions. An internal caching module is used to cache the results of hot IP address queries. The internal caching module is built inside the application system and adopts a hash table data structure to store recent hot data that has been hit or missed in the external cache. It supports fast data query and insertion operations and manages data according to the set cache expiration policy. The IP address matching module is used to perform IP address query operations and coordinate access to internal and external caches. Specifically, it is responsible for receiving the IP address to be matched, obtaining the first segment of the IP address to build the partition key, querying data from the corresponding partition of the external cache module, and writing the query result to the internal cache module. At the same time, when the data of the IP address to be matched exists in the internal cache, the matching result is retrieved from the internal cache first. The cache management module is used to manage the data updates and maintenance of internal and external caches; for internal caches, it executes cache expiration policies; for external caches, it monitors changes in the IP address database and updates the data in Redis in a timely manner.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to perform the IP address library loading and locating method based on the second-level cache mechanism described above.

[0018] The present invention also provides an electronic device, including one or more processors; a storage device for storing one or more computer programs; when the one or more computer programs are executed by the one or more processors, the electronic device enables the IP address library loading and locating method based on the second-level cache mechanism described above.

[0019] To achieve the above objectives, this invention adopts the following technical solution: This invention discloses a method and system for loading and locating an IP address database based on a two-level caching mechanism, a computer-readable storage medium, and an electronic device. The method includes: partitioning the complete IP address database according to the first segment of the IP address and storing it in an ordered set of an external cache, Redis; establishing a local cache within the application to store hot data; during a query, first accessing the internal cache; if no match is found, determining the Redis partition key based on the first segment of the IP address, using the ZRANGEBYSCORE instruction to query the external cache, and then filling the internal cache with the result; managing the internal cache through periodic expiration and an LRU strategy, and synchronously updating the external cache data. The system includes an external cache module, an internal cache module, an IP matching module, and a cache management module. This invention effectively reduces memory usage and database pressure, significantly improves query efficiency and system scalability, and is suitable for high-concurrency IP location scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the IP address matching query method provided in an embodiment of the present invention. Figure 2 A flowchart illustrating the cache update and management method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the IP address database matching system architecture based on a two-level caching mechanism provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will be described in detail below through embodiments.

[0022] The following description, in conjunction with the accompanying drawings, details the specific solutions provided by this invention for a method and system for loading and locating an IP address database based on a two-level caching mechanism, as well as for electronic devices and computer-readable storage media.

[0023] Example 1: Please see Figure 1 This is a flowchart illustrating the IP address matching and query method provided in this application embodiment. Figure 1 As shown, the process of loading and locating the IP address library based on the two-level caching mechanism is specifically illustrated.

[0024] The present invention provides a method for loading and locating an IP address database based on a two-level caching mechanism, the implementation process of which is as follows: Step 1: Initialize the external cache: Store the complete IP address database into an ordered set in Redis, using the first segment of the IP address as the partition key, and store different IP addresses in different partitions in Redis according to the value of their first segment; It should be noted that Redis (Remote Dictionary Server) is an open-source, in-memory key-value storage system that is widely used in high-performance caching, real-time data processing, and distributed architectures.

[0025] When importing IP address database data, a script or data import tool is used to calculate a partition key for each record in the database based on the first segment of its IP address (e.g., the first segment of IP address 192.168.1.1 is 192). A sorted set is created in Redis named after this partition key. The numerical form of the IP address (e.g., converted from IP to integer) is used as the score, and the associated geographic information is serialized into a string and used as a member. All partition keys are stored in a separate Redis set for easier management.

[0026] Its code implementation is as follows: / / Store IP data in Redis cache public void addBaseIpInfo(BaseIpInfo baseIpInfo){ BigInteger srcIP = new BigInteger(baseIpInfo.getSrcIp()); / / Get partition key String ipPrex = getIPPrex(srcIP); / / Find the partition corresponding to the sorted set based on the partition key String redisKey = getRedisKey(ipPrex); Map<String,Double> data = new HashMap<>(); data.put(baseIpInfo.getIpInfo(),srcIP.doubleValue()); / / Save IP information to a sorted set in Redis REDIS_CLUSTER.zadd(redisKey, data); / / Save partition keys to a Redis collection REDIS_CLUSTER.sadd(REDIS_KEY_BASE_IP_KEY,new String[]{ipPrex}); } Step 2: Build an internal cache: Create a cache space within the system to store recently hit and missed hot data from the external cache; It should be noted that the internal cache can use data structures such as hash tables to facilitate fast data lookup and insertion. Within the application, two cache instances can be created using caching libraries such as Guava Cache or Caffeine: 1. CACHE_HIT: Used to cache the most recently successfully hit IP address and its complete information (BaseIpInfo object); 2. CACHE_NOT_HIT: Used to cache IP addresses that have recently been confirmed as non-existent.

[0027] Both caches are configured with a maximum capacity of 1,000,000 entries, expire 1 hour after being written, and use an LRU eviction policy.

[0028] Its code implementation is as follows: / / Create a new LRU cache to store the list of hot IPs that have been hit by the external cache. / / Set the cache data expiration time to 1 hour private static Cache<String, BaseIpInfo> CHACHE_HIT = CacheBuilder.newBuilder().maximumSize(1000000) .expireAfterWrite(1, TimeUnit.HOURS) .build(); / / Create a new LRU cache to store a list of external hot IPs that have not been hit. / / Set the cache data expiration time to 1 hour private static Cache<String, Boolean> CHACHE_NOT_HIT = CacheBuilder.newBuilder().maximumSize(1000000) .expireAfterWrite(1, TimeUnit.HOURS) .build(); Step 3: Matching and querying IP addresses: The application receives the query request and parses the target IP address; first, it performs a match in the internal cache, and if a match is found, it returns the result; if no match is found, it extracts the first segment of the IP address, constructs a partition key, and then determines the target sorted set key name in Redis; using this partition key, it queries the corresponding sorted set partition in Redis to obtain information related to the target IP address.

[0029] The code implementation for IP address matching is as follows: / / IP address matching public BaseIpInfo getBaseIpInfo(String ip){ / / Prioritize retrieving from the internal cache of already matched hot IPs BaseIpInfo baseIpInfo = CHACHE_HIT.getIfPresent(ip); if(baseIpInfo != null){ return baseIpInfo; } / / If the IP address is not found in the internal hotspot IP cache list, return an empty value. Boolean isNotExist = CHACHE_NOT_HIT.getIfPresent(ip); if(isNotExist){ return null; } / / Load from external cache return this.loadBaseIpInfoFromExternalCache(ip); } / / Load from external cache private BaseIpInfo loadBaseIpInfoFromExternalCache(String ip){ BaseIpInfo baseIpInfo = baseIPExternalCache.getBaseIpInfo(ip); / / If the IP address is not found in the external cache, add it to the list of hot IP addresses that were not found in the cache. if(baseIpInfo == null){ CHACHE_NOT_HIT.put(ip, true); return null; } / / If an IP address is found in the external cache, it is added to the list of cached hot IP addresses. CHACHE_HIT.put(ip,baseIpInfo); return baseIpInfo; } The code implementation for its IP address lookup is as follows: public BaseIpInfo getBaseIpInfo(String ip){ BigInteger startIp = IPAddressUtil.ipStrToBigInteger(ip); BigInteger endIp = getEndIp(startIp); / / Get the partition key of an ordered set by IP address String redisKey = this.getRedisKey(startIp); / / Query IP data from an ordered set by partition key Set <tuple>resultSet = REDIS_CLUSTER.zrangeByScoreWithScores(redisKey,startIp.toString(),end Ip.toString(),0,1); if(resultSet == null){ logger.warn("BaseIPExternalCache - getBaseIpInfo warn resultSet cannt be null"); return null; } return toBaseIPInfo(resultSet.iterator().next()); } It should be noted that when querying the internal cache as described above, if it is the first time querying an IP address, both `CACHE_HIT` and `CACHE_NOT_HIT` will be null. Further, when matching in the internal cache, `CACHE_HIT` is checked first. If a match is found, the relevant information of the target IP address is returned directly. If `CACHE_HIT` is not found, `CACHE_NOT_HIT` is queried. If a match is found, null is returned directly. If `CACHE_NOT_HIT` is not found, the external cache Redis is queried. If a result is found in Redis, it is deserialized into an object, and the IP address is stored in `CACHE_HIT`. If no result is found in Redis, the IP address is stored in `CACHE_NOT_HIT`. Simultaneously, when querying in Redis, the IP address is converted to an integer value, and the `ZRANGEBYSCORE` instruction is used to perform a range search in the corresponding ordered set to match the range containing the IP address. `ZRANGEBYSCORE` is a Redis command used to manipulate Sorted Sets. Its function is to return all members in the Sorted Set whose scores fall within a specified range. A Sorted Set is a Redis data structure similar to a Set, but each member is associated with a score; members cannot be duplicated, but scores can be repeated, and members are sorted by their scores in ascending order. In this embodiment, an integer value converted from an IP address can be used as the score, which is the basis for achieving fast range matching; members are serialized strings (e.g., JSON strings or some encoding format) of the geographic location information (such as country, province, city, latitude and longitude) corresponding to the IP range.

[0030] As can be seen, after a query is completed in Redis, the query result is written to the internal cache so that subsequent queries for the same IP address can be retrieved from the internal cache, reducing access to Redis. When the query result returns empty, it means that the IP address is not included in the IP address database. The IP address is then written to the internal cache. When the user queries the same IP address again, an empty value is returned directly, avoiding frequent queries to the external cache due to IP addresses not included in the external cache.

[0031] Example 2: Please see Figure 2 This is a flowchart illustrating the cache update and management method provided in the embodiments of this application. Figure 2 As shown, a specific cache update and management method flow is illustrated.

[0032] The above-mentioned method for loading and locating the IP address database based on the second-level caching mechanism also includes cache update and management steps, specifically: for internal cache, setting a cache expiration policy for management; for external cache, periodically checking the update status of IP address database data, and promptly updating the data in the corresponding sorted set partition in Redis when new IP address data needs to be added or existing data needs to be modified.

[0033] The implementation code for updating the external cache is as follows: / / Fully load the IP address database into the external cache public staticvoid loadDataToRedis()throws Exception { int currentPage = 0; while(true){ List <baseipinfo>resultList = queryDataByPage(currentPage); if(resultList == null || resultList.isEmpty()){ break } batchAddBaseIPInfo(resultList); currentPage++ } } / / Incrementally add new IP data to the external cache public static void batchAddBaseIPInfo(List <baseipinfo>resultList){ for(BaseIpInfo baseIpInfo:resultList){ BASE_IP_EXTERNAL_CACHE.addBaseIpInfo(baseIpInfo); } } It should be noted that the cache expiration policy set for the internal cache can be, for example, setting the internal cache data to expire in one hour and using the Least Recently Used (LRU) algorithm. When the internal cache space is full, the least recently used data is deleted to make room for newly hit data.

[0034] For the external cache (Redis), the update status of the IP address database is checked regularly. When new IP address data needs to be added or existing data needs to be modified, the data in the corresponding sorted set partition in Redis is updated in a timely manner. The corresponding operations include fully loading the IP address database into the external cache and incrementally adding IP data to the external cache.

[0035] Example 3: Please see Figure 3 This is a schematic diagram of the IP address database matching system architecture based on a two-level caching mechanism provided in an embodiment of this application. Figure 3 As shown, a specific IP address database matching system based on a two-level caching mechanism is illustrated.

[0036] This invention provides an IP address database loading and location system based on a two-level caching mechanism. This system executes the IP address database loading and location method based on a two-level caching mechanism described in the above embodiments. The system includes: an external caching module, an internal caching module, an IP address matching module, and a cache management module. The external cache module is used to store complete IP address database data. It uses the first segment of the IP address as the partition key to store the address data in different ordered set partitions and provides data query, insertion and update functions. An internal caching module is used to cache the results of hot IP address queries. This internal caching module is built inside the application system and uses a hash table data structure to store recent hot data that has been hit or missed in the external cache. It supports fast data query and insertion operations and manages data according to the set cache expiration policy.

[0037] The IP address matching module is used to perform IP address query operations and coordinate access to internal and external caches. Specifically, it is responsible for receiving the IP address to be matched, obtaining the first segment of the IP address to construct the partition key, querying data from the corresponding partition of the external cache module, and writing the query result to the internal cache module. At the same time, when the data of the IP address to be matched exists in the internal cache, the matching result is retrieved from the internal cache first.

[0038] The cache management module is used to manage the data updates and maintenance of internal and external caches; for internal caches, it executes cache expiration policies; for external caches, it monitors changes in the IP address database and updates the data in Redis in a timely manner.

[0039] Example 4: Please see the appendix Figure 4 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention, such as... Figure 4 As shown, a computer-readable storage medium 4000 stores program code 4100 for performing the method steps according to the present invention. When executed by a processor, the program code 4100 for performing the method steps according to the present invention implements the IP address database loading and location method based on the second-level cache mechanism as described above. The IP address database loading and location method based on the second-level cache mechanism has been described in detail above and will not be repeated here.

[0040] The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. The computer-readable medium 4000 may include computer storage media and communication media, and may also include any medium capable of transferring a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0041] As one possible design, computer-readable medium 4000 may include compact disc read-only memory (CD ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable medium may include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers.

[0042] Example 5: See Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application, such as... Figure 5 As shown, the electronic device 5000 includes a memory 5100, a processor 5200, and a computer program stored in the memory 5100 and executable by the processor, wherein the processor 5200 executes... Figure 4 When the program code 4100 stored in the computer-readable storage medium 4000, which is used to execute the method steps according to the present invention, performs the steps of the method of the present invention, it can realize the IP address database loading and location method based on the two-level cache mechanism. The method of IP address database loading and location based on the two-level cache mechanism has been described in detail above, and will not be repeated here. The memory 5100 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 5100 has storage space 5300 for storing program code 4100 for executing the method steps according to the invention, which is used to perform any of the method steps described above. The program code 4100 for executing the method steps according to the invention can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. Such computer program products are typically... Figure 4 The computer-readable storage medium shown is shown. A computer device may include multiple processors, each of which may be a single-core processor or a multi-core processor. A processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0043] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0044] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0045] The above are preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made to the structure, shape and principle of this application should be covered within the scope of protection of this application.< / baseipinfo> < / baseipinfo> < / tuple>

Claims

1. A method for loading and locating an IP address database based on a two-level caching mechanism, characterized in that, The method shall perform at least the following steps: Step 1: Initialize the external cache: Store the complete IP address database into an ordered set in Redis, using the first segment of the IP address as the partition key, and store different IP addresses in different partitions in Redis according to the value of their first segment; Step 2: Build an internal cache: Create a cache space within the system to store recently hit and missed hot data from the external cache; Step 3: Matching and querying IP addresses: The application receives the query request and parses the target IP address; first, it performs a match in the internal cache. If a match is found, the result is returned; if no match is found, the first segment of the IP address is extracted, a partition key is constructed, and then the target sorted set key name in Redis is determined. Use the partition key to query the corresponding sorted set partition in Redis and retrieve information related to the target IP address.

2. The method for loading and locating an IP address database based on a two-level caching mechanism according to claim 1, characterized in that, The method also includes cache update and management steps, specifically including: For internal caching, manage it by setting a cache expiration policy; For external caching, periodically check for updates to the IP address database, and update the database when new IP addresses are added. When data needs to be added or existing data needs to be modified, the data in the corresponding sorted set partition in Redis should be updated promptly.

3. The method for loading and locating an IP address database based on a two-level caching mechanism according to claim 2, characterized in that, The internal cache is managed by setting a cache expiration policy, specifically including: Set an internal cache data expiration policy of one hour and use the least recently used algorithm; when the internal cache... When the space is full, delete the least recently used data to make room for newly matched data.

4. The method for loading and locating IP address databases based on a two-level caching mechanism according to claim 2, characterized in that, Regarding external caching, the update status of the IP address database is checked periodically. When new IP address data needs to be added or existing data needs to be modified, the data in the corresponding sorted set partition in Redis is updated promptly. Specifically, this includes: When new IP address data needs to be added, the IP address database is fully loaded into the external cache. When existing data needs to be modified, incrementally add IP data to the external cache.

5. The method for loading and locating an IP address database based on a two-level caching mechanism according to claim 1, characterized in that, The internal cache uses a hash table data structure to facilitate fast data lookup and insertion.

6. The method for loading and locating an IP address database based on a two-level caching mechanism according to claim 1, characterized in that, Step 2 specifically includes: within the application, using the Guava Cache or Caffeine caching library, creating two cache instances: CACHE_HIT, used to cache recently successfully hit IP addresses and their complete information; and CACHE_NOT_HIT, used to cache recently confirmed non-existent IP addresses.

7. The method for loading and locating an IP address database based on a two-level caching mechanism according to claim 6, characterized in that, Step 3 specifically includes: when performing internal cache matching, first match CACHE_HIT. If a match is found, directly return the relevant information of the target IP address; if CACHE_HIT is not found, query CACHE_NOT_HIT. If a match is found, directly return empty; if CACHE_NOT_HIT is not found, query Redis. If a result is found in Redis, it is deserialized into an object, and the IP address is stored in CACHE_HIT. If no result is found in Redis, the IP address is stored in CACHE_NOT_HIT. At the same time, when querying in Redis, the IP address is converted into an integer value, and the ZRANGEBYSCORE instruction is used to perform a range search in the corresponding sorted set to match the range containing the IP address.

8. A system for loading and locating an IP address database based on a two-level caching mechanism, the system executing the IP address database loading and locating method based on a two-level caching mechanism as described in any one of claims 1-6; the system comprising: The system comprises an external caching module, an internal caching module, an IP address matching module, and a cache management module; among which, The external cache module is used to store the complete IP address database data. It uses the first segment of the IP address as the partition key to store the IP address database data in different ordered set partitions and provides data query, insertion and update functions. An internal caching module is used to cache the results of hot IP address queries. The internal caching module is built inside the application system and adopts a hash table data structure to store recent hot data that has been hit or missed in the external cache. It supports fast data query and insertion operations and manages data according to the set cache expiration policy. The IP address matching module is used to perform IP address query operations and coordinate access between internal and external caches. Specifically, it is responsible for receiving the IP address to be matched, obtaining the first segment of the IP address to build the partition key, querying data from the corresponding partition of the external cache module, and writing the query result to the internal cache module. At the same time, when the data of the IP address to be matched exists in the internal cache, the matching result is retrieved from the internal cache first. The cache management module is used to manage the data updates and maintenance of internal and external caches; for internal caches, it executes cache expiration policies; for external caches, it monitors changes in the IP address database and updates the data in Redis in a timely manner.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by the processor of the electronic device, causes the electronic device to perform the IP address library loading and locating method based on the second-level cache mechanism as described in any one of claims 1-6.

10. An electronic device, characterized in that, The device includes one or more processors; a storage device for storing one or more computer programs; and when the one or more computer programs are executed by the one or more processors, the electronic device causes the electronic device to implement the IP address library loading and locating method based on the secondary cache mechanism as described in any one of claims 1-6.

Images