Multi-level caching method and device for distributed database and medium

Abstract

The embodiment of the invention discloses a multi-level caching method and device for a distributed database and a medium, relates to the technical field of database management, and is used for solving the problem that the current dynamic collaborative management effect is poor. The method comprises the following steps: monitoring a configuration change event of a configuration center based on a long polling mechanism to obtain configuration update data; issuing a configuration rule parameter corresponding to the configuration updating data to determine a current caching strategy, and performing caching hierarchy switching based on the current caching strategy to realize caching of the current changed data; monitoring the cache to determine whether the current micro-service cluster receives a cache failure instruction or not; if yes, operation indexes of the distributed relational database are received in real time, and whether database exception exists or not is determined, so that the current caching strategy is backtracked and updated; and asynchronously synchronizing the current changed data to the distributed cache database based on the backtracking updated cache strategy.

Description

Technical Field

[0001] This specification relates to the technical field of data caching, and particularly to a multi-level caching method, device, and medium for a distributed database. Background Art

[0002] Large-scale microservice cluster architectures are widely used in various complex application scenarios to meet the growing business needs and high-concurrency access. At the same time, as the core of data storage and management, distributed databases are facing huge challenges. On the one hand, under the microservice architecture, different service modules access the database extremely frequently, and a large number of read and write operations will bring heavy load pressure to the database, affecting the overall performance and response speed of the system, and it is difficult to meet business scenarios with high real-time requirements. On the other hand, the complexity and variability of business logic make the dynamic adjustment of database configurations a normal state. Therefore, in this context, it is very important to dynamically configure and manage the database and cache system to quickly optimize the data caching strategy and achieve data caching synchronization.

[0003] Traditional solutions rely on static configuration files or manual operations. When it is necessary to adjust database connection parameters or Redis caching strategies, the service often needs to be restarted to take effect. This mechanism will increase the risk of business interruption in high-concurrency scenarios. Service restart may directly cause transaction failures or data loss. At the same time, the configuration switching in multiple environments lacks unified management, and relying on manual operations is prone to configuration errors or omissions, increasing the complexity of operation and maintenance. In terms of cache management, the single cache layer design with a fixed strategy is difficult to cope with dynamic business needs. In high-frequency access scenarios, cache penetration problems are likely to occur, and a large number of requests directly impact the database, resulting in a significant increase in response latency, and there is a lack of a real-time invalidation synchronization mechanism, making it difficult to guarantee data consistency. In addition, the monitoring of database exceptions in existing technologies mostly stays at the level of log records, lacking the ability of automatic threshold warning and rapid backtracking. When connection pool blocking or slow queries occur, operation and maintenance personnel need to manually check historical logs, and the fault recovery time is up to several hours, directly affecting system availability. Summary of the Invention

[0004] To solve the above technical problems, one or more embodiments of this specification provide a multi-level caching method, device, and medium for a distributed database.

[0005] One or more embodiments of this specification adopt the following technical solutions: One or more embodiments of this specification provide a multi-level caching method for a distributed database. The method includes: Listening for configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events; Based on the business scenario corresponding to the configuration update data, match the configuration rule parameters in the configuration center, and issue them to determine the current caching policy, so as to switch the cache level based on the current caching policy and implement the caching of the current changed data; Monitor the cache to determine whether the current microservice cluster has received a cache invalidation instruction; If so, receive the running metrics of the distributed relational database in real time, determine whether there is a database anomaly, so as to retrospectively update the current caching policy; Based on the retrospectively updated caching policy, asynchronously synchronize the current changed data to the distributed cache database.

[0006] Optionally, in one or more embodiments of this specification, before listening to the configuration change event of the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change event, the method further includes: Based on the initial configuration set of the configuration center, load the connection parameters of the distributed relational database and the caching policy of the distributed cache database to obtain the loaded data; Based on the namespace parameters and grouping parameters corresponding to the loaded data, perform configuration initialization for the current environment; Execute the pre-defined warm-up SQL rules of the configuration center to synchronously load the query results into the distributed relational database and the distributed cache database for caching, and complete the initial filling.

[0007] Optionally, in one or more embodiments of this specification, listening to the configuration change event of the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change event specifically includes: Obtain the configuration query request initiated by the current microservice cluster to the configuration center to determine whether there is a change in the target configuration corresponding to the configuration query request within the timeout parameter corresponding to the configuration query request; If not, return an empty response and re-initiate the configuration query request; If so, obtain the configuration change event corresponding to the change, and return the configuration update data corresponding to the configuration change event to trigger the non-restart hot update process of the configuration update data; wherein, the configuration update data includes: updated configuration value, version number.

[0008] Optionally, in one or more embodiments of this specification, based on the business scenario corresponding to the configuration update data, match the configuration rule parameters in the configuration center, and issue them to determine the current caching policy, so as to switch the cache level based on the caching policy and implement the caching of the configuration update data, specifically including: Based on the data characteristics corresponding to the configuration update data, determine the business scenario corresponding to the configuration update data; wherein, the business scenario includes: static data scenario, dynamic data scenario; Based on the existing cache policy and cache data, determine whether the configuration update data in the business scenario meets the preset hierarchical switching trigger condition, and if so, obtain the corresponding configuration rule parameters; Enable the current cache policy according to the configuration rule parameters issued by the configuration center, so as to switch the cache level based on the current cache policy and implement the caching of the configuration update data.

[0009] Optionally, in one or more embodiments of this specification, receive the running metrics of the distributed relational database in real time, determine whether there is a database anomaly, so as to roll back and update the current cache policy, specifically including: Receive the running metrics of the distributed relational database in real time to compare the running metrics with the corresponding preset metrics; wherein, the running metrics include: connection pool blocking rate, slow query frequency, transaction commit latency; Based on the comparison result, determine whether there is a database anomaly in the distributed relational database; If so, obtain the preset number of historical version configurations adjacent to the version number in the distributed relational database; Obtain the historical database anomaly data corresponding to each historical version configuration in the preset number of historical version configurations, so as to determine the most stable historical version configuration based on the historical database anomaly data; Roll back to the most stable historical version configuration based on the interface of the configuration center and synchronously update the current cache policy.

[0010] Optionally, in one or more embodiments of this specification, based on the cache policy after rollback update, asynchronously synchronize the current change data to the distributed cache database, specifically including: Listen to the WAL log of the distributed relational database through logical decoding to capture data change events in real time, and perform a structured conversion on the data change events to obtain change messages and send them to the MQ message queue; Sequentially obtain the change messages in the MQ message queue, and based on the business rules corresponding to the change messages, convert the change messages into operation instructions for the distributed cache database, and implement the synchronization of the distributed cache database based on the operation instructions.

[0011] Optionally, in one or more embodiments of this specification, after receiving the running metrics of the distributed relational database in real time, determining whether there is a database anomaly, and facilitating backtracking and updating the current caching policy, the method includes: Obtain the transaction requests corresponding to the distributed relational database, and sequentially write the transaction requests into the cache queue of the distributed cache database; Obtain a distributed lock through the RedLock algorithm to lock the resources corresponding to the transaction requests, and sequentially execute the transaction requests in the cache queue; If the execution is completed, send an update message based on the MQ message queue, asynchronously refresh the cache data of the distributed cache database, implement synchronous caching of the distributed cache database, and release the distributed lock.

[0012] Optionally, in one or more embodiments of this specification, after asynchronously synchronizing the current changed data to the distributed cache database based on the backtrack-updated caching policy, the method further includes: Determine whether there is a data conflict between the distributed cache database and the distributed relational database. If so, obtain the last modification timestamp of the distributed relational database; Use the last modification timestamp as a time reference to align the data between the distributed cache database and the distributed relational database.

[0013] One or more embodiments of this specification provide a multi-level caching device for a distributed database. The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to: execute any of the above methods. A non-volatile computer storage medium provided by one or more embodiments of this specification stores computer-executable instructions, and the computer-executable instructions are configured to: be able to execute any of the above methods.

[0014] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: Monitor the configuration changes of the configuration center through long polling, so that the configuration update data can take effect without restarting the service, avoiding the business interruption problem caused by the need to restart the traditional static configuration, and improving the business continuity. By configuring the rule parameters, the cache policy is dynamically switched, improving the response speed of high-frequency queries. At the same time, based on the multi-level cache mechanism, the cache penetration rate is reduced, the load on the database is also reduced, and the network latency and database pressure are reduced. When a cache invalidation instruction is detected, the relevant cache is immediately refreshed to ensure data consistency. Continuously collect the database operation metrics, compare them with the preset thresholds, locate the database anomalies at the second level, and perform intelligent rollback to ensure the stability of the configuration, taking into account both business certainty and system adaptability. And asynchronously synchronize the changed data to the distributed cache database, achieving efficient concurrent processing in this concurrent scenario. Description of the Drawings

[0015] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings: Figure 1 It is a schematic flowchart of a multi-level cache method for a distributed database provided by an embodiment of this specification; Figure 2 It is a schematic overall diagram of data synchronization for a distributed database provided by an embodiment of this specification; Figure 3 It is a schematic detailed flowchart of a multi-level cache for a distributed database provided by an embodiment of this specification; Figure 4 It is a schematic structural diagram of a multi-level cache device for a distributed database provided by an embodiment of this specification; Figure 5 It is a schematic structural diagram of a non-volatile storage medium provided by an embodiment of this specification. Detailed Embodiments

[0016] Embodiments of this specification provide a multi-level cache method, device, and medium for a distributed database.

[0017] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this specification.

[0018] As Figure 1 shown, the embodiment of this specification provides a schematic flowchart of a multi-level caching method for a distributed database. As Figure 1 known, in one or more embodiments of this specification, a multi-level caching method for a distributed database specifically includes: S101: Listen for configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events.

[0019] To solve the problem that traditional solutions rely on static configuration files or manual operations. When it is necessary to adjust database connection parameters or Redis cache policies, it is often necessary to restart the service to take effect, resulting in a problem that depends on server restart. In the embodiments of this specification, the configuration change events in the configuration center will be listened for according to the long polling mechanism of the current server, so as to obtain the configuration update data corresponding to the configuration change events. Among them, in a certain embodiment, the configuration center is the Nacos configuration center. Before listening for the configuration change events in the configuration center based on the long polling mechanism, as Figure 2 shown, the dynamic hot update of the database and cache policies will also be realized through this configuration center, and the connection parameters of the distributed relational database, that is, the HGDB connection parameters, and the cache rules of the distributed cache database, that is, the Redis cache rules, will be centrally managed.

[0020] Furthermore, in one or more embodiments of this specification, before listening for the configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events, the method further includes the following processes: First, based on the initial configuration set of the configuration center, load the connection parameters of the distributed relational database and the cache policies of the distributed cache database to obtain the loaded data. Then, based on the namespace parameters and grouping parameters corresponding to the loaded data, perform configuration initialization for the current environment. By executing the pre-defined warm-up SQL rules in the configuration center, synchronize the query results to the distributed relational database and the distributed cache database for caching to complete the initialization filling.

[0021] As Figure 3As shown, in one embodiment of this specification, the Nacos configuration center loads database HGDB connection parameters such as connection pool configuration, read-write separation strategy, etc., and Redis cache strategies such as expiration time, eviction mechanism, etc., and then isolates the configurations of the development, test, and production environments according to the namespace Namespace and group Group.

[0022] In this process, loading the distributed relational database connection parameters and the distributed cache database cache strategy based on the initial configuration set of the configuration center can ensure that the microservice cluster obtains unified and accurate basic configuration information during the startup phase, avoiding service exceptions or data chaos problems caused by inconsistent configurations, and laying a stable configuration foundation for subsequent business operations. Using the namespace parameter and group parameter in the loaded data to initialize the environment configuration can clearly distinguish different environments such as development, test, and production, preventing configuration interference between different environments. This isolation mechanism not only improves the standardization of configuration management but also enables independent configuration debugging and verification in each environment, reducing the risk of online failures caused by incorrect environment configurations. In addition, execute the pre-defined warm-up SQL rules of the configuration center, and synchronize the query results to the distributed relational database and the distributed cache database to complete the initialization filling. This operation enables hot data to exist in the cache when the service starts. When a user request arrives, the data can be directly obtained from the cache, reducing the query pressure on the database and significantly improving the data access speed.

[0023] Specifically, in one or more embodiments of this specification, based on the long polling mechanism of the current microservice cluster, listen for configuration change events of the configuration center to obtain the configuration update data corresponding to the configuration change events, specifically including: Obtain the configuration query request initiated by the current microservice cluster to the configuration center, so as to determine whether there is a change in the target configuration corresponding to the configuration query request within the timeout parameter corresponding to the configuration query request. If not, return an empty response and re-initiate the configuration query request. If there is a change, obtain the configuration change event corresponding to the change, and return the configuration update data corresponding to the configuration change event to trigger the non-restart hot update process of the configuration update data; wherein, the configuration update data includes: updated configuration value, version number.

[0024] In a certain application scenario, the implementation method of the microservice cluster monitoring the configuration changes of the configuration center through the long polling mechanism is as follows: after the client initiates a configuration query request, the configuration center server will keep the connection open, for example, set a 30-second timeout, then if the configuration changes during this period, the new data will be returned immediately; if it does not change, it will be blocked until the timeout and return an empty response, and the client will then re-initiate the long polling request. When the configuration change event is monitored, the client will trigger the configuration parsing, new and old value comparison, and local configuration hot update process, and notify the relevant listeners to execute the callback logic, and finally realize the dynamic refresh of the configuration without restarting the service. This mechanism reduces the number of invalid polling times while ensuring real-time performance and reducing the pressure on the server. That is, through Nacos, the database and cache configuration are uniformly managed, the service restart dependency is eliminated, and the configuration entry time is shortened from minutes to seconds, such as the connection pool expansion response time ≤ 3 seconds.

[0025] S102: Based on the business scenario corresponding to the configuration update data, the configuration rule parameters of the configuration center are matched and issued to determine the current cache strategy, so as to switch the cache level based on the current cache strategy to realize the cache of the current changed data.

[0026] Currently, in terms of cache management, the single cache level design with a fixed strategy is difficult to cope with dynamic business needs. Cache penetration problems are prone to occur in high-frequency access scenarios, and a large number of requests directly impact the database, resulting in a significant increase in response delays. In order to avoid the problem that a single cache level design is difficult to cope with dynamic business needs. In the embodiments of this specification, the configuration rule parameters of the configuration center will be matched according to the business scenario corresponding to the configuration update data, and the current cache strategy will be determined, so as to switch the cache level according to the current cache strategy and realize the caching of the current changed data.

[0027] Specifically, in one or more embodiments of this specification, based on the business scenario corresponding to the configuration update data, the configuration rule parameters of the configuration center are matched and issued to determine the current caching policy, and the caching level is switched based on the caching policy to implement the caching of the configuration update data. The specific process is as follows: First, based on the data characteristics corresponding to the configuration update data, such as access frequency, data update frequency, and data volume size, etc., the business scenario corresponding to the configuration update data is determined. Among them, the business scenarios include: static data scenarios with low update frequency and high access volume, such as basic commodity information and dictionary table data; dynamic data scenarios with frequent updates and strong timeliness, such as real-time inventory and order status. Then, according to the existing caching policy and cached data, it is determined whether the configuration update data in the business scenario meets the preset hierarchical switching trigger condition. If it meets, the corresponding configuration rule parameters are obtained. For example, when indicators such as data access frequency and network latency meet the preset hierarchical switching trigger condition, the caching level weight will be dynamically adjusted to obtain the configuration rule parameters. Then, the current caching policy is enabled according to the configuration rule parameters issued by the configuration center, and the caching level is switched according to the current caching policy to implement the caching of the configuration update data.

[0028] In the above process, during the multi-level caching stratification process, the caching level is intelligently switched according to the data characteristics and business scenarios, avoiding the limitations of traditional fixed strategies. And real-time monitoring of indicators and triggering policy adjustments reduce the cache penetration rate and effectively avoid the deterioration of responses caused by a large number of requests directly reaching the database. Allocating cache resources for different scenarios and switching caching policies as needed can avoid problems such as over-caching or insufficient caching and improve resource utilization. And there is no need for manual intervention in caching policy adjustment during this process. Automatic hierarchical switching is achieved through preset rules and dynamic monitoring, reducing the high error rate caused by manual configuration.

[0029] In an application scenario such as Figure 3 The multi-level caching hierarchical loading process shown is as follows: Based on the cache_level rules configured in the configuration center Nacos, such as local and redis, a hybrid management mode of static fallback and dynamic optimization is adopted: The basic rules are preset manually, such as forcibly specifying the multi-level caching policy for steady-state services such as product details. At the same time, the caching level weight is dynamically adjusted by real-time monitoring of indicators such as data access frequency and network latency. For example, when the traffic surges, it automatically degrades to the pure Redis mode. Finally, the second-level hot update and exception rollback of the policy are realized through the configuration center, taking into account business certainty and system adaptability. During the process of dynamic configuration hot update and multi-level caching coordination, through the hierarchical design combining local cache and Redis, the high-frequency query response speed is increased by 80%, from 500ms to within 100ms, and the database load is reduced by more than 70%.

[0030] S103: Monitor the cache to determine whether the current microservice cluster has received a cache invalidation instruction.

[0031] During the process of switching the cache level according to the current cache policy determined in step S102 above to implement the caching of configuration update data, in the embodiments of this specification, the cache will be monitored to determine whether the current microservice cluster has received a cache invalidation instruction. That is, when data is updated in the distributed relational database, a cache invalidation instruction is sent to all microservice nodes through the Redis publish-subscribe function.

[0032] S104: If so, receive the running metrics of the distributed relational database in real time to determine whether there is a database anomaly, so as to retrospectively update the current cache policy.

[0033] If there is an invalidation instruction, then receive the running metrics of the distributed relational database in real time to determine whether there is a database anomaly problem, so as to retrospectively update the current cache policy determined in step S102 above. Specifically, in one or more embodiments of this specification, receiving the running metrics of the distributed relational database in real time to determine whether there is a database anomaly, so as to retrospectively update the current cache policy, specifically includes the following process: First, receive the running metrics of the distributed relational database in real time, and then compare the running metrics with the corresponding preset metrics. Among them, it should be noted that the running metrics include: connection pool blocking rate, slow query frequency, transaction commit latency, etc. According to the determined comparison result, determine whether there is a database anomaly in the distributed relational database. If there is an anomaly problem, then obtain a preset number of historical version configurations adjacent to the version number in the distributed relational database. Then, obtain the historical database anomaly data corresponding to each historical version configuration within the preset number of historical version configurations, so as to determine the most stable historical version configuration based on the historical database anomaly data. Then, roll back to the most stable historical version configuration based on the interface of the configuration center and synchronously update the current cache policy.

[0034] During this process, by monitoring core metrics such as the connection pool blocking rate, slow query frequency, and transaction commit latency in real time, database anomalies can be detected within seconds compared to traditional log analysis methods, avoiding the spread of faults. And by comparing with preset metric thresholds, automatic determination of anomalies is achieved, eliminating the subjectivity of manual judgment and ensuring the accuracy and timeliness of fault location. Retrieve adjacent versions from the database historical version configuration, such as the most recent 10 versions, and then combine historical anomaly data such as the connection pool status and slow query records under each version to automatically match the most stable historical configuration, and the configuration version with the highest matching success rate can be obtained. Then, through second-level rollback and synchronous update of the cache policy, problems such as cache penetration or data inconsistency caused by inconsistent database and cache policies are avoided, and the stability and response efficiency of the system in high-concurrency scenarios are significantly improved through the collaborative optimization of the cache and the database.

[0035] In a certain application scenario, its rollback process is as follows: when it is detected that the metrics exceed the limits, retrieve the most recent 10 historical version configurations from the config_history table of HGDB, select the stable version to roll back through the Nacos interface, and synchronously update the Redis policy. Trigger configuration backtracking based on threshold alarms (the recovery time is compressed from the hour level to within 5 minutes), combine the atomic binding of the RedLock distributed lock and the HGDB transaction, increase the success rate of high-concurrency transactions to 99.9%, avoid data inconsistency problems such as out-of-stock overselling, and the system availability reaches 99.99%.

[0036] S105: Based on the cache policy updated by backtracking, asynchronously synchronize the current changed data to the distributed cache database.

[0037] That is, after obtaining the cache policy updated by backtracking in the above steps, the current changed data will be asynchronously synchronized to the distributed cache database. This process can asynchronously synchronize incremental data to Redis through MQ, reducing the write pressure of the main database by 50%. At the same time, Nacos dynamically adjusts message queue parameters such as the number of consumers and batch size to achieve on-demand resource allocation, increasing the throughput of the system by 3 times in scenarios such as major e-commerce promotions, and reducing manual intervention in operation and maintenance by 80%.

[0038] Specifically, in one or more embodiments of this specification, asynchronously synchronizing the current changed data to the distributed cache database based on the cache policy updated by backtracking specifically includes: Listen to the WAL logs of a distributed relational database through logical decoding to capture data change events in real time, and structurally transform the data change events to obtain change messages and send them to the MQ message queue. Sequentially obtain the change messages in the MQ message queue, and based on the business rules corresponding to the change messages, transform the change messages into operation instructions for the distributed cache database, and implement the synchronization of the distributed cache database based on the operation instructions. That is, in a certain application scenario, the distributed relational database HGDB in the embodiments of this specification captures data change events through logical decoding, transmits them through the MQ message queue, and is converted by the consumer into Redis operation instructions such as HSET and EXPIRE. This process is mainly divided into three stages: First, HGDB continuously listens to the WAL logs of the database through the logical decoding function to capture data change events such as insert, delete, and update operations, and converts them into structured change messages. These messages contain key information such as operation type, table name, primary key, and data content before and after the change. Second, these structured change messages are sent to a message queue such as Kafka or RabbitMQ for reliable transmission. The message queue plays a role of buffering and persistence here, ensuring that even in case of system failures or network fluctuations, the change messages will not be lost. Third, the consumer service fetches these change messages from the message queue and converts them into corresponding Redis operation instructions according to the business rules. For example, an update operation will be converted into an HSET command to modify the Redis hash table, a delete operation will be converted into a DEL command to remove the cached data, and an appropriate expiration time will also be set to manage the cache lifecycle. The whole process realizes the automatic and real-time synchronization between the database and the cache, which not only ensures data consistency but also improves the system response speed.

[0039] Further, in one or more embodiments of this specification, after receiving the running metrics of the distributed relational database in real time to determine whether there is a database anomaly for the purpose of backtracking and updating the current cache policy, the method further includes: Obtain the transaction requests corresponding to the distributed relational database, and write the transaction requests into the cache queue of the distributed cache database in sequence. Obtain a distributed lock through the RedLock algorithm to lock the resources corresponding to the transaction requests, so as to execute the transaction requests in the cache queue in sequence. If the execution is completed, an update message will be sent based on the MQ message queue, and the cache data of the distributed cache database will be asynchronously refreshed to achieve synchronous caching of the distributed cache database, and the distributed lock will be released. That is, in a high-concurrency scenario, all transaction requests are first written into the Redis queue, that is, the cache queue, for buffering to smooth out peaks and valleys and smooth the system load. Subsequently, the consumer process fetches requests from the queue and attempts to obtain a distributed lock through the RedLock algorithm to ensure mutually exclusive access to shared resources. Once the lock is successfully obtained, the consumer will perform transaction operations related to HGDB to ensure data consistency and integrity. After the transaction is successfully committed, the system sends an update message through the message queue MQ to asynchronously refresh the relevant cache data in Redis to maintain the consistency between the cache and the database. Finally, after all key operations are completed, the distributed lock RedLock will be safely released so that other waiting consumers can continue to process subsequent requests, thus realizing an efficient, reliable, and scalable concurrent processing flow.

[0040] Further, in one or more embodiments of the present specification, after asynchronously synchronizing the current changed data to the distributed cache database based on the backtracked and updated cache policy, the method further includes: Determine whether there is a data conflict between the distributed cache database and the distributed relational database. If there is a conflict, obtain the last modification timestamp of the distributed relational database. Using this last modification timestamp as a time reference, align the data of the distributed cache database and the distributed relational database. That is, when there is a data conflict between Redis and HGDB, overwrite the old value based on the last_modified timestamp of HGDB. After the data alignment is completed, record the synchronization status in the sync_audit log table.

[0041] In addition, in a feasible embodiment, a timed task is used to clean up the config_history historical configuration records 30 days ago every day at midnight and archive the MQ message logs that have been successfully synchronized. By continuously looping through the above steps, a full-process closed-loop management of dynamic configuration effect, cache collaborative update, and automatic exception handling can be achieved.

[0042] In the above solution, the dynamic hot update of the database and cache policy is achieved through the Nacos configuration center. The HGDB connection parameters such as the read-write separation policy and the Redis cache rules such as the expiration time and eviction algorithm are centrally managed. The client listens for configuration changes in real time through the long polling mechanism, achieving second-level effectiveness and completely eliminating the dependence on service restart. A dynamic hierarchical mechanism is constructed by combining the local cache Caffeine and the distributed cache Redis. The cache level is automatically switched according to the business scenario - high-frequency static data such as basic information is preferentially read from the local cache, and dynamic data such as real-time inventory ensures global consistency through Redis. The cache penetration rate is reduced to less than 5% through the Pub / Sub function to broadcast cache invalidation instructions in real time, and the database load is reduced by 70%. In terms of exception handling, the system collects the running metrics of the HGDB database in real time. When the preset threshold of Nacos is exceeded, the most recent stable configuration is retrieved from the version log table of HGDB, triggering a second-level rollback and synchronously updating the Redis policy, compressing the fault recovery time from the hour level to within 5 minutes. At the same time, through the deep coordination of the RedLock distributed lock and the HGDB database transaction, the atomicity of high-concurrency operations is ensured. The incremental data is synchronously updated in real time by combining the MQ asynchronous message queue. When there is a conflict, the old value is overwritten based on the last_modified timestamp of HGDB, reducing the write pressure on the main database by 50%, and finally controlling the data consistency delay within 200 ms. This technical solution significantly improves the system elasticity and stability through dynamic configuration management, intelligent cache optimization, and automated operation and maintenance closed-loop. In scenarios such as e-commerce seckill and financial transactions, the system availability reaches 99.99%, the manual intervention of operation and maintenance is reduced by 80%, and the response time is stable below 50 ms, providing reliable technical support for high-concurrency services.

[0043] As Figure 4 shown, the embodiments of this specification provide a structural schematic diagram of a multi-level cache device for a distributed database. As Figure 4 known, in one or more embodiments of this specification, a multi-level cache device for a distributed database, the device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: execute any of the above methods.

[0044] As Figure 5 shown, the embodiments of this specification provide a structural schematic diagram of a non-volatile storage medium. As Figure 5It can be seen that in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 501, and the computer-executable instructions 501 can: execute any of the above-described methods.

[0045] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the apparatus, device, and non-volatile computer storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0046] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0047] The above is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, there can be various changes and modifications to one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included within the scope of the claims of this specification.

Claims

1. A multi-level caching method for a distributed database, characterized in that The method includes: Listening for configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events; Based on the business scenarios corresponding to the configuration update data, matching the configuration rule parameters in the configuration center, and issuing them to determine the current caching policy, and switching the cache level based on the current caching policy to implement caching of the current changed data; Monitoring the cache to determine whether the current microservice cluster receives a cache invalidation instruction; If so, receiving the running metrics of the distributed relational database in real time to determine whether there is a database anomaly, so as to retrospectively update the current caching policy; Based on the retrospectively updated caching policy, asynchronously synchronize the current changed data to the distributed cache database.

2. The multi-level caching method for a distributed database according to claim 1, characterized in that, Before listening for configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events, the method further includes: Loading the connection parameters of the distributed relational database and the caching policy of the distributed cache database based on the initial configuration set in the configuration center to obtain the loaded data; Initializing the configuration of the current environment based on the namespace parameters and grouping parameters corresponding to the loaded data; Executing the pre-defined warm-up SQL rules in the configuration center to synchronously load the query results into the distributed relational database and the distributed cache database for caching to complete the initial filling.

3. A multi-level caching method for a distributed database according to claim 1, wherein Listening for configuration change events in the configuration center based on the long polling mechanism of the current microservice cluster to obtain the configuration update data corresponding to the configuration change events, specifically including: Obtaining the configuration query request initiated by the current microservice cluster to the configuration center to determine whether there is a change in the target configuration corresponding to the configuration query request within the timeout parameter corresponding to the configuration query request; If not, returning an empty response and re-initiating the configuration query request; If so, obtaining the configuration change event corresponding to the change, and returning the configuration update data corresponding to the configuration change event to trigger the non-restart hot update process of the configuration update data; where the configuration update data includes: updated configuration value, version number.

4. A multi-level caching method for a distributed database according to claim 1, characterized in that, Based on the business scenarios corresponding to the configuration update data, matching the configuration rule parameters in the configuration center, and issuing them to determine the current caching policy, and switching the cache level based on the caching policy to implement caching of the configuration update data, specifically including: Determining the business scenarios corresponding to the configuration update data based on the data characteristics corresponding to the configuration update data; where the business scenarios include: static data scenario, dynamic data scenario; Based on the existing caching policy and caching data, determining whether the configuration update data in the business scenario meets the preset level switching trigger condition, and if so, obtaining the corresponding configuration rule parameters; Enabling the current caching policy according to the configuration rule parameters issued by the configuration center, and switching the cache level based on the current caching policy to implement caching of the configuration update data.

5. A multi-level caching method for a distributed database according to claim 3, characterized in that, Receiving the running metrics of the distributed relational database in real time, determining whether there is a database anomaly, so as to retrospectively update the current cache policy, specifically including: Receiving the running metrics of the distributed relational database in real time to compare the running metrics with the corresponding preset metrics; wherein, the running metrics include: connection pool blocking rate, slow query frequency, transaction commit latency; Based on the comparison result, determining whether there is a database anomaly in the distributed relational database; If so, obtaining a preset number of historical version configurations adjacent to the version number in the distributed relational database; Obtaining the historical database anomaly data corresponding to each historical version configuration in the preset number of historical version configurations, so as to determine the most stable historical version configuration based on the historical database anomaly data; Rolling back to the most stable historical version configuration based on the interface of the configuration center and synchronously updating the current cache policy.

6. A multi-level caching method for a distributed database according to claim 1, wherein, Based on the retrospectively updated cache policy, asynchronously synchronizing the current changed data to the distributed cache database, specifically including: Listening to the WAL log of the distributed relational database through logical decoding to capture data change events in real time, and performing a structured conversion on the data change events to obtain a change message and sending it to the MQ message queue; Sequentially obtaining the change messages in the MQ message queue, and based on the business rules corresponding to the change messages, converting the change messages into operation instructions for the distributed cache database, and implementing the synchronization of the distributed cache database based on the operation instructions.

7. A multi-level caching method for a distributed database according to claim 6, characterized in that, After receiving the running metrics of the distributed relational database in real time, determining whether there is a database anomaly, so as to retrospectively update the current cache policy, the method includes: Obtaining the transaction request corresponding to the distributed relational database and sequentially writing the transaction request into the cache queue of the distributed cache database; Obtaining a distributed lock through the RedLock algorithm to lock the resources corresponding to the transaction request, so as to sequentially execute the transaction requests in the cache queue; If the execution is completed, sending an update message based on the MQ message queue and asynchronously refreshing the cache data of the distributed cache database to implement the synchronous cache of the distributed cache database and releasing the distributed lock.

8. A multi-level caching method for a distributed database according to claim 1, characterized in that, After asynchronously synchronizing the current changed data to the distributed cache database based on the retrospectively updated cache policy, the method further includes: Determining whether there is a data conflict between the distributed cache database and the distributed relational database. If so, obtaining the last modification timestamp of the distributed relational database; Using the last modification timestamp as a time reference to align the data between the distributed cache database and the distributed relational database.

9. A multi-level caching device for a distributed database, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: execute the method according to any one of claims 1-8 above.

10. A non-volatile storage medium stores computer-executable instructions, characterized in that, The computer-executable instructions can: execute the method according to any one of claims 1-8 above.

Images