Distributed cache snapshot management method, device and equipment and readable storage medium

Abstract

The invention relates to a distributed cache snapshot management method and device, equipment and a readable storage medium. The method comprises the following steps of: extracting a plurality of snapshot tasks from a task queue, and respectively distributing the plurality of extracted snapshot tasks to a plurality of idle working threads in a thread pool; determining a Redis instance corresponding to each snapshot task based on a mapping relationship between the working thread and the Redis instance; for each snapshot task, through a Redis instance corresponding to the snapshot task, executing a snapshot command corresponding to the incremental snapshot every first preset duration to obtain an incremental snapshot result, and executing a snapshot command corresponding to the full snapshot every second preset duration to obtain a full snapshot result; and generating a snapshot report according to the incremental snapshot results and the total snapshot results corresponding to the plurality of snapshot tasks. By adopting the method, the data persistence efficiency under a high-load condition can be improved.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a distributed cache snapshot management method, apparatus, device and readable storage medium. Background Art

[0002] In the field of modern database technology, performance and response speed are key factors affecting system availability and user experience. With the explosive growth of data volume and the complexity of application scenarios, the traditional snapshot mechanism of Redis (an open source high-performance key-value storage system) may cause performance degradation under high load conditions, especially in applications with high real-time and high concurrency requirements. For example, in high-concurrency scenarios such as online retail, real-time data analysis, and online games, these problems will increase system response time, affecting user experience and system stability. Therefore, a more efficient distributed cache snapshot management method is urgently needed to improve the efficiency of data persistence under high load conditions. Summary of the invention

[0003] Based on this, it is necessary to provide a distributed cache snapshot management method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems, which can improve the data persistence efficiency under high load conditions.

[0004] In a first aspect, the present application provides a distributed cache snapshot management method, including:

[0005] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0006] Based on the mapping relationship between worker threads and Redis instances, determine the Redis instance corresponding to each snapshot task;

[0007] For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result;

[0008] Generate a snapshot report based on the incremental snapshot results and full snapshot results corresponding to multiple snapshot tasks.

[0009] In one of the embodiments, executing a snapshot command corresponding to the full snapshot every second preset time period to obtain a full snapshot result includes:

[0010] Get the last execution time of the full snapshot;

[0011] Determine the current execution time according to the last execution time and the second preset duration;

[0012] If the system load information at the current execution time meets the full execution condition, the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0013] In one embodiment, the distributed cache snapshot management method further includes:

[0014] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0015] In one embodiment, the distributed cache snapshot management method further includes:

[0016] Determine a plurality of consecutive preset execution times according to the last execution time and the second preset duration, with the second preset duration being an interval between adjacent preset execution times;

[0017] If multiple consecutive preset execution times do not meet the full execution conditions, the first preset duration is adjusted to the first adjustment duration, and the second preset duration is adjusted to the second adjustment duration; the snapshot command corresponding to the incremental snapshot is executed every first adjustment duration to obtain the incremental snapshot result, and the snapshot command corresponding to the full snapshot is executed every second preset duration to obtain the full snapshot result.

[0018] In one embodiment, the distributed cache snapshot management method further includes:

[0019] Get the last execution time of the incremental snapshot;

[0020] If the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

[0021] In one embodiment, the distributed cache snapshot management method further includes:

[0022] Determine a plurality of consecutive preset execution times according to the last execution time and the first preset duration, with the first preset duration being an interval between adjacent preset execution times;

[0023] If the number of write operations corresponding to each of the plurality of consecutive preset execution moments is lower than the second preset number, the first preset duration is adjusted to a third adjustment duration; and the snapshot command corresponding to the incremental snapshot is executed every third adjustment duration to obtain the incremental snapshot result.

[0024] In one embodiment, executing a snapshot command corresponding to an incremental snapshot every first preset time period to obtain an incremental snapshot result includes:

[0025] Every first preset time period, the snapshot command corresponding to the incremental snapshot is asynchronously executed to obtain the incremental snapshot result.

[0026] In a second aspect, the present application also provides a distributed cache snapshot management device, including:

[0027] An allocation module is used to extract multiple snapshot tasks from the task queue and allocate the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0028] A determination module is used to determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance;

[0029] An execution module is used to execute, for each snapshot task, through the Redis instance corresponding to the snapshot task, the snapshot command corresponding to the incremental snapshot at a first preset time interval to obtain the incremental snapshot result, and the snapshot command corresponding to the full snapshot at a second preset time interval to obtain the full snapshot result;

[0030] The generation module is used to generate a snapshot report according to the incremental snapshot results and the full snapshot results corresponding to the multiple snapshot tasks.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0033] Based on the mapping relationship between worker threads and Redis instances, determine the Redis instance corresponding to each snapshot task;

[0034] For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result;

[0035] Generate a snapshot report based on the incremental snapshot results and full snapshot results corresponding to multiple snapshot tasks.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0038] Based on the mapping relationship between worker threads and Redis instances, determine the Redis instance corresponding to each snapshot task;

[0039] For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result;

[0040] Generate a snapshot report based on the incremental snapshot results and full snapshot results corresponding to multiple snapshot tasks.

[0041] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0043] Based on the mapping relationship between worker threads and Redis instances, determine the Redis instance corresponding to each snapshot task;

[0044] For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result;

[0045] Generate a snapshot report based on the incremental snapshot results and full snapshot results corresponding to multiple snapshot tasks.

[0046] The above-mentioned distributed cache snapshot management method, device, computer equipment, computer-readable storage medium and computer program product, by allocating multiple snapshot tasks in the task queue to multiple idle working threads, thereby executing each snapshot task in parallel through multiple threads, reducing the snapshot operation time of a single node, significantly reducing the impact of snapshot operations on system performance, and improving the overall response speed and concurrent processing capabilities of the system; each working thread is connected to a specified Redis instance, and data can be stored in multiple Redis instances in a dispersed manner to avoid the resource bottleneck of a single node. At the same time, for each snapshot task, different interval durations are used to execute incremental snapshots and full snapshots respectively. The interval duration can be flexibly adjusted according to actual needs, which is conducive to ensuring the data persistence efficiency of the system under high load conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] FIG1 is an application environment diagram of a distributed cache snapshot management method according to an embodiment;

[0049] FIG2 is a schematic diagram of a process flow of a distributed cache snapshot management method in one embodiment;

[0050] FIG3 is a schematic diagram of the overall process of a distributed cache snapshot management method in one embodiment;

[0051] FIG4 is a timing diagram of a parallel snapshot scheduling algorithm in one embodiment;

[0052] FIG5 is a structural block diagram of a distributed cache snapshot management device in one embodiment;

[0053] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The distributed cache snapshot management method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment is illustrated by applying the method to the server 104. It can be understood that the method can also be applied to the terminal 102, and can also be applied to a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. The server 104 extracts multiple snapshot tasks from the task queue, and assigns the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; based on the mapping relationship between the working thread and the Redis instance, the Redis instance corresponding to each snapshot task is determined; for each snapshot task, the snapshot command corresponding to the incremental snapshot is executed every first preset time to obtain the incremental snapshot result through the Redis instance corresponding to the snapshot task, and the snapshot command corresponding to the full snapshot is executed every second preset time to obtain the full snapshot result; according to the incremental snapshot results and full snapshot results corresponding to each of the multiple snapshot tasks, a snapshot report is generated. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0056] In an exemplary embodiment, as shown in FIG2 , a distributed cache snapshot management method is provided, which is described by taking the method applied to the server 104 in FIG1 as an example, and includes the following steps 202 to 208 . Among them:

[0057] Step 202: extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively.

[0058] The snapshot task refers to a task submitted by a user for executing a snapshot operation, and the snapshot task submitted by the user is added to the task queue. In some embodiments, multiple snapshot tasks in the task queue support dynamic addition and deletion.

[0059] Multiple worker threads or worker processes are created in the thread pool. Usually, a fixed-size thread pool or process pool is created when the application starts, and an appropriate number of worker threads or worker processes are configured to execute snapshot tasks in parallel.

[0060] Take multiple worker threads configured in the thread pool as an example for explanation. Idle worker threads refer to worker threads in the thread pool that are in an idle state.

[0061] The server assigns the extracted multiple snapshot tasks to multiple idle worker threads. Each worker thread is used to execute the assigned snapshot tasks. The parallel execution of multiple worker threads is helpful to reduce the execution time of snapshot tasks and reduce the impact of snapshot operations on system performance, thereby helping to improve data persistence efficiency under high load conditions. In addition, snapshot tasks can be assigned to idle worker threads, which is helpful to improve the overall response speed and concurrent processing capabilities of the system.

[0062] Step 204: Determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance.

[0063] A Redis instance is a running Redis server process with independent memory space and processing power for storing and managing data. Each Redis instance has its own configuration settings, including ports, memory limits, and persistence policies.

[0064] Each worker thread is connected to a specified Redis instance, and the mapping relationship is used to characterize the connection relationship between the worker thread and the corresponding Redis instance. Since multiple snapshot tasks are respectively assigned to multiple idle worker threads, the Redis instance corresponding to each snapshot task can be determined based on the mapping relationship.

[0065] Step 206: For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result.

[0066] Among them, for each snapshot task, the snapshot command is executed through the Redis instance corresponding to the snapshot task, so that the data is stored in multiple Redis instances in a dispersed manner, avoiding the resource bottleneck of a single node, and managing the sharding logic through the application layer to ensure data distribution and access.

[0067] In some embodiments, multiple Redis instances are deployed and the consistent hashing algorithm is used to ensure that data is evenly distributed to each Redis instance. The above data sharding management logic is implemented at the application layer, including data read and write routing, dynamic adjustment of sharding strategies, and fault handling mechanisms. This method of sharding data divides data or tasks into multiple parts, each of which is distributed to different nodes or instances for processing or storage, which can improve the scalability and performance of the system.

[0068] Incremental snapshot means that when creating a snapshot, only the data blocks that have changed since the last snapshot are backed up, rather than the entire data set. Incremental snapshot can significantly reduce the use of storage space and speed up the creation of snapshots. The incremental snapshot result contains all the data blocks that have changed since the last snapshot.

[0069] A full snapshot is a comprehensive backup of data at a specified point in time, preserving the state of all data. The full snapshot result contains an image of the data at a specified point in time, which can be a copy or replica of the data. In some embodiments, the full snapshot result includes not only the current data, but also historical data, ensuring that the state can be restored to any point in time when needed.

[0070] The first preset duration is the execution interval of the snapshot command corresponding to the incremental snapshot, and the second preset duration is the execution interval of the snapshot command corresponding to the full snapshot. The second preset duration is greater than the first preset duration. For example, the second preset duration is 24 hours and the first preset duration is 1 hour. By setting different interval durations, it is helpful to meet the complex application requirements in a distributed environment, reduce the impact of snapshot operations on the normal operation of the system, and improve the user experience.

[0071] In some embodiments, the server may use at least one of the AOF (Append Only File) mechanism or the RDB (Redis Database File, memory snapshot) mechanism to perform snapshot operations. AOF is a persistence mechanism of Redis that records write operations in log files to ensure that data can be recovered in the event of a failure. RDB is another persistence mechanism of Redis that provides data backup by periodically generating binary snapshot files of data in memory.

[0072] Step 208: Generate a snapshot report according to the incremental snapshot results and the full snapshot results corresponding to each of the plurality of snapshot tasks.

[0073] Among them, the server can summarize the snapshot results of all snapshot tasks, including incremental snapshot results and full snapshot results, and generate a snapshot report, which is used to characterize the execution status of all snapshot tasks. In some embodiments, the server includes a monitoring module for real-time monitoring of the status of each snapshot task, recording the execution time and execution results of each snapshot task, and if the execution fails, recording the reason for the failure, for formulating strategies based on the reason for the failure and rescheduling the snapshot task. The server can add various information recorded by the monitoring module to the snapshot report.

[0074] Figure 3 is a schematic diagram of the overall process of a distributed cache snapshot management method in an embodiment. In which, the snapshot task submitted by the user is added to the task queue, the thread pool is initialized, and multiple snapshot tasks extracted from the task queue are respectively assigned to multiple idle working threads in the thread pool, and based on the mapping relationship between the working thread and the Redis instance, through the Redis instance corresponding to each snapshot task, the snapshot task is executed and the snapshot result is returned, and each snapshot result is summarized to generate a snapshot report.

[0075] In the above-mentioned distributed cache snapshot management method, by allocating multiple snapshot tasks in the task queue to multiple idle working threads, each snapshot task is executed in parallel through multiple threads, thereby reducing the snapshot operation time of a single node, significantly reducing the impact of snapshot operations on system performance, and improving the overall response speed and concurrent processing capabilities of the system; each working thread is connected to a specified Redis instance, and data can be stored in multiple Redis instances in a dispersed manner to avoid the resource bottleneck of a single node. At the same time, for each snapshot task, different interval durations are used to execute incremental snapshots and full snapshots respectively. The interval duration can be flexibly adjusted according to actual needs, which is conducive to ensuring the data persistence efficiency of the system under high load conditions.

[0076] In an exemplary embodiment, a snapshot command corresponding to a full snapshot is executed every second preset time period to obtain a full snapshot result, including: obtaining the last execution time of the full snapshot; determining the current execution time according to the last execution time and the second preset time period; if the system load information at the current execution time meets the full execution condition, executing the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0077] Among them, the last execution time of the full snapshot refers to the time when the full snapshot was last executed, and the current execution time refers to the time after the second preset time has passed since the last execution time. The server can use the sum of the last execution time and the second preset time as the current execution time.

[0078] The system load information is used to indicate the load of the system. For example, the system load information includes the system load. The full execution condition may be that the system load is lower than a preset load. If the system load at the current execution time is lower than the preset load, the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0079] In this embodiment, by determining each execution moment of the full snapshot and judging whether the system load information at the execution moment satisfies the full execution condition, it is determined whether to execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result. This helps to ensure that the full snapshot is executed when the system load is good. In high-load scenarios, system stability is guaranteed, the impact of snapshot operations on the normal operation of the system is reduced, and user experience is improved.

[0080] In an exemplary embodiment, the distributed cache snapshot management method also includes: if the system load information at the current execution time does not meet the full execution condition, obtaining real-time system load information until the real-time system load information meets the full execution condition, and executing the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0081] Among them, at the execution time of the full snapshot, if the system load information does not meet the full execution conditions, the execution time of the full snapshot can be postponed until the real-time system load information meets the full execution conditions, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result, which further ensures that the full snapshot is executed when the system load is good, avoids the problem of excessive system load at the execution time, affecting the normal operation of the system, and ensures system stability.

[0082] In an exemplary embodiment, the distributed cache snapshot management method also includes: determining multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjusting the first preset duration to the first adjustment duration, and adjusting the second preset duration to the second adjustment duration; executing the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and executing the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0083] Among them, for the full snapshot process, multiple consecutive preset execution times refer to the execution times set by the last execution time of the full snapshot and the second preset duration. For example, the last execution time of the full snapshot is 2 a.m. on Monday, and the second preset duration is 24 hours. Multiple consecutive preset execution times may include 2 a.m. on Tuesday, Wednesday, Thursday, etc.

[0084] Multiple consecutive preset execution times do not meet the full execution condition, which means that at the execution times of multiple consecutive full snapshots, the system load information does not meet the full execution condition. The second preset duration can be extended to reduce the execution frequency of full snapshots. At the same time, the first preset duration can also be extended to reduce the execution frequency of incremental snapshots.

[0085] The first adjustment duration may be greater than the first preset duration, and the second adjustment duration may be greater than the second preset duration. The server performs a snapshot operation based on the first adjustment duration and the second adjustment duration.

[0086] In this embodiment, when multiple consecutive preset execution times of the full snapshot cannot meet the full execution conditions, it indicates that the system load may be too large. By extending the first preset time length and the stacked preset time length, the execution interval of the incremental snapshot and the full snapshot is increased, which helps to reduce the impact of the snapshot operation on the normal operation of the system and improve the user experience.

[0087] In an exemplary embodiment, the distributed cache snapshot management method also includes: obtaining the last execution time of the incremental snapshot; if the number of write operations from the last execution time to the current time exceeds a first preset number, executing the snapshot command corresponding to the incremental snapshot to obtain the incremental snapshot result.

[0088] The last execution time of the incremental snapshot refers to the time when the incremental snapshot was last executed, and the current time refers to the current time of the system. The number of write operations refers to the number of write operations performed in the Redis instance. The first preset number is a preset number, for example, it can be 100,000 times.

[0089] The number of write operations from the last execution time to the current time exceeds the first preset number, indicating that the number of write operations within the period has reached a certain number, and the server can execute the snapshot command corresponding to the incremental snapshot to obtain the incremental snapshot result.

[0090] In this embodiment, whether to perform an incremental snapshot operation is determined by judging whether the number of write operations from the last execution time to the current time exceeds a first preset number, which is beneficial to ensuring data consistency and reliability and meeting complex application requirements in a distributed environment.

[0091] In an exemplary embodiment, the distributed cache snapshot management method also includes: determining multiple consecutive preset execution times based on the last execution time and a first preset duration, with adjacent preset execution times separated by a first preset duration; if the number of write operations corresponding to each of the multiple consecutive preset execution times is lower than a second preset number, adjusting the first preset duration to a third adjustment duration; and executing the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain an incremental snapshot result.

[0092] For the incremental snapshot process, the multiple consecutive preset execution times refer to the execution times set by the last execution time of the incremental snapshot and the first preset duration. For example, if the last execution time of the incremental snapshot is 2 a.m. and the second preset duration is 1 hour, the multiple consecutive preset execution times may include 3 a.m., 4 a.m., 5 a.m., etc.

[0093] The number of write operations corresponding to each of the plurality of consecutive preset execution moments is lower than the second preset number, and the second preset number may be lower than the first preset number. For example, the second preset number may be 30,000 times, indicating that the number of write operations counted at the execution moments of the plurality of consecutive incremental snapshots is low, and the first preset duration may be extended to obtain a third adjustment duration. Executing the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result is beneficial to increasing the number of write operations during the execution interval of the incremental snapshot.

[0094] In this embodiment, by determining multiple preset execution times of incremental snapshots, and when the number of write operations corresponding to multiple consecutive preset execution times is lower than the second preset number, the first preset duration is extended and the incremental snapshot operations are reduced, which is beneficial to improving system efficiency.

[0095] In an exemplary embodiment, executing a snapshot command corresponding to an incremental snapshot at a first preset time interval to obtain an incremental snapshot result includes: asynchronously executing a snapshot command corresponding to an incremental snapshot at a first preset time interval to obtain an incremental snapshot result.

[0096] Among them, for the snapshot writing method, an asynchronous writing method can be adopted, which is conducive to improving the snapshot writing efficiency. In some embodiments, AOF persistence is enabled in the Redis configuration file and is set to asynchronous mode.

[0097] In some embodiments, AOF rewrite operations may be performed periodically to compress historical snapshot operations into new AOF files to prevent the AOF file from being too large and affecting performance.

[0098] In this embodiment, the snapshot is written to Redis by adopting an asynchronous writing method, which is conducive to improving the snapshot writing efficiency, thereby improving the data persistence efficiency under high load conditions.

[0099] To explain the distributed cache snapshot management method and its effect in detail, a most detailed embodiment is described below:

[0100] The distributed cache snapshot management method in a high-concurrency scenario is used as an example to illustrate. For example, high-concurrency scenarios may include online retail, real-time data analysis, online games, etc. The distributed cache snapshot management method mainly includes the following steps:

[0101] 1. Data sharding: Store data in multiple Redis instances to avoid resource bottlenecks of a single node. Manage sharding logic at the application layer to ensure data distribution and access.

[0102] a. Deploy multiple Redis instances and use the consistent hashing algorithm to ensure that data is evenly distributed to each Redis instance.

[0103] b. Implement sharding management logic at the application layer, including data read and write routing, dynamic adjustment of sharding strategies, and fault handling mechanisms.

[0104] 2. Parallel snapshot: Execute snapshot operations of each shard in parallel through multi-threading or multi-process to reduce the snapshot time of a single node.

[0105] a. Parallel snapshot scheduler: A parallel snapshot scheduler is implemented in an independent service, which is responsible for managing the snapshot task queue and scheduling the execution of multiple snapshot tasks.

[0106] b. Task queue management: Initialize the task queue, add the Redis instance information that needs to execute the snapshot to the queue, and support dynamic addition and deletion of tasks.

[0107] c. Create a fixed-size thread pool or process pool and configure an appropriate number of worker threads or processes to execute snapshot tasks in parallel.

[0108] d. The parallel snapshot scheduler extracts tasks from the task queue and assigns them to idle worker threads or processes.

[0109] e. Each worker thread connects to the specified Redis instance and executes the corresponding snapshot command (such as BGSAVE).

[0110] f. The monitoring module monitors the status of each snapshot task in real time, records the execution time and results of the task; if the task fails, records the reason for the failure and reschedules the task according to the strategy.

[0111] g. The scheduler summarizes the snapshot results of all snapshot tasks and generates a snapshot report.

[0112] 3. Asynchronous I / O operation: Use Redis's AOF persistence mechanism to configure asynchronous writing and improve snapshot writing efficiency.

[0113] a. Enable AOF persistence in the Redis configuration file and set it to asynchronous mode (appendonly yes, appendfsync everysec).

[0114] b. Perform AOF rewrite operations (BGREWRITEAOF) regularly to compress historical operations into new AOF files to prevent the AOF files from being too large and affecting performance.

[0115] 4. Incremental snapshot: By combining AOF and RDB files, an incremental snapshot mechanism is implemented to reduce the frequency and overhead of full snapshots.

[0116] a. Set Redis to enable both RDB and AOF persistence mechanisms (save 900 1, save 300 10, save60 10000, appendonly yes).

[0117] b.Perform RDB snapshots (BGSAVE) regularly to generate full snapshot files.

[0118] c. Use AOF to record incremental operations and implement incremental snapshots.

[0119] 5. Scheduling algorithm: Design a scheduling algorithm to dynamically determine the frequency of executing full snapshots and incremental snapshots based on the system load and data changes to balance system performance and data consistency. Figure 4 is a timing diagram of a parallel snapshot scheduling algorithm in one embodiment.

[0120] a. Set the initial frequency of full snapshots and incremental snapshots (for example, full snapshots every 24 hours and incremental snapshots every hour), and configure the system load threshold and data change threshold.

[0121] b. Monitor the system's CPU, memory, and I / O usage in real time, and record system load information. For example, you can monitor once every 5 minutes.

[0122] c. Count the number of write operations in the Redis instance every hour and record the amount of data changes.

[0123] d. Check the system load at 2 a.m. every day. If the system load is less than 70%, perform a full snapshot. If the system load is more than 70%, delay the full snapshot until the load is less than 70%.

[0124] e. Execute incremental snapshots every hour on the hour. When the number of write operations detected exceeds 100,000, an incremental snapshot is immediately executed.

[0125] f. Dynamically adjust the frequency of full and incremental snapshots, and optimize snapshot scheduling based on system load and data changes. If three consecutive full snapshots are delayed, adjust the full snapshot frequency to once every 30 hours, and increase the incremental snapshot frequency to once every half hour. If the number of write operations is less than 10,000 times for three consecutive hours, adjust the incremental snapshot frequency to once every 2 hours.

[0126] Through the above design and operation steps, an efficient and stable distributed cache shard snapshot method is implemented, which improves the data persistence efficiency of Redis under high load conditions, reduces system performance bottlenecks, and ensures data consistency and reliability.

[0127] The above distributed cache snapshot management method distributes multiple snapshot tasks in the task queue to multiple idle working threads, thereby executing each snapshot task in parallel through multiple threads, reducing the snapshot operation time of a single node, significantly reducing the impact of snapshot operations on system performance, and improving the overall response speed and concurrent processing capabilities of the system; each working thread is connected to a specified Redis instance, and data can be stored in multiple Redis instances in a dispersed manner to avoid the resource bottleneck of a single node. At the same time, for each snapshot task, different intervals are used to execute incremental snapshots and full snapshots respectively. The interval can be flexibly adjusted according to actual needs, which is conducive to ensuring the data persistence efficiency of the system under high load conditions. In addition, through parallel snapshots and asynchronous write optimization, the impact of snapshot operations on system performance is significantly reduced, and the overall response speed and concurrent processing capabilities of the system are improved; through incremental snapshot mechanisms and dynamic scheduling algorithms, the use of storage resources is effectively reduced, and the system maintenance costs and operating costs are reduced; the data persistence consistency and reliability are guaranteed, and it can run stably even under high load and abnormal conditions, ensuring the integrity and availability of business data.

[0128] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiment of the present application also provides a distributed cache snapshot management device for implementing the above-mentioned distributed cache snapshot management method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more distributed cache snapshot management device embodiments provided below can refer to the above-mentioned limitations on the distributed cache snapshot management method, and will not be repeated here.

[0130] In an exemplary embodiment, as shown in FIG5 , a distributed cache snapshot management device 100 is provided, comprising: an allocation module 120 , a determination module 140 , an execution module 160 and a generation module 180 , wherein:

[0131] The allocation module 120 is used to extract multiple snapshot tasks from the task queue and allocate the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively;

[0132] A determination module 140 is used to determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance;

[0133] An execution module 160 is used to execute, for each snapshot task, through the Redis instance corresponding to the snapshot task, the snapshot command corresponding to the incremental snapshot at a first preset time interval to obtain an incremental snapshot result, and the snapshot command corresponding to the full snapshot at a second preset time interval to obtain a full snapshot result;

[0134] The generating module 180 is used to generate a snapshot report according to the incremental snapshot results and the full snapshot results corresponding to the plurality of snapshot tasks.

[0135] The above-mentioned distributed cache snapshot management device distributes multiple snapshot tasks in the task queue to multiple idle working threads, thereby executing each snapshot task in parallel through multiple threads, reducing the snapshot operation time of a single node, significantly reducing the impact of snapshot operations on system performance, and improving the overall response speed and concurrent processing capabilities of the system; each working thread is connected to a specified Redis instance, and data can be stored in multiple Redis instances in a dispersed manner to avoid the resource bottleneck of a single node. At the same time, for each snapshot task, different interval durations are used to execute incremental snapshots and full snapshots respectively. The interval duration can be flexibly adjusted according to actual needs, which is conducive to ensuring the data persistence efficiency of the system under high load conditions.

[0136] In one embodiment, a snapshot command corresponding to the full snapshot is executed every second preset time period to obtain a full snapshot result, and the execution module 160 is also used to: obtain the last execution time of the full snapshot; determine the current execution time according to the last execution time and the second preset time period; if the system load information at the current execution time meets the full execution condition, execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0137] In one embodiment, the distributed cache snapshot management device 100 further includes a second execution module, and the second execution module is used to:

[0138] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0139] In one embodiment, the distributed cache snapshot management device 100 also includes a first adjustment module, which is used to: determine multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjust the first preset duration to the first adjustment duration, and adjust the second preset duration to the second adjustment duration; execute the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0140] In one embodiment, the distributed cache snapshot management device 100 also includes a third execution module, which is used to: obtain the last execution time of the incremental snapshot; if the number of write operations from the last execution time to the current time exceeds a first preset number, execute the snapshot command corresponding to the incremental snapshot to obtain the incremental snapshot result.

[0141] In one embodiment, the distributed cache snapshot management device 100 also includes a second adjustment module, which is used to: determine multiple consecutive preset execution times based on the last execution time and the first preset duration, with the first preset duration between adjacent preset execution times; if the number of write operations corresponding to each of the multiple consecutive preset execution times is lower than the second preset number, adjust the first preset duration to a third adjustment duration; execute the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result.

[0142] In one embodiment, the snapshot command corresponding to the incremental snapshot is executed every first preset time period to obtain the incremental snapshot result. The execution module 160 is also used to: asynchronously execute the snapshot command corresponding to the incremental snapshot every first preset time period to obtain the incremental snapshot result.

[0143] Each module in the above-mentioned distributed cache snapshot management device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0144] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store incremental snapshot results, full snapshot results, snapshot reports, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distributed cache snapshot management method is implemented.

[0145] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0146] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0147] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance; for each snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time period through the Redis instance corresponding to the snapshot task to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time period to obtain the full snapshot result; generate a snapshot report according to the incremental snapshot results and full snapshot results corresponding to each of the multiple snapshot tasks.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] Obtain the last execution time of the full snapshot; determine the current execution time according to the last execution time and the second preset duration; if the system load information at the current execution time meets the full execution condition, execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0151] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0152] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0153] Determine multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjust the first preset duration to the first adjustment duration, and adjust the second preset duration to the second adjustment duration; execute the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] The last execution time of the incremental snapshot is obtained; if the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

[0156] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0157] Determine multiple consecutive preset execution times according to the last execution time and the first preset duration, and the adjacent preset execution times are separated by the first preset duration; if the number of write operations corresponding to the multiple consecutive preset execution times is lower than the second preset number, adjust the first preset duration to a third adjustment duration; execute the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result.

[0158] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0159] Every first preset time period, the snapshot command corresponding to the incremental snapshot is asynchronously executed to obtain the incremental snapshot result.

[0160] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance; for each snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time period through the Redis instance corresponding to the snapshot task to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time period to obtain the full snapshot result; generate a snapshot report according to the incremental snapshot results and full snapshot results corresponding to each of the multiple snapshot tasks.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] Obtain the last execution time of the full snapshot; determine the current execution time according to the last execution time and the second preset duration; if the system load information at the current execution time meets the full execution condition, execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0165] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0167] Determine multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjust the first preset duration to the first adjustment duration, and adjust the second preset duration to the second adjustment duration; execute the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0169] The last execution time of the incremental snapshot is obtained; if the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0171] Determine multiple consecutive preset execution times according to the last execution time and the first preset duration, and the adjacent preset execution times are separated by the first preset duration; if the number of write operations corresponding to the multiple consecutive preset execution times is lower than the second preset number, adjust the first preset duration to a third adjustment duration; execute the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result.

[0172] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0173] Every first preset time period, the snapshot command corresponding to the incremental snapshot is asynchronously executed to obtain the incremental snapshot result.

[0174] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0175] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance; for each snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time period through the Redis instance corresponding to the snapshot task to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time period to obtain the full snapshot result; generate a snapshot report according to the incremental snapshot results and full snapshot results corresponding to each of the multiple snapshot tasks.

[0176] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0177] Obtain the last execution time of the full snapshot; determine the current execution time according to the last execution time and the second preset duration; if the system load information at the current execution time meets the full execution condition, execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0178] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0179] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181] Determine multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjust the first preset duration to the first adjustment duration, and adjust the second preset duration to the second adjustment duration; execute the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0182] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0183] The last execution time of the incremental snapshot is obtained; if the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] Determine multiple consecutive preset execution times according to the last execution time and the first preset duration, and the adjacent preset execution times are separated by the first preset duration; if the number of write operations corresponding to the multiple consecutive preset execution times is lower than the second preset number, adjust the first preset duration to a third adjustment duration; execute the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result.

[0186] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0187] Every first preset time period, the snapshot command corresponding to the incremental snapshot is asynchronously executed to obtain the incremental snapshot result.

[0188] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0189] Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance; for each snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time period through the Redis instance corresponding to the snapshot task to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time period to obtain the full snapshot result; generate a snapshot report according to the incremental snapshot results and full snapshot results corresponding to each of the multiple snapshot tasks.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] Obtain the last execution time of the full snapshot; determine the current execution time according to the last execution time and the second preset duration; if the system load information at the current execution time meets the full execution condition, execute the snapshot command corresponding to the full snapshot to obtain the full snapshot result.

[0192] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0193] If the system load information at the current execution time does not meet the full execution condition, the real-time system load information is obtained until the real-time system load information meets the full execution condition, and the snapshot command corresponding to the full snapshot is executed to obtain the full snapshot result.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] Determine multiple consecutive preset execution times based on the last execution time and the second preset duration, with the second preset duration between adjacent preset execution times; if multiple consecutive preset execution times do not meet the full execution condition, adjust the first preset duration to the first adjustment duration, and adjust the second preset duration to the second adjustment duration; execute the snapshot command corresponding to the incremental snapshot every first adjustment duration to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset duration to obtain the full snapshot result.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0197] The last execution time of the incremental snapshot is obtained; if the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0199] Determine multiple consecutive preset execution times according to the last execution time and the first preset duration, and the adjacent preset execution times are separated by the first preset duration; if the number of write operations corresponding to the multiple consecutive preset execution times is lower than the second preset number, adjust the first preset duration to a third adjustment duration; execute the snapshot command corresponding to the incremental snapshot every third adjustment duration to obtain the incremental snapshot result.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0202] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0203] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0204] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A distributed cache snapshot management method, characterized in that: The method comprises: Extract multiple snapshot tasks from the task queue, and assign the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; Based on the mapping relationship between worker threads and Redis instances, determine the Redis instance corresponding to each snapshot task; For each snapshot task, through the Redis instance corresponding to the snapshot task, execute the snapshot command corresponding to the incremental snapshot every first preset time to obtain the incremental snapshot result, and execute the snapshot command corresponding to the full snapshot every second preset time to obtain the full snapshot result; Generate a snapshot report based on the incremental snapshot results and full snapshot results corresponding to multiple snapshot tasks.

2. The method according to claim 1, characterized in that: The executing the snapshot command corresponding to the full snapshot every second preset time period to obtain the full snapshot result includes: Get the last execution time of the full snapshot; Determine the current execution time according to the last execution time and the second preset duration; If the system load information at the current execution time satisfies the full execution condition, a snapshot command corresponding to the full snapshot is executed to obtain a full snapshot result.

3. The method according to claim 2, characterized in that The method further comprises: If the system load information at the current execution time does not meet the full execution condition, real-time system load information is obtained until the real-time system load information meets the full execution condition, and a snapshot command corresponding to the full snapshot is executed to obtain a full snapshot result.

4. The method according to claim 2, characterized in that: The method further comprises: Determine a plurality of consecutive preset execution times according to the last execution time and the second preset duration, wherein the second preset duration is separated from adjacent preset execution times; If multiple consecutive preset execution times do not meet the full execution conditions, the first preset duration is adjusted to the first adjustment duration, and the second preset duration is adjusted to the second adjustment duration; the snapshot command corresponding to the incremental snapshot is executed every first adjustment duration to obtain the incremental snapshot result, and the snapshot command corresponding to the full snapshot is executed every second preset duration to obtain the full snapshot result.

5. The method according to claim 1, characterized in that: The method further comprises: Get the last execution time of the incremental snapshot; If the number of write operations from the last execution time to the current time exceeds a first preset number, a snapshot command corresponding to the incremental snapshot is executed to obtain an incremental snapshot result.

6. The method according to claim 5, characterized in that The method further comprises: Determine a plurality of consecutive preset execution times according to the last execution time and the first preset duration, wherein the first preset duration is separated from adjacent preset execution times; If the number of write operations corresponding to each of a plurality of consecutive preset execution moments is lower than the second preset number, the first preset duration is adjusted to a third adjustment duration; and the snapshot command corresponding to the incremental snapshot is executed every the third adjustment duration to obtain the incremental snapshot result.

7. The method according to claim 1, characterized in that The step of executing the snapshot command corresponding to the incremental snapshot every first preset time period to obtain the incremental snapshot result includes: Every first preset time period, the snapshot command corresponding to the incremental snapshot is asynchronously executed to obtain the incremental snapshot result.

8. A distributed cache snapshot management device, characterized in that: The device comprises: An allocation module is used to extract multiple snapshot tasks from the task queue and allocate the extracted multiple snapshot tasks to multiple idle working threads in the thread pool respectively; A determination module is used to determine the Redis instance corresponding to each snapshot task based on the mapping relationship between the working thread and the Redis instance; An execution module is used to execute, for each snapshot task, through the Redis instance corresponding to the snapshot task, the snapshot command corresponding to the incremental snapshot at a first preset time interval to obtain the incremental snapshot result, and the snapshot command corresponding to the full snapshot at a second preset time interval to obtain the full snapshot result; The generation module is used to generate a snapshot report according to the incremental snapshot results and the full snapshot results corresponding to the multiple snapshot tasks.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images