A dynamic rebalancing strategy for a time series database cluster

Abstract

The application relates to a dynamic rebalancing strategy of a time series database cluster, and belongs to the technical field of IT software development, and comprises the following steps: S1. taking a monitoring alarm system as a trigger source, and waiting for the triggering of cluster rebalancing; S2. dynamically selecting a rebalancing strategy; S3. performing a disk data rebalancing process; and S4. performing a cluster shard rebalancing process, wherein the monitoring system is used to monitor the key indicators of each node in real time, such as disk usage, the number of shards and the like, and then the disk data rebalancing and cluster shard rebalancing strategies are dynamically adjusted according to the indicators, so that the resource utilization is improved, the performance is optimized, the stability and availability of the system are improved, and when the node disk usage exceeds a preset threshold, the data is automatically migrated from the node with high usage to the node with low usage, so that the storage and query performance are optimized, the storage efficiency is improved, the storage pressure is reduced, and the write throughput of the time series database is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer science and information technology, and particularly relates to a dynamic rebalancing strategy for a time series database cluster. BACKGROUND

[0002] In the era of big data, it is particularly important to process and store large-scale time series data. Time series databases are widely used due to their efficient processing capabilities for time series data. However, for large-scale time series data, a cluster architecture is usually adopted to provide excellent horizontal scalability and high availability. However, as the amount of data and query load continues to change, it is crucial to continuously maintain the balanced state of the cluster (i.e., redistribute data among nodes).

[0003] The main purpose of the rebalancing operation is to ensure that each node can bear an even load, preventing some nodes from being overloaded, thereby maintaining the running performance and stability of the entire cluster. At the same time, when adding or reducing nodes, rebalancing is also an important step to ensure uniform data distribution and high availability. However, existing rebalancing techniques for time series database clusters usually require manual intervention. For example, administrators need to continuously monitor the cluster state and manually select appropriate rebalancing strategies based on real-time changes in data and load. This method not only consumes a large amount of human resources, but also often fails to achieve the desired effect due to the inability to respond to changes in system state in real time. In addition, existing rebalancing methods often only focus on disk capacity or write throughput, ignoring query availability or query throughput, which can lead to decreased performance and even service interruptions in high query load environments.

[0004] Therefore, how to automatically implement rebalancing of a time series database cluster in a real production environment while considering disk capacity, write throughput, query availability, and query throughput, thereby reducing manual intervention and improving the running efficiency and stability of the cluster, is an important problem currently faced by the field of time series database management and cluster computing. SUMMARY

[0005] In view of the deficiencies of the prior art, the purpose of the present application is to provide a dynamic rebalancing strategy for a time series database cluster, which optimizes the storage and query performance of the cluster by adaptively selecting disk data rebalancing or shard rebalancing. The main steps include triggering of cluster rebalancing, dynamic selection of rebalancing strategy, disk data rebalancing process, and cluster shard rebalancing process.

[0006] The present application proposes a dynamic rebalancing strategy for a time series database cluster, including the following steps:

[0007] S1. Use a monitoring alarm system as a trigger source to wait for the triggering of cluster rebalancing;

[0008] S2. Perform dynamic selection of the rebalancing strategy;

[0009] S3. Perform the disk data rebalancing process;

[0010] S4. Perform the cluster shard rebalancing process.

[0011] Furthermore, the specific steps of the dynamic selection of the rebalancing strategy include:

[0012] S21. Obtain the disk usage rate T1 of each node from the monitoring;

[0013] S22. Obtain the maximum number of shards S max and the minimum number of shards S min ;

[0014] S23. Determine whether T1 is greater than the disk threshold T0. If it is greater than the disk threshold T0, execute S3. If it is less than the disk threshold T0, execute S24;

[0015] S24. Determine whether S max -S min is greater than the shard balance factor. If it is greater, execute S4. If it is less, execute S21.

[0016] Furthermore, the specific steps of the disk data rebalancing process include:

[0017] S31. First, define the threshold K of the cluster disk usage rate d―thr ;

[0018] S32. Sort the cluster nodes in descending order of disk usage rate to obtain an ordered node array host[0..n - 1];

[0019] S33. Sort the cluster shards of the host[0] node in descending order to obtain an ordered shard array shard[0..n - 1];

[0020] S34. Define S max , which is used to identify the shards moved during the rebalancing process;

[0021] S35. Define the cluster shard index variable src and initialize src = 0;

[0022] S36. If src < n - 1, obtain shard[src] from host[0] and assign it to S max , then obtain the disk usage rate K disk of host[0] from the monitoring. Otherwise, it indicates that the shard pointed to by the current src exceeds the boundary of the shard array, and the rebalancing process ends;

[0023] S37. If K disk > K d―thr , further determine whether the disk space of node host[n-1] can accommodate S max , otherwise, it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended.

[0024] Further, the S37, if K disk > K d―thr , further determines whether the disk space of node host[n-1] can accommodate S max , otherwise, it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended, further comprising the steps of:

[0025] S38. If the disk space of cluster node host[n-1] cannot accommodate the shard, that is, the sum of the disk space of host[n-1] and the shard size exceeds the disk usage threshold, then src=src+1, and the step S36 is repeated; otherwise, the shard migration is performed.

[0026] Further, in the S38, the shard migration specifically comprises the following steps:

[0027] S381. Truncate the hot shard, prohibit data from being written into the current shard, and create a new shard to accept new written data;

[0028] S382. Copy the cold shard S min to the target node host[dest];

[0029] S383. Delete the cold shard S min of the source node host[0].

[0030] Further, the cluster shard rebalancing process specifically comprises the following steps:

[0031] S41. First, define the cluster shard balance factor K s―thr ;

[0032] S42. Sort the cluster nodes in descending order according to the number of shards to obtain an ordered node array host[0..n-1];

[0033] S43. Define S min , which is used to identify the shard moved by the shard rebalancing;

[0034] S44. Obtain the smallest cluster shard S min from the host list;

[0035] S45. Define and initialize the node index variable: dest=n-1;

[0036] S46. If dest>0, calculate the shard balance factor K of host[0] and host[dest] shard Otherwise, it indicates that the current dest points to host[0] itself, and the rebalancing process is ended.

[0037] S47. If K shard >K s―thr , judge whether host[dest] has a backup of S min , otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process is ended.

[0038] Further, the K shard >K s―thr , judge whether host[dest] has a backup of S min , otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process is ended.

[0039] S48. If host[dest] has a backup of S min , then dest=dest-1, repeat S46, and perform shard migration.

[0040] Further, in the S48, the performing shard migration specifically includes the following steps:

[0041] S481. truncate the hot shard, prohibit data writing in the current shard, and create a new shard to accept new written data;

[0042] S482. copy the cold shard S min to the target node host[dest];

[0043] S483. delete the cold shard S min of the source node host[0].

[0044] Further, in the S1, the cluster includes multiple nodes.

[0045] Further, the monitoring and alarm system includes a real-time monitoring module and an alarm module.

[0046] The present application has the following advantages:

[0047] 1. The monitoring system is used to monitor the key indicators of each node in real time, such as disk usage, number of shards, etc., and then dynamically adjust the disk data rebalancing and cluster shard rebalancing strategy according to these indicators, thereby improving resource utilization, optimizing performance, and improving system stability and availability.

[0048] 2. When the node disk usage rate exceeds the preset threshold, automatically migrate data from the node with high usage rate to the node with low usage rate, thereby optimizing storage and query performance, improving storage efficiency, reducing storage pressure, and improving the write throughput of the time series database.

[0049] 3. When the number of shards between nodes exceeds the preset shard balancing factor, automatically migrate shards from the node with a large number to the node with a small number, thereby ensuring high availability and enhancing cluster stability, improving data availability, enhancing data redundancy, and improving the query throughput of the database.

[0050] 4. During data or shard migration, consider the available space of the target node and whether a copy of the data or shard to be migrated has been stored, thereby selecting the most suitable target node. The effectiveness and safety of data migration can be ensured, thereby improving data migration efficiency, reducing data redundancy, and improving data security.

[0051] 5. During data or shard migration, first prohibit writing to the data or shard to be migrated, then copy these data or shards to the target node, and finally delete the original data or shard on the source node, which can ensure the consistency and integrity of data migration, improve data consistency, ensure data integrity, and avoid data loss. BRIEF DESCRIPTION OF DRAWINGS

[0052] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0053] Figure 1 The rebalancing trigger process diagram for the embodiments of the application;

[0054] Figure 2 The rebalancing dynamic selection process diagram for the embodiments of the application;

[0055] Figure 3 The disk data rebalancing process diagram for the embodiments of the application;

[0056] Figure 4 The shard rebalancing process diagram for the embodiments of the application. DETAILED DESCRIPTION

[0057] In the following, the technical solutions of the present application will be described clearly and completely in conjunction with the drawings, obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. It should be understood that these descriptions are only exemplary, and are not used to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0058] In addition, in the following description, the description of well-known structures and techniques is omitted to avoid unnecessary confusion of the concepts disclosed in the present application.

[0059] The exemplary embodiments will be described in detail below with reference to the accompanying drawings. In the following description, the same numbers are used to represent the same elements throughout the several drawings. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.

[0060] The present application proposes a dynamic rebalancing strategy for a time series database cluster, which is used to solve the problem that a large amount of human resources are consumed, and due to the inability to respond to changes in system state in real time, the ideal effect is often not achieved. In addition, existing rebalancing methods often only focus on disk capacity or write throughput, while ignoring query availability or query throughput, which may lead to a decline in the performance of the cluster in a high query load environment, and even the problem of service interruption.

[0061] Method embodiments

[0062] In order to assist personnel to understand the present application, the terms appearing in this paper are explained.

[0063] 1. Disk data balancing: the data of the cluster is evenly distributed on different nodes, and the disk usage rate of the cluster nodes meets the balancing condition.

[0064] 2. Cluster shard balancing: the shards of the cluster are evenly distributed on different nodes, and the number of shards of the cluster nodes meets the balancing condition.

[0065] In addition, in the present application, as shown in Figures 1-4 The following embodiments are provided:

[0066] Embodiment 1

[0067] A dynamic rebalancing strategy for a time series database cluster, comprising the following steps:

[0068] S1. Take the monitoring alarm system as the trigger source, and wait for the trigger of cluster rebalancing;

[0069] S2. Perform dynamic selection of rebalancing strategy;

[0070] S3. Perform disk data rebalancing process;

[0071] S4. Perform cluster shard rebalancing process.

[0072] Embodiment 2

[0073] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0074] S1. Take the monitoring alarm system as the trigger source, and wait for the trigger of cluster rebalancing;

[0075] S2. Obtain the disk usage T1 of each node from the monitoring;

[0076] S3. Obtain the maximum number of shards S max and the minimum number of shards S min of the node from the monitoring;

[0077] S4. Determine whether T1 is greater than the disk threshold T0, if yes, perform S6, and if no, perform S5;

[0078] S5. Determine whether S max -S min is greater than the shard balancing factor, if yes, perform S7, and if no, perform S2;

[0079] S6. Perform disk data rebalancing process;

[0080] S7. Perform cluster shard rebalancing process.

[0081] Embodiment 3

[0082] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0083] S1. Take the monitoring alarm system as the trigger source, and wait for the trigger of cluster rebalancing;

[0084] S2. Perform dynamic selection of rebalancing strategy;

[0085] S3. First, define the threshold K d―thr of cluster disk usage;

[0086] S4. Sort the cluster nodes in descending order of disk usage to obtain an ordered node array host[0..n-1];

[0087] S5. The cluster shards of the host[0] node are sorted in descending order to obtain an ordered shard array shard[0..n-1];

[0088] S6. S is defined max , used to identify the shard moved in the rebalancing process;

[0089] S7. The cluster shard index variable src is defined, and src=0 is initialized.

[0090] S8. If src max , the disk usage K of host[0] is obtained from the monitoring disk , otherwise it indicates that the shard pointed to by the current src exceeds the shard array boundary, and the rebalancing process is ended;

[0091] S9. If K disk >K d―thr , it is further judged whether the disk space of the node host[n-1] can accommodate S max , otherwise it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended;

[0092] S10. The cluster shard rebalancing process is performed.

[0093] Embodiment 4

[0094] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0095] S1. The monitoring alarm system is used as a trigger source, and the triggering of the cluster rebalancing is waited;

[0096] S2. The dynamic selection of the rebalancing strategy is performed;

[0097] S3. First, the threshold K d―thr of the cluster disk usage is defined;

[0098] S4. The cluster nodes are sorted in descending order of disk usage to obtain an ordered node array host[0..n-1];

[0099] S5. The cluster shards of the host[0] node are sorted in descending order to obtain an ordered shard array shard[0..n-1];

[0100] S6. S is defined max , used to identify the shard moved in the rebalancing process;

[0101] S7. The cluster shard index variable src is defined, and src=0 is initialized.

[0102] S8. If src < n - 1, obtain shard[src] from host[0] and assign it to S max , and then obtain the disk usage rate K of host[0] from the monitoring disk , otherwise it indicates that the shard pointed to by the current src exceeds the boundary of the shard array, and the rebalancing process ends;

[0103] S9. If K disk > K d―thr , then further determine whether the disk space of node host[n - 1] can accommodate S max , otherwise it indicates that the current disk data has reached the balance condition, and the rebalancing process ends;

[0104] S10. If the disk space of the cluster node host[n - 1] cannot accommodate the shard, that is, the sum of the disk space of host[n - 1] and the shard size exceeds the disk usage rate threshold, then src = src + 1, and repeat step S8; otherwise, perform shard migration;

[0105] S11. Perform the cluster shard rebalancing process.

[0106] Embodiment 5

[0107] A dynamic rebalancing strategy for a time - series database cluster includes the following steps:

[0108] S1. Use the monitoring and alarm system as the trigger source and wait for the trigger of cluster rebalancing;

[0109] S2. Perform dynamic selection of the rebalancing strategy;

[0110] S3. First, define the threshold K of the cluster disk usage rate d―thr ;

[0111] S4. Sort the cluster nodes in descending order of disk usage rate to obtain an ordered node array host[0..n - 1];

[0112] S5. Sort the cluster shards of node host[0] in descending order to obtain an ordered shard array shard[0..n - 1];

[0113] S6. Define S max , which is used to identify the shard moved during the rebalancing process;

[0114] S7. Define the cluster shard index variable src and initialize src = 0.

[0115] S8. If src < n - 1, obtain shard[src] from host[0] and assign it to S max , and then obtain the disk usage rate K of host[0] from the monitoringdisk , otherwise, it indicates that the current src points to the slice beyond the slice array boundary, and the rebalancing process is ended;

[0116] S9. If K disk > K d―thr , further judge whether the disk space of the node host[n-1] can accommodate S max , otherwise, it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended;

[0117] S10. If the disk space of the cluster node host[n-1] cannot accommodate the slice, that is, the sum of the disk space of host[n-1] and the size of the slice exceeds the disk usage threshold, then src = src + 1, and the step S8 is repeated; otherwise, the slice migration is performed;

[0118] In addition, in the present application, the slice migration specifically includes:

[0119] S11. The hot slice is truncated, and data writing into the current slice is prohibited, and a new slice is created to accept new written data;

[0120] S12. The cold slice S min is copied into the target node host[dest];

[0121] S13. The cold slice S min of the source node host[0] is deleted;

[0122] S14. The cluster slice rebalancing process is performed.

[0123] Embodiment 6

[0124] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0125] S1. A monitoring alarm system is used as a trigger source, and the triggering of the cluster rebalancing is waited;

[0126] S2. The dynamic selection of the rebalancing strategy is performed;

[0127] S3. The disk data rebalancing process is performed;

[0128] S4. First, a cluster slice balance factor K s―thr is defined;

[0129] S5. The cluster nodes are sorted in descending order according to the number of slices to obtain an ordered node array host[0..n-1];

[0130] S6. S min is defined, which is used to identify the slice moved by the slice rebalancing;

[0131] S7. Get the smallest cluster shard S from the host list min ;

[0132] S8. Define and initialize the node index variable: dest = n-1;

[0133] S9. If dest > 0, calculate the shard balance factor K of host[0] and host[dest] shard , otherwise, it indicates that the current dest points to host[0] itself, and the rebalancing process is ended;

[0134] S10. If K shard > K s―thr , judge whether host[dest] exists S min backup, otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process is ended.

[0135] Embodiment 7

[0136] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0137] S1. Take the monitoring alarm system as a trigger source, and wait for the trigger of cluster rebalancing;

[0138] S2. Perform dynamic selection of the rebalancing strategy;

[0139] S3. Perform the disk data rebalancing process;

[0140] S4. First, define the cluster shard balance factor K s―thr ;

[0141] S5. Sort the cluster nodes in descending order according to the number of shards to obtain an ordered node array host[0..n-1];

[0142] S6. Define S min , which is used to identify the shard of the shard rebalancing movement;

[0143] S7. Get the smallest cluster shard S from the host list min ;

[0144] S8. Define and initialize the node index variable: dest = n-1;

[0145] S9. If dest > 0, calculate the shard balance factor K of host[0] and host[dest] shard , otherwise, it indicates that the current dest points to host[0] itself, and the rebalancing process is ended;

[0146] S10. If K shard > Ks―thr , then determine whether host[dest] has a backup of S min , otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process ends.

[0147] S41. If host[dest] has a backup of S min , then dest = dest-1, repeat S9 to perform shard migration.

[0148] Embodiment 8

[0149] A dynamic rebalancing strategy for a time series database cluster, comprising the following steps:

[0150] S1. Take the monitoring alarm system as the trigger source, and wait for the trigger of cluster rebalancing;

[0151] S2. Perform dynamic selection of the rebalancing strategy;

[0152] S3. Perform the disk data rebalancing process;

[0153] S4. First, define the cluster shard balance factor K s―thr ;

[0154] S5. Sort the cluster nodes in descending order according to the number of shards to obtain an ordered node array host[0..n-1];

[0155] S6. Define S min , which is used to identify the shard of the shard rebalancing movement;

[0156] S7. Get the smallest cluster shard S min from the host list;

[0157] S8. Define and initialize the node index variable: dest = n-1;

[0158] S9. If dest>0, calculate the shard balance factor K shard of host[0] and host[dest], otherwise, it indicates that the current dest points to host[0] itself, and the rebalancing process ends;

[0159] S10. If K shard >K s―thr , then determine whether host[dest] has a backup of S min , otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process ends.

[0160] S11. If host[dest] has a backup of S min , then dest = dest-1, repeat S9 to perform shard migration.

[0161] In addition, in the embodiment, the shard migration specifically includes the following steps:

[0162] S12. Cut off the hot shard, prohibit data writing in the current shard, and create a new shard to accept new written data;

[0163] S13. Copy the cold shard S min to the target node host[dest];

[0164] S14. Delete the cold shard S min .

[0165] Embodiment 9

[0166] A dynamic rebalancing strategy of a time series database cluster, comprising the following steps:

[0167] S1. Take the monitoring alarm system as a trigger source, the cluster includes multiple nodes, the monitoring alarm system includes a real-time monitoring module and an alarm module, and waits for the trigger of cluster rebalancing;

[0168] S2. Perform dynamic selection of the rebalancing strategy;

[0169] S3. Perform a disk data rebalancing process;

[0170] S4. Perform a cluster shard rebalancing process.

[0171] In addition, in the present application, another embodiment is also provided:

[0172] Embodiment 10

[0173] The present application describes in detail how to use a dynamic rebalancing strategy to achieve automatic rebalancing of a time series database cluster through the following examples. Suppose we have a time series database cluster consisting of 5 nodes (Node1, Node2, Node3, Node4, Node5). Each node has its own disk space and stores different number of shards. We set the disk usage threshold to 80%, the shard balancing factor to 3, and the monitoring system to monitor the disk usage of each node in real time.

[0174] Disk data rebalance process: At a certain moment, the disk usage of Node1 has exceeded 80%, the monitoring alarm module triggers the disk data rebalance. We sort the 5 nodes by disk usage, the result is: Node1 (85%), Node2 (80%), Node3 (75%), Node4 (70%), Node5 (65%). We choose Node1 as the source node of data migration, and Node5 as the target node. We first sort all the shards of Node1 in descending order according to their sizes, the result is: Shard5 (50G), Shard4 (30G), Shard3 (20G), Shard2 (10G), Shard1 (5G). We calculate whether the sum of the disk capacity of Node5 and Shard5 exceeds the disk threshold, if it exceeds, we choose Shard4 as the migration shard, and repeat this process to determine the shards migrated from Node1. Then we migrate the shard from Node1 to Node5, ensuring that the disk usage of Node1 can be reduced to below 80%. During the data migration process, we first disable the write to the data to be migrated, then copy the migration shard to Node5, and finally delete the old shard on Node1. After the data migration is completed, we recalculate the disk usage of each node, if the disk usage of all nodes does not exceed the threshold, the disk data rebalance process ends; otherwise, we repeat the above steps until the disk usage of all nodes does not exceed the threshold.

[0175] Cluster shard rebalance process: At a certain moment, the difference between the number of shards owned by Node1 and the number of shards owned by Node5 exceeds 3, the monitoring alarm triggers the cluster shard rebalance. We sort the 5 nodes by the number of shards, the result is: Node1 (15), Node2 (13), Node3 (12), Node4 (10), Node5 (8). We choose Node1 as the source node of data migration, and Node5 as the target node. We select the shard with the smallest data volume from Node1 as the migration shard, ensuring that the difference between the number of shards of Node1 and Node5 is less than or equal to 3. If Node5 has backup data of the migration shard, we choose Node4 as the target shard, and repeat this process to determine the target node. During the data migration process, we first disable the write to the shard to be migrated, then copy the shards to the target node, and finally delete the old shard on Node1. After the data migration is completed, we recalculate the number of shards of each node, if the difference between the number of shards of all nodes does not exceed the shard balancing factor, the cluster shard rebalance process ends; otherwise, we repeat the above steps until the difference between the number of shards of all nodes does not exceed the shard balancing factor.

[0176] Based on the same inventive concept, another embodiment of the present application provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus,

[0177] The memory is used for storing a computer program.

[0178] The processor is used for executing the program stored on the memory, and realizes the dynamic rebalancing strategy of the time series database cluster.

[0179] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The communication interface is used for communication between the terminal and other devices. The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage system located away from the aforementioned processor.

[0180] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0181] In addition, to achieve the above-mentioned purpose, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the dynamic rebalancing strategy of the time series database cluster.

[0182] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.

[0183] Embodiments of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0184] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0185] These computer program instructions can also be loaded into a computer or other programmable data processing terminal devices to cause a series of operational steps to be performed on the computer or other programmable terminal devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0186] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. "And / or" means that either one of the two or both can be selected. Also, the terms "comprises", "comprising", or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus including the element.

[0187] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0188] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the embodiments of the present application, and not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and any changes or replacements easily thought of by those skilled in the art within the technical range disclosed by the present application should be covered within the protection scope of the present application.

Claims

1. A dynamic rebalancing strategy for a time series database cluster, the strategy comprising: The method comprises the following steps: S1. Taking the monitoring alarm system as a trigger source, waiting for triggering of cluster rebalancing; S2. Dynamically selecting a rebalancing strategy; S3. Performing a disk data rebalancing process; S4. Performing a cluster shard rebalancing process; The dynamic selection of the rebalancing strategy specifically comprises the following steps: S21. Obtaining disk usage T1 of each node from the monitoring; S22. Obtain the maximum number of shards S of the node from the monitoring max and the minimum number of shards S min ; S23. Judging whether T1 is greater than a disk threshold T0, if yes, performing S3, and if no, performing S24; S24. Determine S max -S min whether greater than the slice balance factor, and if so, perform S4, and if not, perform S21; The disk data rebalancing process specifically comprises the following steps: S31. First define the threshold K of cluster disk usage d-thr ; S32. Sorting the cluster nodes in descending order of disk usage to obtain an ordered node array host[0..n-1]; S33. Sorting cluster shards of the host[0] node in descending order to obtain an ordered shard array shard[0..n-1]; S34. Definition S max , for identifying moved fragments in a rebalancing process; S35. Defining a cluster shard index variable src and initializing src=0; S36. If src < n-1, get shard[src] from host[0] and assign it to S max Then get the disk usage K of host[0] from the monitoring disk Otherwise, it means the shard pointed by current src is out of the shard array boundary, end the rebalance process; S37. If K disk > K d-thr , further judge whether the disk space of node host[n-1] can accommodate S max , otherwise it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended; the If K disk > K d-thr , further judge whether the disk space of node host[n-1] can accommodate S max , otherwise it indicates that the current disk data has reached the balance condition, and the rebalancing process is ended, and the method further comprises the following steps: S38. If disk space of the cluster node host[n-1] cannot accommodate the shard, i.e., the sum of the disk space of the host[n-1] and the shard size exceeds the disk usage threshold, then src=src+1, and repeating step S36; otherwise, performing shard migration; In the S38, the shard migration specifically comprises the following steps: S381. Truncating a hot shard, prohibiting data from being written into the current shard, and creating a new shard to accept new written data; S382. Duplicate cold slice S min into target node host[dest] ; S383. Delete the cold shard S of the source node host[0] min .

2. The dynamic rebalancing strategy of a time series database cluster according to claim 1, characterized in that, The cluster shard rebalancing process specifically comprises the following steps: S41. First define the cluster shard balancing factor K s-thr ; S42. Sorting the cluster nodes in descending order of the number of shards to obtain an ordered node array host[0..n-1]; S43. Define S min , for identifying the slices that are moving in slice rebalancing. S44. Get the smallest cluster shard S from the host list min ; S45. Defining and initializing a node index variable: dest=n-1; S46. If dest > 0, then compute the shard balancing factor K for host[0] and host[dest] shard Otherwise, it indicates that the current dest points to host[0] itself, and the rebalancing process ends. S47. If K shard > K s-thr , then determine whether a backup of host[dest] exists S min , otherwise, indicate that the current cluster shard has reached the balance condition and end the rebalancing process.

3. The dynamic rebalancing strategy of a time series database cluster according to claim 2, characterized in that, The K shard >K s-thr If the backup of S min does not exist, otherwise, it indicates that the current cluster shard has reached the balance condition, and the rebalancing process ends, including the following steps: S48. If host[dest] exists S min of the backup, then dest = dest - 1, repeat S46, perform the shard migration.

4. The dynamic rebalancing strategy of a time series database cluster according to claim 3, characterized in that, In the S48, the performing of the shard migration specifically comprises the following steps: S481. Truncating a hot shard, prohibiting data from being written into the current shard, and creating a new shard to accept new written data; S482. Replicate cold slice S min into target node host[dest] ; S483. Delete the cold shard S of the source node host[0] min .

5. The dynamic rebalancing strategy of a time series database cluster according to claim 1, characterized in that, In the S1, the cluster comprises a plurality of nodes. 6.The dynamic rebalancing strategy of a time-series database cluster of claim 1, wherein, The monitoring alarm system comprises a real-time monitoring module and an alarm module.

Images