Method and device for dynamically balancing and deploying multiple raft cluster instances in a multi-data center environment

Abstract

The present invention discloses a method and device for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment includes: the deployment platform issues the raft cluster instance deployment service; determining the target data center according to the type of the raft cluster instance deployment service; statistically calculating the current load of the servers in each target data center, and screening and sorting the servers in each target data center according to the current load; and cyclically selecting servers among the target data centers according to the screening and sorting results to deploy raft cluster instances, so as to perform dynamic balanced deployment of raft cluster instances in a multi-data center environment. The method for dynamically balancing the deployment of multiple raft cluster instances of the present invention realizes dynamic balanced deployment of the raft cluster in a multi-data center environment, and ensures that the multi-data centers can provide the maximum raft group fault tolerance.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed storage, and particularly to a method and device for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment. Background Art

[0002] Raft - a distributed consensus algorithm, where a leader is elected among nodes, and the node that obtains the votes of the majority of nodes becomes the leader node, and other nodes act as follower nodes.

[0003] The current production environment is a multi-cloud provider multi-data center, and it is necessary to deploy storage instances of raft nodes. Multiple storage instances form a raft cluster, and it is necessary to balance the storage instances that load each server among multiple raft clusters. The purpose of the raft cluster to be evenly deployed among the servers in each data center is to improve the fault tolerance of the raft cluster, that is, to minimize the impact of server failures on services to the greatest extent, thereby improving service stability. And improving the fault tolerance requires that when the server resources are sufficient for each shard under the cluster, the instances of the shard are kept on different servers.

[0004] Therefore, how to achieve the dynamic balanced deployment of multiple raft clusters in a multi-data center environment is crucial for improving the fault tolerance of the raft cluster and enhancing service stability, and is the main technical problem to be solved by the present invention.

[0005] In view of this, the present invention patent is specifically proposed. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention proposes a method and device for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment, to achieve the dynamic balanced deployment of the raft cluster in a multi-data center environment, and ensure the invention purpose of maximizing the provision of raft group fault tolerance in a multi-data center. The specific technical solutions are as follows:

[0007] The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment includes:

[0008] The deployment platform issues the raft cluster instance deployment service;

[0009] Determine the target data center according to the type of the raft cluster instance deployment service;

[0010] Statistically calculate the current load of the servers in each target data center, and screen and sort the servers in each target data center according to the current load;

[0011] Deploy the raft cluster instances by circularly selecting servers among the target data centers according to the screening and sorting results, and perform the dynamic balanced deployment of the raft cluster instances in a multi-data center environment.

[0012] As an alternative implementation of the present invention, the types of raft cluster instance deployment services issued by the deployment platform include cluster creation services, and the target computer room determined according to the cluster creation service is the computer room where all current raft clusters are deployed;

[0013] The steps of statistically calculating the current load of servers in each target computer room and screening and sorting the servers in each target computer room according to the current load include:

[0014] Statistically calculate the number of instances of the server that have been deployed in all current raft clusters;

[0015] Group the servers by computer room;

[0016] Sort the servers in each group in descending order of the number of deployed instances;

[0017] The steps of circularly selecting servers between each target computer room according to the screening and sorting results to deploy raft cluster instances and performing dynamic load balancing deployment of raft cluster instances in a multi-computer room environment include:

[0018] According to the number of shards of the newly created cluster and the number of instances of each shard, circularly select servers between groups and within groups until all the servers required by the cluster are allocated.

[0019] As an alternative implementation of the present invention, the steps of circularly selecting servers between groups and within groups according to the number of shards of the newly created cluster and the number of instances of each shard include:

[0020] Specify a preferred computer room according to the cluster creation service, and set the group where the preferred computer room is located as the preferred group;

[0021] According to the number of shards of the newly created cluster and the number of instances of each shard, first select servers from the preferred group according to the sorting within the group;

[0022] Starting from the preferred group, sequentially select the server with the highest sorting from the servers with the fewest selected times in each group, and circularly select servers between groups and within groups.

[0023] As an alternative implementation of the present invention, the types of raft cluster instance deployment services issued by the deployment platform include cluster expansion services, and the target computer room determined according to the cluster expansion service is the computer room where the servers currently deployed in the expanded cluster are located;

[0024] The steps of statistically calculating the current load of servers in each target computer room and screening and sorting the servers in each target computer room according to the current load include:

[0025] Set all the servers in all target computer rooms as expanded servers, and group the expanded servers by computer room;

[0026] Sort the grouped expanded servers in descending order according to the number of instances already deployed in the current entire cluster;

[0027] The dynamic balanced deployment of raft cluster instances in a multi-data center environment by circularly selecting servers among target data centers according to the screening and sorting results includes:

[0028] For a shard of the expanded cluster to be added, first determine whether there is an expanded server in the group of servers where the current shard is deployed and has not deployed an instance of this shard;

[0029] If the judgment result is yes, select the expanded server that has not deployed an instance of this shard and is ranked the highest for deployment. If the judgment result is no, select the expanded server ranked the highest in the group of servers where the current shard is deployed;

[0030] Add each shard of the expanded cluster one by one until all shards of the expanded cluster have completed the expansion task.

[0031] As an alternative embodiment of the present invention, after each shard of the expanded cluster is allocated, it is necessary to re-sort the grouped expanded servers, and after the re-sorting is completed, allocate the next shard of the expanded cluster.

[0032] As an alternative embodiment of the present invention, the types of raft cluster instance deployment services issued by the deployment platform include cluster scaling-down services, and the target data center determined according to the cluster scaling-down service is the data center where the servers that continue to serve in the scaled-down cluster are located;

[0033] The statistical calculation of the current load of servers in each target data center and the screening and sorting of servers in each target data center according to the current load include:

[0034] Group all servers in all target data centers by data center;

[0035] Sort the grouped servers in descending order according to the number of instances already deployed in the current entire cluster;

[0036] The dynamic balanced deployment of raft cluster instances in a multi-data center environment by circularly selecting servers among target data centers according to the screening and sorting results includes:

[0037] Calculate the affected shards after the offline servers in the scaled-down cluster and the number of instances corresponding to the affected shards;

[0038] According to the sorted servers, perform individual compensation deployments for the affected shards.

[0039] As an alternative embodiment of the present invention, the step of compensating each affected shard according to the sorted servers includes:

[0040] When compensating for the instances in the affected shards, select other servers in the same group in the computer room where the server corresponding to the affected shard is located to take over;

[0041] Determine whether there are servers in the same group that have not deployed instances of the affected shard;

[0042] If the judgment result is yes, select the server in the same group that has not deployed the instance of the affected shard and has the earliest sorting for compensatory deployment. If the judgment result is no, select the server in the same group with the earliest sorting in the group where the server deploying the affected shard is located for compensatory deployment.

[0043] As an alternative embodiment of the present invention, the method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment in this embodiment is characterized in that it further includes: adding the instances of compensatory deployment to the shards of the cluster, and then taking offline the instances on the servers that belong to the offline servers in the cluster.

[0044] As an alternative embodiment of the present invention, the method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment in this embodiment includes: each server in each computer room has an agent process, and the agent process registers server information with the deployment platform. The server information includes the server IP and the computer room where the server is located; the deployment platform manages the agent processes on each server and distributes deployment tasks to the agent processes.

[0045] This embodiment also provides a device for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment, including a deployment platform. The deployment method executed by the deployment platform includes:

[0046] Issuing the deployment service of raft cluster instances;

[0047] Determine the target computer room according to the type of the deployment service of raft cluster instances;

[0048] Statistically calculate the current load of each target computer room, and screen and sort the servers in each target computer room according to the current load;

[0049] Select servers to deploy raft cluster instances cyclically among the target computer rooms according to the screening and sorting results, and perform dynamic balancing deployment of raft cluster instances in a multi-computer room environment.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] After the deployment platform issues the raft cluster instance deployment service, regardless of the type of raft cluster instance deployment service, it is necessary to sort the servers in the computer room according to the current load of the servers in the computer room, and then deploy the raft cluster instances according to the sorting result. The basic principle of server sorting in the present invention is that the server with the smaller load is sorted more forward, so as to better utilize the server resources and ensure the load balance among the servers in the computer room. The present invention deploys raft cluster instances by circularly selecting servers among the target computer rooms according to the filtered sorting result, so that the instances of the raft cluster are deployed on the servers in different computer rooms as much as possible, avoiding data loss of the raft cluster caused by the failure of a single or local computer room, and ensuring the load balance among the computer rooms.

[0052] The types of raft cluster instance deployment services issued by the deployment platform in the present invention include cluster creation, cluster expansion, and cluster contraction, covering the entire life cycle of the raft cluster. Therefore, the multi-raft cluster instance dynamic balancing deployment method of the present invention ensures that the load of each server in multiple computer rooms is balanced during the life cycle of the raft cluster.

[0053] Therefore, the multi-raft cluster instance dynamic balancing deployment method of the present invention realizes the dynamic balancing deployment of the raft cluster in a multi-computer room environment, ensuring that the multi-computer rooms can provide the maximum raft group fault tolerance. Brief Description of the Drawings

[0054] Figure 1 Schematic diagram of a multi-raft cluster in an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of a single raft group (one shard) in an embodiment of the present invention;

[0056] Figure 3 Flowchart of the multi-raft cluster instance dynamic balancing deployment method in a multi-computer room environment in an embodiment of the present invention;

[0057] Figure 4 Flowchart of cluster creation in the multi-raft cluster instance dynamic balancing deployment method in a multi-computer room environment in an embodiment of the present invention;

[0058] Figure 5 Flowchart of cluster expansion in the multi-raft cluster instance dynamic balancing deployment method in a multi-computer room environment in an embodiment of the present invention;

[0059] Figure 6 Flowchart of cluster contraction in the multi-raft cluster instance dynamic balancing deployment method in a multi-computer room environment in an embodiment of the present invention. Detailed Description of the Invention

[0060] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention.

[0061] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0062] It should be noted that, without conflict, the embodiments in the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0063] It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

[0064] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the present invention is usually placed during use, or the orientation or positional relationship commonly understood by those skilled in the art. Such terms are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, the terms "first", "second", etc. are only used for descriptive distinction and should not be construed as indicating or implying relative importance.

[0065] See Figure 1 As shown, it is a schematic diagram of a multi-raft cluster in this embodiment. Each raft cluster contains multiple raft groups, and each group is called a shard.

[0066] See Figure 2 As shown, it is a schematic diagram of a single raft group (a shard). A shard contains multiple raft nodes, including 1 master node + N slave nodes.

[0067] Each node belongs to a server. The number of nodes of all raft clusters on a single server is called the load. The deployment platform needs to ensure dynamic balance of the machine load when deploying multiple raft clusters across computer rooms.

[0068] For this purpose, see Figure 3As shown in the figure, this embodiment proposes a method for dynamically balanced deployment of multiple raft cluster instances in a multi-data center environment, including:

[0069] The deployment platform issues the raft cluster instance deployment service;

[0070] Determine the target data center according to the type of the raft cluster instance deployment service;

[0071] Statistically calculate the current load of the servers in each target data center, and screen and sort the servers in each target data center according to the current load;

[0072] Select servers to deploy raft cluster instances cyclically among the target data centers according to the screening and sorting results, and perform dynamic balanced deployment of raft cluster instances in a multi-data center environment.

[0073] The types of the raft cluster instance deployment service issued by the deployment platform in this embodiment include cluster creation, cluster expansion, and cluster contraction, covering the entire life cycle of the raft cluster. Therefore, the method for dynamically balanced deployment of multiple raft cluster instances in this embodiment ensures that the load of each server in multiple data centers is balanced during the life cycle of the raft cluster.

[0074] After the deployment platform issues the raft cluster instance deployment service in this embodiment, regardless of the type of the raft cluster instance deployment service, it is necessary to sort the servers according to the current load of the servers in the data center, and then deploy the raft cluster instances according to the sorting results. The basic principle of server sorting in this embodiment is that the servers with smaller loads are sorted more forward, so as to better utilize server resources and ensure the load balance among the servers in the data center. In this embodiment, the servers are selected cyclically among the target data centers according to the screening and sorting results to deploy raft cluster instances, so that the instances of the raft cluster are deployed on the servers in different data centers as much as possible, avoiding data loss of the raft cluster caused by single or partial data center failures, and ensuring the load balance among the data centers.

[0075] Therefore, the method for dynamically balanced deployment of multiple raft cluster instances in this embodiment realizes dynamic balanced deployment of the raft cluster in a multi-data center environment, and ensures that multiple data centers can provide the maximum raft group fault tolerance.

[0076] The method for dynamically balanced deployment of multiple raft cluster instances in this embodiment adopts different deployment strategies for different types of raft cluster instance deployment services to dynamically adjust the server load to a balanced state. The specific dynamic balanced deployment strategy is as follows:

[0077] See Figure 4As shown in the figure, the types of raft cluster instance deployment services issued by the deployment platform in this embodiment include cluster creation services, and the target computer room determined according to the cluster creation service is the computer room where all current raft clusters are deployed;

[0078] The steps of calculating the current load of each target computer room and screening and sorting the servers in each target computer room according to the current load include:

[0079] Count the number of instances of the server that have been deployed in all current raft clusters;

[0080] Group the servers by computer room;

[0081] Sort the servers in each group in descending order of the number of deployed instances;

[0082] The steps of circularly selecting servers among each target computer room according to the screening and sorting results to deploy raft cluster instances and performing dynamic load balancing deployment of raft cluster instances in a multi-computer room environment include:

[0083] According to the number of shards of the newly created cluster and the number of instances of each shard, circularly select servers between groups and within groups until all the servers required by the cluster are allocated.

[0084] When the multi-raft cluster instance dynamic load balancing deployment method in this embodiment creates a new cluster service, it is necessary to perform load statistics and calculations on all servers in all current computer rooms, and deploy the instances of the newly created cluster one by one on the servers in different computer rooms. When selecting a specific server within a computer room, the server with less load is preferentially selected. In this embodiment, when selecting a computer room, the deployment starts from the specified priority group. Of course, it can also start randomly or in a specified order.

[0085] Furthermore, the steps of circularly selecting servers between groups and within groups according to the number of shards of the newly created cluster and the number of instances of each shard in this embodiment include:

[0086] Specify a priority computer room according to the cluster creation service, and set the group where the priority computer room is located as the priority group;

[0087] According to the number of shards of the newly created cluster and the number of instances of each shard, first select servers from the priority group according to the sorting within the group;

[0088] Starting from the priority group, sequentially select the server with the highest sorting from the servers with the fewest selected times in each group, and circularly select servers between groups and within groups.

[0089] Specifically, for the cluster creation and deployment service in this embodiment, the dynamic load balancing deployment strategy process of the raft cluster instance is as follows:

[0090] Step 1: The statistical server counts the number of instances already deployed in all clusters;

[0091] Step 2: Group the servers by computer room;

[0092] Step 3: Sort the servers in each group. The fewer the number of deployed instances, the higher the sorting position;

[0093] Step 4: According to the specified preferred computer room, set the group where the preferred computer room is located as the preferred group;

[0094] Step 5: Based on the number of shards in the cluster and the number of instances in each shard, first select servers from the preferred group. The higher the sorting position, the earlier it is selected;

[0095] Step 6: Starting from the preferred group, select the server with the highest sorting position from the servers in each group that have been selected the fewest times. Select cyclically between groups and within groups until the servers required by the cluster have been allocated.

[0096] Therefore, when there are already M raft clusters in multiple computer rooms, and a new raft cluster (including several raft groups) is added at this time, the dynamic load balancing deployment method of multiple raft cluster instances in this embodiment needs to calculate the load of each computer room. For a raft group, the lower the load of the machines in each computer room, the more likely it is to be selected first, and select alternately between computer rooms, so as to ensure that the new raft cluster will not damage the machine load balance of the original cluster.

[0097] See Figure 5 As shown, the types of raft cluster instance deployment services issued by the deployment platform in this embodiment include cluster expansion services, and the target computer room determined according to the cluster expansion service is the computer room where the servers currently deployed in the expanded cluster are located;

[0098] The statistical calculation of the current load of each target computer room and the screening and sorting of the servers in each target computer room according to the current load include:

[0099] Set all servers in all target computer rooms as expanded servers and group the expanded servers by computer room;

[0100] Sort the grouped expanded servers in descending order according to the number of instances already deployed in the current all clusters;

[0101] The cyclic selection of servers between each target computer room according to the screening and sorting results to deploy raft cluster instances and the dynamic load balancing deployment of raft cluster instances in a multi-computer room environment include:

[0102] For adding a new shard to the expanded cluster, first judge whether there are expanded servers in the group of servers deployed for the current shard that have not deployed instances of this shard;

[0103] If the judgment result is yes, select the expansion server that has not deployed this shard instance and has the earliest sorting for deployment. If the judgment result is no, select the expansion server with the earliest sorting within the group of servers where the current shard is deployed;

[0104] Add each shard of the expansion cluster one by one until all shards of the expansion cluster have completed the expansion task.

[0105] When the dynamic balanced deployment method of multiple raft cluster instances in this embodiment performs cluster expansion, for the shard instances of the expansion cluster, it preferentially deploys them to the servers that have not been deployed, ensuring that the current cluster is not affected or the impact is reduced as much as possible, and ensuring that multiple computer rooms can provide the maximum raft group fault tolerance.

[0106] Further, after each shard of the expansion cluster is allocated, it is necessary to re-sort the grouped expansion servers, and after the re-sorting is completed, the next shard of the expansion cluster is allocated.

[0107] Specifically, for the cluster expansion deployment service in this embodiment, the process of the dynamic balanced deployment strategy of raft cluster instances is as follows:

[0108] Step 1: Group the expansion servers according to computer rooms;

[0109] Step 2: Sort the grouped servers. The sorting is based on the number of instances already deployed by the servers in the current cluster. The fewer the number of instances, the earlier the sorting;

[0110] Step 3: Allocate servers to the shards of the cluster according to the sorted groups. The allocation principle is that when starting to grab servers from the current group, if there are servers that have not deployed instances for this shard, select the one with the earliest sorting from these servers. If not, directly select the server with the earliest sorting from this group;

[0111] Step 4: After each shard is allocated, it is necessary to re-sort the grouped servers. For the next shard, repeat Step 3 to select servers until all shards of the cluster have completed the expansion task.

[0112] See Figure 6As shown in the figure, the types of raft cluster instance deployment services issued by the deployment platform in this embodiment include cluster scaling-down services. When deploying a cluster scaling-down service, since the distribution of offline servers among shards is random, after the offline servers, there are some shards with a large reduction in the number of instances, while some shards are not affected. The affected shards need to replenish instances to ensure that the fault tolerance of the shards does not decrease due to server offline. Therefore, it is necessary to calculate the migrated instances and then replenish the instances of the damaged shards.

[0113] Since the instances of the offline servers will only be taken over by the servers in the same computer room, the target computer room determined according to the cluster scaling-down service is the computer room where the servers that continue to serve in the scaled-down cluster are located.

[0114] The statistical calculation of the current load of each target computer room and the screening and sorting of the servers in each target computer room according to the current load in this embodiment include:

[0115] Group all the servers in all the target computer rooms by computer room;

[0116] Sort the grouped servers in descending order according to the number of instances already deployed in the current entire cluster.

[0117] The dynamic balanced deployment of raft cluster instances in a multi-computer room environment by cyclically selecting servers to deploy raft cluster instances among each target computer room according to the screening and sorting results in this embodiment includes:

[0118] Calculate the affected shards after the offline servers in the scaled-down cluster and the corresponding number of instances of the affected shards;

[0119] According to the sorted servers, perform one-by-one compensation deployment for the affected shards.

[0120] Furthermore, the one-by-one compensation for the affected shards according to the sorted servers in this embodiment includes:

[0121] When compensating for the instances in the affected shards, select other servers in the same group in the computer room where the server corresponding to the affected shard is located to take over;

[0122] Judge whether there are servers in the same group that have not deployed instances of the affected shard;

[0123] If the judgment result is yes, select the server in the same group that has not deployed instances of the affected shard and is ranked the highest for compensation deployment. If the judgment result is no, select the server in the same group in the computer room where the server corresponding to the affected shard is located and is ranked the highest for compensation deployment.

[0124] Specifically, in this embodiment, for cluster expansion and contraction deployment services, the raft cluster instance dynamic balanced deployment strategy process is as follows:

[0125] Step 1: Calculate the shards affected after the server goes offline and the corresponding number of instances;

[0126] Step 2: Summarize the servers that are still in service in the cluster, group them by computer room, and sort them by the number of instances deployed in the current cluster. The fewer the number of instances, the higher the ranking.

[0127] Step 3: Compensate shard by shard according to the affected shards;

[0128] Step 4: Before each shard is compensated, it will be re-sorted within the group. The instance of the offline server will only be taken over by the server in the same computer room. If there is a server in the same computer room that has not deployed an instance in the shard, the server with the highest ranking will be selected from these servers. If there is no server in the same computer room that has not deployed an instance in the shard, the server with the highest ranking will be selected directly from the group.

[0129] Step 5: Add the compensated instance to the shard, and then take the instance on the offline server offline.

[0130] The method for dynamically balancing the deployment of multiple raft cluster instances in this embodiment also needs to sort the machine loads of each computer room when the cluster is expanded or reduced, and select from each computer room in turn to ensure that the loads within and between computer rooms are balanced.

[0131] The type of raft cluster instance deployment service issued by the deployment platform in this embodiment includes cluster offline service. Since the dynamic balanced deployment method of multiple raft cluster instances in this embodiment does not introduce the influence of changes in other cluster deployment conditions when the cluster is expanded or reduced, directly offline clusters do not affect the overall balance. Therefore, offline clusters can be directly offlined on the platform.

[0132] As an optional implementation of this embodiment, the method for dynamically balanced deployment of multiple raft cluster instances in this embodiment needs to add a distributed lock before the start of each stage to avoid uneven server allocation caused by multiple deployment tasks in parallel, so as to ensure that only one cluster is performing balanced deployment at a time; and release the distributed lock at the end of each stage.

[0133] In this embodiment, each server in each computer room has a dedicated agent process, and the agent process registers server information with the deployment platform. The server information includes the server IP and the computer room where the server is located; the deployment platform manages the agent process on each server and distributes deployment tasks to the agent process.

[0134] As an alternative implementation of this embodiment, without considering the safety of the production environment, it can be deployed in a manner without an agent process. The deployment platform directly executes the script remotely. In terms of the dynamic balancing algorithm, the current solution is the simplest solution, and the same goal can be achieved through redundant steps.

[0135] The method for dynamically balancing the deployment of multiple raft cluster instances in this embodiment is based on the raft protocol to provide data consistency storage services. In a multi-data center environment, the deployment is balanced, and the service stability between raft nodes is maximally guaranteed at the machine deployment level.

[0136] This embodiment also provides a device for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment, including a deployment platform. The deployment method executed by the deployment platform includes:

[0137] Issuing the deployment service of the raft cluster instance;

[0138] Determining the target data center according to the type of the deployment service of the raft cluster instance;

[0139] Statistically calculating the current load of each target data center, and screening and sorting the servers in each target data center according to the current load;

[0140] Circularly selecting servers among the target data centers according to the screening and sorting results to deploy raft cluster instances, and performing dynamic balancing deployment of raft cluster instances in a multi-data center environment.

[0141] After the deployment platform of the device for dynamically balancing the deployment of multiple raft cluster instances in this embodiment issues the deployment service of the raft cluster instance, regardless of the type of the deployment service of the raft cluster instance, it is necessary to sort the servers in the data center according to the current load, and then deploy the raft cluster instance according to the sorting result. The basic principle of server sorting in this embodiment is that the servers with smaller load are sorted more forward, so that the server resources can be better utilized and the load balance among the servers in the data center can be guaranteed. Circularly selecting servers among the target data centers according to the screening and sorting results to deploy raft cluster instances in this embodiment enables the instances of the raft cluster to be deployed on the servers in different data centers as much as possible, avoiding data loss of the raft cluster caused by the failure of a single or local data center, and ensuring the load balance among the data centers.

[0142] In this embodiment, the types of business for which the deployment platform issues raft cluster instance deployments include cluster creation, cluster expansion, and cluster contraction, covering the entire lifecycle of the raft cluster. Therefore, the multi-raft cluster instance dynamic load balancing deployment device in this embodiment ensures that the load of each server in multiple data centers is balanced during the lifecycle of the raft cluster.

[0143] Therefore, the multi-raft cluster instance dynamic load balancing deployment device in this embodiment enables dynamic load balancing deployment of the raft cluster in a multi-data center environment, ensuring that the multi-data centers can provide the maximum raft group fault tolerance.

[0144] The multi-raft cluster instance dynamic load balancing deployment device in the multi-data center environment described in this embodiment further includes an agent process module deployed on each server in each data center. The agent process module registers server information with the deployment platform, and the server information includes the server IP and the data center where the server is located. The deployment platform manages the agent process modules on each server and distributes deployment tasks to the agent process modules.

[0145] This embodiment also provides a storage medium storing computer-executable programs. When the computer-executable programs are executed, the multi-raft cluster instance dynamic load balancing deployment method is implemented.

[0146] The storage medium in this embodiment may include data signals propagated in a baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate, or transmit programs used by or in conjunction with an instruction execution system, device, or component. The program codes contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0147] This embodiment also provides an electronic device, including a processor and a memory. The memory is used to store computer-executable programs. When the computer programs are executed by the processor, the processor executes the multi-raft cluster instance dynamic load balancing deployment method.

[0148] The electronic device is presented in the form of a general computing device. The processor can be one or multiple and work collaboratively. The present invention does not exclude distributed processing, that is, the processors can be dispersed in different physical devices. The electronic device of the present invention is not limited to a single entity and can also be the sum of multiple physical devices.

[0149] The memory stores computer-executable programs, typically machine-readable code. The computer-readable program can be executed by the processor so that the electronic device can execute the method of the present invention or at least some of the steps in the method.

[0150] The memory includes volatile memory, such as random access storage units (RAM) and / or cache storage units, and may also be non-volatile memory, such as read-only storage units (ROM).

[0151] It should be understood that the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements, such as buttons, keyboards, etc. As long as the electronic device can execute the computer-readable program in the memory to implement the method of the present invention or at least some of the steps of the method, it can be considered as the electronic device covered by the present invention.

[0152] From the above description of the embodiments, those skilled in the art can easily understand that the present invention can be implemented by hardware capable of executing specific computer programs, such as the system of the present invention, and the electronic processing units, servers, clients, mobile phones, control units, processors, etc. included in the system. The present invention can also be implemented by computer software that executes the method of the present invention, such as control software executed by a microprocessor, an electronic control unit, a client, a server, etc. However, it should be noted that the computer software that executes the method of the present invention is not limited to being executed in one or specific hardware entities. It can also be implemented in a distributed manner by unspecified specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), or can be distributed and stored on a network, as long as it can enable the electronic device to execute the method according to the present invention.

[0153] The above embodiments are only used to illustrate the present invention and do not limit the technical solutions described in the present invention. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific embodiments. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and their improvements that do not depart from the spirit and scope of the invention are covered by the scope of the claims of the present invention.

Claims

1. Method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment, Characterized in that, Comprising: The deployment platform issues the raft cluster instance deployment service; The types of the raft cluster instance deployment service issued by the deployment platform include cluster creation, cluster expansion, and cluster contraction; Determine the target data center according to the type of the raft cluster instance deployment service; Statistically calculate the current load of the servers in each target data center, and screen and sort the servers in each target data center according to the current load; Select servers to deploy raft cluster instances cyclically among the target data centers according to the screening and sorting results, and perform dynamic balanced deployment of raft cluster instances in a multi-data center environment.

2. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment according to claim 1, Characterized in that, The type of the raft cluster instance deployment service issued by the deployment platform includes cluster creation service, and the target data center determined according to the cluster creation service is the data center where all current raft clusters are deployed; The statistically calculating the current load of the servers in each target data center, and screening and sorting the servers in each target data center according to the current load includes: Statistically calculate the number of instances already deployed by the servers in all current raft clusters; Group the servers according to the data center; Sort the servers in each group in descending order of the number of deployed instances; The selecting servers to deploy raft cluster instances cyclically among the target data centers according to the screening and sorting results, and performing dynamic balanced deployment of raft cluster instances in a multi-data center environment includes: According to the number of shards of the newly created cluster and the number of instances of each shard, select servers cyclically between groups and within groups until all the servers required by the cluster are allocated.

3. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment according to claim 2, Characterized in that, The selecting servers cyclically between groups and within groups according to the number of shards of the newly created cluster and the number of instances of each shard includes: Specify a preferred data center according to the cluster creation service, and set the group where the preferred data center is located as the preferred group; According to the number of shards of the newly created cluster and the number of instances of each shard, first select servers from the preferred group according to the within-group sorting; Starting from the preferred group, sequentially select the server with the least number of selections in each group and the server with the highest ranking, and select servers cyclically between groups and within groups.

4. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment according to claim 1, Characterized in that, The type of the raft cluster instance deployment service issued by the deployment platform includes cluster expansion service, and the target data center determined according to the cluster expansion service is the data center where the servers currently deployed by the expanded cluster are located; The statistically calculating the current load of the servers in each target data center, and screening and sorting the servers in each target data center according to the current load includes: Set all the servers in all target data centers as expansion servers, and group the expansion servers according to the data center; Sort the grouped expansion servers in descending order of the number of instances already deployed in all current clusters; Cyclically selecting servers among each target computer room according to the screening and sorting results to deploy raft cluster instances, and performing dynamic balancing deployment of raft cluster instances in a multi-computer room environment includes: Adding a new shard for an expanded cluster, and determining whether there is an expanded server in the group where the server deployed by the current shard is located that has not deployed the shard instance; If the judgment result is yes, select the expanded server that has not deployed the shard instance and is ranked the highest for deployment. If the judgment result is no, select the expanded server that is ranked the highest in the group where the server deployed by the current shard is located; Add new shards for each shard of the expanded cluster one by one until all shards of the expanded cluster have completed the expansion task.

5. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment according to claim 4, characterized in that, After each shard of the expanded cluster is allocated, it is necessary to re-sort the expanded servers in the group, and after the re-sorting is completed, the next shard of the expanded cluster is allocated.

6. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment according to claim 1, characterized in that, The types of raft cluster instance deployment services issued by the deployment platform include cluster scaling-down services, and the target computer room determined according to the cluster scaling-down services is the computer room where the servers that continue to serve for the scaled-down cluster are located; The method for statistically calculating the current load of the servers in each target computer room and screening and sorting the servers in each target computer room according to the current load includes: Grouping all the servers in all the target computer rooms according to the computer rooms; Sorting the grouped servers in descending order according to the number of instances already deployed in the current entire cluster; Cyclically selecting servers among each target computer room according to the screening and sorting results to deploy raft cluster instances, and performing dynamic balancing deployment of raft cluster instances in a multi-computer room environment includes: Calculating the affected shards after the offline servers of the scaled-down cluster and the number of instances corresponding to the affected shards; According to the sorted servers, perform one-by-one compensation deployment for the affected shards.

7. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment according to claim 6, characterized in that, The method for performing one-by-one compensation for the affected shards according to the sorted servers includes: When compensating for the instances in the affected shards, select other servers in the same group in the computer room where the server corresponding to the affected shard is located to take over; Determine whether there are servers in the same group that have not deployed the instance of the affected shard; If the judgment result is yes, select the server in the same group that has not deployed the instance of the affected shard and is ranked the highest for compensation deployment. If the judgment result is no, select the server in the same group that is ranked the highest in the group where the server deployed by the affected shard is located for compensation deployment.

8. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-computer room environment according to claim 6, characterized in that, It further includes: Adding the instances deployed for compensation to the shards of the cluster, and then taking offline the instances on the offline servers in the cluster.

9. The method for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment according to claim 1, characterized in that, it includes: Each server in each data center has an agent process. The agent process registers server information with the deployment platform. The server information includes the server IP and the data center where the server is located. The deployment platform manages the agent processes on each server and distributes deployment tasks to the agent processes.

10. A device for dynamically balancing the deployment of multiple raft cluster instances in a multi-data center environment, characterized in that, it includes a deployment platform. The deployment method executed by the deployment platform includes: Issuing the raft cluster instance deployment service; The types of raft cluster instance deployment services issued by the deployment platform include cluster creation, cluster expansion, and cluster contraction; Determining the target data center according to the type of the raft cluster instance deployment service; Statistically calculating the current load of each target data center, and screening and sorting the servers in each target data center according to the current load; Selecting servers to deploy raft cluster instances cyclically among the target data centers according to the screening and sorting results, and performing dynamic balance deployment of raft cluster instances in a multi-data center environment.

Images