System and method for accelerating copy balancing of distributed database

Abstract

The present invention discloses a system and method for accelerating replica balancing of a distributed database, belonging to the field of computer communication technology. The technical problem to be solved by the present invention is how to accelerate the replica balancing of each node in the distributed database. The technical solution adopted is as follows: the system comprises a replica balancing unit, a Gossip unit, a node status statistics unit and a load score statistics unit; wherein the replica balancing unit is used to detect a node load indicator and determine whether a balance condition is met, and if the replica balance condition is met, replica migration is triggered; the Gossip unit is used to synchronize data between nodes and send and receive node status statistics according to a topology mode of a Gossip protocol; the node status statistics unit is used to count the node load status and send the status statistics to other nodes via the Gossip unit; and the load score statistics unit is used to calculate the load score based on the load status statistics of each node reported by the Gossip unit, and summarize the load score into a cluster node load score record table.

Description

Technical Field

[0001] The present invention relates to the technical field of computer communications, and in particular to a system and method for accelerating copy balancing of a distributed database. Background Art

[0002] The underlying layer of a distributed database typically organizes data into an ordered key-value pair structure and segments the key space, each segment being called a range. To ensure disaster recovery and high availability, a fixed number of replicas are typically configured for each range, and then consistent replication is performed using the Raft protocol.

[0003] As attached Figure 1 As shown in the figure, a Raft group has three states: Leader, Follower, and Candidate. The Leader is the processor of all requests; the Follower passively accepts requests from the Leader and writes them to the local log file; the Candidate is a candidate and can be transformed into the Leader under certain conditions.

[0004] In a distributed database, multiple replicas of the same Raft group are typically distributed across different nodes. Typically, a Raft group has three replicas, each of which can only serve as a leader or a follower. The Raft protocol requires that both reads and writes be performed through the leader. The leader is responsible for communicating with the other two followers; followers do not communicate with each other. The leader ensures ACID semantics for replica operations through log replication and periodically sends heartbeats to followers. If a follower fails to receive a heartbeat, it will wait for a randomly selected timeout period. After the timeout period expires, it becomes a candidate and a new leader election is held.

[0005] A Range covers a range of keys and is typically of a fixed size. As the size of the Range increases and exceeds the fixed size, it must be split to create a new Range. The newly created Range replicas form a new Raft replica group.

[0006] The purpose of replica balancing is to balance the storage, computing power, and network pressure of each node. If data is too concentrated in certain specific tables, it is easy to cause multiple range splits on certain nodes. The number of replicas of these nodes grows too quickly, resulting in an uneven distribution of replicas between nodes. This may eventually cause individual nodes to reach their load limit prematurely or even crash.

[0007] The typical replica balancing approach involves regularly checking node load indicators and performing replica migration if the indicator exceeds a predetermined threshold. However, to reduce invalid checks, the regular check intervals are long, resulting in poor real-time performance. Furthermore, there's no quick way to find the optimal node for replica migration.

[0008] When continuously writing large amounts of data to a distributed database, replicas are prone to becoming concentrated on individual nodes, and this distribution cannot be uniformly distributed in a short period of time. Distributed database cluster processing logic is complex, and data distribution cannot be predicted in advance. Importing large amounts of data in a short period of time can easily lead to uneven replica distribution, which cannot be evenly distributed in a short period of time, affecting cluster stability.

[0009] Typical replica balancing practices involve regularly checking metrics such as the number of replicas, memory and disk usage, and QPS. Replica migration is initiated if any metric exceeds a predetermined threshold. However, when replicas are unevenly distributed, nodes with fewer replicas often remain idle because they do not meet the requirements for replica migration. However, to reduce ineffective checks, the interval between regular checks should be kept small, preventing replica balancing from being achieved quickly.

[0010] Therefore, how to accelerate the balance of replicas of each node in a distributed database is a technical problem that needs to be solved urgently. Summary of the Invention

[0011] The technical task of the present invention is to provide a system and method for accelerating the balance of replicas of distributed databases, so as to solve the problem of how to accelerate the balance of replicas of each node in the distributed database.

[0012] The technical task of the present invention is achieved in the following manner: a system for accelerating replica balancing of a distributed database, the system comprising:

[0013] The replica balancing unit is used to detect the node load index and determine whether the balance condition is met. If the replica balance condition is met, the replica migration is triggered;

[0014] The Gossip unit is used to synchronize data between nodes and send and receive node status statistics according to the topology mode of the Gossip protocol. The Gossip unit uses the Gossip protocol to transmit the load status statistics of each node in the cluster.

[0015] Node status statistics unit, used to count node load status and send status statistics to other nodes through the Gossip unit;

[0016] The load score statistics unit is used to calculate the load score based on the load status statistics of each node reported by the Gossip unit and summarize it into the cluster node load score record table.

[0017] Preferably, the node status statistics unit counts the number of replicas, the number of leaders, the memory and disk usage, and the QPS index of the node after each regular time interval, and transmits the two information of node status index changes and update time to other nodes through the Gossip unit; wherein, the node status index is used for the subsequent load score calculation and is the basis for the target selection when the replica balancing unit performs replica migration.

[0018] Preferably, the load score statistics unit calculates the load score using the number of node copies, the number of leaders, the memory and disk usage, and the QPS indicators reported by the Gossip unit. The load score is obtained by calculating the average value of each indicator and then summing the average value ratios of each indicator.

[0019] More preferably, the load score statistics unit calculates the load score as follows:

[0020] (1) Calculate the average value: Calculate the average number of cluster node replicas Average number of leaders Average memory and disk usage and and the node average of QPS

[0021] (2) Calculate the node load score: Calculate the current number of replicas r i , Leader number l i , memory and disk usage m i and d i and the node value q of QPS i The node load score is obtained by summing the percentage of the corresponding node average value. The formula is as follows:

[0022]

[0023] (3) Summarize the load scores into the cluster node load score table.

[0024] More preferably, the replica balancing unit performs the balance check in the following cases:

[0025] Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica:

[0026] If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table;

[0027] Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica:

[0028] If it exists, execute the replica balancing process in case ①;

[0029] Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold:

[0030] If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1;

[0031] When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

[0032] More preferably, the replica migration process in the replica balancing unit is specifically as follows:

[0033] (1) Add replicas to new nodes and remove existing replicas; adding replicas will prioritize nodes with a node load score lower than a set node load score threshold (the threshold is set based on cluster size and application requirements or the node with the lowest node load score is directly selected), while removing replicas will prioritize nodes with a node load score higher than a set node load score threshold (the threshold is set based on cluster size and application requirements or the node with the highest node load score is directly selected);

[0034] (2) Determine whether the current leader replica and the replica to be removed belong to the same Raft group:

[0035] ①. If yes, directly execute the copy change process and proceed to step (3) next;

[0036] (3) If the removed replica is the leader replica, the leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica;

[0037] (4) The new leader replica is calculated based on the node load score, and the node where the old leader replica is located is re-added to the priority removal node table. When the replica balancing unit performs replica migration, the old leader replica will be removed first. Among them, the leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing.

[0038] (5) After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration, that is, adding new members and removing old members, and ultimately maintaining the basic number of replicas unchanged.

[0039] More optimally, remove the replica node as follows:

[0040] (1) Arrange the replicas from the cluster node load score record table and remove the node table first, select N nodes whose load scores are higher than the set node load score threshold, and N is 1 / 3 of the total number of nodes;

[0041] (2) Determine whether the leader copy of the selected node exists in the same group of copies among N nodes:

[0042] If it exists, the replica migration process is initiated.

[0043] Preferably, add a replica node as follows:

[0044] (1) Arrange the replicas from the cluster node load score record table and add the node table first. Select N nodes whose load scores are lower than or higher than the set node load score threshold. N is 1 / 3 of the total number of nodes.

[0045] (2) Randomly select one of the N nodes to add a replica. The purpose of randomness is to avoid the situation where the balance target is stacked on some nodes when the selection results of multiple nodes in the cluster are the same.

[0046] A method for accelerating replica balancing in a distributed database comprises: using a load score statistics unit to calculate node load scores, and storing the calculated load score results for each node in a cluster node load score record table; when the replica balancing unit performs replica migration, the load score record table is used to quickly select nodes where replicas are prioritized for addition and removal, thereby accelerating the migration of replicas from high-load nodes to low-load nodes; the details are as follows:

[0047] S1, calculate node load score;

[0048] S2. Balance check;

[0049] S3, replica migration.

[0050] Preferably, the calculation node load score in step S1 is specifically as follows:

[0051] The load score is calculated using the number of node replicas, number of leaders, memory and disk usage, and QPS reported by the Gossip unit. The load score is obtained by calculating the average of each indicator and then summing the average percentage of each indicator. The formula is as follows:

[0052]

[0053] in, Indicates the average number of cluster node replicas; Indicates the average number of Leader replicas; Indicates the average memory usage; Indicates the average disk usage; Indicates the node average value of QPS; r i Indicates the number of previous copies; l i Represents the number of Leader copies; m i Indicates the memory usage value; d i Indicates the disk usage value; q i Node value indicating QPS;

[0054] The balance check in step S2 is as follows:

[0055] Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica:

[0056] If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table;

[0057] Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica:

[0058] If it exists, execute the replica balancing process in case ①;

[0059] Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold:

[0060] If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1;

[0061] When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

[0062] The replica migration in step S3 is as follows:

[0063] S301. Add a replica to a new node and remove the original replica. Adding a replica prioritizes nodes whose node load score is lower than a set node load score threshold (the threshold is set based on cluster size and application requirements or the node with the lowest node load score is directly selected). Removing a replica prioritizes nodes whose node load score is higher than a set node load score threshold (the threshold is set based on cluster size and application requirements or the node with the highest node load score is directly selected).

[0064] S302: Determine whether the current leader replica and the replica to be removed belong to the same Raft group:

[0065] ① If yes, directly execute the copy change process and then go to step S303;

[0066] S303: The removed replica is the leader replica. The leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica.

[0067] S304: The new leader replica calculates the node load score and re-adds the node where the old leader replica is located to the priority removal node table. When the replica balancing unit performs replica migration, the old leader replica will be removed first. The leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing.

[0068] S305. After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration, that is, adding new members and removing old members, and ultimately maintaining the basic number of replicas unchanged.

[0069] Gossip is a decentralized distributed protocol characterized by peer-to-peer communication, exponential message propagation, and excellent scalability. It is commonly used to synchronize data between replica nodes in distributed database systems.

[0070] The system and method for accelerating replica balancing of a distributed database of the present invention have the following advantages:

[0071] (1) The present invention uses a load score statistics unit to calculate the node load score and saves the calculated load score results of each node in a cluster node load score record table. When performing replica migration, the replica balancing unit uses the load score record table to quickly select the nodes that are prioritized for adding replicas and the nodes that are prioritized for removing replicas, thereby accelerating the migration of replicas from high-load nodes to low-load nodes.

[0072] (2) The present invention can accelerate the uniform distribution of replicas, solving the problem of large-scale accumulation of replicas on individual nodes when a distributed database imports a large amount of data and being unable to balance them in a short period of time. It can accelerate the balancing of replicas on each node of the distributed database;

[0073] (3) The present invention transmits node load status statistics through the Gossip unit to accelerate cluster node status synchronization;

[0074] (4) Real-time detection of node load status through the replica balancing unit, triggering replica migration as soon as the balance condition is met;

[0075] (5) The present invention realizes the migration of all role replicas through the conversion of the Leader replica role. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The present invention will be further described below with reference to the accompanying drawings.

[0077] Attachment Figure 1 This is the Raft state transition diagram in the background technology;

[0078] Attachment Figure 2 A block diagram of a system for accelerating replica balancing in a distributed database;

[0079] Attachment Figure 3 This is a schematic diagram of replica load score statistics;

[0080] Attachment Figure 4 Schematic diagram of target selection for replica balancing cells. DETAILED DESCRIPTION

[0081] The system and method for accelerating replica balancing of a distributed database according to the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] Example 1:

[0083] As attached Figure 2 As shown, the distributed database accelerated replica balancing system of the present invention includes:

[0084] The replica balancing unit is used to detect the node load index and determine whether the balance condition is met. If the replica balance condition is met, the replica migration is triggered;

[0085] The Gossip unit is used to synchronize data between nodes and send and receive node status statistics according to the topology mode of the Gossip protocol. The Gossip unit uses the Gossip protocol to transmit the load status statistics of each node in the cluster.

[0086] Node status statistics unit, used to count node load status and send status statistics to other nodes through the Gossip unit;

[0087] The load score statistics unit is used to calculate the load score based on the load status statistics of each node reported by the Gossip unit and summarize it into the cluster node load score record table.

[0088] The node status statistics unit in this embodiment counts the number of replicas, the number of leaders, the memory and disk usage, and the QPS index of the node after each regular time interval, and transmits the two information of node status index changes and update time to other nodes through the Gossip unit; among them, the node status index is used for the subsequent load score calculation and is the basis for the target selection when the replica balancing unit performs replica migration.

[0089] The load score statistics unit in this embodiment calculates the load score using the number of node replicas, the number of leaders, the memory and disk usage, and the QPS indicators reported by the Gossip unit. The load score is obtained by calculating the average of each indicator and then summing the average ratios of each indicator.

[0090] As attached Figure 3 As shown, the load score statistics unit in this embodiment calculates the load score as follows:

[0091] (1) Calculate the average value: Calculate the average number of cluster node replicas Average number of leaders Average memory and disk usage and and the node average of QPS

[0092] (2) Calculate the node load score: Calculate the current number of replicas r i , Leader number l i , memory and disk usage m i and d i and the node value q of QPS i The node load score is obtained by summing the percentage of the corresponding node average value. The formula is as follows:

[0093]

[0094] (3) Summarize the load scores into the cluster node load score table.

[0095] The replica balancing unit in this embodiment performs balance checks in the following situations:

[0096] Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica:

[0097] If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table;

[0098] Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica:

[0099] If it exists, execute the replica balancing process in case ①;

[0100] Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold:

[0101] If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1;

[0102] When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

[0103] As attached Figure 4 As shown, the replica migration process in the replica balancing unit in this embodiment is specifically as follows:

[0104] (1) Add replicas to new nodes and remove existing replicas; adding replicas is prioritized on the node with the lowest node load score, while removing replicas is prioritized on the node with the highest node load score;

[0105] (2) Determine whether the current leader replica and the replica to be removed belong to the same Raft group:

[0106] ①. If yes, directly execute the copy change process and proceed to step (3) next;

[0107] (3) If the removed replica is the leader replica, the leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica;

[0108] (4) The new leader replica is calculated based on the node load score, and the node where the old leader replica is located is re-added to the priority removal node table. When the replica balancing unit performs replica migration, the old leader replica will be removed first. Among them, the leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing.

[0109] (5) After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration, that is, adding new members and removing old members, and ultimately maintaining the basic number of replicas unchanged.

[0110] The details of removing the replica node in this embodiment are as follows:

[0111] (1) Arrange the replicas from the cluster node load score record table and remove the node table first, select N nodes whose load scores are higher than the set node load score threshold, and N is 1 / 3 of the total number of nodes;

[0112] (2) Determine whether the leader copy of the selected node exists in the same group of copies among N nodes:

[0113] If it exists, the replica migration process is initiated.

[0114] The details of adding a replica node in this embodiment are as follows:

[0115] (1) Arrange the replicas from the cluster node load score record table and add the node table first. Select N nodes whose load scores are lower than or higher than the set node load score threshold. N is 1 / 3 of the total number of nodes.

[0116] (2) Randomly select one of the N nodes to add a replica. The purpose of randomness is to avoid the situation where the balance target is stacked on some nodes when the selection results of multiple nodes in the cluster are the same.

[0117] Example 2:

[0118] The present invention provides a method for accelerating replica balancing in a distributed database. The method comprises: using a load score statistics unit to calculate node load scores, and storing the calculated load score results of each node in a cluster node load score record table; when the replica balancing unit performs replica migration, the load score record table is used to quickly select replica priority adding nodes and replica priority removing nodes, thereby accelerating the migration of replicas from high-load nodes to low-load nodes. The method is specifically as follows:

[0119] S1, calculate node load score;

[0120] S2. Balance check;

[0121] S3, replica migration.

[0122] The calculation node load score in step S1 of this embodiment is specifically as follows:

[0123] The load score is calculated using the number of node replicas, number of leaders, memory and disk usage, and QPS reported by the Gossip unit. The load score is obtained by calculating the average of each indicator and then summing the average percentage of each indicator. The formula is as follows:

[0124]

[0125] in, Indicates the average number of cluster node replicas; Indicates the average number of Leader replicas; Indicates the average memory usage; Indicates the average disk usage; Indicates the node average value of QPS; r i Indicates the number of previous copies; l i Represents the number of Leader copies; m i Indicates the memory usage value; d i Indicates the disk usage value; q i Node value indicating QPS;

[0126] The balance check in step S2 of this embodiment is specifically as follows:

[0127] Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica:

[0128] If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table;

[0129] Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica:

[0130] If it exists, execute the replica balancing process in case ①;

[0131] Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold:

[0132] If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1;

[0133] When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

[0134] The replica migration in step S3 of this embodiment is specifically as follows:

[0135] S301. Add a replica to a new node and remove the original replica. Adding a replica is prioritized on the node with the lowest node load score, while removing a replica is prioritized on the node with the highest node load score.

[0136] S302: Determine whether the current leader replica and the replica to be removed belong to the same Raft group:

[0137] ① If yes, directly execute the copy change process and then go to step S303;

[0138] S303: The removed replica is the leader replica. The leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica.

[0139] S304: The new leader replica calculates the node load score and re-adds the node where the old leader replica is located to the priority removal node table. When the replica balancing unit performs replica migration, the old leader replica will be removed first. The leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing.

[0140] S305. After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration, that is, adding new members and removing old members, and ultimately maintaining the basic number of replicas unchanged.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed database accelerated replica balancing system, characterized in that: The system includes, The replica balancing unit is used to detect the node load index and determine whether the balance condition is met. If the replica balance condition is met, the replica migration is triggered; The Gossip unit is used to synchronize data between nodes and send and receive node status statistics according to the topology mode of the Gossip protocol. The Gossip unit uses the Gossip protocol to transmit the load status statistics of each node in the cluster. Node status statistics unit, used to count node load status and send status statistics to other nodes through the Gossip unit; The load score statistics unit is used to calculate the load score based on the load status statistics of each node reported by the Gossip unit and summarize it into the cluster node load score record table; The replica migration process in the replica balancing unit is as follows: (1) Add replicas to new nodes and remove existing replicas; adding replicas will prioritize nodes with a node load score lower than the set node load score threshold, while removing replicas will prioritize nodes with a node load score higher than the set node load score threshold; (2) Determine whether the current leader replica and the replica to be removed belong to the same Raft group: ①. If yes, directly execute the copy change process and proceed to step (3) next; (3) If the removed replica is the leader replica, the leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica; (4) The new leader replica is calculated based on the node load score, and the node where the old leader replica is located is re-added to the priority node removal table. When the replica balancing unit performs replica migration, the old leader replica will be removed first. Among them, the leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing. (5) After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration.

2. The distributed database accelerated replica balancing system according to claim 1, characterized in that: The node status statistics unit counts the number of replicas, the number of leaders, the memory and disk usage, and the QPS index of the node after each regular time interval, and transmits the node status index change and update time to other nodes through the Gossip unit; among them, the node status index is used for the subsequent load score calculation and is the basis for the target selection when the replica balancing unit performs replica migration.

3. The distributed database accelerated replica balancing system according to claim 1, characterized in that: The load score statistics unit calculates the load score using the number of node copies, number of leaders, memory and disk usage, and QPS indicators reported by the Gossip unit. The load score is obtained by calculating the average value of each indicator and then summing the proportion of the average values ​​of each indicator.

4. The system for accelerating replica balancing of a distributed database according to claim 1 or 3, characterized in that: The load score statistics unit calculates the load score specifically as follows: (1) Calculate the average value: Calculate the average number of cluster node replicas , average number of leaders , memory and disk usage averages and and the node average of QPS ; (2) Calculate the node load score: calculate the current number of replicas , Number of Leaders , memory and disk usage and and QPS node value The node load score is obtained by summing the percentage of the corresponding node average value. The formula is as follows: ； (3) Summarize the load scores into the cluster node load score table.

5. The distributed database accelerated replica balancing system according to claim 4, characterized in that: The replica balancing unit performs balance checks in the following situations: Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica: If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table; Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica: If it exists, execute the replica balancing process in case ①; Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold: If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1; When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

6. The distributed database accelerated replica balancing system according to claim 1, characterized in that: To remove a replica node: (1) Arrange the copies from the cluster node load score record table and remove the node table first, select N nodes whose load scores are higher than the set node load score threshold, and N is 1 / 3 of the total number of nodes; (2) Determine whether the Leader copy of the selected node exists in the same group of copies among N nodes: If it exists, the replica migration process is initiated.

7. The system for accelerating replica balancing of a distributed database according to claim 1, characterized in that: Adding a replica node is as follows: (1) Arrange the replicas from the cluster node load score record table and add the node table first. Select N nodes whose load scores are lower than or higher than the set node load score threshold. N is 1 / 3 of the total number of nodes. (2) Randomly select one of the N nodes to add a replica.

8. A method for accelerating replica balancing in a distributed database, characterized in that: The method is to calculate the node load score by using a load score statistics unit, and save the calculation results of the load score of each node in a cluster node load score record table; When the replica balancing unit performs replica migration, it quickly selects the nodes to be added and removed first based on the load score record table, thereby accelerating the migration of replicas from high-load nodes to low-load nodes. The details are as follows: S1, calculate node load score; S2. Balance check; S3, replica migration; details are as follows: S301. Add a replica to a new node and remove the original replica. Adding a replica is prioritized on nodes whose node load score is lower than a set node load score threshold, while removing a replica is prioritized on nodes whose node load score is higher than the set node load score threshold. S302: Determine whether the current leader replica and the replica to be removed belong to the same Raft group: ① If yes, directly execute the copy change process and then go to step S303; S303: The removed replica is the leader replica. The leader role needs to be transferred, and subsequent replica migrations are initiated by the new leader replica. S304: The new leader replica calculates the node load score and re-adds the node where the old leader replica is located to the priority node removal table. The replica balancing unit will prioritize removing the old leader replica when performing replica migration. The leader replica is the coordinator of replica migration, and non-leader replicas do not actively handle replica balancing. S305. After the replica balancing unit completes the selection of adding replica nodes and removing replica nodes, it submits the replica nodes to be added and the corresponding nodes and the replica nodes to be deleted to the Raft layer, and uses Raft atomic operations to modify the replica configuration.

9. The method for accelerating replica balancing of a distributed database according to claim 8, characterized in that ,The calculation node load scores in step S1 are as follows: The load score is calculated using the number of node replicas, number of leaders, memory and disk usage, and QPS indicators reported by the Gossip unit. The load score is obtained by calculating the average of each indicator and then summing the average percentage of each indicator. The formula is as follows: ; in, Indicates the average number of cluster node replicas; Indicates the average number of Leader replicas; Indicates the average memory usage; Indicates the average disk usage; Indicates the node average QPS; Indicates the number of previous copies; Indicates the number of Leader copies; Indicates the memory usage value; Indicates the disk usage value; Node value representing QPS; The balance check in step S2 is as follows: Case 1: When a replica is added, check in the replica priority removal node table whether there is a node with a replica in the same group as the current replica: If it exists, perform replica balancing, select nodes from the replica priority node table to add, and remove the replicas of the corresponding nodes from the priority node table; Case 2: When a normal replica switches to the Leader, the replica priority removal node table is checked to see if there is a replica in the same group as the current replica: If it exists, execute the replica balancing process in case ①; Case 3: Regularly check whether the number of replicas, number of leaders, memory and disk usage, and QPS have reached the threshold: If two indicators exceed the threshold, replica balancing is performed, and the replica with a QPS higher than the threshold is selected to execute the replica balancing process in case 1; When the replica balancing unit performs a balance check, if any of the following conditions (1), (2), or (3) is met, the conditions for replica migration are met.

Images