Load balancing method and device of distributed database, electronic equipment and medium

Abstract

The present disclosure provides a load balancing method and device for a distributed database, electronic equipment and medium, relates to the technical field of big data and cloud computing, and particularly relates to the technical field of data storage. The specific implementation scheme is as follows: a heartbeat message sent by a first storage module is received. Then, for each shard replica in the first storage module, whether the shard replica belongs to a user and whether the shard replica is a primary shard replica are determined according to attribute information of the shard replica. Then, for each determined user, if the number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold, a first notification message is sent to the first storage module, wherein the first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to a second storage module, and the number of primary shard replicas of the user stored in the second storage module is less than that in the first storage module. Thus, the access performance of large-table data is ensured.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of big data and cloud computing, and particularly relates to the technical field of data storage. BACKGROUND

[0002] In a distributed database, a data table is generally split into multiple parts for storage, each part is referred to as a shard, each shard includes a part of row data of the data table, and multiple shard replicas exist for each shard to ensure data reliability. SUMMARY

[0003] The present disclosure provides a load balancing method and device for a distributed database, an electronic device, and a medium.

[0004] In a first aspect, the present disclosure provides a load balancing method for a distributed database, comprising:

[0005] receiving a heartbeat message sent by a first storage module in a storage cluster, wherein the heartbeat message comprises attribute information of each shard replica stored in the first storage module;

[0006] for each shard replica in the first storage module, determining a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica;

[0007] for each determined user, if a number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold, sending a first notification message to the first storage module; wherein the first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to a second storage module; and a number of primary shard replicas of the user stored in the second storage module is less than a number of primary shard replicas of the user stored in the first storage module.

[0008] In a second aspect, the present disclosure provides a load balancing device for a distributed database, comprising:

[0009] a receiving module configured to receive a heartbeat message sent by a first storage module in a storage cluster, wherein the heartbeat message comprises attribute information of each shard replica stored in the first storage module;

[0010] a determining module configured to, for each shard replica in the first storage module, determine a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica;

[0011] The sending module is configured to, for each user determined by the determining module, if the number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold, send a first notification message to the first storage module; wherein the first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to a second storage module; the number of primary shard replicas of the user stored in the second storage module is less than the number of primary shard replicas of the user stored in the first storage module.

[0012] In a third aspect, an electronic device is provided, including:

[0013] at least one processor; and

[0014] a memory connected with the at least one processor; wherein

[0015] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0016] In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect.

[0017] In a fifth aspect, a computer program product is provided, including a computer program, and the computer program, when executed by a processor, implements the method of the first aspect.

[0018] It should be understood that the contents described in this part are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0020] Figure 1 is an exemplary schematic diagram of a distributed database structure provided by an embodiment of the present disclosure;

[0021] Figure 2 is a flowchart of a load balancing method of a distributed database provided by an embodiment of the present disclosure;

[0022] Figure 3 is a flowchart of a method for determining the identity of a storage module corresponding to a primary shard replica provided by an embodiment of the present disclosure;

[0023] Figure 4This is a flowchart of a method for determining the identifier of a storage module corresponding to a fragmented copy, provided in an embodiment of this disclosure;

[0024] Figure 5 This is a flowchart of a method for determining the average value of shard replicas and the average value of primary shard replicas according to an embodiment of this disclosure;

[0025] Figure 6 This is an exemplary schematic diagram of a load balancing process for a distributed database provided in an embodiment of this disclosure;

[0026] Figure 7 This is a schematic diagram of the structure of a load balancing device for a distributed database provided in an embodiment of this disclosure;

[0027] Figure 8 This is a block diagram of an electronic device used to implement the load balancing method for a distributed database according to embodiments of the present disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] In distributed databases, sharded replicas ensure consistency among all replicas of the same shard through distributed consensus protocols, such as the Raft consensus protocol. Based on the requirements of distributed consensus protocols, each shard within the same shard has one primary replica and one or more secondary replicas. The primary replica is called the leader, and the secondary replicas are called followers. Reading and writing data within a shard takes place on the primary replica. When the amount of data in a shard exceeds a threshold, a shard split is triggered, dividing the shard into two new shards, each corresponding to a portion of the original shard's rows. For example, if a shard contains more than 2 million rows, or its size exceeds 500 megabytes (MB), it is split into two sub-shards of equal size.

[0030] like Figure 1 As shown, the Spanner-like distributed database includes three modules: a database (db) module 101, a metadata module 102, and a storage module 103. Spanner is a scalable, multi-version, globally distributed, synchronously replicated database.

[0031] The db module 101 is configured to read data from or write data to the store module 103.

[0032] The meta module 102 is configured to manage metadata of the shard replica and schedule a storage location of the shard replica.

[0033] The store module 103 is configured to store the shard replica of the data table.

[0034] It should be noted that, Figure 1 The number of modules in the distributed database is not limited to this.

[0035] If multiple rows of data in the data table belong to the same user, the user is referred to as a big user of the table. Because the big user of the table has more rows of data, the data of the big user of the table is distributed on multiple shards. If the primary shard replicas of the big user of the table are concentrated in a small number of store modules, for example, concentrated in one or two store modules, the access load of the one or two store modules is increased, the response is slow, and the time consumed for responding to a user request is increased or even timed out. The access performance of the data of the big user of the table is difficult to guarantee, and the store resources of the entire store cluster are not fully utilized.

[0036] The traditional load balancing method for the distributed database includes the following three methods: shard load balancing, raft leader load balancing, and access hotspot load balancing.

[0037] In the shard load balancing method, the number of shard replicas stored in each store module is balanced, so that the amount of data carried by each store module is more balanced.

[0038] In the raft leader load balancing method, the leader data stored in each store module is balanced, so that the leaders are distributed among the store modules.

[0039] In the access hotspot load balancing method, the store module reports information about the current access load, such as read-write speed, to the meta module. Then, the meta detects the access hotspot leader according to the information about the current access load of each store module, and distributes the access hotspot leader among the store modules.

[0040] However, none of the three methods achieves load balancing from the user dimension, and cannot guarantee the access performance of the data of the big user of the table.

[0041] In order to ensure the access performance of data of each user, the embodiment of the present disclosure provides a load balancing method of a distributed database, which is applied to a meta module in Figure 1 . As shown in Figure 2 , the method comprises the following steps:

[0042] S201, receiving a heartbeat message sent by a first storage module in a storage cluster.

[0043] The first storage module is any storage module in the storage cluster. For the convenience of description, the storage module sending the heartbeat message is referred to as the first storage module. For example, the first storage module is any one of the three store modules shown in Figure 1 , and the storage cluster is a cluster composed of the three store modules in Figure 1 .

[0044] The heartbeat message comprises attribute information of each shard replica stored in the first storage module. The attribute information can indicate the user to which the data in the shard replica belongs, and whether the shard replica is a primary shard replica. In addition, the attribute information can also indicate other aspects of information, for example, the attribute information can also indicate the size of the shard replica, which is not limited in the embodiment of the present disclosure.

[0045] S202, for each shard replica in the first storage module, determining the user to which the shard replica belongs and whether the shard replica is a primary shard replica according to the attribute information of the shard replica.

[0046] In an implementation manner, the meta module can determine the user to which each shard replica belongs and whether the shard replica is a primary shard replica in sequence according to the arrangement order of each shard replica in the first storage module.

[0047] S203, for each determined user, if the number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold, sending a first notification message to the first storage module.

[0048] The first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to a second storage module, and the number of primary shard replicas of the user stored in the second storage module is less than the number of primary shard replicas of the user stored in the first storage module.

[0049] In an implementation manner, for each user determined in S202, the number of primary shard replicas of the user stored in the first storage module is counted, and it is determined whether the number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold. If yes, the first notification message is sent to the first storage module. If no, the adjustment of the storage location of the primary shard replicas of the user stored in the first storage module is not performed.

[0050] Optionally, the number of master shard copies transferred by the first storage module to other storage modules can be a fixed value, for example, the fixed value is 1. Alternatively, the number of master shard copies transferred by the first storage module to other storage modules can be a value determined by the meta module according to the number of master shard copies of the user stored in the first storage module. For example, if the ratio of the number of master shard copies of the user stored in the first storage module to the first preset threshold exceeds a preset ratio, it is determined that the number of master shard copies to be transferred is 2; otherwise, it is determined that the number of master shard copies to be transferred is 1. The specific determination method can also refer to the description in the following.

[0051] After the first storage module receives the first notification message, for each master shard copy that needs to be transferred, if the second storage module stores a slave shard copy of the master shard copy, the first storage module can change the master shard copy stored by itself into a slave shard copy, and notify the second storage module to change the slave shard copy of the master shard copy stored by itself into a master shard copy, thereby realizing the transfer of the storage location of the master shard copy.

[0052] Alternatively, if the second storage module does not store a slave shard copy of the master shard copy, the first storage module can send the master shard copy to the second storage module, so as to store the master shard copy in the second storage module.

[0053] In the embodiments of the present disclosure, the meta module can determine the user to which each shard copy stored by the storage module belongs, and whether each shard copy is a master shard copy, through the heartbeat message reported by the storage module. Then, when the storage module stores more master shard copies of the same user, part of the master shard copies are transferred to the storage module that stores fewer shard copies of the user, so as to realize the load balancing of the master shard copies in the user dimension, that is, to realize the dispersed distribution of the master shard copies of the same user in each storage module. Thus, the access performance of the data of each user is ensured.

[0054] In an embodiment of the present disclosure, the attribute information of each shard copy in the heartbeat message includes the primary key prefix and the identity information of the shard copy. The primary key prefix of the shard copy includes a user identifier, which indicates the user to which the row data included in the shard copy belongs. The identity information is used to indicate whether the identity of the shard copy is a master shard copy or a slave shard copy, that is, the identity information indicates whether the shard copy is a leader or a follower of the shard.

[0055] The embodiment of the present disclosure sets a key for the data table when splitting the data table, and splits the data table according to the key to obtain multiple shards. For example, the key is of an int type, and the value of the key in the data table is in the range of [0, 200]. When splitting the data table, the row data with the key in the range of [0, 100] is taken as shard 1, and the row data with the key in the range of [101, 200] is taken as shard 2.

[0056] In the embodiment of the present disclosure, the key of the data table is set by a person in advance, and the key includes a user identifier field. Optionally, the key in the embodiment of the present disclosure can also include other fields.

[0057] For example, the fields in a shopping information table include a user identifier (userid), an order identifier (orderid), a product identifier (productid), a cost (cost), a discount (discount), and a purchasing time (purchasing_time).

[0058] When the key only includes the user identifier field, the key of the shopping information table is userid. When the key includes the user identifier field and other fields, the key of the shopping information table can be userid+orderid.

[0059] Setting multiple keys for the data table can make the splitting granularity of the data table finer. For example, the row data with userid 1 and orderid in the range of [0, 5] is taken as shard 1, and the row data with userid 1 and orderid in the range of [6, 10] is taken as shard 2.

[0060] As can be seen, each shard replica corresponds to a range of keys of the data table, and the key prefix in the embodiment of the present disclosure can include the maximum value, the minimum value, or the intermediate value of the key range of the row data stored by the shard replica in the data table. For example, the key of the data table is userid+orderid, the userid of the row data stored by shard 1 is 1 and the orderid is in the range of [0, 5], and the key prefix of shard 1 is userid: 1, orderid: 0. Optionally, when the key prefix is set to include the minimum value of the key range of the row data stored by the shard replica in the data table, the key prefix can be referred to as a start key.

[0061] On this basis, the manner in which S202 in the above determines the user to which the shard replica belongs and whether the shard replica is a primary shard replica can be implemented as follows: determining the user to which the shard replica belongs according to the key prefix of the shard replica, and determining whether the shard replica is a primary shard replica according to the identity information of the shard replica.

[0062] For example, when the identity information is leader, the shard replica is determined as the primary shard replica; when the identity information is follower, the shard replica is determined as the secondary shard replica. Alternatively, when the identity information is 1, the shard replica is determined as the primary shard replica; when the identity information is 0, the shard replica is determined as the secondary shard replica.

[0063] Optionally, since the primary key of the data table can be one or more, and the purpose of the embodiment of the present disclosure is to perform load balancing based on the user to which the shard replica belongs, that is, the meta module does not perform load balancing according to all primary keys, but only performs load balancing according to the primary keys in the user dimension. Therefore, the embodiment of the present disclosure presets the primary key prefix load balancing dimension as 1, wherein the primary key prefix load balancing dimension refers to the number of primary keys on which load balancing is based. When the meta module determines the user to which the shard replica belongs, it only considers the primary keys in the user dimension in the primary key prefix based on the primary key prefix load balancing dimension, that is, 1.

[0064] Specifically, the meta module can determine the value of the user dimension in the primary key prefix, that is, the user identifier in the primary key prefix, based on the preset primary key prefix load balancing dimension, and then take the user represented by the user identifier as the user to which the shard replica belongs.

[0065] For example, when the primary key prefix in the heartbeat message only includes userid: 1, the meta module takes the value of userid, that is, 1, as the user to which the shard replica belongs.

[0066] Alternatively, when the primary key prefix in the heartbeat message includes userid: 1, orderid: 0, since the preset primary key prefix load balancing dimension is 1, the meta module ignores the value of orderid and takes the value of userid, that is, 1, as the user to which the shard replica belongs.

[0067] By using the above method, the embodiment of the present disclosure can obtain the user to which the shard replica belongs according to the primary key prefix of the shard replica, and determine whether the shard replica is a primary shard replica according to the identity information, so as to obtain the number of shard replicas and the number of primary shard replicas of each user stored in the first storage module, facilitating subsequent determination of whether the number of primary shard replicas and the number of shard replicas of the same user stored in the first storage module are excessive, so as to perform load balancing in the user dimension.

[0068] In an embodiment of the present disclosure, the first notification message includes the identities of the second storage modules to which the first specified number of primary shard replicas to be transferred correspond respectively. After receiving the first notification message, the first storage module can transfer the first specified number of primary shard replicas to the second storage modules corresponding to the first specified number of primary shard replicas respectively according to the identities of the second storage modules.

[0069] Before the first storage module is sent the first notification message in S203, refer to Figure 3 The meta module can also determine the identity of the second storage module corresponding to each of the first specified number of primary shard replicas through the following steps:

[0070] S301. Select one primary shard replica of the user stored by the first storage module according to a preset order.

[0071] The preset order can be the arrangement order of the primary shard replicas of the user stored by the first storage module or a random order, and the present disclosure does not make a specific limitation on this. The primary shard replica is selected according to the preset order so as not to repeatedly determine the identity of the second storage module corresponding to the same primary shard replica.

[0072] S302. Determine the storage module in which the slave shard replica of the selected primary shard replica is located, and select the storage module in which the number of primary shard replicas of the user is currently the least from the determined storage modules.

[0073] The meta module obtains the shard to which the shard replica stored in each storage module belongs, the user to which the shard replica belongs, and whether the shard replica is a primary shard replica through the heartbeat messages reported by each storage module, so that the meta module can filter the storage module in which the slave shard replica included in the shard to which the selected primary shard replica belongs from the selected primary shard replica. Further, the storage module in which the number of primary shard replicas of the user is currently the least is selected from the filtered storage modules, so that the distribution of the primary shard replicas of the user in the storage modules is more uniform.

[0074] The primary shard replica and its slave shard replica belong to the same shard, and the primary shard replica and its slave shard replica are different shard replicas of the shard, so the stored row data is the same. The second storage module is selected from the storage module in which the slave shard replica of the primary shard replica is stored, so that the same shard replicas of the same shard are stored in the first storage module and the second storage module. The transfer of the primary shard replica between the same shard replicas of the same shard is equivalent to changing the primary shard replica in the first storage module to a slave shard replica and changing the slave shard replica in the second storage module to a primary shard replica, so as to realize the transfer of the primary shard replica without actually transferring the shard replica. Thus, while achieving the dispersed distribution of the primary shard replicas of the same user in the storage modules, the communication resources consumed by the transfer of the shard replicas can be reduced.

[0075] Optionally, when there is more than one storage module in which the number of primary shard replicas of the user is currently the least, one storage module can be randomly selected from the storage modules in which the number of primary shard replicas of the user is currently the least.

[0076] S303, increase the number of the user's master shard replicas stored in the selected storage module by 1, and return the identifier of the selected storage module as the identifier of the second storage module corresponding to the master shard replica to S301, wherein the step of selecting one master shard replica of the user stored in the first storage module according to the preset order is repeated until the identifiers of the first specified number of master shard replicas corresponding to the second storage modules are determined.

[0077] For example, the leaders of user 1 stored in the first storage module include leaderl, leader2, leader3, leader4 and leader5. When the first specified number is 2, leaderl is selected first, and the storage modules where the followers of leaderl are stored are determined to be storeA and storeB. Assuming that the number of leaders stored in storeA is 0 and the number of leaders stored in storeB is 3, it is determined that leaderl corresponds to storeA. The number of leaders of the user stored in storeA is updated to 1 + 0 = 1. Then leader2 is selected, and the stores where the followers of leaderl are stored are determined to be storeA and storeB. At this time, the number of leaders of the user stored in storeA is 1 and the number of leaders of the user stored in storeB is 3, so it is determined that leader2 corresponds to storeA.

[0078] Through the above method, in the embodiment of the present disclosure, the meta module can determine the transferred storage module for each master shard replica to be scheduled respectively, so that the master shard replicas of the same user are distributed after transfer. Moreover, in the embodiment of the present disclosure, since the second storage module stores the slave shard replicas of the master shard replicas, the master shard replicas and the slave shard replicas belong to the same shard, so the row data stored is the same. Therefore, when the master shard replicas are transferred, only the identity change of the shard replicas needs to be completed, and the storage location of the shard replicas does not need to be transferred, thereby improving the efficiency of load balancing.

[0079] In an embodiment of the present disclosure, the first preset threshold in S203 can be the average value of the master shard replicas corresponding to the user. The average value of the master shard replicas corresponding to the user is the quotient between the total number of master shard replicas of the user and the number of storage modules in the storage cluster.

[0080] For example, the total number of primary shard replicas of user 1 is 5, the number of storage modules in the storage cluster is 3, and the average value of the primary shard replicas corresponding to user 1 is 5 / 3. Correspondingly, when the meta module executes S203, it is assumed that the first storage module includes shard 1, shard 2, and shard 3. Among them, shard 1, shard 2, and shard 3 all belong to user 1, and shard 1 and shard 3 are primary shard replicas. At this time, 2>5 / 3, that is, the number of primary shard replicas of user 1 stored in the first storage module is greater than the average value of the primary shard replicas corresponding to user 1.

[0081] On this basis, before sending the first notification message to the first storage module in S203, the meta module can also calculate the first specified number, including: calculating the difference between the number of primary shard replicas of the user stored in the first storage module and the average value of the primary shard replicas corresponding to the user, and taking the calculation result as the first specified number.

[0082] For example, the leader of user 2 stored in the first storage module includes leader 1, leader 2, and leader 3, the average value of the leader corresponding to user 2 is 2, and the first specified number is determined to be 3-2=1.

[0083] By using the above method, in the embodiment of the disclosure, the meta module can determine the number of primary shard replicas stored in the first storage module that exceeds the average value of the primary shard replicas corresponding to the user, so as to subsequently transfer the primary shard replicas exceeding the average value to other storage modules, thereby realizing uniform distribution of the primary shard replicas of the same user on the storage modules.

[0084] Since the storage module reports the heartbeat message at a high frequency, if load balancing is performed every time the heartbeat message is received, the load balancing operation will be too frequent, increasing the calculation pressure of the meta module.

[0085] In an embodiment of the disclosure, in order to reduce the calculation pressure of the meta module, after receiving the heartbeat message sent by the first storage module in S201, the meta module can also execute: increasing the number of times of sending the heartbeat message by the first storage module by 1, and then in response to the increased number of times reaching a specified number of times, executing the above S202. Otherwise, wait for the next time the first storage module sends a heartbeat message.

[0086] The specified number of times can be pre-set according to actual conditions. For example, the specified number of times is a multiple of 50, that is, the specified number of times is 50, 100, 150, and the like. If the number of times of sending the heartbeat message by the first storage module is 49, after receiving the heartbeat message sent by the first storage module, 49+1=50 is calculated, at this time, the number of times of sending the heartbeat message by the first storage module reaches the specified number of times, and the load balancing operation is performed in S202.

[0087] Through the above method, the load balancing operation is performed only when the number of times of receiving the heartbeat message reported by the first storage module reaches the specified number of times in the meta module, thereby reducing the frequency of performing the load balancing operation in the meta module, and reducing the calculation pressure of the meta module.

[0088] In an embodiment of the present disclosure, the scheduling strategy in S203 is a primary shard replica scheduling strategy, that is, the primary shard replica of the same user is scheduled in the storage module, so as to realize uniform distribution of the primary shard replica of the same user in the storage module.

[0089] In addition, an embodiment of the present disclosure also provides a replica scheduling strategy, that is, the shard replica of the same user is scheduled in the storage module, so as to realize scattered distribution of the shard replica of the same user in the storage module. The shard replica of the user includes the primary shard replica and the slave shard replica of the user.

[0090] The meta module can choose to use the leader scheduling strategy or the replica scheduling strategy. That is, after determining the user to which the shard replica belongs and whether the shard replica is the primary shard replica in S202, the meta module can further execute the following steps:

[0091] In response to the preset scheduling strategy corresponding to the specified number of times being the primary shard replica scheduling strategy, S203 is executed. In response to the preset scheduling strategy corresponding to the specified number of times being the replica scheduling strategy, the replica scheduling strategy is executed, that is, for each determined user, if the number of shard replicas of the user stored in the first storage module is greater than the preset second threshold, a second notification message is sent to the first storage module.

[0092] The second notification message is used to notify the first storage module to transfer part of the shard replicas of the user to the third storage module; the number of shard replicas of the user stored in the third storage module is less than the number of shard replicas of the user stored in the first storage module.

[0093] The scheduling strategy corresponding to the specified number of times can be pre-set according to actual conditions. For example, when the specified number of times is a multiple of 50 and not a multiple of 100, the corresponding copy scheduling strategy is adopted; when the specified number of times is a multiple of 50 and a multiple of 100, the corresponding leader scheduling strategy is adopted. That is, when the specified number of times is 50, 150, 250, etc., the corresponding copy scheduling strategy is adopted; when the specified number of times is 100, 200, 300, etc., the corresponding leader scheduling strategy is adopted.

[0094] In the copy scheduling strategy, for each user determined in S202, the number of shard copies of the user stored in the first storage module is counted. It is determined whether the number of shard copies of the user stored in the first storage module is greater than a preset second threshold. If yes, a second notification message is sent to the first storage module. If no, the storage location of the shard copy of the user stored in the first storage module is not adjusted.

[0095] By using the above method, the meta module in the embodiment of the present disclosure can select the leader scheduling strategy and the copy scheduling strategy, so as to schedule the shard copy of the user in the leader dimension or the copy shard dimension. Since in the copy scheduling strategy, the shard copies of the same user can be distributed in each storage module, the access performance of the data of the user is ensured.

[0096] In an embodiment of the present disclosure, the second notification message includes the identifier of the third storage module corresponding to each of the second specified number of shard copies to be transferred. So that after the first storage module receives the second notification message, the first storage module can transfer the second specified number of shard copies to the third storage module corresponding to each of the second specified number of shard copies according to the identifier of the third storage module corresponding to each of the second specified number of shard copies.

[0097] Therefore, before the second notification message is sent to the first storage module, referring to Figure 4 , the meta module can also determine the identifier of the third storage module corresponding to each of the second specified number of shard copies by the following steps:

[0098] S401, obtaining the number of shard copies of the user stored in each storage module in the storage cluster.

[0099] The meta module obtains the user to which the shard copy stored in each storage module belongs through the heartbeat message reported by each storage module in the storage cluster, so as to obtain the number of shard copies of the user stored in each storage module in the storage cluster.

[0100] S402, selecting one shard copy of the user stored in the first storage module according to a preset order, and determining the storage module in which the number of shard copies of the user is the least.

[0101] The preset order can be an arrangement order of the shard copies of the user stored by the first storage module or a random order, and the present embodiment is not limited in this regard. The shard copy is selected according to the preset order so as not to repeatedly determine the identifier of the third storage module corresponding to the same shard copy.

[0102] Optionally, when there is more than one storage module currently storing the least number of shard copies of the user, a storage module is randomly selected from the storage modules currently storing the least number of shard copies of the user.

[0103] S403, the number of shard copies of the user stored by the determined storage module is increased by 1, and the identifier of the determined storage module is taken as the identifier of the third storage module corresponding to the shard copy, and the step of selecting a shard copy of the user stored by the first storage module according to the preset order is returned to S402 until the identifiers of the third storage modules corresponding to the second specified number of shard copies are determined.

[0104] For example, the shard copies of user 1 stored in the first storage module include shard copy 1, shard copy 2 and shard copy 3. When the second specified number is 1, shard copy 1 is selected. Assuming that the number of shard copies of user 1 stored in storeA in the storage cluster is 0 and the number of shard copies of user 1 stored in storeB is 1, it is determined that shard copy 1 corresponds to storeA.

[0105] Through the above method, in the present embodiment, the meta module can determine the transferred storage module for each shard copy to be scheduled, so that the shard copies of the same user are distributed on the storage modules after being transferred.

[0106] In an embodiment of the present disclosure, the second preset threshold in the above copy scheduling strategy is the average value of the shard copies corresponding to the user. The average value of the shard copies corresponding to the user is the quotient between the total number of shard copies of the user and the number of storage modules in the storage cluster.

[0107] For example, assuming that the total number of shard copies of user 1 is 5 and the number of storage modules in the storage cluster is 2, i.e., the average value of the shard copies corresponding to the user is 5 / 2. Assuming that the first storage module includes shard 1, shard 2 and shard 3, and shard 1, shard 2 and shard 3 all belong to user 1. At this time, 3>5 / 2, i.e., the number of shard copies of user 1 stored by the first storage module is greater than the average value of the shard copies corresponding to user 1.

[0108] Before sending the second notification message to the first storage module, the meta module can also calculate a second specified quantity, including: calculating the difference between the number of shard replicas of the user stored in the first storage module and the average number of shard replicas corresponding to the user, and using the calculation result as the second specified quantity.

[0109] For example, the first storage module stores fragmented copies of user 2, including fragmented copy 1, fragmented copy 2 and fragmented copy 3. The average number of fragmented copies corresponding to user 2 is 2, and the second specified number is determined to be 3-2=1.

[0110] Using the above method, in this embodiment of the disclosure, the meta module can determine the number of shard replicas stored in the first storage module that exceed the average number of shard replicas corresponding to the user, so that the shard replicas exceeding the average number can be transferred to other storage modules, thereby achieving a uniform distribution of shard replicas of the same user across the storage modules.

[0111] In one embodiment of this disclosure, due to Figure 1 The leader scheduling strategy shown requires the average value of the primary shard replicas for each user, and the replica scheduling strategy requires the average value of the shard replicas. Therefore, this is executed in the meta module. Figure 1 Before the load balancing method shown, you can also use... Figure 5 The method shown calculates the average value of the primary shard replicas and the average value of the shard replicas for each user. Alternatively, the meta module can also periodically... Figure 5 The method shown updates the average primary shard replica count and the average shard replica count for the user. This embodiment of the disclosure executes the meta module. Figure 5 The timing is not specifically limited.

[0112] Optionally, after collecting heartbeat messages sent by each storage module in S501 for a period of time, the meta module executes S502-S504 to ensure that the meta module fully obtains the shard replica information stored by all storage modules in the storage cluster.

[0113] See Figure 5 The following explains in detail how the meta module calculates the average value of primary shard replicas and the average value of shard replicas for each user:

[0114] S501: Receive heartbeat messages sent by each storage module in the storage cluster.

[0115] The heartbeat message sent by each storage module includes the primary key prefix and identity information of each shard replica stored in that storage module.

[0116] S502, for each storage module, determining a user to which each shard replica in the storage module belongs according to a primary key prefix of each shard replica included in the heartbeat message sent by the storage module, and determining whether each shard replica is a primary shard replica according to identity information of each shard replica.

[0117] The specific implementation of S502 is the same as S202 described above, and reference can be made to the related description of S202 described above, which will not be described here again.

[0118] S503, for each user, taking a sum of the number of shard replicas of the user stored by each storage module in the storage cluster as a total number of shard replicas of the user, and calculating a quotient between the total number of shard replicas of the user and the number of storage modules in the storage cluster, taking the calculation result as a shard replica average value corresponding to the user.

[0119] For example, the storage cluster includes store1, store2 and store3, for each user, it is determined that the number of shard replicas of the user stored by store1 is 2, the number of shard replicas of the user stored by store2 is 1, and the number of shard replicas of the user stored by store3 is 0, so the sum of the number of shard replicas of the user is 3. The shard replica average value corresponding to the user = (2+1) / 3 = 1.

[0120] Optionally, an upper integer of the quotient between the total number of shard replicas of the user and the number of storage modules in the storage cluster can be calculated, so that the calculated shard replica average value corresponding to the user is an integer.

[0121] S504, taking a sum of the number of primary shard replicas of the user stored by each storage module in the storage cluster as a total number of primary shard replicas of the user, and calculating a quotient between the total number of primary shard replicas of the user and the number of storage modules in the storage cluster, taking the calculation result as a primary shard replica average value corresponding to the user.

[0122] For example, the storage cluster includes store1, store2 and store3, for each user, it is determined that the number of primary shard replicas of the user stored by store1 is 2, the number of primary shard replicas of the user stored by store2 is 1, and the number of primary shard replicas of the user stored by store3 is 0, so the sum of the number of primary shard replicas of the user is 3. The primary shard replica average value corresponding to the user = (2+1) / 3 = 1.

[0123] Optionally, an upper integer of the quotient between the total number of primary shard replicas of the user and the number of storage modules in the storage cluster can be calculated, so that the calculated primary shard replica average value corresponding to the user is an integer.

[0124] By using the above method, in the embodiments of the present disclosure, the meta module can collect the users to which the shard replica stored by each storage module in the storage cluster belongs, and whether the shard replica is a primary shard replica, so as to determine the average value of the shard replica corresponding to each user and the average value of the primary shard replica corresponding to each user, so as to be used in subsequent load balancing.

[0125] Referring to Figure 6 , the load balancing method of the distributed database provided by the embodiments of the present disclosure is exemplified in combination with an application scenario as follows:

[0126] The store cluster includes store1, store2, store3, store4, store5 and store6, Figure 6 The rectangle in each store module in the above table indicates a shard replica stored by the store module, and the shard number in the shard replica indicates the shard to which the shard replica belongs. When the shard replica is marked with "leader", it indicates that the shard replica is a primary shard replica, and when the shard replica is not marked with "leader", it indicates that the shard replica is a follower shard replica.

[0127] The store module reports the primary key prefix and identity information of the shard replica stored by the store module to the meta module through a heartbeat message. For example, in the heartbeat message reported by store1, the primary key prefix start_key of the shard replica of shard 1 is: userid: 1, orderid: 3, and the identity information is leader; the primary key prefix start_key of the shard replica of shard 2 is: userid: 1, orderid: 5, and the identity information is follower; and the primary key prefix start_key of the shard replica of shard 3 is: userid: 1, orderid: 9, and the identity information is leader.

[0128] Since the primary key prefix load balancing dimension is 1, the meta module only considers the userid in the primary key prefix and ignores the orderid, so that the shards corresponding to the user with userid 1 include shard 1, shard 2 and shard 3. The meta module determines that the shard replicas of the user stored by store1 include the shard replica of shard 1, the shard replica of shard 2 and the shard replica of shard 3, which is recorded as userid = 1, store1: [1, 2, 3].

[0129] The shard replicas of the user stored by store2 include the shard replica of shard 1 and the shard replica of shard 3, which is recorded as userid = 1, store2: [1, 3].

[0130] The shard replica of the user stored in store3 includes the shard replica of shard 1 and the shard replica of shard 2, denoted as userid = 1, store3: [1, 2].

[0131] The shard replica of the user stored in store4 includes the shard replica of shard 2, denoted as userid = 1, store4: [2].

[0132] The shard replica of the user is not stored in store5.

[0133] The shard replica of the user stored in store6 includes the shard replica of shard 3, denoted as userid = 1, store6: [3].

[0134] The shard replica distribution of the user dimension is obtained: userid = 1, shard list (regionlist) = {store1: [1, 2, 3], store2: [1, 3], store3: [1, 2], store4: [2], store6: [3]}. The leader number distribution of the user dimension is obtained: userid = 1, leader list (leaderlist) = {store1: 2, store3: 1}. The shard replica distribution and the leader distribution of other users are also counted in the same way.

[0135] The average shard replica corresponding to the user is calculated as The average primary shard replica corresponding to the user is calculated as

[0136] When the subsequent meta module receives the heartbeat message sent by store1, the scheduling strategy corresponding to the number of times of sending the heartbeat message by store1 is the leader scheduling strategy. Since the number of leaders stored in store1 is 2, which is equal to the average primary shard replica 1 corresponding to the user, i.e., 1 leader needs to be transferred. A leader of the user is randomly selected from store1. It is assumed that the leader of shard 3 is selected. The follower of shard 3 is stored in store2 and store6, and the number of leaders stored in store2 and store6 is 0. At this time, a store module is randomly selected from store2 and store6. It is assumed that store6 is selected. A first notification message is sent to store1. The first notification message is used to notify store1 to schedule the leader of shard 3 to store6. When store1 receives the first notification message, the shard replica of shard 3 stored in store1 is changed to a follower, and store6 is notified to change the shard replica of shard 3 stored in store6 to a leader.

[0137] After the meta module receives the heartbeat message sent by the store 1, when the scheduling strategy corresponding to the number of times that the store 1 sends the heartbeat message is the replica scheduling strategy, since the store 1 stores the shard replica number of the user is 3-the shard replica average value corresponding to the user 2=1, that is, 1 shard replica needs to be transferred. A shard replica is randomly selected from the store 1, and it is assumed that the shard replica of shard 2 is selected. The store 5 in the store cluster stores the least number of shard replicas of the user, and therefore the store 5 is selected. A second notification message is sent to the store 1, and the second notification message is used to notify the store 1 to transfer the shard replica of shard 2 to the store 5. When the store 1 receives the second notification message, the shard replica of shard 2 stored by the store 1 is transferred to the store 5.

[0138] After the shard replicas and the leader of the user with the userid 1 are evenly distributed in the store cluster by the above method, the access operation of the user data is evenly distributed to each store module, thereby guaranteeing the access performance of the user data.

[0139] Figure 6 Only the scheduling of the shard replicas and the leader of the user with the userid 1 is shown, and the shard replicas and the leader of other users are also scheduled in the same way.

[0140] Based on the same inventive concept, corresponding to the method embodiment, the embodiment of the disclosure provides a load balancing device of a distributed database, applied to a meta module. As shown in Figure 7 The device comprises a receiving module 701, a determining module 702 and a sending module 703.

[0141] The receiving module 701 is used to receive a heartbeat message sent by a first storage module in a storage cluster, and the heartbeat message comprises attribute information of each shard replica stored in the first storage module.

[0142] The determining module 702 is used to determine, for each shard replica in the first storage module, a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica.

[0143] The sending module 703 is configured to send a first notification message to the first storage module for each user determined by the determining module 702, if the number of the primary shard replicas of the user stored in the first storage module is greater than the average value of the primary shard replicas corresponding to the user; wherein the first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to the second storage module; and the number of the primary shard replicas of the user stored in the second storage module is less than the number of the primary shard replicas of the user stored in the first storage module.

[0144] In an embodiment of the present disclosure, wherein the attribute information of each shard replica includes a primary key prefix and identity information of the shard replica, the primary key prefix of the shard replica includes a user identifier; the determining module 702 is specifically configured to:

[0145] determine a user to which the shard replica belongs according to the primary key prefix of the shard replica;

[0146] determine whether the shard replica is a primary shard replica according to the identity information of the shard replica.

[0147] In an embodiment of the present disclosure, the apparatus further includes an accumulating module.

[0148] The accumulating module is configured to, after receiving the heartbeat message sent by the first storage module in the storage cluster, increase the number of times of sending the heartbeat message by the first storage module by 1, and in response to the increased number of times reaching a specified number of times, invoke the determining module 702 to perform the step of determining, for each shard replica in the first storage module, a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica; otherwise, wait for the first storage module to send the heartbeat message next time.

[0149] In an embodiment of the present disclosure, the apparatus can further include a calling module.

[0150] The calling module is configured to, after determining, for each shard replica in the first storage module, a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica, in response to a preset scheduling strategy corresponding to a specified number of times being a primary shard replica scheduling strategy, invoke the sending module 703 to perform the step of sending a first notification message to the first storage module for each user determined by the determining module 702, if the number of the primary shard replicas of the user stored in the first storage module is greater than a preset first threshold value;

[0151] The sending module 703 is further configured to, in response to the preset scheduling strategy corresponding to the specified number being the replica scheduling strategy, send a second notification message to the first storage module for each user determined, if the number of shard replicas of the user stored in the first storage module is greater than the preset second threshold value; the second notification message is used to notify the first storage module to transfer part of the shard replicas of the user to the third storage module, and the number of shard replicas of the user stored in the third storage module is less than the number of shard replicas of the user stored in the first storage module.

[0152] In an embodiment of the present disclosure, the first notification message includes the identification of the second storage module corresponding to each of the first specified number of master shard replicas to be transferred; the apparatus can further include a selection module;

[0153] The selection module is configured to, before sending the first notification message to the first storage module, select one master shard replica of the user stored in the first storage module in a preset order; determine the storage module in which the slave shard replica of the selected master shard replica is located, and select the storage module in which the number of master shard replicas of the user currently stored is the least from the determined storage module;

[0154] The determination module 702 is further configured to add 1 to the number of master shard replicas of the user stored in the storage module selected by the selection module, and return the identification of the selected storage module as the identification of the second storage module corresponding to the master shard replica, and return to the step of selecting one master shard replica of the user stored in the first storage module in a preset order until the identification of the second storage module corresponding to each of the first specified number of master shard replicas is determined.

[0155] In an embodiment of the present disclosure, the first preset threshold value is an average value of the master shard replicas corresponding to the user, and the average value of the master shard replicas corresponding to the user is the quotient between the total number of master shard replicas of the user and the number of storage modules in the storage cluster; the apparatus can further include a calculation module;

[0156] The calculation module is configured to, before sending the first notification message to the first storage module, calculate the difference between the number of master shard replicas of the user stored in the first storage module and the average value of the master shard replicas corresponding to the user, and take the calculation result as the first specified number.

[0157] In an embodiment of the present disclosure, the second notification message includes the identification of the third storage module corresponding to each of the second specified number of shard replicas to be transferred; the apparatus can further include an acquisition module;

[0158] The acquisition module is configured to, before sending the second notification message to the first storage module, acquire the number of shard replicas of the user stored in each storage module in the storage cluster;

[0159] The determining module 702 is further configured to determine, in the preset order, a storage module storing one shard copy of the user from the first storage module, and determine a storage module storing the least number of shard copies of the user; add 1 to the number of shard copies of the user stored in the determined storage module, and take the identifier of the determined storage module as the identifier of a third storage module corresponding to the shard copy, and return to the step of selecting one shard copy of the user stored in the first storage module in the preset order until the identifiers of the first specified number of third storage modules corresponding to the shard copies are determined.

[0160] In an embodiment of the present disclosure, the second preset threshold is an average value of shard copies corresponding to the user, and the average value of shard copies corresponding to the user is a quotient between the total number of shard copies of the user and the number of storage modules in the storage cluster; the device can further include a calculating module;

[0161] The calculating module is configured to calculate a difference between the number of shard copies of the user stored in the first storage module and the average value of shard copies corresponding to the user before sending the second notification message to the first storage module, and take the calculation result as the second specified number.

[0162] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of shard information are in line with relevant laws and regulations and do not violate public order and good customs.

[0163] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0164] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0165] As Figure 8As shown, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded into a random access memory (RAM) 803 from a storage unit 808. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0166] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, and the like; an output unit 807, such as various types of displays, a speaker, and the like; a storage unit 808, such as a magnetic disk, an optical disk, and the like; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0167] The computing unit 801 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 801 performs various methods and processes described above, such as the load balancing method of a distributed database. For example, in some embodiments, the load balancing method of a distributed database is implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the load balancing method of a distributed database described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the load balancing method of a distributed database by any other appropriate means, such as by means of firmware.

[0168] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0169] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0170] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0171] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0172] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0173] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0174] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without departing from the desired results of the technology disclosed in the present disclosure, which are not limited herein.

[0175] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and scope of the disclosure. Any modifications, equivalent substitutions, improvements, and the like, made within the spirit and principles of the disclosure, are intended to be included in the scope of the disclosure.

Claims

1. A load balancing method of a distributed database, comprising: receiving a heartbeat message sent by a first storage module in a storage cluster, wherein the heartbeat message comprises attribute information of each shard replica stored in the first storage module; when a number of times of sending the heartbeat message by the first storage module reaches a specified number, for each shard replica in the first storage module, determining, according to the attribute information of the shard replica, a user to which the shard replica belongs and whether the shard replica is a primary shard replica; in response to a preset scheduling policy corresponding to the specified number being a primary shard replica scheduling policy, for each determined user, if a number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold, sending a first notification message to the first storage module; wherein the first notification message is used to notify the first storage module to transfer part of the primary shard replicas of the user to a second storage module; the number of primary shard replicas of the user stored in the second storage module is less than the number of primary shard replicas of the user stored in the first storage module; in response to the preset scheduling policy corresponding to the specified number being a replica scheduling policy, for each determined user, if a number of shard replicas of the user stored in the first storage module is greater than a preset second threshold, sending a second notification message to the first storage module; wherein the second notification message is used to notify the first storage module to transfer part of the shard replicas of the user to a third storage module; the number of shard replicas of the user stored in the third storage module is less than the number of shard replicas of the user stored in the first storage module.

2. The method of claim 1, wherein, The attribute information of each shard replica comprises a primary key prefix and identity information of the shard replica, and the primary key prefix of the shard replica comprises a user identifier; The determining, according to the attribute information of the shard replica, of the user to which the shard replica belongs and whether the shard replica is a primary shard replica comprises: determining, according to the primary key prefix of the shard replica, the user to which the shard replica belongs; determining, according to the identity information of the shard replica, whether the shard replica is a primary shard replica. 3.The method of claim 1, after receiving the heartbeat message sent by the first storage module in the storage cluster, the method further comprises: incrementing the number of times of sending the heartbeat message by the first storage module by 1, and in response to the incremented number reaching the specified number, performing the step of determining, for each shard replica in the first storage module, according to the attribute information of the shard replica, the user to which the shard replica belongs and whether the shard replica is a primary shard replica; otherwise, waiting for the first storage module to send the heartbeat message next time.

4. The method of any of claims 1-3, wherein the first notification message comprises an identification of the second storage module to which each of the first specified number of copies of the primary shard is to be transferred. Before the sending of the first notification message to the first storage module, the method further comprises: selecting, in a preset order, one primary shard replica of the user stored in the first storage module; determining a storage module in which a slave shard replica of the selected primary shard replica is located, and selecting, from the determined storage module, a storage module in which the number of primary shard replicas of the user currently stored is the least. add 1 to the number of the primary shard copies of the user stored in the selected storage module, and return the identity of the selected storage module as the identity of the second storage module corresponding to the primary shard copy to the step of selecting one primary shard copy of the user stored in the first storage module according to the preset order until the identities of the first specified number of primary shard copies each corresponding to the second storage module are determined. 5.The method of claim 4, wherein the preset first threshold is an average value of the primary shard copies corresponding to the user, and the average value of the primary shard copies corresponding to the user is a quotient between the total number of the primary shard copies of the user and the number of the storage modules in the storage cluster; and before the first notification message is sent to the first storage module, the method further comprises: calculating a difference between the number of the primary shard copies of the user stored in the first storage module and the average value of the primary shard copies corresponding to the user, and taking the calculation result as the first specified number. 6.The method of claim 1, wherein the second notification message comprises an identification of a third storage module corresponding to each of the second specified number of replicas of the shard to be migrated. before the second notification message is sent to the first storage module, the method further comprises: obtaining the number of the shard copies of the user stored in each storage module in the storage cluster; selecting one shard copy of the user stored in the first storage module according to the preset order, and determining a storage module currently storing the least number of the shard copies of the user; adding 1 to the number of the shard copies of the user stored in the determined storage module, and returning the identity of the determined storage module as the identity of the third storage module corresponding to the shard copy to the step of selecting one shard copy of the user stored in the first storage module according to the preset order until the identities of the first specified number of shard copies each corresponding to the third storage module are determined. 7.The method of claim 6, wherein the preset second threshold is an average value of the shard copies corresponding to the user, and the average value of the shard copies corresponding to the user is a quotient between the total number of the shard copies of the user and the number of the storage modules in the storage cluster; and before the second notification message is sent to the first storage module, the method further comprises: calculating a difference between the number of the shard copies of the user stored in the first storage module and the average value of the shard copies corresponding to the user, and taking the calculation result as the second specified number. 8.A load balancing apparatus of a distributed database, comprising: a receiving module configured to receive a heartbeat message sent by a first storage module in a storage cluster, wherein the heartbeat message comprises attribute information of each shard copy stored in the first storage module; a determining module configured to, when the number of times of sending the heartbeat message by the first storage module reaches a specified number of times, determine, for each shard copy in the first storage module, a user to which the shard copy belongs and whether the shard copy is a primary shard copy according to the attribute information of the shard copy. The sending module is configured to, in response to the preset scheduling strategy corresponding to the specified number of times being the primary shard replica scheduling strategy, send a first notification message to the first storage module for each user determined by the determining module, if the number of primary shard replicas of the user stored in the first storage module is greater than a preset first threshold value; wherein the first notification message is used to notify the first storage module to move part of the primary shard replicas of the user to a second storage module; and the number of primary shard replicas of the user stored in the second storage module is less than the number of primary shard replicas of the user stored in the first storage module. The sending module is further configured to, in response to the preset scheduling strategy corresponding to the specified number of times being the replica scheduling strategy, send a second notification message to the first storage module for each user determined by the determining module, if the number of shard replicas of the user stored in the first storage module is greater than a preset second threshold value; wherein the second notification message is used to notify the first storage module to move part of the shard replicas of the user to a third storage module, and the number of shard replicas of the user stored in the third storage module is less than the number of shard replicas of the user stored in the first storage module.

9. The apparatus of claim 8, wherein, The attribute information of each shard replica includes a primary key prefix and identity information of the shard replica, and the primary key prefix of the shard replica includes a user identifier. The determining module is specifically configured to: determine a user to which the shard replica belongs according to the primary key prefix of the shard replica; and 10. The apparatus of claim 8, further comprising: determine whether the shard replica is a primary shard replica according to the identity information of the shard replica. An accumulation module is further included.

11. The apparatus of any of claims 8-10, wherein the first notification message comprises an identification of a second storage module corresponding to each of the first specified number of primary shard replicas to be transferred; and the apparatus further comprises: The accumulation module is configured to, after receiving a heartbeat message sent by a first storage module in a storage cluster, increase the number of times of sending the heartbeat message by the first storage module by 1, and in response to the increased number of times reaching a specified number of times, invoke the determining module to perform the step of determining, for each shard replica in the first storage module, a user to which the shard replica belongs and whether the shard replica is a primary shard replica according to attribute information of the shard replica; otherwise, wait for the first storage module to send a heartbeat message next time. A selection module is further included. The selection module is configured to, before sending the first notification message to the first storage module, select one primary shard replica of the user stored in the first storage module in a preset order, determine a storage module in which a slave shard replica of the selected primary shard replica is located, and select a storage module in which the number of primary shard replicas of the user is currently the least from the determined storage modules.

12. The apparatus of claim 11, wherein the preset first threshold is a user corresponding primary shard replica average value, and the user corresponding primary shard replica average value is a quotient between a total number of primary shard replicas of the user and a number of storage modules in the storage cluster. The determining module is further configured to increase the number of primary shard replicas of the user stored in the storage module selected by the selection module by 1, return the identifier of the selected storage module as the identifier of a second storage module corresponding to the primary shard replica, and return the step of selecting one primary shard replica of the user stored in the first storage module in a preset order until the identifiers of the first specified number of primary shard replicas each corresponding to a second storage module are determined. A calculation module is further included. The computing module is configured to calculate a difference between the number of primary shard replicas of the user stored in the first storage module and an average value of the primary shard replicas corresponding to the user before the first notification message is sent to the first storage module, and use the calculation result as the first specified number.

13. The apparatus of claim 8, wherein the second notification message comprises an identification of a third storage module corresponding to each of the second specified number of replicas of the shard to be migrated; and the apparatus further comprises: The obtaining module is configured to obtain the number of shard replicas of the user stored in each storage module in the storage cluster before the second notification message is sent to the first storage module. The determining module is further configured to select one shard replica of the user stored in the first storage module in a preset order, and determine a storage module currently storing the least number of shard replicas of the user. The number of shard replicas of the user stored in the determined storage module is increased by 1, and the identifier of the determined storage module is used as the identifier of a third storage module corresponding to the shard replica, and the step of selecting one shard replica of the user stored in the first storage module in a preset order is returned until the identifiers of the first specified number of shard replicas each corresponding to a third storage module are determined. The computing module is configured to calculate a difference between the number of primary shard replicas of the user stored in the first storage module and an average value of the primary shard replicas corresponding to the user before the first notification message is sent to the first storage module, and use the calculation result as the first specified number.

14. The apparatus of claim 13, wherein the preset second threshold is an average value of the shard replicas corresponding to the user, and the average value of the shard replicas corresponding to the user is a quotient between a total number of the shard replicas of the user and a number of the storage modules in the storage cluster; and the apparatus further comprises:

15. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7. The computer instructions are used to enable the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, ​ ​

Images