Load balancing method and device for distributed database

Abstract

The present application embodiment discloses a load balancing method and device for a distributed database. A specific implementation of the method includes: in response to detecting that some master nodes in a master node cluster have an increase in the amount of tasks to be synchronized, selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue; selecting a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue; for the first master node in the backlog processing queue, selecting a second master node corresponding to the first master node from the balanced replacement queue, exchanging the data partition of the selected second master node with the data partition of the first master node, and in response to determining that the amount of tasks to be synchronized in the first master node has decreased, removing the first master node from the backlog processing queue. This implementation reduces the risk of inconsistent data storage in a master-slave node cluster.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a load balancing method and device for a distributed database. Background Art

[0002] In order to ensure the high reliability of cluster data, existing distributed databases usually need to build a master-slave cluster disaster recovery mode across computer rooms (that is, the distributed database includes a master node cluster and a slave node cluster for disaster recovery), and perform real-time data synchronization between the master and slave clusters.

[0003] At present, data synchronization of the master-slave cluster of distributed databases often does not consider the synchronization performance of cluster nodes. Therefore, if the disaster recovery synchronization speed of a slave node in the master-slave cluster is slow, it often leads to a backlog of data synchronization tasks. Furthermore, if the backlog of data synchronization tasks is serious, it will also lead to the risk of inconsistent data in the master-slave cluster for a long time, thus affecting the disaster recovery effect of the distributed database. Summary of the invention

[0004] The embodiments of the present application provide a load balancing method and device for a distributed database.

[0005] In a first aspect, an embodiment of the present application provides a load balancing method for a distributed database, wherein the distributed database includes a master node cluster and a slave node cluster for disaster recovery, the method comprising: in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue; selecting a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of first master nodes is less than or equal to the number of second master nodes; for the first master node in the backlog processing queue, selecting a second master node corresponding to the first master node from the balanced replacement queue, exchanging the data partition of the selected second master node with the data partition of the first master node; in response to determining that the amount of tasks to be synchronized in the first master node has decreased, removing the first master node from the backlog processing queue.

[0006] In some embodiments, the above method also includes: building a synchronization task queue in the master node of the master node cluster, wherein the synchronization task queue is used to cache tasks to be synchronized; for the master node in the master node cluster, in response to detecting that the number of tasks to be synchronized in the synchronization task queue of the master node continues to increase within a preset time period, determining that the master node has an increase in the number of tasks to be synchronized.

[0007] In some embodiments, in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, before selecting a first master node from the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold to generate a backlog processing queue, the above method also includes: detecting the tasks to be synchronized of the master nodes in the master node cluster; determining the time difference between the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster this time and the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster last time; in response to determining that the time difference is greater than a third preset threshold, determining whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster.

[0008] In some embodiments, in response to detecting that some master nodes in a master node cluster have an increase in the amount of tasks to be synchronized, selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue, including: in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, determining whether the master node cluster is divided into at least two groups; in response to determining that the master node cluster is divided into at least two groups, detecting an increase in the amount of tasks to be synchronized of all master nodes in at least two groups; determining a first group in which some master nodes have an increase in the amount of tasks to be synchronized, from the at least two groups, generating a master node list, wherein the master node list is composed of master nodes with an increase in the amount of tasks to be synchronized in the first group; selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node list to generate a backlog processing queue.

[0009] In some embodiments, for the first master node in the backlog processing queue, a second master node corresponding to the first master node is selected from the balanced replacement queue, and the data partition of the selected second master node is exchanged with the data partition of the first master node, including: for the first master node in the backlog processing queue, in response to determining that the first master node is the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, selecting the second master node with the smallest amount of backlog tasks from the balanced replacement queue; determining the selected second master node as the second master node corresponding to the first master node, determining the data partition with the largest amount of tasks to be synchronized in the first master node, and determining the data partition with the smallest amount of tasks to be synchronized in the second master node corresponding to the first master node; exchanging the data partition of the determined first master node with the data partition of the determined second master node.

[0010] In some embodiments, the above method further includes: in response to determining that the backlog processing queue is empty, reselecting a first master node in the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold.

[0011] In a second aspect, an embodiment of the present application provides a load balancing device for a distributed database, wherein the distributed database includes a master node cluster and a slave node cluster for disaster recovery, and the device includes: a backlog processing queue generation unit, configured to select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized to generate a backlog processing queue; a balanced replacement queue generation unit, configured to select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of first master nodes is less than or equal to the number of second master nodes; a first master node removal unit, configured to select a second master node corresponding to the first master node in the backlog processing queue from the balanced replacement queue, exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue in response to determining that the amount of tasks to be synchronized in the first master node has decreased.

[0012] In some embodiments, the above-mentioned device also includes: a synchronization task queue construction unit, configured to construct a synchronization task queue in a master node of a master node cluster, wherein the synchronization task queue is used to cache tasks to be synchronized; a unit for determining the growth in the amount of tasks to be synchronized, configured to determine, for a master node in the master node cluster, that there is an increase in the amount of tasks to be synchronized in the synchronization task pair queue of the master node, in response to detecting that the amount of tasks to be synchronized in the master node continues to increase within a preset time period.

[0013] In some embodiments, the above-mentioned device also includes: a detection unit, configured to detect the tasks to be synchronized of the master nodes in the master node cluster; a time difference determination unit, configured to determine the time difference between the time of detecting the tasks to be synchronized of the master nodes in the master node cluster this time and the time of detecting the tasks to be synchronized of the master nodes in the master node cluster last time; a master node determination unit, configured to determine whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster in response to determining that the time difference is greater than a third preset threshold.

[0014] In some embodiments, the backlog processing queue generation unit is further configured to: in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, determine whether the master node cluster is divided into at least two groups; in response to determining that the master node cluster is divided into at least two groups, detect the increase in the amount of tasks to be synchronized of all master nodes in the at least two groups; determine a first group from the at least two groups where there is an increase in the amount of tasks to be synchronized for some master nodes, and generate a master node list, wherein the master node list is composed of master nodes with an increase in the amount of tasks to be synchronized in the first group; and select the first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node list to generate a backlog processing queue.

[0015] In some embodiments, the first master node removal unit is further configured to: for the first master node in the backlog processing queue, in response to determining that the first master node is the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, select the second master node with the smallest backlog of tasks from the balanced replacement queue; determine the selected second master node as the second master node corresponding to the first master node, determine the data partition with the largest amount of tasks to be synchronized in the first master node, and determine the data partition with the smallest amount of tasks to be synchronized in the second master node corresponding to the first master node; and exchange the data partition of the determined first master node with the data partition of the determined second master node.

[0016] In some embodiments, the above-mentioned device also includes: a reselection unit, configured to reselect a first master node in the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold in response to determining that the backlog processing queue is empty.

[0017] The load balancing method and device of the distributed database provided by the embodiment of the present application, in response to detecting that the amount of tasks to be synchronized in some master nodes in the master node cluster increases, can select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue, and then select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, and finally select a second master node corresponding to the first master node from the balanced replacement queue for the first master node in the backlog processing queue, so as to exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue when it is determined that the amount of tasks to be synchronized in the first master node is reduced. In this way, load balancing of tasks to be synchronized in the master node cluster can be achieved, the risk of inconsistent data storage in the master-slave node cluster in the distributed database is reduced, and the security of data storage in the distributed database is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0019] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;

[0020] Figure 2 is a flow chart of an embodiment of a load balancing method for a distributed database according to the present application;

[0021] Figure 3 is a flow chart of another embodiment of a load balancing method for a distributed database according to the present application;

[0022] Figure 4 It is a structural diagram of an embodiment of a load balancing device for a distributed database according to the present application;

[0023] Figure 5 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0024] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of a load balancing method for a distributed database or a load balancing device for a distributed database of the present application can be applied.

[0027] like Figure 1 As shown, the system architecture 100 may include a master node cluster 101 of a distributed database, a network 102, and a slave node cluster 103 for disaster recovery in the distributed database. The network 102 is used to provide a medium for a communication link between the master node cluster 101 and the slave node cluster 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0028] The master node cluster 101 of the distributed database can store pre-written data, etc. The master node cluster 101 of the distributed database can include n master nodes, where n can be a positive integer. The master node in the master node cluster 101 can be hardware or software. When the master node is a hardware device, it can be an electronic device such as a server or a terminal device for storing data. When the master node is software, it can be installed in an electronic device such as a server or a terminal device. It can be implemented as multiple software or software modules (for example, used to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0029] In a distributed database, the master node in the master node cluster 101 can interact with the slave nodes in the slave node cluster 103 through the network 102 to receive or send data, etc. The slave nodes in the slave node cluster 103 can provide disaster recovery services for the distributed database, such as backing up the data stored in the master node, etc. The slave node cluster 103 of the distributed database may include m slave nodes, where m may be a positive integer.

[0030] It should be noted that the load balancing method for a distributed database provided in the embodiment of the present application is generally executed by a master node in the master node cluster 101, and accordingly, the load balancing device for a distributed database is generally arranged in a master node in the master node cluster.

[0031] It should be noted that the slave nodes in the slave node cluster 102 can be hardware or software. When the slave nodes are hardware, the slave node cluster 102 can be implemented as a distributed server cluster consisting of multiple servers. When the slave nodes are software, the slave node cluster 102 can be implemented as multiple software or software modules (for example, for providing distributed services). No specific limitation is made here.

[0032] It should be understood that Figure 1 The number of master node clusters, networks, and slave node clusters in the embodiment is only for illustration. Any number of master node clusters, networks, and slave node clusters may be provided as required.

[0033] Continue to refer Figure 2 , shows a process 200 of an embodiment of a load balancing method for a distributed database according to the present application. The load balancing method for a distributed database includes the following steps:

[0034] Step 201, in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, select a first master node from the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold to generate a backlog processing queue.

[0035] In this embodiment, the distributed database may include a master node cluster and a slave node cluster (for example Figure 1 Here, the master node cluster may include at least two master nodes, and the slave node cluster may include at least two slave nodes, wherein each slave node in the slave node cluster may synchronize and store the tasks to be synchronized in each master node of the master node cluster through a wired connection or a wireless connection, so as to achieve the disaster recovery effect of the distributed database. The execution subject of the load balancing method of the distributed database (for example Figure 1 When it is detected that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, the execution subject may select a master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master nodes with an increase in the amount of tasks to be synchronized. The execution subject may then determine the selected master node as the first master node, and generate a backlog processing queue for each first master node. It is understandable that if there are unfinished tasks to be synchronized in the master node, and the number of unfinished tasks to be synchronized continues to increase, it can be determined that the master node has an increase in the amount of tasks to be synchronized.

[0036] In some optional implementations of the present embodiment, if the above-mentioned execution subject detects that all the master nodes in the above-mentioned master node cluster have an increase in the amount of tasks to be synchronized, there is no need to generate a backlog processing queue. It can be understood that if all the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, it can be explained that the master-slave cluster synchronization in the distributed database has a problem that cannot be solved by the load balancing method of the distributed database in this application, so there is no need to execute the solution provided in this application. Of course, if the above-mentioned execution subject detects that there is no master node in the above-mentioned master node cluster that has an increase in the amount of tasks to be synchronized, there is no need to execute the solution provided in this application.

[0037] In some optional implementations of the present embodiment, each master node in the above-mentioned master node cluster can construct a synchronization task queue for caching tasks to be synchronized. Then, for any master node in the master node cluster, the above-mentioned execution subject can detect the tasks to be synchronized in the synchronization task queue in the master node. If it is determined that the amount of tasks to be synchronized in the master node continues to increase within a preset time period (for example, within the past hour), it can be determined that the master node has an increase in the amount of tasks to be synchronized. Otherwise, it can be determined that the master node does not have an increase in the amount of tasks to be synchronized.

[0038] In some optional implementations of the present embodiment, the above-mentioned execution subject may further perform the following steps before executing step 201: detecting the tasks to be synchronized of the master nodes in the master node cluster; determining the time difference between the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster this time and the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster last time; in response to determining that the time difference is greater than the third preset threshold, determining whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster. It can be understood that if the above-mentioned time difference is greater than the third preset threshold, it can be explained that the distributed database has not executed the load balancing scheme of the present application for a long time. In this case, executing the load balancing scheme of the present application can solve the problem of data inconsistency caused by slow synchronization of the master-slave cluster; if the above-mentioned time difference is less than or equal to the third preset threshold, it can be explained that the time between the distributed database and the last execution of the load balancing scheme of the present application is short. In order to avoid performance jitter caused by frequent adjustments, the above-mentioned execution subject can temporarily suspend the execution of the load balancing scheme of the present application.

[0039] Step 202 : Select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue.

[0040] In this embodiment, the above execution entities (for example Figure 1 The master node in the master node cluster shown in the figure can detect the amount of tasks to be synchronized of each master node in the master node cluster, and when it is determined that the amount of tasks to be synchronized in the master node is less than the second preset threshold value, the master node is determined as the second master node. Then, a balanced replacement queue is generated for each determined second master node. Among them, the second master node and the above-mentioned first master node are different master nodes in the master node cluster, and the number of second master nodes in the balanced replacement queue can be the same as the number of first master nodes in the above-mentioned backlog processing queue. Alternatively, the number of second master nodes in the balanced replacement queue can be greater than the number of first master nodes in the above-mentioned backlog processing queue. Optionally, the above-mentioned execution body can determine the above-mentioned second master nodes in the master node cluster without the first master node, so as to ensure that the second master node is different from the above-mentioned first master node. It can be understood that the load in each second master node in the balanced replacement queue is usually small, so each second master node can take on more tasks to be synchronized.

[0041] Step 203 , for any first master node in the backlog processing queue, load balancing of the tasks to be synchronized in the first master node can be completed through steps 2031 and 2032 .

[0042] Step 2031: Select a second master node corresponding to the first master node from the balanced replacement queue, and exchange the data partition of the selected second master node with the data partition of the first master node.

[0043] In this embodiment, for any first master node in the backlog queue, the above-mentioned execution subject can select a second master node corresponding to the first master node from the balanced replacement queue. As an example, the second master node corresponding to the first master node can be any second master node in the balanced replacement queue, or the second master node corresponding to the first master node can also be any second master node in the balanced replacement queue whose amount of tasks to be synchronized is less than a preset value. Then, the above-mentioned execution subject can exchange the data partition of the selected second master node corresponding to the first master node with the data partition of the first master node, so that the selected second master node can process the larger amount of tasks to be synchronized in the first master node, and the first master node can process the smaller amount of tasks to be synchronized in the selected second master node, thereby improving the processing speed of the tasks to be synchronized in the first master node, so that the master-slave cluster can be synchronized quickly, and the load balancing effect of each master node is improved.

[0044] In some optional implementations of this embodiment, for each first master node in the backlog processing queue, the execution subject may detect the amount of tasks to be synchronized in each first master node, and determine the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, for example, the first master node with the largest amount of tasks to be synchronized is the first master node A. For each second master node in the balanced replacement queue, the execution subject may detect the amount of tasks to be synchronized in each second master node, and select the second master node with the smallest amount of tasks to be synchronized in the balanced replacement queue, for example, the second master node with the smallest amount of tasks to be synchronized is the second master node B. Afterwards, the execution subject may determine the selected second master node as the second master node corresponding to the first master node, that is, the second master node B may be determined as the second master node corresponding to the first master node A. Then, the execution subject may determine the data partition with the largest total amount of tasks to be synchronized in the first master node, and determine the data partition with the smallest total amount of tasks to be synchronized in the second master node, that is, determine the data partition with the largest total amount of tasks to be synchronized in the first master node A and the data partition with the smallest total amount of tasks to be synchronized in the second master node B. Finally, the execution subject can exchange the determined data partition of the first master node with the data partition of the second master node, that is, the determined data partition of the first master node A can be exchanged with the data partition of the second master node B. This method can further improve the load balancing performance of the distributed database.

[0045] Step 2032: In response to determining that the amount of tasks to be synchronized in the first master node is reduced, the first master node is removed from the backlog processing queue.

[0046] In this embodiment, after exchanging the data partition of the selected second master node with the data partition of the first master node, the execution subject may continue to detect the amount of tasks to be synchronized in the first master node, and remove the first master node from the backlog processing queue when it is determined that the amount of tasks to be synchronized in the first master node has decreased. It is understandable that if it is detected that the amount of tasks to be synchronized in the first master node has decreased, it can be determined that there is no increase in the amount of tasks to be synchronized in the first master node, that is, the first master node is not a master node with a backlog of tasks to be synchronized, and the first master node can be removed from the backlog processing queue at this time.

[0047] In some optional implementations of this embodiment, the execution subject may continuously monitor the number of first master nodes in the backlog processing queue, and when it is determined that the backlog processing queue is empty, the current load balancing operation may be terminated, and the next round of load balancing operation may be started to reselect the first master node in the master node cluster. Optionally, if any first master node is still in the backlog processing queue after a preset number of data partition exchanges, the first master node may be directly removed from the backlog processing queue, forcibly terminating the current load balancing operation.

[0048] The load balancing method and device of the distributed database provided by the above-mentioned embodiment of the present application, in response to detecting that the amount of tasks to be synchronized in some master nodes in the master node cluster increases, can select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue, and then select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, and finally select a second master node corresponding to the first master node from the balanced replacement queue for the first master node in the backlog processing queue, so as to exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue when it is determined that the amount of tasks to be synchronized in the first master node is reduced. In this way, load balancing of tasks to be synchronized in the master node cluster can be achieved, the risk of inconsistent data stored in the master node cluster and the slave node cluster in the distributed database is reduced, and the security of data storage in the distributed database is improved.

[0049] Please continue to refer to Figure 3 , which shows a process 300 of another embodiment of a load balancing method for a distributed database. The process 300 of the load balancing method for a distributed database includes the following steps:

[0050] Step 301, in response to detecting that some master nodes in a master node cluster have an increase in the amount of tasks to be synchronized, determining whether the master node cluster is divided into at least two groups.

[0051] In this embodiment, the master node cluster of the distributed database can be divided into different groups according to functions, etc. The execution subject of the load balancing method of the distributed database (for example Figure 1 When detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, it can continue to determine whether the master node cluster is divided into at least two groups. Here, each group may include at least one master node.

[0052] Step 302: In response to determining that the master node cluster is divided into at least two groups, detecting an increase in the amount of tasks to be synchronized of all master nodes in the at least two groups.

[0053] In this embodiment, after determining that the master node cluster is divided into at least two groups, the execution subject can continue to monitor the growth of the amount of tasks to be synchronized of all master nodes included in each of the at least two groups. Here, the execution subject needs to detect the number of tasks to be synchronized in each master node, and whether the tasks to be synchronized in each master node are increasing and the growth rate.

[0054] Step 303: Determine, from the at least two groups, a first group in which there is an increase in the amount of tasks to be synchronized of some master nodes, and generate a list of master nodes.

[0055] In this embodiment, based on the increase in the amount of tasks to be synchronized of all master nodes in at least two groups detected in step 302, the above-mentioned execution subject can determine the first group from each group. The first group here can be a group in which there is an increase in the amount of tasks to be synchronized of some master nodes in the above-mentioned at least two groups. It can be understood that if the amount of tasks to be synchronized of all master nodes in a group increases, then the group is not the above-mentioned first group; if the amount of tasks to be synchronized of all master nodes in a group does not increase, then the group is not the above-mentioned first group. Then, the above-mentioned execution subject can form a master node list with the master nodes in the first group that have an increase in the amount of tasks to be synchronized.

[0056] Step 304 , selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node list to generate a backlog processing queue.

[0057] In this embodiment, based on the master node list generated in step 303, the execution subject can select a master node whose amount of tasks to be synchronized is greater than a first preset threshold in the master node list, and determine the selected master node as the first master node. Finally, all the first master nodes can be aggregated to generate a backlog processing queue.

[0058] Step 305 , selecting a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue.

[0059] In this embodiment, the execution subject can detect the amount of tasks to be synchronized of each master node in the master node cluster, and when it is determined that the amount of tasks to be synchronized in the master node is less than the second preset threshold, the master node is determined as the second master node. Then, a balanced replacement queue is generated for each determined second master node. Among them, the second master node and the first master node are different master nodes in the master node cluster, and the number of second master nodes in the balanced replacement queue can be the same as the number of first master nodes in the backlog processing queue. Alternatively, the number of second master nodes in the balanced replacement queue can be greater than the number of first master nodes in the backlog processing queue.

[0060] Step 306 , for any first master node in the backlog processing queue, load balancing of the tasks to be synchronized in the first master node can be completed through steps 3061 and 3062 .

[0061] Step 3061: Select a second master node corresponding to the first master node from the balanced replacement queue, and exchange the data partition of the selected second master node with the data partition of the first master node.

[0062] In this embodiment, for any first master node in the backlog queue, the above-mentioned execution subject can select a second master node corresponding to the first master node from the balanced replacement queue. As an example, the second master node corresponding to the first master node can be any second master node in the balanced replacement queue, or the second master node corresponding to the first master node can also be any second master node in the balanced replacement queue whose amount of tasks to be synchronized is less than a preset value. Then, the above-mentioned execution subject can exchange the data partition of the selected second master node corresponding to the first master node with the data partition of the first master node, so that the selected second master node can process the larger amount of tasks to be synchronized in the first master node, and the first master node can process the smaller amount of tasks to be synchronized in the selected second master node, thereby improving the processing speed of the tasks to be synchronized in the first master node, so that the master-slave cluster can be synchronized quickly, and the load balancing effect of each master node is improved.

[0063] Step 3062: In response to determining that the amount of tasks to be synchronized in the first master node is reduced, remove the first master node from the backlog processing queue.

[0064] In this embodiment, after exchanging the data partition of the selected second master node with the data partition of the first master node, the execution subject may continue to detect the amount of tasks to be synchronized in the first master node, and remove the first master node from the backlog processing queue when it is determined that the amount of tasks to be synchronized in the first master node has decreased. It is understandable that if it is detected that the amount of tasks to be synchronized in the first master node has decreased, it can be determined that there is no increase in the amount of tasks to be synchronized in the first master node, that is, the first master node is not a master node with a backlog of tasks to be synchronized, and the first master node can be removed from the backlog processing queue at this time.

[0065] from Figure 3 It can be seen that Figure 2 Compared with the corresponding embodiment, the process 300 of the load balancing method of the distributed database in this embodiment highlights the step of generating a backlog processing queue. Therefore, the solution described in this embodiment can directly filter the groups with an overall increase in the amount of tasks to be synchronized in the group by group in the case of master node cluster grouping, thereby improving the efficiency of generating the backlog processing queue and further improving the effect of load balancing.

[0066] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a load balancing device for a distributed database. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0067] like Figure 4 As shown, the load balancing device 400 of the distributed database of this embodiment includes: a backlog processing queue generation unit 401, a balanced replacement queue generation unit 402, and a first master node removal unit 403. Among them, the backlog processing queue generation unit 401 is configured to select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized; the balanced replacement queue generation unit 402 is configured to select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of first master nodes is less than or equal to the number of second master nodes; the first master node removal unit 403 is configured to select a second master node corresponding to the first master node from the balanced replacement queue for the first master node in the backlog processing queue, exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue in response to determining that the amount of tasks to be synchronized in the first master node is reduced.

[0068] In some optional implementations of the present embodiment, the device 400 also includes: a synchronization task queue construction unit, configured to construct a synchronization task queue in a master node of the master node cluster, wherein the synchronization task queue is used to cache tasks to be synchronized; a unit for determining the growth in the amount of tasks to be synchronized, configured to determine, for a master node in the master node cluster, that there is an increase in the amount of tasks to be synchronized in the synchronization task pair queue of the master node, in response to detecting that the amount of tasks to be synchronized in the master node continues to increase within a preset time period.

[0069] In some optional implementations of the present embodiment, the device 400 also includes: a detection unit, configured to detect the tasks to be synchronized of the master nodes in the master node cluster; a time difference determination unit, configured to determine the time difference between the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster this time and the moment of detecting the tasks to be synchronized of the master nodes in the master node cluster last time; and a master node determination unit, configured to determine whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster in response to determining that the time difference is greater than a third preset threshold.

[0070] In some optional implementations of the present embodiment, the backlog processing queue generation unit 401 is further configured to: in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, determine whether the master node cluster is divided into at least two groups; in response to determining that the master node cluster is divided into at least two groups, detect the increase in the amount of tasks to be synchronized of all master nodes in the at least two groups; determine from the at least two groups a first group where there is an increase in the amount of tasks to be synchronized of some master nodes, and generate a master node list, wherein the master node list is composed of the master nodes with an increase in the amount of tasks to be synchronized in the first group; select the first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node list to generate a backlog processing queue.

[0071] In some optional implementations of the present embodiment, the first master node removal unit 403 is further configured to: for the first master node in the backlog processing queue, in response to determining that the first master node is the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, select the second master node with the smallest backlog of tasks from the balanced replacement queue; determine the selected second master node as the second master node corresponding to the first master node, determine the data partition with the largest amount of tasks to be synchronized in the first master node, and determine the data partition with the smallest amount of tasks to be synchronized in the second master node corresponding to the first master node; and exchange the data partition of the determined first master node with the data partition of the determined second master node.

[0072] In some optional implementations of this embodiment, the device 400 also includes: a reselection unit, configured to reselect a first master node in the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold in response to determining that the backlog processing queue is empty.

[0073] The units described in the device 400 and the reference Figure 2 and Figure 3 Therefore, the operations and features described above for the method are also applicable to the device 400 and the units included therein, and will not be described in detail here.

[0074] Reference below Figure 5 , which shows an electronic device (eg Figure 1 A schematic diagram of the structure of a computer system 500 having a master node (as shown). Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0075] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0076] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0077] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0078] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0079] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0080] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The described units may also be set in a processor, for example, may be described as: a processor includes a backlog processing queue generation unit, a balanced replacement queue generation unit, and a first master node removal unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the backlog processing queue generation unit may also be described as "a unit that selects a first master node from the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold to generate a backlog processing queue in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized."

[0081] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device: in response to detecting that some master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue; select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of first master nodes is less than or equal to the number of second master nodes; for the first master node in the backlog processing queue, select a second master node corresponding to the first master node from the balanced replacement queue, exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue in response to determining that the amount of tasks to be synchronized in the first master node is reduced.

[0082] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A load balancing method for a distributed database, wherein: The distributed database includes a master node cluster and a slave node cluster for disaster recovery, and the method includes: In response to detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, selecting a first master node from the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold to generate a backlog processing queue; Selecting a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of the first master nodes is less than or equal to the number of the second master nodes; For the first master node in the backlog processing queue, a second master node corresponding to the first master node is selected from the balanced replacement queue, and the data partition of the selected second master node is exchanged with the data partition of the first master node. In response to determining that the amount of tasks to be synchronized in the first master node is reduced, the first master node is removed from the backlog processing queue.

2. The method according to claim 1, wherein: The method further comprises: Constructing a synchronization task queue in the master node of the master node cluster, wherein the synchronization task queue is used to cache tasks to be synchronized; For a master node in the master node cluster, in response to detecting that the amount of tasks to be synchronized in the synchronization task pair queue of the master node continues to increase within a preset time period, it is determined that the master node has an increase in the amount of tasks to be synchronized.

3. The method according to claim 1, wherein: In response to detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, before selecting a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue, the method further includes: Detecting tasks to be synchronized of the master nodes in the master node cluster; Determine a time difference between a time of detecting tasks to be synchronized of the master nodes in the master node cluster this time and a time of detecting tasks to be synchronized of the master nodes in the master node cluster last time; In response to determining that the time difference is greater than a third preset threshold, it is determined whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster.

4. The method according to claim 1, wherein: In response to detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, selecting a first master node from the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold to generate a backlog processing queue, comprising: In response to detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, determining whether the master node cluster is divided into at least two groups; In response to determining that the master node cluster is divided into at least two groups, detecting an increase in the amount of tasks to be synchronized of all master nodes in the at least two groups; Determine a first group in which some master nodes have an increased amount of tasks to be synchronized from the at least two groups, and generate a master node list, wherein the master node list is composed of the master nodes in the first group with an increased amount of tasks to be synchronized; A first master node whose amount of tasks to be synchronized is greater than a first preset threshold is selected from the master node list to generate a backlog processing queue.

5. The method according to claim 1, wherein: The step of selecting, for a first master node in the backlog processing queue, a second master node corresponding to the first master node from the balanced replacement queue, and exchanging a data partition of the selected second master node with a data partition of the first master node includes: For the first master node in the backlog processing queue, in response to determining that the first master node is the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, select the second master node with the smallest amount of backlog tasks from the balanced replacement queue; determine the selected second master node as the second master node corresponding to the first master node, determine the data partition with the largest amount of tasks to be synchronized in the first master node, and determine the data partition with the smallest amount of tasks to be synchronized in the second master node corresponding to the first master node; exchange the data partition of the determined first master node with the data partition of the determined second master node.

6. The method according to any one of claims 1 to 5, wherein: The method further comprises: In response to determining that the backlog processing queue is empty, a first master node having an amount of tasks to be synchronized greater than a first preset threshold is reselected in the master node cluster.

7. A load balancing device for a distributed database, wherein: The distributed database includes a master node cluster and a slave node cluster for disaster recovery, and the device includes: a backlog processing queue generating unit, configured to select a first master node whose amount of tasks to be synchronized is greater than a first preset threshold from the master node cluster to generate a backlog processing queue in response to detecting that the amount of tasks to be synchronized increases in some master nodes in the master node cluster; a balanced replacement queue generating unit, configured to select a second master node whose amount of tasks to be synchronized is less than a second preset threshold from the master node cluster to generate a balanced replacement queue, wherein the number of the first master nodes is less than or equal to the number of the second master nodes; The first master node removal unit is configured to select a second master node corresponding to the first master node from the balanced replacement queue for the first master node in the backlog processing queue, exchange the data partition of the selected second master node with the data partition of the first master node, and remove the first master node from the backlog processing queue in response to determining that the amount of tasks to be synchronized in the first master node has decreased.

8. The device according to claim 7, wherein: The device also includes: A synchronization task queue construction unit is configured to construct a synchronization task queue in a master node of the master node cluster, wherein the synchronization task queue is used to cache tasks to be synchronized; The unit for determining the growth of the amount of tasks to be synchronized is configured to determine, for a master node in the master node cluster, that the amount of tasks to be synchronized in the synchronization task pair queue of the master node has increased continuously within a preset time period, in response to detecting that the amount of tasks to be synchronized in the master node's synchronization task pair queue has continued to increase within a preset time period.

9. The device according to claim 7, wherein: The device also includes: A detection unit, configured to detect tasks to be synchronized of a master node in the master node cluster; a time difference determining unit, configured to determine a time difference between a time of detecting tasks to be synchronized of master nodes in the master node cluster this time and a time of detecting tasks to be synchronized of master nodes in the master node cluster last time; The master node determination unit is configured to determine whether there is an increase in the amount of tasks to be synchronized of some master nodes in the master node cluster in response to determining that the time difference is greater than a third preset threshold.

10. The device according to claim 7, wherein: The backlog processing queue generating unit is further configured to: In response to detecting that some of the master nodes in the master node cluster have an increase in the amount of tasks to be synchronized, determining whether the master node cluster is divided into at least two groups; In response to determining that the master node cluster is divided into at least two groups, detecting an increase in the amount of tasks to be synchronized of all master nodes in the at least two groups; Determine a first group in which some master nodes have an increased amount of tasks to be synchronized from the at least two groups, and generate a master node list, wherein the master node list is composed of the master nodes in the first group with an increased amount of tasks to be synchronized; A first master node whose amount of tasks to be synchronized is greater than a first preset threshold is selected from the master node list to generate a backlog processing queue.

11. The device according to claim 7, wherein: The first master node removal unit is further configured to: For the first master node in the backlog processing queue, in response to determining that the first master node is the first master node with the largest amount of tasks to be synchronized in the backlog processing queue, selecting a second master node with the smallest amount of backlog tasks from the balanced replacement queue; Determine the selected second master node as the second master node corresponding to the first master node, determine the data partition with the largest amount of tasks to be synchronized in the first master node, and determine the data partition with the smallest amount of tasks to be synchronized in the second master node corresponding to the first master node; The data partition of the determined first master node and the data partition of the determined second master node are exchanged.

12. The device according to any one of claims 7 to 11, wherein: The device also includes: The reselection unit is configured to reselect a first master node in the master node cluster whose amount of tasks to be synchronized is greater than a first preset threshold in response to determining that the backlog processing queue is empty.

13. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

14. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images