A load balancing method, device, electronic device and storage medium based on improved consistent hash ring

Abstract

The present application discloses a load balancing method and device based on an improved consistent hash ring, which is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes. The method includes: obtaining data to be stored and determining a hash value of the data to be stored; determining the first node that is closest to the hash value of the data to be stored in a clockwise direction on a created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a symmetric point on the initial hash ring; if the candidate node is determined to be any server node, the data to be stored is stored in the server node; if the candidate node is determined to be a virtual node, the data to be stored is stored in the server node corresponding to the virtual node.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to a load balancing method and device based on an improved consistent hash ring. Background Art

[0002] In related technologies, distributed clusters often use consistent hashing algorithms for data storage. The consistent hashing algorithm organizes the entire hash value space into a virtual ring, which is Figure 1 The hash ring shown is organized in a clockwise direction, representing 0 to 2 32 For a value of -1, the point directly above the hash ring represents 0, the first point to the right of 0 represents 1, and so on, until 2 32 -1, 0 and 2 32 -1 overlap, perform hash calculation on each server node in the distributed cluster, and determine the position of each server node on the hash ring according to the hash value corresponding to each server node, such as Figure 2 As shown, suppose there are three server nodes: A, B, and C. Figure 2 The positions of A, B, and C on the hash ring are shown. In this way, when storing or accessing data, the corresponding target server node can be located in the following way: hash the data to obtain the hash value of the data, determine the position of the hash value of the data on the hash ring, and the first server node encountered in a clockwise direction from this position is the target server node. The data is stored in the target server node or the data is accessed from the target server node.

[0003] However, when the server nodes are too concentrated on the hash ring, data skew will occur due to the uneven distribution of server nodes, so that the data to be stored is concentrated on some server nodes, while other server nodes store too little data, causing load imbalance problems, such as Figure 2 The data stored in server node A will be more than the data stored in server node B. Therefore, the consistent hash algorithm introduces a virtual node mechanism to virtualize each server node into multiple virtual nodes. When the hash value of the data to be stored falls within the range of the virtual node, the data to be stored is stored in the server node corresponding to the virtual node. Figure 3 As shown, assuming that Figure 2 The server node A in the virtualization is four virtual nodes: A#1, A#2, A#3, A#4, the server node B is four virtual nodes: B#1, B#2, B#3, B#4, the server node C is four virtual nodes: C#1, C#2, C#3, C#4, Figure 3The position of each virtual node on the hash ring is shown in FIG. 1 . When the hash value of the data to be stored is located between virtual nodes B#4 and A#1, or between B#1 and A#2, or between C#3 and A#3, or between C#4 and A#4 on the hash ring, the data to be stored is stored in server node A. When the hash value of the data to be stored is located between virtual nodes C#2 and B#1, or between A#3 and B#2, or between C#1 and B#3, or between A#4 and A#4 on the hash ring, the data to be stored is stored in server node A. When the hash value of the data to be stored is between B#2 and C#1, or between A#1 and C#2, or between A#2 and C#3, or between B#3 and C#4 on the hash ring, the data to be stored is stored in server node C. In this way, the data is evenly distributed on server nodes A, B, and C as much as possible to achieve load balancing. In the prior art, more virtual nodes are usually set to achieve better load balancing effects. However, when there are a large number of server nodes in a distributed cluster, or the number of server nodes increases or decreases, the redistribution of data will affect more server nodes due to drastic changes in the number of virtual nodes. For example, when a server node is abnormal, the data of the server node needs to be migrated to the server node corresponding to the next virtual node in the clockwise direction of the hash change of its virtual node, still with Figure 3 For example, suppose that server node A is abnormal. Since the next node of virtual node A#1 is C#2, the next node of A#2 is C#3, the next node of A#3 is B#2, and the next node of A#4 is B#4, the data on server node A needs to be migrated to B and C. During the data migration process, server nodes B and C are affected. When the number of server nodes in a distributed cluster is very large, data migration will seriously affect the overall availability of the distributed system. The data migration process will affect the query hit rate of the abnormal server node data. In addition, when there are too many virtual nodes (sometimes up to tens of thousands), it will affect the data storage performance and data synchronization performance, and reduce the data storage efficiency and data synchronization efficiency. Summary of the invention

[0004] In order to solve the problem that the existing load balancing method based on consistent hash ring cannot balance the overall availability and load balancing of the distributed system, and affects the data storage efficiency and data synchronization efficiency, the embodiment of the present application provides a load balancing method and device based on an improved consistent hash ring.

[0005] In a first aspect, an embodiment of the present application provides a load balancing method based on an improved consistent hash ring, which is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes, and the method includes:

[0006] Acquire the data to be stored, and determine the hash value of the data to be stored;

[0007] Determine the first node that is closest to the hash value of the data to be stored in the clockwise direction on the created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a symmetric point on the initial hash ring;

[0008] If it is determined that the candidate node is any server node, storing the data to be stored in the server node;

[0009] If it is determined that the candidate node is a virtual node, the data to be stored is stored in a server node corresponding to the virtual node.

[0010] In a possible implementation manner, when creating a virtual node corresponding to each server node, the method further includes:

[0011] If it is determined that there are other server nodes at the symmetric point of any server node on the initial hash ring, then determine the arc between the symmetric point of any server node and the next node in the clockwise direction of the initial hash ring;

[0012] Move any server node onto the arc.

[0013] In a possible implementation manner, moving any server node to the arc specifically includes:

[0014] Move any server node to the midpoint of the arc.

[0015] In a possible implementation manner, the hash value corresponding to any server node after the move is determined in the following manner:

[0016] According to the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, the hash value corresponding to any server node after the move is determined.

[0017] In a possible implementation, creating an initial hash ring based on the hash value corresponding to each server node specifically includes:

[0018] Obtaining IP information of each server node;

[0019] Obtaining a corresponding hash value of each server node based on the IP information of each server node and a preset hash function;

[0020] The starting point is 0 and the end point is 2 32 A hash ring is constructed, and a corresponding server node is constructed at a point on the hash ring corresponding to the hash value corresponding to each server node, to obtain the initial hash ring.

[0021] In a possible implementation manner, creating a corresponding virtual node for each server node at a respective symmetric point on the initial hash ring specifically includes:

[0022] Determine the hash value corresponding to the corresponding virtual node according to the hash value corresponding to each server node;

[0023] A corresponding virtual node is constructed at a point on the initial hash ring corresponding to the hash value corresponding to each virtual node.

[0024] In a possible implementation, the hash value corresponding to the corresponding virtual node is determined according to the hash value corresponding to each server node:

[0025] The hash value corresponding to the virtual node corresponding to each server node is calculated by the following formula:

[0026]

[0027] Among them, H′ i Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N;

[0028] H i Indicates the hash value corresponding to the i-th server node.

[0029] In a possible implementation, determining the hash value corresponding to any server node after the move according to the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring specifically includes:

[0030] The hash value corresponding to any server node after the move is calculated by the following formula:

[0031]

[0032] Among them, H moveIndicates the hash value corresponding to any server node after the move;

[0033] H represents the hash value corresponding to any server node;

[0034] H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

[0035] In a possible implementation, the method further includes:

[0036] If a new server node is added to the distributed cluster, a virtual node is created for the new server node to update the target hash ring;

[0037] If any server node in the distributed cluster is abnormal, the data stored in the server node is migrated to the next server node in the clockwise direction of the target hash ring; and

[0038] The virtual node corresponding to any server node is deleted from the target hash ring.

[0039] In a second aspect, an embodiment of the present application provides a load balancing device based on an improved consistent hash ring, which is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes, and the device includes:

[0040] An acquisition unit, used to acquire the data to be stored and determine the hash value of the data to be stored;

[0041] A determination unit, used to determine the first node that is closest to the hash value of the data to be stored in the clockwise direction on the created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a respective symmetric point on the initial hash ring;

[0042] A first storage unit, configured to store the data to be stored in any server node if it is determined that the candidate node is any server node;

[0043] The second storage unit is used to store the data to be stored in a server node corresponding to the virtual node if it is determined that the candidate node is a virtual node.

[0044] In a possible implementation manner, the device further includes:

[0045] A processing unit is used to, when creating a corresponding virtual node for each server node, determine an arc between the symmetrical point of any server node and the next node in the clockwise direction of the initial hash ring if it is determined that there are other server nodes at the symmetrical point of any server node on the initial hash ring; and move any server node to the arc.

[0046] In a possible implementation, the processing unit is specifically configured to move any one of the server nodes to the midpoint of the arc.

[0047] In a possible implementation, the processing unit is specifically used to determine the hash value corresponding to any server node after moving in the following manner: based on the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, determine the hash value corresponding to any server node after moving.

[0048] In a possible implementation manner, the device further includes:

[0049] The first creation unit is used to obtain the IP information of each server node; calculate the corresponding hash value of each server node based on the IP information of each server node and a preset hash function; take the starting point as 0 and the end point as 2 32 A hash ring is constructed, and a corresponding server node is constructed at a point on the hash ring corresponding to the hash value corresponding to each server node, to obtain the initial hash ring.

[0050] In a possible implementation, the first creation unit is specifically configured to determine a hash value corresponding to a corresponding virtual node according to a hash value corresponding to each server node; and to construct a corresponding virtual node at a point on the initial hash ring corresponding to the hash value corresponding to each virtual node.

[0051] In a possible implementation manner, the first creation unit is specifically configured to calculate a hash value corresponding to the virtual node corresponding to each server node by using the following formula:

[0052]

[0053] Among them, H′ i Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N;

[0054] H i Indicates the hash value corresponding to the i-th server node.

[0055] In a possible implementation manner, the processing unit is specifically configured to calculate a hash value corresponding to any server node after the move by using the following formula:

[0056]

[0057] Among them, H move Indicates the hash value corresponding to any server node after the move;

[0058] H represents the hash value corresponding to any server node;

[0059] H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

[0060] In a possible implementation manner, the device further includes:

[0061] A second creation unit is configured to create a virtual node for a newly added server node if a server node is added to the distributed cluster, so as to update the target hash ring;

[0062] A migration unit, configured to migrate data stored in any server node to the next server node in the clockwise direction of the target hash ring if any server node in the distributed cluster is abnormal;

[0063] A deleting unit is used to delete the virtual node corresponding to any server node from the target hash ring.

[0064] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the load balancing method based on the improved consistent hash ring described in the present application is implemented.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the load balancing method based on an improved consistent hash ring described in the present application.

[0066] The beneficial effects of this application are as follows:

[0067] The load balancing solution based on the improved consistent hash ring provided in the embodiment of the present application is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes, obtains data to be stored, determines the hash value of the data to be stored, and determines that the first node whose hash value of the data to be stored is closest to the candidate node in the clockwise direction on the created target hash ring is a candidate node, wherein the target hash ring is created in the following manner: an initial hash ring is created based on the hash value corresponding to each server node, and a corresponding virtual node is created for each server node at a symmetric point on the initial hash ring to obtain a target hash ring. If the candidate node is determined to be any server node, the data to be stored is stored in the server node. If the candidate node is determined to be a virtual node, the data to be stored is stored in the server node corresponding to the virtual node. Compared with the existing consistent hash algorithm that virtualizes multiple virtual nodes from a server node, the embodiment of the present application targets each server in a distributed cluster (i.e., a distributed system). A virtual node is created for each node respectively, and the position of the virtual node of each server node is set at the symmetric point of the server node on the hash ring according to the linear symmetry characteristics of the ring. An improved consistent hash ring is generated by each server node and its corresponding virtual node. Data is stored based on the improved consistent hash ring. Since the virtual nodes of each server node correspond to their respective symmetric points on the hash ring, the arrangement order of each server node can be guaranteed to be consistent with the arrangement order of the virtual nodes corresponding to each server node. When a server node is abnormal, the data on the server node only needs to be migrated to the next server node in the clockwise direction on the hash ring without affecting other server nodes. Similarly, when a new server node is added, it only affects the data storage of an adjacent server node, thereby improving the overall availability and high scalability of the distributed system, improving the data storage efficiency and data synchronization efficiency, and balancing the performance between load balancing and the overall availability of the distributed system.

[0068] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0070] Figure 1This is one of the hash ring diagrams;

[0071] Figure 2 This is the second diagram of the hash ring;

[0072] Figure 3 This is the third hash ring diagram;

[0073] Figure 4 A schematic diagram of an application scenario of a load balancing method based on an improved consistent hash ring provided in an embodiment of the present application;

[0074] Figure 5 A schematic diagram of the implementation process of the load balancing method based on the improved consistent hash ring provided in the embodiment of the present application;

[0075] Figure 6 A schematic diagram of the implementation process of creating a target hash ring provided in an embodiment of the present application;

[0076] Figure 7 A schematic diagram of the implementation process of creating an initial hash ring provided in an embodiment of the present application;

[0077] Figure 8 An example diagram of an initial hash ring provided in an embodiment of the present application;

[0078] Fig. 9 A schematic diagram of an implementation process for creating a corresponding virtual node for each server node provided in an embodiment of the present application;

[0079] Fig.10 Provided for the embodiments of this application Figure 8 Schematic diagram of the target hash ring corresponding to the initial hash ring shown in;

[0080] Fig.11 Another target hash ring example diagram provided in an embodiment of the present application;

[0081] Fig.12 Another example diagram of a hash ring provided in an embodiment of the present application;

[0082] Fig.13 Another example diagram of a target hash ring provided in an embodiment of the present application;

[0083] Fig.14 A schematic diagram of the structure of a load balancing device based on an improved consistent hash ring provided in an embodiment of the present application;

[0084] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to solve the problems in the background technology, the embodiments of the present application provide a load balancing method and device based on an improved consistent hash ring.

[0086] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0087] First reference Figure 4 , which is a schematic diagram of an application scenario of the load balancing method based on the improved consistent hash ring provided in an embodiment of the present application, which is a distributed cluster (distributed system) 100 provided in an embodiment of the present application, and the distributed cluster 100 includes a control node 101 and at least two server nodes 102: server nodes 1 to N, N is an integer greater than or equal to 2, the control node 101 creates an initial hash ring based on the hash values ​​corresponding to the server nodes 1 to N, and creates a corresponding virtual node for each of the server nodes 1 to N at the symmetric point on the initial hash ring, and obtains a target hash ring containing 2N nodes, and performs distributed data storage or data migration based on the target hash ring, and provides data query, etc.

[0088] The control node 101 may be a server or a terminal device, which is not limited in the embodiment of the present application.

[0089] The server node 102 may be an independent physical server or a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, and a cloud storage.

[0090] Based on the above application scenarios, the following will refer to the attached Figures 5 to 13 The exemplary embodiments of the present application are described in more detail. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in any way. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0091] like Figure 5 As shown, it is a schematic diagram of the implementation process of the load balancing method based on the improved consistent hash ring provided in the embodiment of the present application. The load balancing method based on the improved consistent hash ring can be applied to the above-mentioned distributed cluster 100, and specifically can include the following steps:

[0092] S21. Acquire data to be stored, and determine a hash value of the data to be stored.

[0093] In specific implementation, after obtaining the data to be stored, the hash value of the data to be stored can be calculated according to the following formula:

[0094] H data = hash(data)%2 32

[0095] Among them, H data Indicates the hash value of the data to be stored;

[0096] data indicates the data to be stored.

[0097] During implementation, the hash value of the data to be stored is calculated using the same hash function (hash()) used to calculate the hash value corresponding to the server node.

[0098] S22. Determine the first node that is closest to the hash value of the data to be stored in the created target hash ring in the clockwise direction as a candidate node.

[0099] In specific implementation, the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at the respective symmetric point on the initial hash ring.

[0100] Specifically, you can follow the following Figure 6 The process shown creates a target hash ring and may include the following steps:

[0101] S31. Create an initial hash ring based on the hash value corresponding to each server node.

[0102] When implementing it specifically, you can follow the following steps: Figure 7 The process shown creates the initial hash ring, including the following steps:

[0103] S41. Obtain IP information of each server node.

[0104] During specific implementation, the IP address of each server node is obtained.

[0105] S42: Obtain the corresponding hash value of each server node based on the IP information of each server node and a preset hash function.

[0106] In specific implementation, the hash ring value space is set to: 0~2 32 -1, you can hash the IP address of the server node to get the corresponding hash value, and then compare the hash value to 2 32 Modulo the hash value of the server node to get its position on the hash ring.

[0107] Specifically, the corresponding hash value of each server node can be calculated by the following formula:

[0108] H i =hash(IP i )%2 32

[0109] Among them, H i represents the hash value of the i-th server node, i = 1, 2, ..., N;

[0110] IP i Represents the IP address of the i-th server node.

[0111] The hash function used when calculating the hash value of a server node can be selected at will, and the embodiments of the present application do not limit this. In addition to calculating the hash value corresponding to the server node based on the IP address of the server node, other information of the server node, such as the name of the server node, can also be used to calculate the hash value corresponding to the server node, and the embodiments of the present application do not limit this.

[0112] S43, starting point is 0, end point is 2 32 Construct a hash ring, and construct the corresponding server node at the point on the hash ring corresponding to the hash value of each server node to obtain the initial hash ring.

[0113] In the specific implementation, the number of server nodes in the distributed cluster N=3 is taken as an example for explanation, including server nodes A, B, and C. The positions of the hash values ​​of A, B, and C on the hash ring are as follows: Figure 8 As shown, the hash ring of server nodes A, B, and C is marked, that is, the initial hash ring created.

[0114] S32. Create a corresponding virtual node for each server node at its respective symmetric point on the initial hash ring to obtain a target hash ring.

[0115] In specific implementation, if it is determined that there are no other server nodes at the symmetric points of the server node on the initial hash ring, that is, the symmetric points of the server node on the initial hash ring do not coincide with other server nodes, then a corresponding virtual node is created for each server node at the symmetric points on the initial hash ring; if it is determined that there are other server nodes at the symmetric points on the initial hash ring, then after creating a virtual node for the server node that does not have other server nodes at the symmetric points on the initial hash ring, move the server node clockwise to a certain position, and then create a corresponding virtual node for the moved server node at the symmetric point on the initial hash ring. The symmetric points of the server nodes are linear symmetric points with the center of the hash ring as the center point.

[0116] Specifically, the any server node can be moved in the following manner: determine the arc between the symmetric point between the any server node and the next node in the clockwise direction of the initial hash ring, and move the any server node to the arc. Among them, the any server node and the next node in the clockwise direction of the initial hash ring can be physical server nodes or virtual nodes. The any server node can be moved to the midpoint of the arc determined above to achieve a better load balancing effect.

[0117] Specifically, you can follow the following Fig. 9 The process shown creates a corresponding virtual node for each server node, including the following steps:

[0118] S51. Determine the hash value corresponding to the corresponding virtual node according to the hash value corresponding to each server node.

[0119] In specific implementation, the hash value corresponding to the virtual node corresponding to each server node can be calculated by the following formula:

[0120]

[0121] Among them, H′ i Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N;

[0122] H i Indicates the hash value corresponding to the i-th server node.

[0123] For any server node after the above move, the hash value corresponding to any server node after the move is determined in the following manner: based on the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, the hash value corresponding to any server node after the move is determined.

[0124] Specifically, the hash value corresponding to any server node after the move can be calculated by the following formula:

[0125]

[0126] Among them, H move Indicates the hash value corresponding to any server node after the move;

[0127] H represents the hash value corresponding to any server node;

[0128] H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

[0129] The above formula is the situation where any server node is moved to the midpoint of the arc between the any server node and the next node of the any server node in the clockwise direction of the initial hash ring. In the embodiment of the present application, the position of the any server node on the arc can be set according to actual conditions, and the embodiment of the present application is not limited to this.

[0130] S52: Construct a corresponding virtual node at a point on the initial hash ring corresponding to the hash value of each virtual node.

[0131] In specific implementation, Figure 8 Taking the initial hash ring shown in the figure as an example, the virtual nodes created for server nodes A, B, and C are: A#, B#, and C# respectively. Fig.10 The hash ring shown is Figure 8 The corresponding target hash ring, from Fig.10 From the arrangement order of each server node and its corresponding virtual node, it can be seen that the arrangement order of nodes is always consistent along the clockwise direction of the target hash ring. Assuming that server node A is the starting point, the arrangement order of each node is: A, B, C#; A#, B#, C. If server node B is the starting point, the arrangement order of each node is: B, C#, A#; B#, C, A. If server node C is the starting point, the arrangement order of each node is: C, A, B; C#, A#, B#. Fig.10 In the example shown in , server nodes A, B, and C are all distributed on the semi-arc of the hash ring, which belongs to the situation where the server nodes are unevenly distributed. Here is another example: Fig.11 As shown in Figure 1, server nodes A, B, and C are evenly distributed on the hash ring. After creating corresponding virtual nodes A#, B#, and C# for each server node, the target hash ring is Fig.11As shown in the figure, assuming that server node A is the starting point, the order of the nodes is: A, C#, B; A#, C, B#. If server node B is the starting point, the order of the nodes is: B, A#, C; B#, A, C#. If server node C is the starting point, the order of the nodes is: C, B#, A; C#, B, A#. It can be seen that along the clockwise direction of the target hash ring, the order of the nodes remains consistent. Assume that server node C already exists at the symmetrical point of server node A on the hash ring, such as Fig.12 As shown, A can be moved to the midpoint of the arc between it and its next node B. A virtual node A# is created at the symmetric point of the hash ring for the moved server node A, and a virtual node C# is created for the server node C at the position before A moves. The target hash ring after the creation is as follows: Fig.13 As shown. The position of server node C can also be moved to the arc between C and B# without moving A, which is not limited in the present embodiment. Similarly, the order of nodes in the created target hash ring remains consistent. When determining the candidate node corresponding to the data to be stored, for example Fig.10 In the example, assuming that the hash value of the data to be stored falls on the arc between virtual node A# and virtual node B#, virtual node B# is determined as a candidate node. assuming that the hash value of the data to be stored falls on the arc between virtual node B# and server node C, server node C is determined as a candidate node.

[0132] During implementation, if it is determined that the candidate node is any (physical) server node, step S23 is executed; if it is determined that the candidate node is a virtual node, step S24 is executed.

[0133] S23: If it is determined that the candidate node is any server node, the data to be stored is stored in the server node.

[0134] Continuing with the above example, assuming that the hash value of the data to be stored falls on the arc between virtual node B# and server node C, the data to be stored will be stored in server node C.

[0135] S24: If it is determined that the candidate node is a virtual node, the data to be stored is stored in the server node corresponding to the virtual node.

[0136] Continuing with the above example, assuming that the hash value of the data to be stored falls on the arc between virtual node A# and virtual node B#, the data to be stored is stored on server node B corresponding to virtual node B#.

[0137] Similarly, when a user queries data from a distributed cluster, the server node where the data to be queried is located can be determined according to the above process for storing data, which will not be elaborated here.

[0138] In one implementation, if a new server node is added to the distributed cluster, a virtual node is created for the new server node to update the target hash ring.

[0139] In one embodiment, if any server node in the distributed cluster has an abnormality, the data stored in the server node is migrated to the next server node in the clockwise direction of the target hash ring, and the virtual node corresponding to the server node is deleted from the target hash ring.

[0140] In specific implementation, if any server node in the distributed cluster has an abnormality, if the next node of any server node in the clockwise direction of the target hash ring is a physical server node, then the server node is migrated to the physical server node; if the next node of any server node in the clockwise direction of the target hash ring is a virtual node, then the data stored in the server node is migrated to the server node corresponding to the virtual node.

[0141] Continuing with the above example, if Fig.10 If an exception occurs in server node A, the data stored in server node A is migrated to server node B, and A's virtual node A# is deleted.

[0142] Since each virtual node essentially indicates its corresponding server node, the order of nodes remains consistent along the clockwise direction of the target hash ring. If a server node is abnormal, the data stored on the abnormal server node only needs to be migrated to the next server node in the clockwise direction of the target hash ring, without having to migrate to other server nodes. Thus, the overall availability and high scalability of the distributed system can be achieved while achieving load balancing as much as possible, and improving data storage efficiency. Adding a new server node or an abnormality in a server node will only affect its next server node, but not other server nodes, improving data synchronization efficiency.

[0143] Based on the same inventive concept, an embodiment of the present application also provides a load balancing device based on an improved consistent hash ring. Since the principle of solving the problem by the above-mentioned load balancing device based on the improved consistent hash ring is similar to the above-mentioned load balancing method based on the improved consistent hash ring, the implementation of the above-mentioned device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0144] like Fig.14As shown, it is a structural diagram of a load balancing device based on an improved consistent hash ring provided in an embodiment of the present application, which is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes, and the load balancing device based on the improved consistent hash ring includes:

[0145] An acquisition unit 61 is used to acquire data to be stored and determine a hash value of the data to be stored;

[0146] A determination unit 62 is used to determine that the first node that is closest to the hash value of the data to be stored in the clockwise direction on the created target hash ring is a candidate node, wherein the target hash ring is created by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a respective symmetric point on the initial hash ring;

[0147] A first storage unit 63, configured to store the data to be stored in any server node if it is determined that the candidate node is any server node;

[0148] The second storage unit 64 is configured to store the data to be stored in a server node corresponding to the virtual node if it is determined that the candidate node is a virtual node.

[0149] In a possible implementation manner, the device further includes:

[0150] A processing unit is used to, when creating a corresponding virtual node for each server node, determine an arc between the symmetrical point of any server node and the next node in the clockwise direction of the initial hash ring if it is determined that there are other server nodes at the symmetrical point of any server node on the initial hash ring; and move any server node to the arc.

[0151] In a possible implementation, the processing unit is specifically configured to move any one of the server nodes to the midpoint of the arc.

[0152] In a possible implementation, the processing unit is specifically used to determine the hash value corresponding to any server node after moving in the following manner: based on the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, determine the hash value corresponding to any server node after moving.

[0153] In a possible implementation manner, the device further includes:

[0154] The first creation unit is used to obtain the IP information of each server node; calculate the corresponding hash value of each server node based on the IP information of each server node and a preset hash function; take the starting point as 0 and the end point as 2 32 A hash ring is constructed, and a corresponding server node is constructed at a point on the hash ring corresponding to the hash value corresponding to each server node, to obtain the initial hash ring.

[0155] In a possible implementation, the first creation unit is specifically configured to determine a hash value corresponding to a corresponding virtual node according to a hash value corresponding to each server node; and to construct a corresponding virtual node at a point on the initial hash ring corresponding to the hash value corresponding to each virtual node.

[0156] In a possible implementation manner, the first creation unit is specifically configured to calculate a hash value corresponding to the virtual node corresponding to each server node by using the following formula:

[0157]

[0158] Among them, H′ i Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N;

[0159] H i Indicates the hash value corresponding to the i-th server node.

[0160] In a possible implementation manner, the processing unit is specifically configured to calculate a hash value corresponding to any server node after the move by using the following formula:

[0161]

[0162] Among them, H move Indicates the hash value corresponding to any server node after the move;

[0163] H represents the hash value corresponding to any server node;

[0164] H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

[0165] In a possible implementation manner, the device further includes:

[0166] A second creation unit is configured to create a virtual node for a newly added server node if a server node is added to the distributed cluster, so as to update the target hash ring;

[0167] A migration unit, configured to migrate data stored in any server node to the next server node in the clockwise direction of the target hash ring if any server node in the distributed cluster is abnormal;

[0168] A deleting unit is used to delete the virtual node corresponding to any server node from the target hash ring.

[0169] Based on the same technical concept, the present application embodiment also provides an electronic device 700, referring to Fig.15 As shown, the electronic device 700 is used to implement the load balancing method based on the improved consistent hash ring described in the above method embodiment. The electronic device 700 of this embodiment may include: a memory 701, a processor 702, and a computer program stored in the memory and executable on the processor, such as a load balancing program based on the improved consistent hash ring. When the processor executes the computer program, the steps in the above load balancing method embodiments based on the improved consistent hash ring are implemented, such as Figure 5 Step S21 is shown.

[0170] The specific connection medium between the memory 701 and the processor 702 is not limited in the embodiment of the present application. Fig.15 In the embodiment, the memory 701 and the processor 702 are connected via a bus 703. The bus 703 is Fig.15 The connection between other components is shown by bold lines, which is only for schematic illustration and is not intended to be limiting. The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0171] The memory 701 may be a volatile memory, such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 701 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 701 may be a combination of the above memories.

[0172] Processor 702 is used to implement a load balancing method based on an improved consistent hash ring provided in the present application.

[0173] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.

[0174] In some possible implementations, various aspects of the load balancing method based on an improved consistent hash ring provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the load balancing method based on an improved consistent hash ring according to various exemplary implementations of the present application described above in this specification.

[0175] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0176] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0180] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A load balancing method based on an improved consistent hash ring, characterized in that: Applied to a distributed cluster, the distributed cluster includes at least two server nodes, the method includes: Acquire the data to be stored, and determine the hash value of the data to be stored; Determine the first node that is closest to the hash value of the data to be stored in the clockwise direction on the created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a symmetric point on the initial hash ring; If it is determined that the candidate node is any server node, storing the data to be stored in the server node; If it is determined that the candidate node is a virtual node, the data to be stored is stored in a server node corresponding to the virtual node.

2. The method according to claim 1, characterized in that When creating a virtual node corresponding to each server node, the method further includes: If it is determined that there are other server nodes at the symmetric point of any server node on the initial hash ring, then determine the arc between any server node and the next node in the clockwise direction of the initial hash ring; Move any server node to the midpoint of the arc.

3. The method according to claim 2, characterized in that The hash value corresponding to any server node after the move is determined in the following manner: According to the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, the hash value corresponding to any server node after the move is determined.

4. The method according to claim 1, characterized in that Create an initial hash ring based on the hash value corresponding to each server node, including: Obtaining IP information of each server node; Obtaining a corresponding hash value of each server node based on the IP information of each server node and a preset hash function; The starting point is 0 and the end point is 2 32 -1 builds a hash ring, and builds a corresponding server node at a point on the hash ring corresponding to the hash value corresponding to each server node to obtain the initial hash ring.

5. The method according to claim 4, characterized in that Creating a corresponding virtual node for each server node at a respective symmetric point on the initial hash ring, specifically including: Determine the hash value corresponding to the corresponding virtual node according to the hash value corresponding to each server node; A corresponding virtual node is constructed at a point on the initial hash ring corresponding to the hash value corresponding to each virtual node.

6. The method according to claim 5, characterized in that Determine the hash value corresponding to the corresponding virtual node according to the hash value corresponding to each server node: The hash value corresponding to the virtual node corresponding to each server node is calculated by the following formula: Among them, H i ′ Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N; H i Indicates the hash value corresponding to the i-th server node.

7. The method according to claim 3, characterized in that Determining the hash value corresponding to the moved any server node according to the hash value of the any server node and the hash value of the next node in the clockwise direction of the initial hash ring, specifically includes: The hash value corresponding to any server node after the move is calculated by the following formula: Among them, H move Indicates the hash value corresponding to any server node after the move; H represents the hash value corresponding to any server node; H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

8. The method according to claim 1, characterized in that Also includes: If a new server node is added to the distributed cluster, a virtual node is created for the new server node to update the target hash ring; If any server node in the distributed cluster is abnormal, the data stored in the server node is migrated to the next server node in the clockwise direction of the target hash ring; as well as The virtual node corresponding to any server node is deleted from the target hash ring.

9. A load balancing device based on an improved consistent hash ring, characterized in that: Applied to a distributed cluster, the distributed cluster includes at least two server nodes, and the device includes: An acquisition unit, used to acquire the data to be stored and determine the hash value of the data to be stored; A determination unit, used to determine the first node that is closest to the hash value of the data to be stored in the clockwise direction on the created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a respective symmetric point on the initial hash ring; A first storage unit, configured to store the data to be stored in any server node if it is determined that the candidate node is any server node; The second storage unit is used to store the data to be stored in a server node corresponding to the virtual node if it is determined that the candidate node is a virtual node.

10. The device according to claim 9, characterized in that Also includes: A processing unit is used to, when creating a corresponding virtual node for each server node, determine an arc between any server node and the next node in the clockwise direction of the initial hash ring if it is determined that there are other server nodes at a symmetrical point on the initial hash ring for any server node; and move any server node to the midpoint of the arc.

11. The device according to claim 10, characterized in that The processing unit is specifically used to determine the hash value corresponding to any server node after moving in the following manner: according to the hash value of any server node and the hash value of the next node in the clockwise direction of the initial hash ring, determine the hash value corresponding to any server node after moving.

12. The device according to claim 9, characterized in that Also includes: A first creation unit, used to obtain IP information of each server node; Obtaining a corresponding hash value of each server node based on the IP information of each server node and a preset hash function; The starting point is 0 and the end point is 2 32 -1 builds a hash ring, and builds a corresponding server node at a point on the hash ring corresponding to the hash value corresponding to each server node to obtain the initial hash ring.

13. The device according to claim 12, characterized in that The first creation unit is specifically used to determine the hash value corresponding to the corresponding virtual node according to the hash value corresponding to each server node; and to construct the corresponding virtual node at the point on the initial hash ring corresponding to the hash value corresponding to each virtual node.

14. The device according to claim 13, characterized in that The first creation unit is specifically configured to calculate a hash value corresponding to the virtual node corresponding to each server node by using the following formula: Among them, H i ′ Indicates the hash value corresponding to the virtual node corresponding to the i-th server node, i = 1, 2, ..., N; H i Indicates the hash value corresponding to the i-th server node.

15. The device according to claim 11, characterized in that The processing unit is specifically used to calculate the hash value corresponding to any server node after the move by the following formula: Among them, H move Indicates the hash value corresponding to any server node after the move; H represents the hash value corresponding to any server node; H next Indicates the hash value corresponding to the next node of any server node in the clockwise direction of the initial hash ring.

16. The device according to claim 9, characterized in that Also includes: A second creation unit is configured to create a virtual node for a newly added server node if a server node is added to the distributed cluster, so as to update the target hash ring; A migration unit, configured to migrate data stored in any server node to the next server node in the clockwise direction of the target hash ring if any server node in the distributed cluster is abnormal; A deleting unit is used to delete the virtual node corresponding to any server node from the target hash ring.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the load balancing method based on the improved consistent hash ring is implemented as described in any one of claims 1 to 8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the load balancing method based on an improved consistent hash ring as described in any one of claims 1 to 8 are implemented.

Images