Master node selection method and device in distributed system, electronic equipment and storage medium

Abstract

The application provides a master node selection method and device in a distributed system, electronic equipment and a storage medium. The method comprises the following steps: for each effective node in the current distributed system, determining a first proportion of the total resource amount of the effective node and the remaining resource amount of the effective node; determining a candidate node from the effective nodes based on the first proportion of each effective node, wherein the first proportion of the candidate node is higher than a proportion threshold; determining whether the candidate node is a master node according to the total number of all effective nodes and a number threshold, wherein when the total number is less than or equal to twice the number threshold, the candidate node is the master node, and the number threshold is determined based on the divisible share of the total remaining resource amount of all effective nodes. The master node selected by the scheme has more available hardware resources, and communication between nodes is not required, thereby further saving system resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed computing, in particular to a master node selection method in a distributed system, a master node selection device in a distributed system, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] A node is a basic component unit of a distributed system, usually refers to a computer or server that runs independently in a network. These nodes complete tasks through cooperation, and have decentralization, parallelism and fault tolerance. In a distributed system, a master node is a key role, responsible for coordinating and managing other nodes in the system.

[0003] In some distributed systems, master node election is performed according to the ZooKeeper Atomic Broadcast (ZAB) protocol. This way selects the master node through a majority voting mechanism, and ensures the data consistency of the distributed system through transaction log synchronization and atomic broadcast.

[0004] In other distributed systems, the Bully Algorithm is used for master node election. In this algorithm, master node election is based on the unique identifier of the node, such as server ID or priority. Higher identifier values have higher priority and are more likely to be selected as master nodes. In some cases, the node with the highest identifier value is selected as the master node.

[0005] On the one hand, the master node determined by the related distributed node election method may not be the optimal solution, making it difficult to effectively utilize the resources of the distributed system. On the other hand, in the execution process of the related distributed node election method, communication is required between each node in the distributed system, wasting system resources. SUMMARY

[0006] The present application is proposed in view of the above problems.

[0007] According to a first aspect of the present application, a method for selecting a master node in a distributed system is provided. The method comprises: performing the following operations for active nodes in a current distributed system: determining, for each active node in the current distributed system, a first proportion of a remaining resource amount of the active node and a total resource amount of the active node; determining, based on the first proportion of each active node, a candidate node from the active nodes, wherein the first proportion of the candidate node is higher than a proportion threshold; determining, according to a total number of all active nodes and a number threshold, whether the candidate node is a master node, wherein when the total number is less than or equal to twice the number threshold, the candidate node is the master node, and the number threshold is determined based on a divisible share of a total remaining resource amount of all active nodes.

[0008] Exemplarily, the determining the first proportion of the remaining resource amount of the active node and the total resource amount of the active node comprises: determining a second proportion of a remaining storage space and a total storage space, a third proportion of a remaining running memory and a total running memory, and / or a fourth proportion of a remaining processor core number and a total processor core number; and determining the first proportion according to the second proportion, the third proportion and / or the fourth proportion.

[0009] Exemplarily, the determining the first proportion according to the second proportion, the third proportion and / or the fourth proportion comprises: determining a first weight, a second weight and / or a third weight according to a type of a task executed by the distributed system; and determining the first proportion according to a first product, a second product and / or a third product, wherein the first product is a product of the second proportion and the first weight, the second product is a product of the third proportion and the second weight, and the third product is a product of the fourth proportion and the third weight.

[0010] Exemplarily, the type is an I / O intensive type or a computation intensive type, and the method further comprises: when the type is the I / O intensive type, the first weight is maximum and the third weight is minimum; and when the type is the computation intensive type, the second weight is maximum and the first weight is minimum.

[0011] Exemplarily, the method further comprises: monitoring the active nodes in the distributed system in real time; and when the active nodes change, performing the operations again for the active nodes in the current distributed system.

[0012] Exemplarily, before the performing the operations again or in the process of the performing the operations again and before the determining whether the candidate node is the master node, the method further comprises: updating the number threshold according to the changed active nodes.

[0013] According to the total number of all the valid nodes and the number threshold, the method further includes: when the total number is equal to a second value, determining that the candidate node is not the master node and determining that the valid node with the lowest first proportion is the invalid node, wherein the second value is a sum of a first value and a power of 2, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0014] According to the total number of all the valid nodes and the number threshold, the method further includes: when the total number is equal to a second value, determining that the candidate node is not the master node and determining that the valid node with the lowest first proportion is the invalid node, wherein the second value is a sum of a first value and a power of 2, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0015] According to the total number of all the valid nodes and the number threshold, the method further includes: when the total number is equal to a second value, determining that the candidate node is not the master node and determining that the valid node with the lowest first proportion is the invalid node, wherein the second value is a sum of a first value and a power of 2, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0016] According to the total number of all the valid nodes and the number threshold, the method further includes: when the total number is equal to a second value, determining that the candidate node is not the master node and determining that the valid node with the lowest first proportion is the invalid node, wherein the second value is a sum of a first value and a power of 2, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0017] According to the total number of all the valid nodes and the number threshold, the method further includes: when the total number is equal to a second value, determining that the candidate node is not the master node and determining that the valid node with the lowest first proportion is the invalid node, wherein the second value is a sum of a first value and a power of 2, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0018] According to the second aspect of the present application, a master node selection device in a distributed system is further provided, which includes:

[0019] The proportion determining module is configured to determine, for each valid node in the current distributed system, a first proportion of a remaining resource amount of the valid node and a total resource amount of the valid node;

[0020] The selection module is configured to determine a candidate node from the valid nodes based on the first proportion of each valid node, wherein the first proportion of the candidate node is higher than a proportion threshold.

[0021] The judgment module is configured to determine, according to a total number of all the valid nodes and a number threshold, whether the candidate node is the master node, wherein when the total number is less than or equal to twice the number threshold, the candidate node is determined as the master node, and the number threshold is determined based on a divisible share of a total remaining resource amount of all the valid nodes.

[0022] According to a third aspect of the present application, there is provided an electronic device comprising a processor and a memory, wherein the memory stores computer program instructions which, when executed by the processor, cause the processor to perform the method for selecting a master node in a distributed system as described above.

[0023] According to a fourth aspect of the present application, there is provided a storage medium having stored thereon program instructions which, when executed by a processor, cause the processor to perform the method for selecting a master node in a distributed system as described above.

[0024] According to a fifth aspect of the present application, there is provided a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the method for selecting a master node in a distributed system as described above.

[0025] In the above technical solution, for each valid node in the current distributed system, a first proportion of a remaining resource amount of the valid node and a total resource amount of the valid node is determined, then the valid nodes whose first proportion is higher than a proportion threshold are selected as candidate nodes, and finally whether the candidate nodes are selected as master nodes is determined according to a total number of all valid nodes and a number threshold, wherein when the total number is less than or equal to twice the number threshold, the candidate nodes are selected as master nodes. In this way, the nodes with more available hardware resources can be selected as master nodes from the valid nodes by using the resources of the valid nodes themselves. In addition, in the execution process of the above master node selection method, the nodes in the distributed system communicate in disorder, thereby saving system resources.

[0026] The above description is merely a summary of the technical solutions of the present application. In order to enable a clearer understanding of the technical means of the present application and to be implemented according to the content of the description, and in order to enable the above and other purposes, features and advantages of the present application to be more apparent and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0027] The above and other purposes, features and advantages of the present application will become more apparent from the following detailed description of the embodiments of the present application, taken in conjunction with the accompanying drawings. The accompanying drawings are provided to assist in the understanding of the embodiments of the present application, and constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation on the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 a schematic flowchart of a method for selecting a master node in a distributed system according to an embodiment of the present application is shown;

[0029] Figure 2 a schematic flowchart of determining a first proportion of a remaining resource amount of a valid node and a total resource amount of the valid node according to an embodiment of the present application is shown;

[0030] Figure 3 A schematic flowchart of determining the first proportion according to an embodiment of the present application is shown;

[0031] Figure 4 A schematic flowchart of determining the quantity threshold according to an embodiment of the present application is shown;

[0032] Figure 5 A schematic block diagram of the master node selection apparatus of the distributed system according to an embodiment of the present application is shown; and

[0033] Figure 6 A schematic block diagram of the electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions, and advantages of the present application more apparent, the following will describe example embodiments according to the present application in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present application.

[0035] In order to at least partially solve the above problems, a master node selection method in a distributed system is proposed. The distributed system can include a plurality of active nodes. The plurality of active nodes work through the system to improve the computing efficiency, reliability, and scalability of the distributed system. Among the plurality of active nodes, one can be a master node in the distributed system. The master node is used to coordinate and manage the operation of the entire distributed system, and can allocate tasks to each active node, monitor the state thereof, and maintain the overall consistency of the system. According to the master node selection method in the distributed system according to the embodiments of the present application, one of the current active nodes in the distributed system can be selected as the master node to coordinate and manage the operation of the entire distributed system.

[0036] Figure 1 A schematic flowchart of the master node selection method in the distributed system according to an embodiment of the present application is shown. As shown in Figure 1 In the master node selection method in the distributed system, an operation is performed on the active nodes in the current distributed system, and the operation at least includes steps S110 to S130.

[0037] In step S110, for each active node in the current distributed system, a first proportion of a remaining resource amount of the active node and a total resource amount of the active node is determined.

[0038] In a distributed system, the running state of each node can be a closed state, a normal running state, and a fault state. When the distributed system performs a task, the nodes in the normal running state can cooperate with each other to perform the task. These nodes that can normally run and can participate in performing the task are valid nodes. The nodes in the closed state or the fault state are not used to perform the task, and the nodes that do not participate in performing the task can be regarded as invalid nodes.

[0039] In each valid node in the distributed system, there is own hardware resource. The hardware resource can include storage space, running memory, processor core number, data transmission channel number, and the total amount of these hardware resources can be referred to as total resource amount. The remaining resource amount is the unused resource amount in the total resource amount. Taking the total storage space as an example, the remaining resource amount includes the unused remaining storage space. Similarly, the remaining resource amount can also include the remaining running memory, the remaining processor core number, and / or the remaining data channel number. It can be understood that the roles of various hardware resources are different based on different tasks. In order to save the calculation amount, when performing the master node selection method according to the embodiments of the present application, one or more hardware resources in the valid node can be ignored.

[0040] For example, when the total resource amount includes any one of the resource amounts of the hardware resources such as the total storage space, the total running memory, the total processor core number, or the total data transmission channel number, the ratio between the remaining resource amount of the valid node and the total resource amount of the valid node can be taken as the first ratio.

[0041] For example, when the total resource amount includes multiple types of resource amounts of the hardware resources such as the total storage space, the total running memory, the total processor core number, or the total data transmission channel number, for each type of hardware resource, the ratio between the remaining resource amount and the total resource amount corresponding to the hardware resource can be determined. Then, the first ratio is calculated according to the ratio corresponding to each type of hardware resource. For example, the total resource amount of node a includes the total storage space b1 and the total running memory b2, if the remaining storage space of node a is a1 and the remaining running memory of node a is a2, then the first ratio of the remaining storage space and the remaining running memory can be calculated according to a1 / b1 and a2 / b2 respectively. The same applies to other types of hardware resource resource amounts, which will not be described here in detail. It can be understood that the influence of the remaining resource amount of different hardware resources on the comprehensive performance of the node can be the same or different, so the weight corresponding to each type of hardware resource can be determined respectively, and then the first ratio of each type of hardware resource is calculated according to the ratio between the remaining resource amount and the total resource amount corresponding to each type of hardware resource and the respective weight. The weight corresponding to each type of hardware resource can be a fixed value, or can be determined according to the type of the task performed by the distributed system.

[0042] In some embodiments, the total resource amount of different effective nodes is usually set to be the same or similar when the distributed system performs a task, and therefore the first ratio represents the comprehensive performance of the effective nodes for performing the task. The higher the first ratio, the higher the comprehensive performance of the effective nodes for performing the task, and vice versa.

[0043] In step S120, a candidate node is determined from the effective nodes based on the first ratio of each effective node, wherein the first ratio of the candidate node is higher than a ratio threshold.

[0044] The ratio threshold can be fixed, for example, 50%, 30%, etc. The ratio threshold can also be determined according to the first ratio of all effective nodes, for example, when the first ratio of all effective nodes ranges from 30% to 50%, the ratio threshold can be a value in the range of 30%-50% (for example, 40%, 45%, etc.). The ratio threshold determined according to the first ratio of all effective nodes can more reasonably filter the candidate nodes from the effective nodes.

[0045] Taking only the storage space of the hardware resources of the effective nodes into account, the maximum storage space of different effective nodes can usually be the same, and a part of the storage space of each effective node can be used. When the distributed system performs a task and does not read and write the existing data in the storage space, the remaining storage space is usually used to perform the task. In particular, the master node in the distributed system needs more sufficient remaining storage space to be responsible for coordinating and managing other nodes in the distributed system. The higher the first ratio of the effective node, the stronger the comprehensive performance of the effective node, and the effective node is more suitable as a candidate node for the master node. The master node can preferentially select the candidate node.

[0046] Similarly, when the total resource amount includes total running memory, total number of processor cores, and / or total number of data transmission channels, etc. hardware resources, the higher the first ratio of the effective node, the stronger the comprehensive performance of the effective node, and the effective node is more suitable as a candidate node for the master node. When the first ratio of the effective node is higher than the ratio threshold, it indicates that the effective node has strong comprehensive performance and is more suitable as a master node. The effective node with the first ratio higher than the ratio threshold can all be candidate nodes to determine whether it can be a master node.

[0047] Exemplarily, one or more effective nodes with the first ratio higher than the ratio threshold can be randomly selected as candidate nodes.

[0048] Alternatively, the effective node with the highest first ratio can be determined as a candidate node.

[0049] The effective nodes can be sorted according to their first proportions, or the first proportions of the respective effective nodes are compared in sequence to determine the effective node with the highest first proportion and higher than the proportion threshold. When there are multiple effective nodes with the highest first proportion and higher than the proportion threshold, all of the effective nodes can be selected as the candidate nodes, or part of the effective nodes (e.g., one) can be selected as the candidate nodes. In this case, the part of the effective nodes can be randomly selected as the candidate nodes, or the part of the effective nodes can be selected as the candidate nodes according to other information of the hardware resources of the effective nodes, such as the number, the starting time, and the like of the effective nodes. In this way, the number of the determined candidate nodes can be reduced, and the number of the determined master nodes can be reduced, so that the complexity of cooperation between the master nodes in performing the distributed task can be prevented from being excessively reduced.

[0050] In step S130, whether the candidate node is determined as the master node is determined according to the total number of all the effective nodes and the number threshold. When the total number is less than or equal to twice the number threshold, the candidate node is determined as the master node. The number threshold is determined based on the divisible share of the total remaining resource amount of all the effective nodes.

[0051] It is assumed that each effective node aims to maximize its own survival target (also referred to as benefit, such as becoming a master node or avoiding becoming an ineffective node), and tries to allocate more resources to itself based on this. The aforementioned determined candidate node can give a resource allocation proposal, and the candidate node tries to maximize the allocation of resources to itself to become a master node on the premise that the candidate node will not become an ineffective node. All the effective nodes can choose to support or oppose the proposal. When more than half of the effective nodes support the proposal of the candidate node, if the candidate node allocates the most resources in the proposal, the candidate node can become a master node. However, each effective node can expect to allocate more resources on the premise that the effective node will not become an ineffective node, so the effective node may not necessarily support the proposal of the resource allocation of the candidate node, and the candidate node can also propose a resource allocation proposal in combination with this situation.

[0052] Suppose that the distributed system includes five effective nodes, and a total of 100 total remaining resources can be allocated. The five effective nodes can include a first node, a second node, a third node, a fourth node, and a fifth node, and the first proportions of the first to fifth nodes decrease in sequence. The candidate node of the effective node can make a resource allocation proposal, and if the current candidate node cannot make a suitable proposal, the candidate node becomes an ineffective node, a new candidate node makes a resource allocation proposal, and the like.

[0053] In the above example, assume that the nodes 1 to 3 each cannot make a suitable proposal as an alternative node and become invalid nodes. At this time, only the nodes 4 and 5 are valid nodes. When the node 4 makes a proposal for resource allocation as an alternative node, the node 5 will oppose the proposal of the node 4 to monopolize all the remaining resources, and the node 4 will become an invalid node because it cannot obtain more than half of the support. At this time, only the node 5 remains as an alternative node, and the node 5 makes a proposal for resource allocation again without any node opposing it, and more than half of the support, so it will become the master node.

[0054] Assume that the nodes 1 to 2 each cannot make a suitable proposal as an alternative node and become invalid nodes. When the node 3 makes a proposal as an alternative node, the node 4 can support the proposal of the node 3 without becoming an invalid node. The node 3 knows this, and makes a proposal of "100, 0, 0", in which "100" means that 100 resources are allocated to the node 3, and the two "0"s mean that 0 resources are allocated to the nodes 4 and 5, respectively. The node to which 0 resources are allocated does not become an invalid node. The node 3 knows that the node 4 will still vote in favor although it does not obtain resources, and in addition to its own vote, its proposal can pass, so it can avoid becoming an invalid node and become the master node because the node 3 allocates the most resources.

[0055] Similarly, if the node 1 has become an invalid node and the node 2 is not an invalid node, the nodes 2, 3, 4, and 5 are valid nodes. The node 2 can make a proposal of "98, 0, 1, 1" as an alternative node, that is, 98 resources are allocated to the node 2, 0 resources are allocated to the node 3, and 1 resource is allocated to each of the nodes 4 and 5. The proposal of the node 2 can make the nodes 4 and 5 obtain more resources than the proposal of the node 3, that is, the proposal of the node 2 is more advantageous to the nodes 4 and 5 than the proposal of the node 3, so the nodes 4 and 5 will support the node 2 and do not want the node 2 to fail in the election and do not want the node 3 to become the master node and allocate resources.

[0056] Based on the above election process, the proposal of the node 2 can be the final election proposal, and the node 2 will allocate 98 resources and become the master node because the node 2 allocates more resources than the other nodes.

[0057] Those skilled in the art can understand that the above gives an example of including 5 effective nodes, and in the instance of other number of effective nodes, the analysis process is similar. In summary, the alternative node gives its proposal, and other effective nodes can choose to support and oppose. According to this rule, if there are X effective nodes, and the total remaining resources can be divided into A parts, after the proposal, the following cases can exist:

[0058] (1) When X≤2A, the maximum system resource utilization degree of the alternative node is wherein represents rounding up, and X and A are positive integers. The maximum system resource utilization degree Y indicates the additional resources that the alternative node can allocate relative to other nodes, and at this time Y can generally be positive.

[0059] (2) If X=2A+2 N (N is an integer greater than or equal to 0), in order to avoid becoming an invalid node, the alternative node will make its maximum system resource utilization degree Y to be 0, so as to obtain the support of more than half of the effective nodes and avoid becoming an invalid node.

[0060] (3) When X>2A and X≠2A+2 N , the proposal of the alternative node will not be supported by more than half of the nodes.

[0061] The number threshold for determining the master node in step S130 can be calculated according to the total remaining resource amount and the first proportion. The number threshold can be an integer or a non-integer. For example, the number threshold can be determined according to the product between the total remaining resource amount and the first proportion. Illustratively, the product can be determined as the number threshold. Alternatively, the product can be rounded, and the rounding result can be taken as the number threshold.

[0062] Illustratively, the number threshold can be a positive integer less than or equal to the maximum number of parts into which the total remaining resource amount of all effective nodes can be divided. It can be understood that hardware resources can be divided for use by different effective nodes. However, hardware resources are not infinitely divisible, and there is a minimum unit. When divided to the minimum unit, it cannot be further divided into different effective nodes. Taking the number of processor cores as an example, a processor core generally cannot be further divided. In other words, the number of processor cores is the maximum number of parts into which it can be divided.

[0063] Alternatively, the number threshold can be a larger value in the above value range. Alternatively, the number threshold can be further determined according to the total number of current effective nodes. The larger the total number, the larger the number threshold can be; otherwise, vice versa. Thus, for the above example of X effective nodes and A total remaining resources, the number threshold T can be equal to the maximum number of parts A into which the total remaining resource amount can be divided, i.e., T=A.

[0064] According to the relationship between X and A and based on the equivalence relationship between A and T, it can be determined whether the candidate node can be the master node. When X≤2T, the maximum system resource utilization degree Y of the candidate node is greater than 0. At this time, the proposal of the candidate node is passed, and the candidate node allocates more resources than other effective nodes, and the comprehensive performance is greater than that of other effective nodes, so the candidate node can be used as the master node. Thus, the candidate node can be used as the master node, and all effective nodes can be allocated resources according to the proposal of the candidate node.

[0065] In the above technical solution, for each effective node in the current distributed system, the first proportion of the remaining resource amount of the effective node and the total resource amount of the effective node is determined, then the effective node with the first proportion higher than the proportion threshold is determined as the candidate node, and finally, according to the total number of all effective nodes and the number threshold, it is determined whether the candidate node is the master node, wherein when the total number is less than or equal to twice the number threshold, the candidate node is the master node. In this way, the resources of the effective nodes can be used to select the node with more available hardware resources as the master node from the effective nodes. In addition, in the execution process of the above master node selection method, whether the node can be the effective node can be directly determined according to the number threshold and the total number of effective nodes, so the candidate node does not need to communicate with each node in the distributed system, thereby saving system resources.

[0066] Figure 2 A schematic flowchart of determining the first proportion of the remaining resource amount of the effective node and the total resource amount of the effective node according to one embodiment of the application is shown. As shown in Figure 2 The above step S110 can include steps S111 to S112.

[0067] In step S111, the second proportion of the remaining storage space and the total storage space, the third proportion of the remaining running memory and the total running memory, and / or the fourth proportion of the remaining processor core number and the total processor core number of the effective node are determined.

[0068] The second proportion is the evaluation result of the storage space of the effective node, which represents the storage performance of the effective node that can be used to execute the task of the distributed system. The third proportion is the evaluation result of the running memory of the effective node, which represents the memory performance of the effective node that can be used to execute the task of the distributed system. The fourth proportion is the evaluation result of the processor core number of the effective node, which represents the processor performance of the effective node that can be used to execute the task of the distributed system.

[0069] In step S112, the first proportion is determined according to the second proportion, the third proportion and / or the fourth proportion.

[0070] For example, the determined second proportion, third proportion and / or fourth proportion can be added to obtain the first proportion.

[0071] For example, in a complex case, the tasks performed by the distributed system can have different requirements for the storage space, running memory and / or number of processor cores. Therefore, for the second proportion, third proportion and / or fourth proportion, a respective corresponding weight can be determined. Then the second proportion, third proportion and / or fourth proportion are multiplied by the respective corresponding weight. The multiplied results are added to obtain the first proportion.

[0072] In the above technical solution, the second proportion of the remaining storage space and total storage space of the effective node, the third proportion of the remaining running memory and total running memory, and / or the fourth proportion of the remaining number of processor cores and total number of processor cores are determined, and then the first proportion is determined according to the second proportion, third proportion and / or fourth proportion. The storage space, running memory and processor core number are very important for the node to perform tasks. The first proportion determined based on these hardware resources can better reflect the ability of each effective node to participate in performing the tasks of the distributed system, thereby ensuring that the selected master node is more ideal. In addition, in combination with different types of hardware resources, the determined first proportion can better represent the performance of the effective node.

[0073] Figure 3 A schematic flowchart of determining the first proportion according to an embodiment of the present application is shown. As shown in Figure 3 The above step S112 can include steps S112a to S112b.

[0074] In step S112a, a first weight, a second weight and / or a third weight are determined according to the type of task performed by the distributed system.

[0075] There are many types of tasks performed by the distributed system, such as compute-intensive, I / O-intensive, storage-intensive, etc. Different types of tasks can have different requirements for the hardware resources of the effective node, i.e., the requirements for the storage space, running memory and number of processor cores, etc. can be different. The requirements for various hardware resources corresponding to the type of task can be determined according to the type of task performed by the distributed system.

[0076] The first weight represents the requirement of the task for the storage space of the effective node. The second weight represents the requirement of the task for the running memory of the effective node. The third weight represents the requirement of the task for the processor of the effective node.

[0077] Optionally, the specific values of the first weight, second weight and third weight are set according to experience, and the sum of the first weight, second weight and third weight can be equal to 1.

[0078] Optionally, the specific values of the first weight, the second weight and the third weight are pre-set according to experience, and the sum of the first weight, the second weight and the third weight can be equal to a preset value. The preset value can be determined according to the proportion of the hardware resources such as storage space, running memory and processor core number in the hardware resources of the effective node. For example, when the hardware resources of the effective node further include the number of data channels, if the proportion of the hardware resources such as storage space, running memory and processor core number in the hardware resources of the effective node is 80%, the sum of the first weight, the second weight and the third weight can be 80%. The same applies when the proportion of the hardware resources such as storage space, running memory and processor core number in the hardware resources of the effective node is other values, which will not be described here. The proportion of the hardware resources such as storage space, running memory and processor core number in the hardware resources of the effective node can be a fixed value, or can be determined according to the type of the task executed by the distributed system.

[0079] For example, when the type of the task executed by the distributed system is I / O intensive, the first weight is the largest and the third weight is the smallest.

[0080] When the distributed system executes an I / O intensive task, the demand for storage space is the largest and the demand for processor core number is the smallest among the hardware resources such as storage space, running memory and processor core number of the effective node. Therefore, for the case that the type of the task executed by the distributed system is I / O intensive, the first weight can be set to the largest and the third weight can be set to the smallest. To improve the influence of storage space on the first proportion and reduce the influence of processor core number on the first proportion. For example, when the type of the task executed by the distributed system is I / O intensive, the first weight can be 50%, the second weight can be 30%, and the third weight can be 20%.

[0081] For example, when the type of the task executed by the distributed system is computation intensive, the second weight is the largest and the first weight is the smallest.

[0082] When the distributed system executes a computation intensive task, the demand for running memory is the largest and the demand for storage space is the smallest among the hardware resources such as storage space, running memory and processor core number of the effective node. Therefore, for the case that the type of the task executed by the distributed system is computation intensive, the first weight can be set to the smallest and the second weight can be set to the largest. To improve the influence of running memory on the first proportion and reduce the influence of storage space on the first proportion.

[0083] For example, when the type of the task executed by the distributed system is computation intensive, the first weight can be 20%, the second weight can be 50%, and the third weight can be 30%.

[0084] At step S112b, a first proportion is determined according to the first product, the second product and / or the third product, wherein the first product is a product of the second proportion and the first weight, the second product is a product of the third proportion and the second weight, and the third product is a product of the fourth proportion and the third weight.

[0085] When the hardware resource to be considered only contains one of the storage space, the running memory and the number of processor cores of the effective node, the corresponding first product, second product or third product can be taken as the first proportion. When the hardware resource to be considered contains multiple ones of the storage space, the running memory and the number of processor cores of the effective node, multiple ones of the corresponding first product, second product and / or third product can be added to obtain the first proportion.

[0086] For example, when the hardware resource to be considered contains the storage space, the running memory and the number of processor cores of the effective node, the first proportion corresponding to the effective node can be determined according to the following formula 1:

[0087] P = K1*C + K2*E + K3*F Formula 1

[0088] Wherein, K1 represents the first weight, K2 represents the second weight, K3 represents the third weight, C represents the second proportion, E represents the third proportion, and F represents the fourth proportion.

[0089] In another example, when the hardware resource to be considered contains any two of the storage space, the running memory and the number of processor cores of the effective node, the first proportion corresponding to the effective node can be determined according to the following formula 2:

[0090] P = K1*C + K2*E Formula 2

[0091] Wherein, K1 represents one of the first weight, the second weight and the third weight, K2 represents another one of the first weight, the second weight and the third weight, C represents one of the second proportion, the third proportion and the fourth proportion, and E represents another one of the second proportion, the third proportion and the fourth proportion. K1 and C correspond to the same type of hardware resource, K2 and E correspond to the same type of hardware resource, and K1 and K2 correspond to different types of hardware resources. For example, when the hardware resource to be considered contains the storage space and the running memory of the effective node, in the above formula 2, K1 can represent the first weight, K2 can represent the second weight, C can represent the second proportion, and E can represent the third proportion.

[0092] In the technical solution, the first weight, the second weight and / or the third weight are determined according to the type of the task executed by the distributed system, and then the first proportion is determined according to the first product, the second product and / or the third product, wherein the first product is the product of the second proportion and the first weight, the second product is the product of the third proportion and the second weight, and the third product is the product of the fourth proportion and the third weight. In this way, the required resources and the first proportion of the required resources that are more suitable for the type of the task executed by the distributed system can be determined. Furthermore, the master node selected based on the first proportion can be more suitable for the type of the task.

[0093] Figure 4 A schematic flowchart of determining the quantity threshold is shown according to an embodiment of the present application. As shown in the figure, the master node selection method of the distributed system can further include steps S130a-S130c for determining the quantity threshold. Figure 4

[0094] In step S130a, the divisible share of the total remaining resource amount of the distributed system is determined according to the remaining resource amount of each valid node.

[0095] The remaining resource amount of each valid node of the same type can be added to determine the total available resource amount of each type of hardware resource, respectively. For example, the remaining storage space of each valid node can be added to obtain the available storage space. For another example, the remaining running memory of the valid node can also be added to obtain the total available running memory. Similarly, for other types of hardware resources of the valid node, the total available resource amount of the hardware resource of the type is also determined. Then the total available resource amount of each type can be divided into M shares, respectively, and one share of the total available resource amount of all types is added to one share of the total remaining resource amount, so that the total remaining resource amount can be determined, i.e., the total remaining resource amount can be divided into M shares, and each share of the resource contains one share of the total available resource amount of all types. For example, for storage space and running memory, the storage space and running memory are divided into 10 shares, respectively, each share of the storage space is 10 kb, and each share of the running memory is 1 kb, so that each share of the resource can contain 10 kb of storage space and 1 kb of running memory.

[0096] In step S130b, for each valid node, a fourth product between the first proportion corresponding to the valid node and the divisible share of the total remaining resource amount is determined.

[0097] The fourth product represents the amount of resources that can be allocated to the valid node in the distributed system. The higher the fourth product, the more resources that can be allocated to the valid node. The lower the fourth product, the fewer resources that can be allocated to the valid node.

[0098] ​At step S130c, the sum of the fourth products corresponding to each of the effective nodes is calculated and rounded up to obtain a quantity threshold.

[0099] The quantity threshold determined according to the sum of the fourth products of all the effective nodes represents the total demand of the remaining resources of the current all effective nodes, i.e. how many parts are divided. The resource amount corresponding to each part of the remaining resources can be set according to the task performed by the distributed system, and in the case of ensuring that each effective node can be allocated a certain resource, each part of the remaining resources can contain a specified amount of hardware resources. The quantity threshold thus determined can generally ensure that each effective node can be allocated a certain amount of hardware resources, thereby ensuring that each effective node can successfully perform the task of the distributed system, rather than becoming an ineffective node.

[0100] In the above technical solution, according to the remaining resource amount of each effective node, the divisible parts of the total remaining resource amount are determined, then for each effective node, the fourth product between the first proportion corresponding to the effective node and the divisible parts of the total remaining resource amount is determined, then the sum of the fourth products corresponding to each of the effective nodes is calculated and rounded up to obtain a quantity threshold. The quantity threshold is obtained by rounding up the sum of the fourth products corresponding to each of the effective nodes, and the double of the quantity threshold thus determined is necessarily greater than or equal to twice the total number of effective nodes, so according to the quantity threshold, the effective node with the most remaining resources can be guaranteed to be selected as the master node. And after the candidate node is determined as the master node, each effective node can also be allocated part of the hardware resources without being allocated less than one part of the hardware resources due to the proposal of the master node, and the resource allocation scheme of the master node is more reasonable.

[0101] Optionally, the master node selection method in the distributed system according to some embodiments of the present application can further include steps S140 and S150.

[0102] At step S140, the effective nodes in the distributed system are monitored in real time.

[0103] Optionally, each effective node in the distributed system can periodically send a heartbeat signal to its monitoring center. The monitoring center can receive and record the heartbeat signal. If there is an effective node whose heartbeat signal is not received within a timeout period, it means that the effective node has become an ineffective node. It can be understood that the ineffective node does not participate in the execution of the task of the distributed system. The ineffective node can be a node in a fault state or a closed state. The ineffective node can also be a node in a normal operating state but does not participate in the execution of the task of the distributed system. When a node becomes an ineffective node, it means that the number of effective nodes decreases. When a heartbeat signal is received from a new node, it means that the number of effective nodes increases. In this way, the effective nodes in the distributed system can be monitored in real time.

[0104] Optionally, the monitoring center can periodically send a health check request to the nodes. When the nodes return the status information, the monitoring center can determine whether the nodes are valid nodes according to the returned status information, and determine which nodes are included in the valid nodes. In this way, the valid nodes in the distributed system can also be monitored in real time.

[0105] Optionally, the valid nodes can send the running logs to a centralized log system, and then the running logs can be analyzed by using a tool (such as ELK Stack) to determine whether the valid nodes in the distributed system change.

[0106] The valid nodes in the distributed system can also be monitored in real time in other ways than the above-described manner, which is not limited here.

[0107] In step S150, when the valid nodes change, the above-described operations are performed again for the valid nodes in the current distributed system.

[0108] The change of the valid nodes can include a change in the number of valid nodes, a change in the remaining resource amount of the valid nodes, a change in the divisible share of the total resource amount of the valid nodes, and the like. After the valid nodes change, the currently determined master node can not be the most suitable master node, and the operations in steps S110 to S130 can be performed again to select the most suitable node from the current valid nodes as the master node.

[0109] For example, when the valid nodes increase, the performance of the increased valid nodes can be higher than that of the currently determined master node. Therefore, when the valid nodes increase, the operations in steps S110 to S130 can be performed again according to the current valid nodes, that is, the master node is re-determined.

[0110] Similarly, when the valid nodes decrease, the decreased valid nodes will affect the overall performance of the distributed system. In particular, when the decreased node is the master node, according to some embodiments of the present application, a new master node can be selected in time. Therefore, when the valid nodes decrease, the operations in steps S110 to S130 can be performed again according to the current valid nodes, that is, the master node is re-determined.

[0111] Similarly, when the remaining resource amount of the valid nodes changes, the divisible share of the total resource amount of the valid nodes changes, or the valid nodes change in other ways, which will not be described in detail here.

[0112] In the above technical solution, the valid nodes in the distributed system are monitored in real time, and when the valid nodes change, the above-described operations are performed again for the valid nodes in the current distributed system. In this way, when the valid nodes change, the most suitable master node at this time can be selected in time, and the system resources can be effectively utilized.

[0113] Exemplarily, when the task performed by the distributed system changes, the above operation can also be performed again for the active nodes in the current distributed system. The master node more suitable for the current task is selected from the active nodes.

[0114] Exemplarily, before the operation is performed again, or in the process of performing the operation again and before it is determined whether the candidate node is the master node, the above method further comprises a step S160 of updating the quantity threshold according to the changed active nodes.

[0115] In the process of monitoring the active nodes in the distributed system in real time, the active nodes can change. According to some embodiments of the present application, different total quantities of active nodes can correspond to different quantity thresholds respectively. The corresponding quantity threshold can be determined according to the total quantity of the active nodes after increase or decrease, so as to adapt to the case of different total quantities of active nodes. For example, when the total quantity of active nodes increases, the quantity threshold can increase accordingly, and vice versa.

[0116] For the changed active nodes, the total remaining resource quantity and the remaining resource quantity corresponding to the active nodes can change. The quantity threshold and the first proportion of each active node in the current can be updated synchronously according to the total remaining resource quantity and the remaining resource quantity corresponding to the active nodes. For example, when the divisible quantity of the total resource quantity corresponding to the active nodes increases, the quantity threshold can increase accordingly, and vice versa.

[0117] It can be understood that by updating the quantity threshold, the rationality of the selection of the master node when the active nodes change can be ensured.

[0118] Then, as described above, in the steps 110 to S130, the candidate node can be determined according to the first proportion first, and then when the total quantity of active nodes is less than or equal to twice the quantity threshold, the candidate node is selected as the master node.

[0119] Exemplarily, when the total quantity of active nodes is greater than twice the quantity threshold, it is determined that the selection of the master node fails, and an alarm signal is output. When the user receives the alarm signal, he / she can adjust the quantity threshold to perform the step S130 again to select the master node.

[0120] Alternatively, when the total number of the active nodes is greater than twice the number threshold, it is determined that the master node selection fails, and the number threshold can be automatically adjusted by a certain step to perform the step S130 again to select the master node. For example, the step can be fixed, or determined according to the number threshold, or determined according to the total number of the current active nodes. For example, the current number threshold can be reduced by a fixed value of 1, 5, 10, etc. after each determination of the master node selection failure. For another example, the number threshold can be reduced by a ratio after each determination of the master node selection failure, where the ratio can be 1 / 2, 1 / 3, etc. For example, after the determination of the master node selection failure, the number threshold can be adjusted, where the adjusted number threshold is not less than half of the total number of the current active nodes.

[0121] For example, the step S130 can further include a step S131 of determining that the candidate node is not the master node and is an invalid node when the total number is greater than twice the number threshold and is not equal to a second value, where the second value is the sum of 2 raised to the power of N and the first value, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0122] According to the above description of the total number of the active nodes and the number threshold, when the total number of the active nodes is greater than twice the number threshold and is not equal to a second value, i.e., when X>2A and X≠2A+2 N , the proposal of the candidate node will not be supported by more than half of the nodes. At this time, the candidate node can be determined as an invalid node.

[0123] It can be understood that after the determination that the candidate node is not the master node and is an invalid node, the active nodes will be reduced. At this time, the process of the steps S140 to S150 described above can be performed to attempt to determine the master node in the current active nodes again.

[0124] In the above technical solution, when the total number of the active nodes is greater than twice the number threshold and is not equal to a second value, the candidate node is directly determined as an invalid node, and then the master node is determined again according to the reduced active nodes. In this way, it is further ensured that the selected master node can utilize more hardware resources and is a more ideal master node.

[0125] For example, the step S130 can further include a step S132 of determining that the candidate node is not the master node and determining the active node with the lowest first ratio as an invalid node when the total number is equal to a second value, where the second value is the sum of 2 raised to the power of N and the first value, the first value is twice the number threshold, and N is an integer greater than or equal to 0.

[0126] According to the above description of the total number of the active nodes and the number threshold, when the total number is equal to a second value, i.e., when X=2A+2N When the alternative node's proposal can get more than half of the valid nodes' support, but the maximized system resource utilization Y is 0, the alternative node is not suitable to be the master node. For example, in a distributed system including three valid nodes c1, c2, c3, if the number threshold is 1, and c1 is an alternative node, in order to get more than half of the valid nodes' support, c1 can propose a scheme of "0, 1, 0". In the scheme, c1 and c3 get 0 share of resources, and c2 gets 1 share of hardware resources. At this time, the scheme of c1 can get the support of c2 and c3, but c2 and c3 do not get additional resources, so c1 is not suitable to be the master node. In order to avoid c1 being determined as an alternative node again, c1 can be determined as an invalid node.

[0127] When the alternative node is determined not to be the master node and is an invalid node, at this time, the valid nodes are reduced, the master node can be determined again in the current valid nodes by performing the processes of steps S140 to S150.

[0128] In the above technical solution, when the total number of valid nodes is equal to the second value, the alternative node is determined not to be an invalid node, and the valid node with the lowest first proportion is eliminated, and then the master node is determined again according to the reduced valid nodes. Thus, it is further ensured that the selected master node can utilize more hardware resources, and is a more ideal master node.

[0129] Figure 5 A schematic block diagram of a master node selection device of a distributed system according to an embodiment of the present application is shown. As shown in the figure, the master node selection device of the distributed system includes a proportion determination module 510, a selection module 520, and a judgment module 530. Figure 5 The proportion determination module 510 is configured to determine, for each valid node in the current distributed system, a first proportion of a total resource amount of the valid node to a remaining resource amount of the valid node.

[0130] The proportion determination module 510 is configured to determine, for each valid node in the current distributed system, a first proportion of a total resource amount of the valid node to a remaining resource amount of the valid node.

[0131] The selection module 520 is configured to determine, from the valid nodes, an alternative node based on the first proportion of each valid node being the highest, wherein the first proportion of the alternative node is higher than a proportion threshold.

[0132] The judgment module 530 is configured to determine, according to a total number of all valid nodes and a number threshold, whether the alternative node is a master node, wherein when the total number is less than or equal to twice the number threshold, the alternative node is the master node, and the number threshold is determined based on a positive integer less than or equal to a maximum shareable share of a total remaining resource amount of all valid nodes.

[0133] Exemplarily, the proportion determining module 510 can include a first determining sub-module and a second determining sub-module. The first determining sub-module is configured to determine a second proportion of the remaining storage space and the total storage space of the effective node, a third proportion of the remaining running memory and the total running memory, and / or a fourth proportion of the remaining processor core number and the total processor core number. The second determining sub-module is configured to determine the first proportion according to the second proportion, the third proportion and / or the fourth proportion.

[0134] Exemplarily, the second determining sub-module includes a weight determining sub-module and a third determining sub-module. The weight determining sub-module is configured to determine a first weight, a second weight and / or a third weight according to the type of the task executed by the distributed system. The third determining sub-module is configured to determine the first proportion according to the first product, the second product and / or the third product, wherein the first product is the product of the second proportion and the first weight, the second product is the product of the third proportion and the second weight, and the third product is the product of the fourth proportion and the third weight.

[0135] Exemplarily, the type is I / O intensive or computation intensive. When the type is I / O intensive, the first weight is the largest and the third weight is the smallest; when the type is computation intensive, the second weight is the largest and the first weight is the smallest.

[0136] Exemplarily, the master node selection apparatus in the distributed system further includes a monitoring module. The monitoring module is configured to monitor the effective nodes in the distributed system in real time. The proportion determining module 510, the selection module 520 and the judging module 530 are further configured to perform respective operations on the effective nodes in the current distributed system when the effective nodes change.

[0137] Exemplarily, the master node selection apparatus in the distributed system further includes an updating module. The updating module is configured to update the quantity threshold according to the changed effective nodes before performing the operation again or in the process of performing the operation again and before determining whether the candidate node is the master node.

[0138] Exemplarily, the judging module 530 can further include a first judging sub-module. The first judging sub-module is configured to determine that the candidate node is not the master node and is an invalid node when the total quantity is greater than twice the quantity threshold and is not equal to a second value, wherein the second value is the sum of 2 raised to the power of N and the first value, the first value is twice the quantity threshold, and N is an integer greater than or equal to 0.

[0139] Exemplarily, the judging module 530 can further include a second judging sub-module. The second judging sub-module is configured to determine that the candidate node is not the master node and determine that the effective node with the lowest first proportion is the invalid node when the total quantity is equal to the second value, wherein the second value is the sum of 2 raised to the power of N and the first value, the first value is twice the quantity threshold, and N is an integer greater than or equal to 0.

[0140] Exemplarily, the master node selection apparatus in the distributed system further comprises a resource statistics submodule, a resource conversion calculation submodule, and a numerical value generation submodule. The resource statistics submodule is configured to determine the divisible share of the total remaining resource amount of the distributed system according to the remaining resource amount of each valid node. The resource conversion calculation submodule is configured to determine, for each valid node, a fourth product between the first proportion corresponding to the valid node and the divisible share of the total remaining resource amount. The numerical value generation submodule is configured to calculate the sum of the fourth products corresponding to each valid node respectively and round up the sum to obtain the quantity threshold.

[0141] Exemplarily, the selection module 520 comprises a screening submodule. The screening submodule is configured to determine the valid node with the highest first proportion as the candidate node.

[0142] Exemplarily, the quantity threshold is a positive integer less than or equal to the maximum divisible share of the total remaining resource amount of all valid nodes.

[0143] According to another aspect of the present application, an electronic device is also provided. Figure 6 A schematic block diagram of an electronic device according to an embodiment of the present application is shown. As shown, the electronic device comprises a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, are configured to perform the master node selection method in a distributed system as described above. Figure 6

[0144] In addition, according to still another aspect of the present application, a storage medium is also provided, on which program instructions are stored, which, when executed by a computer or a processor, cause the computer or the processor to perform the corresponding steps of the master node selection method in a distributed system according to the above-mentioned embodiments of the present application, and to implement the corresponding modules in the master node selection apparatus in a distributed system according to the above-mentioned embodiments of the present application. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above-mentioned storage media. The computer-readable storage medium can be any combination of one or more computer-readable storage media.

[0145] According to another aspect of the present application, a computer program product is also provided, comprising computer program instructions, which, when executed, are configured to perform the master node selection method in a distributed system as described above.

[0146] ​Those skilled in the art can understand the specific implementation and beneficial effects of the above-mentioned distributed system master node selection device, electronic equipment, storage medium and computer program product by reading the above description of the master node selection method in the distributed system. For brevity, it will not be repeated here.

[0147] Although example embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the example embodiments are only exemplary and are not intended to limit the scope of the present application thereto. Those of ordinary skill in the art can make various changes and modifications without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed in the appended claims.

[0148] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0149] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0150] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not shown in detail in order not to obscure the understanding of the present specification.

[0151] Similarly, it is to be understood that the embodiments of the present application can be alternately grouped together in a single embodiment, figure, or description of an embodiment for the purpose of brevity and understanding in the description of the example embodiments of the present application, and that individual features of the application should not be construed as being a requirement of the claimed application unless explicitly recited in each of the claims. Rather, an aspect of the application is to address its specified problems and is not a constructive combination of all features of some one disclosed embodiment. Rather, the claims, as recited in the particular claims section, define the application.

[0152] Those skilled in the art will appreciate that all features described herein (including all features of the accompanying claims, abstract and drawings) can be taken in combination with any and all other features described herein, and that the ordering of certain processes or units of any method or apparatus so disclosed is not an inherent part of the process or unit in question and that each process or unit can be performed or implemented in any order unless explicitly claimed otherwise.

[0153] Furthermore, those skilled in the art will recognize that braces surrounding two or more items in a list of items, and the use of the term "comprising" to refer to items in such lists, means that items in such lists are to be treated like they were individually written out using the term "and" to join them together. For example, a list of items comprising A, B, and C is to be treated as if it said A, B, and C, or A, or B, or C, or 2A, or 2B, or 2C, or an A and a B, or an A and a C, or a B and a C, or an A and a B and a C, or 2A and 2B, or 2A and 2C, or 2B and 2C, or 2A and 2B and 2C, and so on.

[0154] Various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some of the modules in the master node selection apparatus in a distributed system according to embodiments of the present application. The present application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of the present application can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.

[0155] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or means can be listed, comprising means performing the same function. The use of the word 'a' or 'an' does not exclude the presence of a plurality of such devices or means. The word 'first','second', 'third', and the like in the description do not necessarily have a chronological order.

[0156] The above description is only specific embodiments or specific implementations of the present application, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, and all such changes or replacements should be covered within the protection scope of the present application. The protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for selecting a master node in a distributed system, characterized in that, The method includes performing the following operations on valid nodes in the current distributed system: For each valid node in the current distributed system, determine the first ratio of the remaining resources of that valid node to the total resources of that valid node; Based on the first proportion of each valid node, candidate nodes are determined from the valid nodes, wherein the first proportion of the candidate nodes is higher than the proportion threshold. Based on the total number of all valid nodes and a quantity threshold, it is determined whether the candidate node should be the master node. Specifically, when the total number is less than or equal to twice the quantity threshold, the candidate node is designated as the master node. When the total number is greater than twice the quantity threshold but not equal to a second value, the candidate node is determined not to be the master node and is an invalid node. When the total number is equal to the second value, the candidate node is determined not to be the master node, and the valid node with the lowest first proportion is determined to be an invalid node. The quantity threshold is determined based on the number of divisible portions of the total remaining resources of all valid nodes. The second value is the sum of 2 raised to the power of N and the first value, where the first value is twice the quantity threshold, and N is an integer greater than or equal to 0.

2. The method according to claim 1, characterized in that, The determination of the first ratio between the remaining resources of the effective node and the total resources of the effective node includes: Determine a second ratio of the remaining storage space to the total storage space of the valid node, a third ratio of the remaining running memory to the total running memory, and / or a fourth ratio of the remaining number of processor cores to the total number of processor cores; The first ratio is determined based on the second ratio, the third ratio, and / or the fourth ratio.

3. The method according to claim 2, characterized in that, Determining the first ratio based on the second ratio, the third ratio, and / or the fourth ratio includes: The first weight, the second weight, and / or the third weight are determined based on the type of task performed by the distributed system. The first ratio is determined based on the first product, the second product, and / or the third product, wherein the first product is the product of the second ratio and the first weight, the second product is the product of the third ratio and the second weight, and the third product is the product of the fourth ratio and the third weight.

4. The method according to claim 3, characterized in that, The type is I / O intensive or compute intensive, and the method further includes: When the type is I / O intensive, the first weight is the largest and the third weight is the smallest; When the type is computationally intensive, the second weight is the largest and the first weight is the smallest.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Real-time monitoring of active nodes in a distributed system; When the valid node changes, the operation is performed again for the valid node in the current distributed system.

6. The method according to claim 5, characterized in that, Before the operation is performed again, or during the operation and before determining whether the candidate node is the primary node, the method further includes: Update the quantity threshold based on the changed valid nodes.

7. The method according to claim 1, characterized in that, The method further includes: Based on the remaining resources of each valid node, determine the number of parts into which the total remaining resources of the distributed system can be divided; For each valid node, determine the fourth product between the first proportion corresponding to that valid node and the number of divisible parts of the total remaining resources; Calculate the sum of the fourth products corresponding to each valid node and round the sum up to obtain the quantity threshold.

8. The method according to claim 1, characterized in that, The quantity threshold is a positive integer that is less than or equal to the maximum number of divisible parts of the total remaining resources of all valid nodes.

9. A master node selection device in a distributed system, characterized in that, include: The ratio determination module is used to determine, for each valid node in the current distributed system, the first ratio of the remaining resources of that valid node to the total resources of that valid node. The selection module is used to select the valid node with the highest first proportion as the candidate node; The judgment module is used to determine whether a candidate node should be the master node based on the total number of all valid nodes and a quantity threshold. Specifically, when the total number is less than or equal to twice the quantity threshold, the candidate node is designated as the master node. When the total number is greater than twice the quantity threshold but not equal to a second value, the candidate node is determined not to be the master node and is an invalid node. When the total number is equal to the second value, the candidate node is determined not to be the master node, and the valid node with the lowest first proportion is determined to be an invalid node. The quantity threshold is a positive integer less than or equal to the maximum divisible number of the total remaining resources of all valid nodes. The second value is the sum of 2 raised to the power of N and the first value, where the first value is twice the quantity threshold, and N is an integer greater than or equal to 0.

10. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the master node selection method in a distributed system as described in any one of claims 1 to 8.

11. A storage medium on which program instructions are stored, characterized in that, The program instructions are used at runtime to execute the master node selection method in a distributed system as described in any one of claims 1 to 8.

12. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the master node selection method in a distributed system as described in any one of claims 1 to 8.

Images