Leader node transfer method, distributed system, electronic equipment and storage medium

Abstract

The invention discloses a leader node transfer method, a distributed system, electronic equipment and a storage medium, and relates to the field of distributed systems.The method is applied to a leader node in the distributed system and comprises the steps that parameter values of performance parameters of the leader node are determined according to the hardware performance, the load condition and the network delay of the leader node, obtaining a target parameter value; obtaining N parameter values corresponding to N following nodes in the distributed system; determining candidate leading nodes from the N following nodes and the leading nodes according to the target parameter value and the N parameter values; and under the condition of determining to carry out leader node transfer, executing leader node transfer operation, so that the candidate leader node becomes a new leader node in the distributed system. By adopting the scheme, the performance of the distributed system is improved, and the problem that the performance of the distributed system is easy to be poorer due to the fact that the leader node is fixed in a consensus algorithm is solved.

Description

Technical Field

[0001] The present application relates to the field of distributed systems, and in particular to a method for transferring a leader node, a distributed system, an electronic device, and a storage medium. Background Art

[0002] In distributed systems, consistency—that is, how to maintain a consistent state of data across multiple nodes after updates—is a core issue, attracting considerable attention from both academia and industry. Algorithms that address this issue are called consensus algorithms, of which Raft is one. In Raft, the entire cluster typically has an odd number of nodes, such as 3, 5, or 7. Nodes are classified into three roles: Leader, Follower, and Candidate. The Leader receives client requests and copies log entries to Follower nodes. Followers passively receive messages and respond accordingly. The Candidate role indicates that a node is attempting to become a Leader.

[0003] In the Raft algorithm, when a node receives a voting request, if the log of the sending node is more recent than its own log and the node has not yet voted in the current election, it will vote for the sender of the request. Therefore, during the leader election process, only the node timeout and log validity affect the election results. That is, after the leader is elected, if no nodes in the entire system have timed out and the leader node is not faulty, the leader node will not change. If the leader performs poorly at this time, the performance of the entire distributed system will be affected.

[0004] Regarding the problem in related technologies where the leader node is fixed in the consensus algorithm, which leads to poor performance of the distributed system, no effective solution has been proposed yet. Summary of the Invention

[0005] The present application provides a method for transferring a leader node, a distributed system, an electronic device, and a storage medium, to at least solve the problem that in a consensus algorithm, the leader node is fixed, resulting in poor performance of the distributed system.

[0006] The present application provides a method for transferring a leader node, which is applied to a leader node in a distributed system, comprising: determining parameter values ​​of performance parameters of the leader node based on the hardware performance, load condition, and network delay of the leader node to obtain a target parameter value; and obtaining N parameter values ​​corresponding to N follower nodes in the distributed system, wherein the i-th parameter value among the N parameter values ​​is the parameter value of the performance parameter of the i-th follower node determined by the i-th follower node among the N follower nodes based on the hardware performance, load condition, and network delay of the i-th follower node; determining a candidate leader node from the N follower nodes and the leader node based on the target parameter value and the N parameter values; and when it is determined that the leader node is to be transferred, executing a leader node transfer operation so that the candidate leader node becomes the new leader node in the distributed system.

[0007] The present application also provides a distributed system, including: a leader node, used to determine the parameter value of the leader node on the performance parameter based on the hardware performance, load condition and network delay of the leader node, and obtain the target parameter value; N follower nodes, wherein each follower node in the N follower nodes is used to determine the parameter value of the follower node on the performance parameter based on the hardware performance, load condition and network delay of the follower node, and send the parameter value to the leader node; the leader node is also used to determine a candidate leader node from the N follower nodes and the leader node based on the target parameter value and the obtained N parameter values, and when it is determined to transfer the leader node, perform a leader node transfer operation to make the candidate leader node become the new leader node in the distributed system.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for transferring a leader node when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for transferring a leader node are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned leader node transfer methods when executed by a processor.

[0011] This application forms a series of comparable parameter values ​​by quantifying the hardware performance, load conditions and network delay of the leader node and follower nodes, and then determines a candidate leader node based on these parameter values. When the transfer conditions are met, the leader node transfer operation is performed, thereby avoiding the performance bottleneck caused by the long-term fixation of the leader node, ensuring that the distributed system can adaptively select the leader node according to the real-time status of the node, thereby improving the performance of the distributed system, and solving the problem that in the consensus algorithm, the leader node is fixed, which leads to poor performance of the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a hardware structure block diagram of a method for transferring a leader node according to an embodiment of the present application;

[0014] Figure 2 This is a flowchart (1) of a method for transferring a leader node according to an embodiment of the present application;

[0015] Figure 3 is a scene module diagram according to an embodiment of the present application;

[0016] Figure 4 This is a flowchart (II) of a method for transferring a leader node according to an embodiment of the present application;

[0017] Figure 5 is a flow chart of a leader node transfer operation according to an embodiment of the present application;

[0018] Figure 6 This is a structural block diagram of a distributed system according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0021] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the leader node transfer method depends, the specific application environment architecture or specific hardware architecture is described here.

[0023] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 FIG. 1 is a hardware structure diagram of a method for transferring a leader node according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 for communication functions and an input and output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0024] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the startup method of the operating system in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0025] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by a communication provider of the server device. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0026] To better understand the Raft algorithm, the following is a detailed explanation: In the initial state, all nodes are followers and have a random timeout period. When a node's timeout countdown ends, it becomes a candidate and sends a voting request to all nodes. If a node obtains votes from more than half of the nodes, it becomes the leader and sends heartbeat messages to other nodes at fixed time intervals. After receiving the heartbeat messages, the other nodes reset their own timeout countdowns, thereby maintaining the follower status. When the entire cluster is operating normally, the leader node receives client requests, generates log entries, stores them locally, and copies the log entries to other nodes. When more than half of the nodes confirm receipt of the log entry, the leader submits the log entry and notifies other nodes to submit the log entry (a committed log entry means that it will not be lost in the future, no matter which node becomes the leader). If the Leader node fails, the Follower will not receive the heartbeat information before the timeout countdown ends. In this case, it will become a Candidate and send a voting request. The node that obtains more than half of the votes will become the new Leader, and the other nodes will maintain the role of Follower.

[0027] It should be noted that in related technologies, the best node can only be elected as the leader when the old leader fails and other nodes begin re-election. If the current leader's performance is poor but it is not faulty and the system can continue to operate, the current leader will not be replaced, which will affect the performance of the entire system. For example, the minimum timeout period of a node in the system is 300ms. The current leader node A takes approximately 100ms to complete the processing of a request (from receiving the request to completing the log entry submission), while the other node B, due to its better performance, only needs 30ms to complete these operations. If the leader node is replaced from A to B, the processing time of each request can be shortened by 70ms. However, with the current technical solution, although the processing speed of node A is slow, it will not cause a timeout, so node B will never be able to become the new leader, and the entire system can only complete each request in 100ms.

[0028] It should be noted that a node suitable for being a leader should have good performance, low load, and low network latency. However, in a distributed system running the Raft algorithm, the status of each node will change in real time. Possible reasons are as follows: (1) System expansion and contraction, machine maintenance and repair will lead to the addition and removal of nodes. Even if a machine does not exit the system, the performance will change due to hot plugging of hardware such as hard disks and memory; (2) Due to changes in tasks, the load of each node may change at any time; (3) Changes in the physical network, the addition and removal of nodes, and changes in traffic may all lead to changes in network latency. Due to the above reasons, the node most suitable for being a leader will also change. If the leader cannot switch to the best node in time, the speed and throughput of the entire system in processing requests will decrease. In order to solve this problem, this application monitors the status of nodes in the system in real time and adaptively adjusts the leader node so that the node with the best overall situation can become the leader in time.

[0029] Specifically, this application provides a mechanism for adaptively optimizing the selection of leader nodes in a distributed system consensus algorithm, which can effectively cope with the challenges of dynamic changes in node performance in the system, ensure that the distributed system always runs under the leader node in the best state, and significantly improve the overall performance and stability of the distributed system. In this embodiment, a leader node transfer method is provided, which is applied to the leader node in a distributed system. Figure 2 This is a flow chart (1) of a method for transferring a leader node according to an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps S202-S208:

[0030] Step S202: determining the parameter value of the performance parameter of the leader node according to the hardware performance, load condition and network delay of the leader node to obtain the target parameter value;

[0031] It's important to note that the leader node calculates its target parameter value using a specific quantitative evaluation method, taking into account its own hardware performance, current workload, and network latency with other nodes in the distributed system. This target parameter value represents a snapshot of the leader node's performance at the current moment. The calculation of target parameter values ​​isn't limited to static metrics; it also considers dynamic factors to ensure the accuracy and timeliness of the evaluation results.

[0032] Step S204: Obtain N parameter values ​​corresponding to N follower nodes in the distributed system, wherein the i-th parameter value among the N parameter values ​​is the parameter value of the performance parameter of the i-th follower node determined by the i-th follower node among the N follower nodes based on the hardware performance, load condition, and network delay of the i-th follower node;

[0033] It should be noted that the method for the follower node to determine its own performance parameter value is the same as that of the leader node.

[0034] It should be noted that there is no specific order in which step S202 and step S204 are performed.

[0035] Step S206: determining a candidate leader node from the N follower nodes and the leader node according to the target parameter value and the N parameter values;

[0036] It should be noted that in this step, the leader node comprehensively compares and analyzes the parameter values ​​of all nodes it has collected, including its own target parameter values ​​and the parameter values ​​of the follower nodes. Using predefined decision logic or algorithms, it determines a candidate with the best performance and the most suitable candidate to become the leader node.

[0037] Step S208: When it is determined to transfer the leader node, a leader node transfer operation is performed to make the candidate leader node become a new leader node in the distributed system.

[0038] The above steps quantify the hardware performance, load conditions, and network latency of the leader node and follower nodes to form a series of comparable parameter values. A candidate leader node is then determined based on these parameter values. When the transfer conditions are met, the leader node transfer operation is executed. This avoids the performance bottleneck caused by the long-term fixation of the leader node, ensures that the distributed system can adaptively select the leader node according to the real-time status of the node, thereby improving the performance of the distributed system and solving the problem that in the consensus algorithm, the leader node is fixed, which leads to poor performance of the distributed system.

[0039] In an exemplary embodiment, the above step S202 can be implemented by the following steps S11-S14:

[0040] Step S11: using a hardware evaluation tool to determine a first parameter value according to the hardware information of the leader node, wherein the first parameter value is used to reflect the hardware performance of the leader node;

[0041] Alternatively, you can use standardized hardware performance evaluation tools, such as UnixBench and SPEC CPU, to comprehensively assess the hardware capabilities of the leader node. These tools provide detailed performance metrics for processors, memory, and disk I / O, and quantify the node's hardware processing capabilities by running a series of benchmark tests.

[0042] Step S12: determining the processor usage, memory usage, and input / output usage of the leader node; performing a weighted sum of the processor usage, memory usage, and input / output usage to obtain a second parameter value, wherein the load condition includes the processor usage, memory usage, and input / output usage of the leader node;

[0043] It's important to note that the leader node's real-time load also affects its effectiveness as the system commander. This step involves monitoring and calculating the leader node's processor utilization, memory utilization, and input / output (I / O) utilization. These load metrics can be obtained using built-in system monitoring commands or third-party tools.

[0044] Step S13: determining a third parameter value according to a network delay between the leader node and each of the N follower nodes;

[0045] It's important to note that network latency is a crucial factor in distributed systems, directly impacting the efficiency and stability of inter-node communication. In step S13, the leader node calculates the network latency between the leader and each follower node by recording the timestamps of messages sent and received between the leader and each follower node. The leader then takes the average or median of these timestamps to generate a more accurate indicator of network latency.

[0046] It should be noted that there is no specific order in which steps S11 to S13 are to be performed.

[0047] Step S14: determining a target parameter value according to the first parameter value, the second parameter value, and the third parameter value.

[0048] It should be noted that this embodiment implements a comprehensive quantitative assessment of leader node performance, including hardware performance, load, and network latency. This multi-dimensional assessment mechanism accurately determines the values ​​of the leader node's performance parameters. Furthermore, based on real-time performance data, the distributed system can more flexibly respond to hardware changes, task fluctuations, and network environment uncertainties, maintaining optimal performance.

[0049] In an exemplary embodiment, the above step S14 includes: determining the target parameter value by the following formula:

[0050]

[0051] Wherein, S is the target parameter value, S1 is the first parameter value, S2 is the second parameter value, and S3 is the third parameter value.

[0052] It should be noted that, from the above formula, it can be seen that the node with better hardware performance, lower load, and lower network latency has a higher total score.

[0053] In an exemplary embodiment, obtaining N parameter values ​​corresponding to N follower nodes in a distributed system includes: obtaining the i-th parameter value and obtaining N parameter values ​​by the following method: obtaining the i-th parameter value from the message that the i-th follower node replies to the leader node.

[0054] It should be noted that when a follower node in a distributed system replies to a leader node's message, it may include parameter values ​​determined by itself in the reply message. The reply message includes but is not limited to:

[0055] Heartbeat Response Message: The leader periodically sends heartbeat signals to all followers to confirm their active status. When a follower receives a heartbeat signal from the leader, it resets its own timeout timer and sends a heartbeat response message (often called an AppendEntriesRpc response) to the leader to confirm receipt of the heartbeat signal. In this response, the follower can append customized parameter values ​​to inform the leader of its current status.

[0056] Log Replication Response Message: When a leader receives a write request from a client, it converts it into a log entry and stores it in its local log. This log entry is then replicated to all follower nodes. After successfully accepting the log entry, the follower node sends a response confirmation (also called an AppendEntriesRpc response) to the leader, indicating successful log replication. Similarly, the follower can include its own parameter values ​​in this response message.

[0057] It should be noted that this embodiment provides an efficient, real-time data collection mechanism that allows the leader node to seamlessly extract parameter values ​​reflecting the performance status of each node from regular communications with follower nodes, without requiring additional query or reporting processes. This approach reduces system overhead, improves the frequency and accuracy of data collection, and ensures that the leader node can make more accurate leadership transition decisions based on the latest and most comprehensive performance information, thereby dynamically optimizing the operational efficiency and overall performance of the distributed system.

[0058] In an exemplary embodiment, a candidate leader node is determined from N follower nodes and a leader node based on a target parameter value and N parameter values, including: selecting a highest parameter value from the target parameter value and the N parameter values, wherein the higher the parameter value of the node on the performance parameter, the better the performance of the node; and determining the node corresponding to the highest parameter value from the N follower nodes and the leader node as the candidate leader node.

[0059] It should be noted that the leader node can select the highest parameter value from the target parameter value and N parameter values. This highest parameter value represents the performance of the best-performing node in the system. It should be noted that the higher the parameter value of the node in the performance parameter, the stronger its hardware capabilities, the lower the load, the shorter the network latency, and thus the better the overall performance.

[0060] It's important to note that once the highest parameter value is determined, the leader node will check whether it was generated by the leader itself or by a follower node. If the node with the highest parameter value is one of the follower nodes, that follower node will be marked as a candidate leader node. If the leader node's target parameter value is the highest parameter value, the current leader node remains in its position and does not need to be replaced.

[0061] It should be noted that by implementing the above mechanism, this application can accurately and in real time evaluate the performance of all nodes in the system and adaptively determine candidate leadership nodes based on these quantitative parameters, ultimately achieving optimal selection and timely transfer of leadership nodes. This dynamic decision-making mechanism significantly improves the performance stability and efficiency of distributed systems.

[0062] In an exemplary embodiment, the method further includes: determining whether to transfer the leader node by the following method 1, method 2, method 3, or method 4:

[0063] Method 1: If the candidate leader node is not the current leader node and the time the current leader node has been the leader node is greater than the preset time, the leader node is transferred;

[0064] It should be noted that this method emphasizes the consideration of the leadership node's tenure. Specifically, as long as the candidate leadership node is different from the current leadership node, and the time the current leadership node has served in the leadership role exceeds the system-preset time threshold, the transfer of the leadership node is triggered. The preset time (for example, 10 minutes) is set to avoid too frequent leadership changes, which bring unnecessary fluctuations and overhead to the system. This method is simple and direct, and is suitable for most distributed systems with high stability requirements. When the node performance changes reach a certain level, the leadership can be adjusted in a timely manner, thereby optimizing the overall performance of the system.

[0065] Method 2: If the candidate leader node is neither the current leader node nor the previous leader node, and the time since the current leader node became the leader node is greater than the preset time, the leader node is transferred;

[0066] It's important to note that in Method 2, in addition to the time threshold requirements in Method 1, an additional condition is added: the candidate leader node cannot be the current leader node or the leader node from the previous term. This restriction prevents the leader node from frequently "swinging" between two specific nodes, preventing potential performance or stability issues. This method is particularly suitable for system environments that have experienced rapid leadership node rotation. By limiting the possibility of the former leader node becoming a candidate again, the system can more smoothly transition to the new leadership structure.

[0067] Method 3: If the candidate leader node is not the current leader node, the time since the current leader node became the leader node is greater than the preset time, and the target parameter value of the current leader node is less than the target parameter value and half of the highest parameter value among N parameter values, the leader node is transferred;

[0068] It's important to note that Method 3 introduces a more stringent performance criterion. In addition to the time threshold requirement, it also incorporates a performance gap between the current leader and the node with the highest parameter value. Leadership transfer is triggered only when the current leader's target parameter value falls below half of the highest parameter value. This means that if the current leader's performance deteriorates significantly, leaving a significant gap with other nodes in the system, urgent action will be taken to transition leadership.

[0069] Method 4: When the candidate leader node is not the current leader node, the time since the current leader node became the leader node is greater than the preset time, and the parameter values ​​of the performance parameters of more than half of the N follower nodes and the leader node are greater than the target parameter value, the leader node is transferred.

[0070] It should be noted that method 4 focuses on the overall performance of the system. As long as the candidate leader node is different from the current leader node, the incumbent time of the current leader node exceeds the preset threshold, and the parameter values ​​of the performance parameters of more than half of the nodes in the system (including the candidate leader node) are higher than the target parameter values, it is determined that the leader node needs to be transferred. This method embodies the principle of "democracy" and ensures that the transfer of leadership is not only based on the excellent performance of a single node, but also takes into account the overall performance level of the majority of nodes in the system, thereby promoting the balanced development of the performance of the entire system and improving the overall processing capacity and stability of the system.

[0071] It should be noted that in this embodiment, the above-mentioned strategies enable appropriate leadership node transfer decisions to be made under varying system environments and performance contexts. These rules not only guarantee the system's basic requirements for leadership node performance, but also balance long-term stability, emergency response in extreme situations, and overall system performance. This demonstrates a high degree of flexibility and intelligence, effectively resolving the issue of leader node selection and performance matching in traditional Raft algorithms.

[0072] In an exemplary embodiment, the leader node transfer operation may be performed through the following steps S21-S22:

[0073] Step S21: Stop accepting client requests and replicate logs to the candidate leader node. During the process of replicating logs to the candidate leader node, heartbeat information is sent to each of the N follower nodes as normal.

[0074] Step S22: When the log is successfully copied to the candidate leader node, a heartbeat message is sent to each of the N follower nodes, and a trigger message is sent to the candidate leader node, where the trigger message is used to instruct the candidate leader node to send a voting request to become the leader node.

[0075] It should be noted that the leadership node transfer operation proposed in this application effectively realizes the smooth transition of leadership and performance optimization in the distributed system through steps S21 and S22. Specifically, the current leadership node suspends receiving new client requests. This measure is intended to avoid data conflicts or consistency issues during the transfer process. Next, the leadership node copies the log entries to the selected candidate leadership node. This process is a necessary preparation before the transfer of leadership, which ensures that the candidate leadership node has a high probability of winning the election and becoming the new leadership node. It is worth noting that while copying the log, the system continues to send heartbeat information to all follower nodes. This mechanism maintains the active state of the cluster, avoids other follower nodes from erroneously triggering leadership elections due to not receiving heartbeats for a long time, and ensures the stability and consistency of the entire transfer process.

[0076] Furthermore, once log replication is successful, indicating that the candidate leader has fully synchronized with the cluster state, the leader node immediately sends a heartbeat message to all follower nodes. This resets each node's timeout, reducing the likelihood of elections caused by timeouts and accelerating the formal transfer of leadership. Subsequently, the system sends a trigger message to the candidate leader, instructing it to initiate the leadership election process. Because the candidate node performs best in log synchronization and performance evaluation, its election request virtually guarantees success, ensuring a smooth and rapid transition of leadership to the best-performing node.

[0077] It should be noted that the above steps ensure smooth and efficient transfer of the leader node, and achieve data consistency and system stability by pausing the acceptance of new requests, synchronizing logs, and maintaining heartbeats.

[0078] In an exemplary embodiment, the method further includes: performing weighted summation of the processor usage, memory usage, and input / output usage to obtain a second parameter value, and the method further includes: determining the current operating scenario of the distributed system; and determining the weight values ​​corresponding to the processor usage, memory usage, and input / output usage based on the current operating scenario.

[0079] It should be noted that in the process of scoring nodes and determining whether they are suitable as leader nodes, this application introduces an innovative mechanism, namely scenario-based weighted parameter evaluation. The core of this mechanism is that it not only takes into account the processor utilization, memory utilization, and input and output (I / O) utilization of the node, but also dynamically adjusts the weights of these performance indicators according to the current operating scenario of the distributed system. Specifically, the system will first identify the current operating status or workload type. For example, the system may be in different scenarios such as high-concurrency request processing, large data transmission, or memory-intensive computing. After identifying the current scenario, the system will automatically calculate and assign weight values ​​for each performance indicator based on the characteristics of the scenario to reflect which indicators have a more critical impact on system performance under the current workload.

[0080] For example, in scenarios with high concurrent requests, processor utilization may become a key factor in determining system response speed and throughput, so the system will give it a higher weight. However, when performing large-scale data transfers, I / O utilization may receive a significantly higher weight, as it directly impacts data transfer efficiency and system bottlenecks. This allows the system to more intelligently assess the overall performance of nodes, ensuring the selection of the most suitable leader node in various operating scenarios, ensuring efficient and stable cluster operation.

[0081] In this embodiment, by adjusting the weights of performance indicators according to the actual operation scenario, the performance bottlenecks of the nodes under the current workload can be identified more accurately, thereby improving the pertinence and accuracy of the scoring mechanism.

[0082] Obviously, the embodiments described above are only part of the embodiments of the present application, rather than all the embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present application. Specifically:

[0083] This application proposes an adaptive Raft performance improvement algorithm. This method monitors the hardware performance, load, and network latency of each node in the distributed system in real time, selects the node that is most suitable for the leader, and automatically transfers the leader when certain conditions are met to ensure the performance of the leader node, thereby ensuring that the entire system always runs in the best condition. For a better understanding, Figure 3 The scenario module diagram of this application is shown.

[0084] For better understanding, Figure 4 This is a flow chart (II) of a method for transferring a leader node according to an embodiment of the present application, such as Figure 4 As shown, the solution proposed in this application includes the following parts:

[0085] (1) Node scoring:

[0086] Each node scores itself locally. The score consists of three parts: hardware performance, load, and network latency. The specific scoring method is as follows:

[0087] Hardware performance: Use the UnixBench evaluation tool for evaluation and scoring (other evaluation tools such as SPEC CPU are also acceptable). The result is recorded as S1. The evaluation tool is only run during initialization and hardware changes. S1 is not updated at other times.

[0088] S1=Score Unixbench ;

[0089] Load situation: including CPU usage, memory usage, and I / O usage. The CPU usage is obtained using commands such as top and vmstat. CPU Indicates; memory usage is obtained using commands such as free and top, and is expressed as ρ mem Indicates; I / O usage is obtained using iostat, using ρ IO The load score is recorded as S2, and the calculation formula is as follows. Each node updates S2 every time the timeout is reset:

[0090] S2=0.6×ρ CPU +0.2×ρmem +0.2×ρ IO ;

[0091] Network latency: The sending time of each message (including the message sent by the leader and the message replied by other nodes) is recorded. After receiving the message, each node calculates the difference between the time it receives the message and the time it sends the message, which is the network latency with the node that sends the message (the delay between the current node and node i is called latency). i The average network delay between a node and all other nodes is recorded as the node's network delay score, denoted as S3. S3 is updated every time a message is received. In the following formula, n is the number of nodes other than the current node.

[0092]

[0093] It should be noted that the calculation formula for the total score is:

[0094]

[0095] This formula means that nodes with better performance, lower load, and lower network latency will have higher overall scores.

[0096] (2) Optimal node decision:

[0097] When each node replies to a Leader message, it attaches its own score S to the message and sends it to the Leader. The Leader compares its own score with the scores of each node to select the best node and determine whether a leadership transfer is necessary. To prevent frequent leadership transfers and the leader switching back and forth between two nodes, the system only makes a leadership transfer decision when more than 10 minutes have passed since the last leadership transfer, the current leader score is less than 1 / 2 of the highest score, and the highest-scoring node is not the previous leader.

[0098] (3) Leader transfer:

[0099] When a leader transfer decision is made, the system will execute the leader transfer. The specific steps are as follows: Figure 5 As shown, it should be noted that Figure 5In step 2, since the log replication time may be long, the leader continues to send heartbeat messages to other nodes to prevent other nodes from starting elections and disrupting the leader transfer. In step 3, after the log is copied, the leader first sends a heartbeat message. This operation can reset the timers of other nodes, thereby preventing other nodes from starting elections. Then the leader immediately sends a message to trigger the election of the target node (that is, the candidate leader node), so that the target node has a high probability of winning the election. In step 4, if the target node does not win the election, it means that during this process, the situation of each node has changed (for example, the network is congested), and the target node may no longer be the best node. At this time, the target node will become a follower again, and the original leader will continue to maintain its role until the next decision to transfer the leader is made and the relevant operations are performed again.

[0100] It should be noted that the technical solution of this application has the following technical effects:

[0101] (1) The mechanism of scoring each node in real time can timely and accurately reflect the changes in hardware, load, network, etc., and select the node most suitable for the leader.

[0102] (2) The mechanism of adaptive leader transfer can ensure that the leader is always the node with the best overall performance, ensuring that the entire system runs at the best performance. In related technologies, since the processing of each client request, the replication and submission of log entries need to go through the leader, when the leader node has poor performance, or the load is high, or the network delay is large, the performance of the entire system will be slowed down; however, in the technical solution of this application, the leader node always performs the best. According to the characteristics of the consensus algorithm, as long as half of the nodes complete the replication and submission of log entries, the system can continue to process subsequent transactions. A small number of nodes with poor performance will not affect the overall performance. Therefore, the technical solution proposed in this application can enable the system to always run at the best performance.

[0103] It should be noted that this application is suitable for all consensus algorithms that use the Leader mechanism, and can also be applied to other scenarios that require a Leader role, not just the Raft algorithm.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0105] The embodiment of the present application also provides a distributed system, Figure 6 This is a block diagram of a distributed system according to an embodiment of the present application. Figure 6 As shown, the system includes:

[0106] The leader node 602 is configured to determine a parameter value of a performance parameter of the leader node 602 according to the hardware performance, load condition, and network delay of the leader node 602 to obtain a target parameter value;

[0107] N follower nodes 604, wherein each follower node 604 is configured to determine a parameter value of a performance parameter of the follower node 604 according to the hardware performance, load condition, and network delay of the follower node 604, and send the parameter value to the leader node 602;

[0108] The leader node 602 is also used to determine a candidate leader node from the N follower nodes 604 and the leader node 602 based on the target parameter value and the N parameter values ​​obtained, and when it is determined that the leader node is to be transferred, perform a leader node transfer operation to make the candidate leader node become the new leader node in the distributed system.

[0109] The above system quantifies the hardware performance, load conditions and network delay of the leader node and follower nodes to form a series of comparable parameter values, and then determines a candidate leader node based on these parameter values. When the transfer conditions are met, the leader node transfer operation is executed, thereby avoiding the performance bottleneck caused by the long-term fixation of the leader node, ensuring that the distributed system can adaptively select the leader node according to the real-time status of the node, thereby improving the performance of the distributed system and solving the problem that in the consensus algorithm, the leader node is fixed, which leads to poor performance of the distributed system.

[0110] In an exemplary embodiment, the leader node 602 is further used to use a hardware evaluation tool to determine a first parameter value based on the hardware information of the leader node, wherein the first parameter value is used to reflect the hardware performance of the leader node; and to determine the processor usage, memory usage, and input / output usage of the leader node; to perform weighted summation on the processor usage, memory usage, and input / output usage to obtain a second parameter value, wherein the load condition includes the processor usage, memory usage, and input / output usage of the leader node; and to determine a third parameter value based on the network delay between the leader node and each of the N follower nodes; and to determine a target parameter value based on the first parameter value, the second parameter value, and the third parameter value.

[0111] In an exemplary embodiment, the leader node 602 is further configured to determine the target parameter value using the following formula:

[0112]

[0113] Wherein, S is the target parameter value, S1 is the first parameter value, S2 is the second parameter value, and S3 is the third parameter value.

[0114] In an exemplary embodiment, the leader node 602 is further configured to obtain parameter values ​​corresponding to the follower nodes from the message in which the follower nodes reply to the leader node.

[0115] In an exemplary embodiment, the leader node 602 is further used to select a highest parameter value from the target parameter value and N parameter values, wherein the higher the parameter value of the node on the performance parameter, the better the performance of the node; and the node corresponding to the highest parameter value among the N follower nodes and the leader node is determined as a candidate leader node.

[0116] In an exemplary embodiment, the leader node 602 is also used to determine whether to transfer the leader node in the following manner: when the candidate leader node is not the current leader node and the time when the current leader node becomes the leader node is greater than a preset time, the leader node is transferred; when the candidate leader node is not the current leader node or the previous leader node, and the time when the current leader node becomes the leader node is greater than a preset time, the leader node is transferred; when the candidate leader node is not the current leader node, the time when the current leader node becomes the leader node is greater than a preset time, and the target parameter value of the current leader node is less than the target parameter value and half of the highest parameter value among N parameter values, the leader node is transferred; when the candidate leader node is not the current leader node, the time when the current leader node becomes the leader node is greater than a preset time, and the parameter values ​​of the performance parameters of more than half of the N follower nodes and the leader node are greater than the target parameter value.

[0117] In an exemplary embodiment, the leader node 602 is further used to perform a leader node transfer operation in the following manner: stop accepting client requests, copy logs to the candidate leader node, and normally send heartbeat signals to each of the N follower nodes during the process of copying logs to the candidate leader node; if the logs are successfully copied to the candidate leader node, send a heartbeat signal to each of the N follower nodes once, and send a trigger message to the candidate leader node, wherein the trigger message is used to instruct the candidate leader node to send a voting request to become the leader node.

[0118] For the description of the features in the embodiment corresponding to the distributed system, please refer to the relevant description of the embodiment corresponding to the method for transferring the leader node, which will not be repeated here.

[0119] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the method for transferring a leader node.

[0120] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned embodiments of the leader node transfer method when running.

[0121] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0122] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned embodiments of the method for transferring a leader node are implemented.

[0123] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned leadership node transfer method embodiments.

[0124] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The above is a detailed introduction to the method and device for transferring a leader node, an electronic device, a storage medium, and a computer program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for transferring a leader node, characterized in that: Leader nodes used in distributed systems include: Determining a parameter value of a performance parameter of the leader node according to the hardware performance, load condition, and network delay of the leader node to obtain a target parameter value; and Obtaining N parameter values ​​corresponding to N follower nodes in the distributed system, wherein an i-th parameter value among the N parameter values ​​is a parameter value of the i-th follower node on the performance parameter determined by the i-th follower node among the N follower nodes based on the hardware performance, load condition, and network delay of the i-th follower node; determining a candidate leader node from the N follower nodes and the leader node according to the target parameter value and the N parameter values; In the case where it is determined to perform a leader node transfer, a leader node transfer operation is performed to make the candidate leader node become a new leader node in the distributed system.

2. The method for transferring a leader node according to claim 1, wherein: The step of determining the parameter value of the performance parameter of the leader node according to the hardware performance, load condition, and network delay of the leader node to obtain the target parameter value includes: Using a hardware evaluation tool, determining a first parameter value according to the hardware information of the leader node, wherein the first parameter value is used to reflect the hardware performance of the leader node; and Determining a processor usage rate, a memory usage rate, and an input / output usage rate of the leader node; performing a weighted summation of the processor usage rate, the memory usage rate, and the input / output usage rate to obtain a second parameter value, wherein the load condition includes the processor usage rate, the memory usage rate, and the input / output usage rate of the leader node; and determining a third parameter value according to a network delay between the leader node and each of the N follower nodes; The target parameter value is determined according to the first parameter value, the second parameter value, and the third parameter value.

3. The method for transferring a leader node according to claim 2, wherein: The determining the target parameter value according to the first parameter value, the second parameter value, and the third parameter value includes: The target parameter value is determined by the following formula: Wherein, S is the target parameter value, S1 is the first parameter value, S2 is the second parameter value, and S3 is the third parameter value.

4. The method for transferring a leader node according to claim 1, wherein: The obtaining N parameter values ​​corresponding to the N follower nodes in the distributed system includes: The i-th parameter value is obtained in the following manner to obtain the N parameter values: The i-th parameter value is obtained from a message that the i-th follower node replies to the leader node.

5. The method for transferring a leader node according to claim 1, wherein: The determining a candidate leader node from the N follower nodes and the leader node according to the target parameter value and the N parameter values ​​includes: Selecting a highest parameter value from the target parameter value and the N parameter values, wherein the higher the parameter value of the node on the performance parameter, the better the performance of the node; A node corresponding to the highest parameter value among the N follower nodes and the leader node is determined as the candidate leader node.

6. The method for transferring a leader node according to claim 1, wherein: The method further comprises: Whether to transfer the leader node is determined by the following methods: If the candidate leader node is not the current leader node and the time since the current leader node became the leader node is greater than a preset time, transfer the leader node; or If the candidate leader node is neither the current leader node nor the previous leader node, and the time since the current leader node became the leader node is greater than the preset time, transferring the leader node; Transfer the leader node if the candidate leader node is not the current leader node, the time when the current leader node became the leader node is greater than the preset time, and the target parameter value of the current leader node is less than the target parameter value and half of the highest parameter value among the N parameter values; or The leader node is transferred when the candidate leader node is not the current leader node, the time when the current leader node becomes the leader node is greater than the preset time, and the parameter values ​​of the performance parameters of more than half of the N follower nodes and the leader node are greater than the target parameter value.

7. The method for transferring a leader node according to claim 1, wherein: The execution of the leadership node transfer operation includes: Stop accepting client requests, and replicate logs to the candidate leader node, and in the process of replicating logs to the candidate leader node, normally send heartbeat information to each of the N follower nodes; When the log is successfully copied to the candidate leader node, a heartbeat message is sent to each of the N follower nodes, and a trigger message is sent to the candidate leader node, wherein the trigger message is used to instruct the candidate leader node to send a voting request to become the leader node.

8. A distributed system, characterized in that: include: The leader node is configured to determine a parameter value of a performance parameter of the leader node according to the hardware performance, load condition, and network delay of the leader node to obtain a target parameter value; N follower nodes, wherein each of the N follower nodes is configured to determine a parameter value of the performance parameter of the follower node based on the hardware performance, load condition, and network delay of the follower node, and send the parameter value to the leader node; The leader node is also used to determine a candidate leader node from the N follower nodes and the leader node based on the target parameter value and the N parameter values ​​obtained, and when it is determined that the leader node is to be transferred, perform a leader node transfer operation so that the candidate leader node becomes the new leader node in the distributed system.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the leader node transfer method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the leader node transfer method according to any one of claims 1 to 7 are implemented.

Images