Large-scale data consistency method and device based on multi-stage tree structure cluster

Abstract

The invention discloses a large-scale data consistency method and device based on a multi-level tree structure cluster, and the method comprises the steps: constructing the multi-level tree structure cluster, distributing an i-level cluster for a follower in an i-1-level cluster as a corresponding subtree, and enabling a leader in the i-level cluster to be the follower in the i-1-level cluster; the i-level leader triggers own data collection according to the heartbeat detection message, and feeds back the collected data to the i-1-level leader; the (i-1)-level leader carries out consistency comparison on the collected data, if the collected data are not consistent, consistent data construction is carried out, a synchronization log of the constructed data is sent to the i-level leader, and the i-level leader carries out log synchronization after verification; and the i-level leader sends a log copy message to the i-level follower in the sub-tree where the i-level leader is located, and the i-level follower executes log synchronization after verification. The method has relatively low election performance loss, and realizes the capability of adapting to a large-scale distributed system.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed systems, and in particular to a large-scale data consistency method and device based on a multi-level tree structure cluster. Background Art

[0002] With the rapid development of cloud computing and big data technology, enterprises and users have increasing requirements for data processing capabilities and system availability. In this context, distributed systems and microservice architectures have become mainstream technologies. However, data consistency and high availability in distributed systems (especially large-scale systems) are still key issues that need to be solved.

[0003] Traditional distributed systems usually use master-slave replication or distributed consensus algorithms (such as Paxos, Raft, etc.) to ensure data consistency. Among them, the Raft algorithm has attracted widespread attention for its simplicity and ease of understanding. Raft ensures data consistency between multiple nodes in a distributed system through leader election, log replication and security mechanisms.

[0004] However, the current consensus algorithm still has some shortcomings, taking the Raft algorithm as an example:

[0005] First, the election process may cause performance problems: The election process of the Raft algorithm may cause performance problems because when a network partition or node crash occurs, a new leader needs to be re-elected, which not only affects the availability of the system but also causes data inconsistency problems.

[0006] Second, it is not suitable for large-scale distributed systems: In large-scale distributed systems, the election process of the Raft algorithm involves more nodes, and each election requires more message transmission and computing resources. The election process may cause excessive network load, which will reduce the overall performance of the system. At the same time, the leader needs to send heartbeat and log replication messages to a large number of followers. As the number of nodes increases, network traffic will also increase significantly, which may cause network congestion and performance degradation. If the bandwidth of a large-scale distributed system is insufficient, the message transmission speed will slow down, affecting the system's throughput, so it is not suitable for large-scale distributed systems.

[0007] Therefore, how to provide a large-scale data consistency method and device based on a multi-level tree structure cluster that is applicable to a distributed system and has high availability is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0008] In view of the above research status and existing problems, the present invention provides a large-scale data consistency method and device based on a multi-level tree structure cluster, which has low election performance loss and realizes the ability to adapt to large-scale distributed systems.

[0009] The present invention provides a large-scale data consistency method based on a multi-level tree structure cluster, comprising the following steps:

[0010] Construct a multi-level tree structure cluster, the cluster includes: a first-level cluster and an i-level cluster, i∈[2,N], N is a positive integer greater than 2; the consensus node status in each of the clusters includes: leader, follower and candidate; assign an i-level cluster as its corresponding subtree to the follower in the i-1-level cluster, and the leader in the i-level cluster is the follower in the i-1-level cluster;

[0011] The i-1 level leader sends a heartbeat detection message to the i-level leader in its corresponding subtree based on the polling heartbeat duration; the i-level leader triggers its own data collection according to the heartbeat detection message, and feeds back the collected data to the i-1 level leader;

[0012] The i-1 level leader performs consistency comparison on the collected data. If there is inconsistency, it constructs the consistent data and sends the synchronization log of the constructed data to the i-level leader. The i-level leader performs log synchronization after verification.

[0013] The level i leader sends a log replication message to the level i followers in its subtree, and the level i followers perform log synchronization after verification.

[0014] Preferably, the step of constructing a multi-level tree structure cluster includes an election step:

[0015] The current-level candidate is triggered to elect a leader by the current-level follower failing to receive the heartbeat detection message from the current-level leader;

[0016] When the i-1-level candidate becomes the i-1-level leader through the voting mechanism, the remaining i-1-level consensus nodes update their own consensus node status to followers, and the i-1-level leader sends the i-1-level election result notification to the i-level leader in its corresponding subtree; the i-level leader sends the i-1-level election result notification to the i-level followers in its subtree; the i-level followers continue to pass the i-1-level election result notification downward until the consensus node has no corresponding subtree;

[0017] When the i-level candidate becomes the i-level leader through the voting mechanism, the remaining i-level consensus nodes update their own consensus node status to followers, and the i-level leader sends the i-level election result notification to the corresponding i-1-level leader in its subtree, and the i-level leader continues to pass the i-level election result notification upward to the first-level leader; the i-1-level leader sends the i-level election result notification to the remaining i-1-level followers, and the remaining i-1-level followers continue to pass the i-level election result notification downward until there is no corresponding subtree for the consensus node;

[0018] The consensus node that receives the election result notification adjusts the node information according to the election result notification.

[0019] Preferably, the voting mechanism includes:

[0020] The current-level candidate sends an election request message to other consensus nodes at the current level;

[0021] The current and other consensus nodes verify the status information of the current and candidate nodes, and send voting information to the current candidate nodes after verification;

[0022] The current-level candidate accumulates the voting information, and when the amount of the voting information meets the requirement of the minimum number of consensus nodes, the current-level candidate updates its status to a leader, and the remaining current-level consensus nodes update their status to followers.

[0023] Preferably, the election step further includes:

[0024] If the i-1 level leader still does not receive the i-level election result notification sent by the i-level leader in its corresponding subtree after looping a specified number of polling heartbeat duration periods, the following steps are performed:

[0025] The i-1 level leader sends a survival detection message to all consensus nodes in its corresponding subtree, and designates a consensus node among the consensus nodes that have successfully survived the survival verification as the i-level leader of its corresponding subtree;

[0026] The designated consensus node updates its status to the level i leader, sends a level i election result notification to the remaining consensus nodes in the subtree, and returns an update success message to the level i-1 leader; the remaining consensus nodes update their status to the level i follower;

[0027] The i-1 level leader adjusts the node information according to the update success information, and the i-1 level leader continues to pass the update success information upward to the first-level leader; and sends the i-level election result notification to the remaining i-1 level followers; the remaining i-1 level followers continue to pass the i-level election result notification downward until the consensus node has no corresponding subtree;

[0028] The consensus node that receives the i-level election result adjusts the node information according to the i-level election result notification.

[0029] Preferably, the step of designating a consensus node as the level i leader of its corresponding subtree among the consensus nodes that have successfully survived the verification includes any one of the following:

[0030] Random designation: designate a consensus node among the consensus nodes that have successfully verified survival as the i-level leader of its corresponding subtree;

[0031] Comparative log designation: compare the log states of the consensus nodes that have successfully passed the survival verification, and designate the consensus node with the latest log update time as the i-level leader of its corresponding subtree.

[0032] Preferably, the step of constructing a multi-level tree structure cluster includes a log replication step:

[0033] The level i leader receives the log replication message sent by the level i-1 leader, performs log synchronization after verification, and sends a log replication message to the level i follower. The level i follower performs log synchronization after verification and replies with a level i response message to the level i leader;

[0034] The level i leader accumulates the level i response messages, and when the number of the level i response messages meets the requirement of the minimum number of consensus nodes, the level i leader replies with a level i-1 response message to the level i-1 leader;

[0035] The i-1 level leader accumulates the i-1 level response messages, and when the number of the i-1 level response messages meets the requirement of the minimum number of consensus nodes, the i-1 level leader updates the client request corresponding to the log to the state machine for execution.

[0036] Preferably, the requirement for the minimum number of nodes includes: greater than half of the number of consensus nodes in the current level cluster.

[0037] Preferably, the step of sending a heartbeat detection message includes:

[0038] The level i leader receives the heartbeat detection message sent by the level i-1 leader based on the polling heartbeat duration; the level i leader sends a heartbeat detection message to the level i follower based on the polling heartbeat duration;

[0039] The consensus node triggers the generation of candidates from the consensus nodes of the current cluster based on the failure to receive a heartbeat detection message within the polling heartbeat duration.

[0040] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the large-scale data consistency method based on a multi-level tree structure cluster are implemented.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the large-scale data consistency method based on a multi-level tree structure cluster are implemented.

[0042] This paper proposes a large-scale data consistency cluster solution based on multi-level tree structure governance, aiming to solve the performance and scalability problems faced by the traditional Raft algorithm in large-scale distributed systems. By transforming the original single-layer Raft model into a multi-level cluster structure, it has the following beneficial effects compared with the existing technology:

[0043] Adaptation to large-scale distributed systems: The present invention expands the number of clusters to the power of the previous limit through a multi-level tree structure. This scalability enables the present invention to be applied to larger-scale distributed systems to meet the needs of enterprises for high availability and large-scale data processing. The present invention helps to achieve load balancing. In a multi-level structure, log replication and heartbeat message transmission are shared by multiple leader nodes instead of being concentrated on a single node. This effectively disperses the network load and improves the overall performance and stability of the system.

[0044] Improve data consistency: The present invention ensures data consistency in a multi-level cluster through a multi-level log replication process. The i-1 level leader distributes log entries to the i-level leader, which then distributes them to its followers. This level-by-level synchronization mechanism not only improves the reliability of data transmission, but also reduces the burden of a single network request.

[0045] Enhanced system availability: The present invention improves the partition fault tolerance of the system through a multi-level structure and a hierarchical election mechanism. Even in the case of network partitions, each sub-cluster can still operate independently, ensuring that some functions of the system are not affected.

[0046] Reduce election performance loss: The present invention adopts a hierarchical election mechanism and divides the election process into multiple levels of elections. For example, the first-level election is responsible for selecting the leader of the first-level cluster, and the second-level election selects the leader within each second-level cluster. This hierarchical election mechanism reduces the complexity and network load of elections in large-scale clusters, and avoids performance degradation caused by frequent elections. The present invention can achieve rapid fault recovery: when a node in a level i cluster fails, the level i-1 leader can quickly detect and designate a new level i leader, ensuring that the system resumes normal operation in a short time, thereby improving the availability and response speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

[0048] Figure 1 A schematic diagram of the structure of a multi-level tree structure cluster provided by an embodiment of the present invention;

[0049] Figure 2 A flowchart of a large-scale data consistency method based on a multi-level tree structure cluster provided by an embodiment of the present invention;

[0050] Figure 3 A first-level election flow chart of a large-scale data consistency method based on a multi-level tree structure cluster provided by an embodiment of the present invention;

[0051] Figure 4 A secondary election flow chart of a large-scale data consistency method based on a multi-level tree structure cluster provided by an embodiment of the present invention;

[0052] Figure 5 A flowchart of secondary election failure processing for a large-scale data consistency method based on a multi-level tree structure cluster is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] A first aspect of an embodiment of the present invention discloses a large-scale data consistency method based on a multi-level tree structure cluster, comprising the following steps:

[0055] Construct a multi-level tree structure cluster, the cluster includes: a level-1 cluster and an i-level cluster, i∈[2,N], N is a positive integer greater than 2; the consensus node states in each cluster include: leader, follower and candidate; assign an i-level cluster to the follower in the i-1-level cluster as its corresponding subtree, and the leader in the i-level cluster is the follower in the i-1-level cluster; the i-1-level leader sends a heartbeat detection message to the i-level leader in its corresponding subtree based on the polling heartbeat duration; the i-level leader triggers its own data collection based on the heartbeat detection message, and feeds back the collected data to the i-1-level leader; the i-1-level leader compares the collected data for consistency, and if it is inconsistent, it builds consistent data and sends the synchronization log of the constructed data to the i-level leader, and the i-level leader performs log synchronization after verification; the i-level leader sends a log replication message to the i-level follower in its subtree, and the i-level follower performs log synchronization after verification.

[0056] like Figure 1 As shown, an example is given based on a two-level tree structure, and multi-level logic is extended by applying the two-level structure. In the figure, the value of i is 2.

[0057] like Figure 2 As shown, after each follower receives the committed log entry, it applies it to its own replicated state machine to ensure that the state of all nodes is consistent.

[0058] This embodiment takes the Raft algorithm as an example for explanation:

[0059] Raft is a consensus algorithm for managing replication logs. The core concepts of the Raft algorithm include:

[0060] Leader: The leader in the Raft algorithm is responsible for processing client requests, log replication, member changes, etc. In a Raft cluster, there is only one leader.

[0061] Follower: Followers are other nodes in the Raft cluster that receive the leader's log entries and copy them to their own logs. Followers do not actively initiate requests, but respond to requests from the leader.

[0062] Candidate: A candidate is a node in a Raft cluster. During the election process, a node will become a candidate and participate in the leader election.

[0063] Log: The Raft algorithm uses logs to record client requests. The leader sends client requests to followers in the form of log entries, and the followers copy these log entries to their own logs.

[0064] Log replication: The leader sends log entries to followers, and followers append these log entries to their own logs. When most nodes confirm receipt of the log entry, the leader commits the log entry and applies it to the state machine.

[0065] Heartbeat: The leader will periodically send heartbeat messages to followers to maintain its leadership. If a follower does not receive a heartbeat message within a period of time, it will assume that the leader has failed and initiate an election.

[0066] Election: When a follower believes that the leader has failed, it will transform into a candidate and start an election. The candidate will send voting requests to other nodes, and the candidate with the majority of votes will become the new leader.

[0067] Security: The Raft algorithm ensures the security of the system through a series of mechanisms, such as limiting the number of leaders to one at a time and ensuring the consistency of logs.

[0068] CAP: The CAP theorem (Consistency, Availability, Partition tolerance) is a basic concept in distributed systems. It describes that a distributed system can only meet two of its properties when facing network partitions.

[0069] Consistency: Consistency means that all nodes in a distributed system see the same data at the same time. When a node modifies the data, other nodes can immediately see the modification. This consistency model is usually called strong consistency.

[0070] Availability: Availability means that a distributed system can provide services to the outside world at any time, even if some nodes fail. When a node fails, other nodes can still continue to process requests.

[0071] Partition tolerance: Partition tolerance means that a distributed system can continue to provide services when encountering a network partition (i.e., the network connection is interrupted and nodes cannot communicate with each other). A distributed system should be able to maintain normal operation in the event of a network partition.

[0072] Figure 1 There are primary clusters and secondary clusters in the system. Each secondary cluster has a primary cluster node as the master node.

[0073] This multi-level tree structure makes the number of clusters become the power of the previous Raft algorithm limit. Compared with distributing network requests to each first-level node for distribution, this structure can support larger-scale data consistency clusters. Assuming that the previous Raft algorithm can only guarantee the normal application of 32 nodes, it can support 32*32 (1024) nodes for normal application after the transformation.

[0074] In the data consistency process, after the level i follower receives the log replication message sent by the level i leader, it continues to pass it downward until the consensus node has no corresponding subtree. The synchronization log sent to the level i leader by the level i-1 leader after consistency comparison of the collected data includes synchronization log fragments or installation snapshots. The level i leader will also send a heartbeat detection message to the level i followers in its corresponding subtree based on the polling heartbeat duration. As long as the current consensus node has its corresponding subtree as the leader, the consensus node will send a heartbeat detection message to the followers in its corresponding subtree.

[0075] In one embodiment, the step of constructing a multi-level tree structure cluster includes an election step:

[0076] The current-level candidate is triggered to elect a leader by the current-level follower failing to receive the heartbeat detection message from the current-level leader;

[0077] When the i-1 level candidate becomes the i-1 level leader through the voting mechanism, the remaining i-1 level consensus nodes update their own consensus node status to followers. The i-1 level leader sends the i-1 level election result notification to the i-1 level leader in its corresponding subtree; the i-1 level leader sends the i-1 level election result notification to the i-1 level followers in its subtree; the i-1 level followers continue to pass the i-1 level election result notification downward until there is no corresponding subtree for the consensus node. It can be understood that if Figure 3 As shown, the operation of notifying the second-level followers is added to the first-level election process.

[0078] When the i-level candidate becomes the i-level leader through the voting mechanism, the remaining i-level consensus nodes update their own consensus node status to followers. The i-level leader sends the i-level election result notification to the corresponding i-1-level leader in its subtree, and the i-level leader continues to pass the i-level election result notification upward to the first-level leader; the i-1-level leader sends the i-level election result notification to the remaining i-1-level followers, and the remaining i-1-level followers continue to pass the i-level election result notification downward until there is no corresponding subtree for the consensus node. It can be understood that if Figure 4 As shown, the secondary election adds the operation of notifying the first-level leader node on the basis of the election rules. The first-level leader node then notifies other first-level follower nodes, and the first-level follower node then notifies the second-level follower nodes it leads.

[0079] The consensus nodes that receive the election result notification adjust the node information according to the election result notification.

[0080] In this embodiment, the voting mechanism includes:

[0081] The current-level candidate sends an election request message to other consensus nodes at the current level;

[0082] The current and other consensus nodes verify the status information of the current and candidate nodes, such as whether the log directory of the current candidate is more complete than its own log directory. After the verification is passed, the voting information is sent to the current candidate.

[0083] The current-level candidates accumulate voting information. When the number of voting information meets the minimum number of consensus nodes, the current-level candidate updates its status to leader, and the remaining current-level consensus nodes update their status to followers.

[0084] In this embodiment, the requirement for the minimum number of consensus nodes includes: greater than half of the number of consensus nodes in the current level cluster.

[0085] In this embodiment, the election step also includes the step of secondary election failure processing:

[0086] If the i-1-level leader has not received the i-level election result notification sent by the i-level leader in its corresponding subtree after looping the specified number of polling heartbeat duration periods, the following steps are performed:

[0087] The i-1 level leader sends a survival detection message to all consensus nodes in its corresponding subtree, and designates a consensus node among the consensus nodes that have successfully survived the survival verification as the i-level leader of its corresponding subtree;

[0088] The designated consensus node updates its status to level i leader, sends level i election result notification to other consensus nodes in the subtree, and returns update success information to level i-1 leader; other consensus nodes update their status to level i follower;

[0089] The i-1 level leader adjusts the node information according to the update success information, and the i-1 level leader continues to pass the update success information upward to the first-level leader; and sends the i-level election result notification to the remaining i-1 level followers; the remaining i-1 level followers continue to pass the i-level election result notification downward until the consensus node has no corresponding subtree;

[0090] The consensus node that receives the level i election result adjusts the node information according to the level i election result notification.

[0091] It is understandable that if Figure 5As shown in the figure, when the secondary cluster election fails (the number of cluster nodes drops below half, etc.), in order to ensure the normal operation of the primary cluster (election and log replication), a leader will actively detect the survival information of other nodes in the cluster after not receiving the primary leader information reported by the secondary cluster within the limited number of heartbeats, and designate a node to become the secondary leader node. After the secondary leader node is designated, the cluster is adjusted by the two-level leaders to reach a data consistency state.

[0092] It should be noted that the condition for designating a secondary cluster leader is that a single cluster node must not meet half of the nodes, so the primary leader node forcibly designates the secondary cluster node to become the secondary leader, so that data consistency is guaranteed.

[0093] In this embodiment, the step of designating a consensus node as the level i leader of its corresponding subtree among the consensus nodes that have successfully passed the survival verification includes any one of the following:

[0094] Random designation: designate a consensus node among the consensus nodes that have successfully passed the survival verification as the i-level leader of its corresponding subtree;

[0095] Comparative log designation: Compare the log states of the consensus nodes that have successfully verified their survival, and designate the consensus node with the latest log update time as the i-level leader of its corresponding subtree.

[0096] In one embodiment, the step of constructing a multi-level tree structure cluster includes a log replication step:

[0097] The level i leader receives the log replication message sent by the level i-1 leader, performs log synchronization after verification, and sends a log replication message to the level i follower. The level i follower performs log synchronization after verification and replies with a level i response message to the level i leader;

[0098] The level i leader accumulates level i response messages. When the number of level i response messages meets the minimum number of consensus nodes, the level i leader replies with a level i-1 response message to the level i-1 leader.

[0099] The i-1 level leader accumulates i-1 level response messages. When the number of i-1 level response messages meets the requirement of the minimum number of consensus nodes, the i-1 level leader updates the client request corresponding to the log to the state machine for execution.

[0100] In this embodiment, the requirement for the minimum number of nodes includes: greater than half of the number of consensus nodes in the current level cluster.

[0101] In one embodiment, the step of sending a heartbeat detection message includes:

[0102] The level i leader receives the heartbeat detection message sent by the level i-1 leader based on the polling heartbeat duration; the level i leader sends a heartbeat detection message to the level i follower based on the polling heartbeat duration;

[0103] The consensus node triggers the generation of candidates from the consensus nodes of the current cluster based on the failure to receive a heartbeat detection message within the polling heartbeat duration.

[0104] The second aspect of an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a large-scale data consistency method based on a multi-level tree structure cluster according to the first aspect of the embodiment are implemented.

[0105] The third aspect of an embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a large-scale data consistency method based on a multi-level tree structure cluster of the first aspect of the embodiment are implemented.

[0106] The above is a detailed introduction to a large-scale data consistency method and device based on a multi-level tree structure cluster provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0107] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A large-scale data consistency method based on multi-level tree structure clusters, characterized in that: The steps include: Construct a multi-level tree structure cluster, the cluster includes: a first-level cluster and an i-level cluster, i∈[2,N], N is a positive integer greater than 2; the consensus node status in each of the clusters includes: leader, follower and candidate; assign an i-level cluster as its corresponding subtree to the follower in the i-1-level cluster, and the leader in the i-level cluster is the follower in the i-1-level cluster; The i-1 level leader sends a heartbeat detection message to the i-level leader in its corresponding subtree based on the polling heartbeat duration; the i-level leader triggers its own data collection according to the heartbeat detection message, and feeds back the collected data to the i-1 level leader; The i-1 level leader performs consistency comparison on the collected data. If there is inconsistency, it constructs the consistent data and sends the synchronization log of the constructed data to the i-level leader. The i-level leader performs log synchronization after verification. The level i leader sends a log replication message to the level i followers in its subtree, and the level i followers perform log synchronization after verification.

2. A large-scale data consistency method based on multi-level tree structure cluster according to claim 1, characterized in that: The steps of constructing a multi-level tree structure cluster include the election step: The current-level candidate is triggered to elect a leader by the current-level follower failing to receive the heartbeat detection message from the current-level leader; When the i-1 level candidate becomes the i-1 level leader through the voting mechanism, the remaining i-1 level consensus nodes update their own consensus node status to followers, and the i-1 level leader sends the i-1 level election result notification to the i-1 level leader in its corresponding subtree; the i-1 level leader sends the i-1 level election result notification to the i-1 level followers in its subtree; the i-1 level followers continue to pass the i-1 level election result notification downward until there is no corresponding subtree for the consensus node; When the i-level candidate becomes the i-level leader through the voting mechanism, the remaining i-level consensus nodes update their own consensus node status to followers, and the i-level leader sends the i-level election result notification to the corresponding i-1-level leader in its subtree, and the i-level leader continues to pass the i-level election result notification upward to the first-level leader; the i-1-level leader sends the i-level election result notification to the remaining i-1-level followers, and the remaining i-1-level followers continue to pass the i-level election result notification downward until there is no corresponding subtree for the consensus node; The consensus node that receives the election result notification adjusts the node information according to the election result notification.

3. A large-scale data consistency method based on multi-level tree structure cluster according to claim 2, characterized in that: The voting mechanism includes: The current-level candidate sends an election request message to other consensus nodes at the current level; The current and other consensus nodes verify the status information of the current and candidate nodes, and send voting information to the current candidate nodes after verification; The current-level candidate accumulates the voting information, and when the amount of the voting information meets the requirement of the minimum number of consensus nodes, the current-level candidate updates its status to a leader, and the remaining current-level consensus nodes update their status to followers.

4. A large-scale data consistency method based on multi-level tree structure cluster according to claim 2, characterized in that: The election step also includes: If the i-1 level leader still does not receive the i-level election result notification sent by the i-level leader in its corresponding subtree after looping a specified number of polling heartbeat duration periods, the following steps are performed: The i-1 level leader sends a survival detection message to all consensus nodes in its corresponding subtree, and designates a consensus node among the consensus nodes that have successfully survived the survival verification as the i-level leader of its corresponding subtree; The designated consensus node updates its status to the level i leader, sends a level i election result notification to the remaining consensus nodes in the subtree, and returns an update success message to the level i-1 leader; the remaining consensus nodes update their status to the level i follower; The i-1 level leader adjusts the node information according to the update success information, and the i-1 level leader continues to pass the update success information upward to the first-level leader; and sends the i-level election result notification to the remaining i-1 level followers; the remaining i-1 level followers continue to pass the i-level election result notification downward until the consensus node has no corresponding subtree; The consensus node that receives the i-level election result adjusts the node information according to the i-level election result notification.

5. A large-scale data consistency method based on multi-level tree structure cluster according to claim 4, characterized in that: The step of designating a consensus node as the level i leader of its corresponding subtree among the consensus nodes that have successfully survived the verification includes any one of the following: Random designation: designate a consensus node among the consensus nodes that have successfully verified survival as the i-level leader of its corresponding subtree; Comparative log designation: compare the log states of the consensus nodes that have successfully passed the survival verification, and designate the consensus node with the latest log update time as the i-level leader of its corresponding subtree.

6. A large-scale data consistency method based on multi-level tree structure cluster according to claim 1, characterized in that: The steps to build a multi-level tree-structured cluster include the log replication step: The level i leader receives the log replication message sent by the level i-1 leader, performs log synchronization after verification, and sends a log replication message to the level i follower. The level i follower performs log synchronization after verification and replies with a level i response message to the level i leader; The level i leader accumulates the level i response messages, and when the number of the level i response messages meets the requirement of the minimum number of consensus nodes, the level i leader replies with a level i-1 response message to the level i-1 leader; The i-1 level leader accumulates the i-1 level response messages, and when the number of the i-1 level response messages meets the requirement of the minimum number of consensus nodes, the i-1 level leader updates the client request corresponding to the log to the state machine for execution.

7. A large-scale data consistency method based on multi-level tree structure cluster according to claim 6, characterized in that: The requirement for the minimum number of nodes includes: being greater than half of the number of consensus nodes in the current level cluster.

8. A large-scale data consistency method based on multi-level tree structure cluster according to claim 1, characterized in that: The step of sending a heartbeat detection message comprises: The level i leader receives the heartbeat detection message sent by the level i-1 leader based on the polling heartbeat duration; the level i leader sends a heartbeat detection message to the level i follower based on the polling heartbeat duration; The consensus node triggers the generation of candidates from the consensus nodes of the current cluster based on the failure to receive a heartbeat detection message within the polling heartbeat duration.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of a large-scale data consistency method based on a multi-level tree structure cluster are implemented as described in any one of claims 1-8.

10. A non-transitory computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of a large-scale data consistency method based on a multi-level tree structure cluster as described in any one of claims 1 to 8 are implemented.

Images