Transaction processing method, device, electronic device, and storage medium

Abstract

The present application provides a transaction processing method, device, electronic device, and storage medium. The method is applied to a system using a consensus algorithm, and the method includes: receiving pending transactions and assigning the pending transactions to a consensus shard in a created consensus cluster to trigger a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node; executing the consensus resolution and storing the execution result of the consensus resolution to complete the processing of the pending transactions. The method of the present application can solve the problem of how to ensure the high availability of a system based on a consensus algorithm while avoiding waste of system resources and improving the consensus efficiency of the system.

Description

Technical Field

[0001] The present application relates to transaction processing technology based on consensus algorithms, and in particular to a transaction processing method, device, electronic device, and storage medium. Background Art

[0002] With the development of the financial sector, business demands are becoming increasingly demanding. Distributed systems and consortium blockchain systems are currently being used to address these needs. To meet the high-performance demands of these businesses, current distributed and consortium blockchain systems generally employ consensus algorithms based on distributed consistency principles (primarily including the PBFT consensus algorithm and the Raft consensus algorithm). These consensus algorithms offer advantages such as high transaction throughput, transaction confirmation latency as low as seconds, and eventual consistency assurance of output results.

[0003] However, these consensus algorithms lack high fault tolerance, which means that to ensure high availability of consensus-based systems, a sufficient number of consensus nodes must be deployed. Because all consensus nodes participate in the consensus process, increasing system availability increases the number of consensus nodes, leading to an increasing number of faulty consensus nodes, which in turn reduces the efficiency of the consensus system. Furthermore, each consensus node maintains a copy of the data, while generally only three copies are required to ensure high data availability. Therefore, as system availability increases, the waste of system resources increases.

[0004] Therefore, how to ensure the high availability of the system based on the consensus algorithm while avoiding waste of system resources and improving the consensus efficiency of the system still needs to be considered. Summary of the Invention

[0005] The present application provides a transaction processing method, device, electronic device, and storage medium to solve the problem of how to ensure the high availability of a system based on a consensus algorithm while avoiding waste of system resources and improving the consensus efficiency of the system.

[0006] In one aspect, the present application provides a transaction processing method, applied to a system using a consensus algorithm, the method comprising:

[0007] Receive pending transactions and assign them to a consensus shard in a created consensus cluster, triggering a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node;

[0008] Execute the consensus resolution and store the execution result of the consensus resolution to complete the processing of the pending transaction.

[0009] In one embodiment, the system further includes a leader cluster selection process, and the consensus cluster is created by:

[0010] Obtain a consensus shard creation request, where the creation request carries at least the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created;

[0011] According to the request to create the consensus shard, query whether the name of the consensus shard to be created exists in the selected main cluster; if not, create the consensus shard in the consensus cluster according to the name of the consensus shard to be created carried in the creation request and the number of consensus sets contained in the consensus shard to be created, so as to obtain the created consensus cluster.

[0012] In one embodiment, creating a consensus shard in the consensus cluster according to the name of the consensus shard to be created and the number of consensus sets included in the consensus shard to be created carried in the creation request includes:

[0013] Generate a key-value pair for the consensus shard according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request;

[0014] Inserting the key-value pair of the consensus shard into the leader election cluster to create the consensus shard in the consensus cluster.

[0015] In one embodiment, inserting the key-value pair of the consensus shard into the leader election cluster to create the consensus shard in the consensus cluster includes:

[0016] Insert the key-value pair of the consensus shard into the leader election cluster, and trigger the leader election cluster to receive the master node application information sent by a consensus node in the system;

[0017] Triggering the leader election cluster to respond to the master node application information and insert the number information of the consensus node into the key-value pair of the consensus shard to assign the consensus node as the consensus master node of the consensus set to be assigned in the consensus shard, wherein the consensus set to be assigned refers to a consensus set that does not have a consensus master node;

[0018] Assign at least one consensus node to each consensus set with a consensus master node in the consensus shard as at least one backup node of the consensus set, where the consensus master node and the backup node in the consensus set are different;

[0019] When each consensus set in a consensus shard is assigned a consensus master node and at least one backup node, creation of a consensus shard in the consensus cluster is complete.

[0020] In one embodiment, the master node application information includes lease application information and the number information of the consensus node; after the consensus node is assigned as the consensus master node, the consensus node periodically sends lease application information to the leader election cluster; the leader election cluster responds to the lease application information and maintains the consensus node as the consensus master node;

[0021] The method further comprises:

[0022] When the master election cluster fails to receive the lease application information sent by the consensus node for a preset period of time, the master election cluster is triggered to receive the master node application information sent by the backup node in the consensus set;

[0023] The consensus set with the one consensus node as the consensus master node is updated to be the consensus set to be allocated, and the backup node is allocated as the consensus master node of the consensus set to be allocated.

[0024] In one embodiment, each consensus node in the system subscribes to the key-value information of each consensus shard recorded by the leader election cluster. When the key-value information of any consensus shard is updated, each consensus node sends the master node application information to the leader election cluster.

[0025] In one embodiment, after creating the consensus cluster, the method further includes:

[0026] When receiving a new consensus set indication, if it is determined based on the consensus set information in the leader selection cluster that the new consensus set has not been created, then,

[0027] According to the new consensus set information carried by the new consensus set indication and the indicated target consensus shard, the key value information of the target consensus shard in the master election cluster is updated to add the new consensus set to the target consensus shard, wherein the new consensus set includes at least one consensus set.

[0028] In one embodiment, it further includes:

[0029] When receiving a consensus set deletion instruction, if it is determined based on the consensus set information in the master selection cluster that the consensus set has been created, then:

[0030] According to the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard, the key value information of the target consensus shard in the master election cluster is updated to remove the deletion consensus set from the target consensus shard, wherein the deletion consensus set includes at least one consensus set.

[0031] In one embodiment, updating the key value information of the target consensus shard in the master selection cluster according to the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard to remove the deletion consensus set from the target consensus shard includes:

[0032] Determine the consensus master node for deleting the consensus set based on the deletion consensus set information carried by the deletion consensus set instruction and the indicated target consensus shard;

[0033] When the lease application information sent by the consensus master node of the deletion consensus set is not received within the preset time period, the key value information of the deletion consensus set is filtered out from the key value information of the target consensus shard in the master selection cluster, and the key value information of the deletion consensus set is deleted to eliminate the deletion consensus set in the target consensus shard.

[0034] In one embodiment, the method further comprises:

[0035] When receiving a consensus shard deletion instruction, if it is determined based on the consensus shard information in the master selection cluster that the consensus shard to be deleted has been created, then,

[0036] According to the deletion consensus shard information carried by the deletion consensus shard indication, the key value information of the deletion consensus shard is removed from the key value information of the target consensus shard in the master selection cluster, and the key value information of each consensus set corresponding to the deletion consensus shard is removed from the key value information of the target consensus shard in the master selection cluster.

[0037] In one embodiment, the system further includes a transaction receiving module, the transaction receiving module including a plurality of transaction pool shards, and the receiving of pending transactions and assigning the pending transactions to a consensus shard includes:

[0038] Trigger the transaction receiving module to receive pending transactions, and cache the pending transactions to the target transaction pool shard corresponding to the shard number according to the shard number carried by the pending transactions;

[0039] According to the correspondence between the transaction pool shards and the consensus shards, the consensus shard corresponding to the target transaction pool shard is obtained, and the to-be-processed transaction is allocated to a consensus shard corresponding to the target transaction pool shard.

[0040] On the other hand, the present application provides a system using a consensus algorithm, the system comprising at least: a processor, a replication state machine, and a memory; wherein,

[0041] The processor is configured to receive pending transactions and assign the pending transactions to a consensus shard in the created consensus cluster, thereby triggering a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node;

[0042] The replicated state machine is used to execute the consensus resolution;

[0043] The memory is used to store the execution result of the consensus resolution to complete the processing of the pending transaction.

[0044] On the other hand, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0045] The memory stores computer-executable instructions;

[0046] The processor executes the computer-executable instructions stored in the memory to implement the transaction processing method as described in the first aspect.

[0047] On the other hand, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the instructions are executed, the computer executes the transaction processing method as described in the first aspect.

[0048] On the other hand, the present application provides a computer program product, comprising a computer program, which implements the transaction processing method as described in the first aspect when executed by a processor.

[0049] The method provided by the embodiment of the present application creates a consensus cluster, which includes multiple independent consensus shards, and each consensus shard includes multiple consensus sets. All consensus nodes in the distributed system or blockchain system are assigned to the consensus set, and each consensus set is assigned a consensus master node and at least one backup node. When processing pending transactions, the pending transactions are assigned to the corresponding consensus shards, triggering the consensus master node in a consensus set in the consensus shard to process the pending transactions. In this way, only the backup nodes in the consensus set need to back up the data, and the number of data copies that need to be maintained is greatly reduced. As the system availability increases, system resources will not be wasted.

[0050] In addition, each consensus set has a consensus master node and at least one backup node. When the consensus master node fails, the backup node is updated to the consensus master node. Therefore, as the system availability increases, if the consensus master node fails, it is only necessary to switch the backup node to the consensus master node, and there is no need to increase the number of consensus nodes in the system. In layman's terms, as long as more than one consensus node in the system is normal, the entire system can work normally. Therefore, the method provided in the embodiment of the present application also avoids the problem of decreased consensus efficiency caused by the fault tolerance limit of the consensus algorithm, greatly improving the consensus efficiency of the system, thereby improving the transaction processing speed.

[0051] In addition, different pending transactions can be given to different consensus shards for concurrent processing, which greatly improves the scalability of the system and reduces the waste of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0053] Figure 1 A schematic diagram of an application scenario of the transaction processing method provided in this application;

[0054] Figure 2 A flowchart of a transaction processing method provided in one embodiment of the present application;

[0055] Figure 3 A schematic diagram of the architecture of a system using a consensus algorithm provided in one embodiment of the present application;

[0056] Figure 4 A schematic diagram of an architecture of a system using a consensus algorithm provided in another embodiment of the present application;

[0057] Figure 5 A schematic diagram of a system using a consensus algorithm provided in accordance with one embodiment of the present application;

[0058] Figure 6 A schematic diagram of an electronic device provided for one embodiment of the present application.

[0059] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0061] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0062] First, let’s explain the terms involved in this application:

[0063] Consensus algorithm: Each node uses an algorithm to confirm a batch of transactions and ensure that all nodes have the same confirmation result for this batch of data. This algorithm is called a consensus algorithm. Currently, common consensus algorithms include the Practical Byzantine Fault Tolerance (PBFT) consensus algorithm and the Replication and Fault Tolerant (Raft) consensus algorithm.

[0064] Consensus algorithm fault tolerance: When more than a specified number of nodes in a consensus system (a system using a consensus algorithm) fail, the consensus system will be unable to confirm transactions and data, causing system anomalies. The maximum percentage of failed nodes that a consensus system can tolerate is called the consensus algorithm's fault tolerance. Among commonly used consensus algorithms, the PBFT consensus algorithm has a fault tolerance of one-third (1 / 3), and the Raft consensus algorithm has a fault tolerance of one-half (1 / 2).

[0065] Consensus node: A node that executes the consensus algorithm. The main functions of a consensus node include: batching transactions into blocks, running the consensus algorithm to reach consensus on the consensus resolution containing the batched transactions, passing the consensus resolution to the replicated state machine for execution, and storing the state results of the replicated state machine after execution.

[0066] Lease: In key-value distributed storage systems (such as etcd), the duration of a primary key's ownership. If the lease is not renewed regularly, the key will be cleared after the lease expires. Leases are primarily used in scenarios such as master node selection in distributed systems.

[0067] Etcd: Etcd is an open source, highly available, distributed key-value storage system developed in Go. It can be used for service registration, service discovery, and master node election. It has features such as full replication, high availability, and high consistency.

[0068] Consensus Set: A technical term used in this application. A consensus set consists of a master consensus node and several backup nodes. If a master consensus node fails, a backup node can be selected to replace it. All nodes in a consensus set share storage.

[0069] Consensus Shard: This is a technical term used in this application. Each consensus shard contains several consensus sets, and consensus is achieved based on the dimensions of the consensus set.

[0070] Transaction pool: A module used to receive and verify client transactions. A transaction pool contains several transaction queues. Verified transaction requests from different consensus shards are cached in different transaction queues.

[0071] With the development of the financial sector, business demands are becoming increasingly demanding. Distributed systems and consortium blockchain systems are currently being used to address these needs. To meet the high-performance demands of these businesses, current distributed and consortium blockchain systems generally employ consensus algorithms based on distributed consistency principles (primarily including the PBFT consensus algorithm and the Raft consensus algorithm). These consensus algorithms offer advantages such as high transaction throughput, transaction confirmation latency as low as seconds, and eventual consistency assurance of output results.

[0072] However, these consensus algorithms have low fault tolerance. The PBFT consensus algorithm can only tolerate 1 / 3 of its nodes failing, and the Raft consensus algorithm, while sacrificing its tolerance for Byzantine faults, can only tolerate 50% of its nodes failing. For example, a consensus system based on the PBFT algorithm (a system that uses a consensus algorithm) with 4 nodes can only tolerate 1 node failure, while a system with 7 nodes can only tolerate 2 node failures. A consensus system based on the Raft algorithm can only tolerate 1 node failure with 3 nodes, and 2 node failures with 5 nodes. This means that to ensure high availability of a consensus algorithm-based system, a sufficient number of consensus nodes must be deployed.

[0073] Because all consensus nodes must participate in the consensus process, as system availability increases, the number of consensus nodes will continue to increase, and the number of faulty consensus nodes will continue to increase, which will lead to a decrease in the consensus efficiency of the consensus system. On the other hand, each consensus node maintains a copy of the data, and generally only three copies of a piece of data are needed to ensure high data availability. Therefore, as system availability increases (the number of tolerable faulty nodes increases), the waste of system resources will become more and more serious.

[0074] How to ensure the high availability of the system based on the consensus algorithm while avoiding waste of system resources and improving the consensus efficiency of the system still needs to be considered.

[0075] Based on this, the present application provides a transaction processing method, device, electronic device, and storage medium. The transaction processing method is applied to a system using a consensus algorithm. The transaction processing method includes: receiving a pending transaction and assigning the pending transaction to a consensus shard in a created consensus cluster to trigger a consensus master node in a consensus set in the consensus shard to process the pending transaction and generate a consensus resolution. The consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node. The consensus resolution is then executed, and the execution result of the consensus resolution is stored to complete the processing of the pending transaction.

[0076] Specifically, a consensus cluster is first created, consisting of multiple independent consensus shards, each of which contains multiple consensus sets. All consensus nodes in the system using the consensus algorithm are assigned to each consensus set, with each set assigned a consensus master node and at least one backup node. When processing pending transactions, they are assigned to the corresponding consensus shard and processed by a consensus master node in a consensus set within that consensus shard. This allows data backup to be performed only by the backup nodes in the consensus set, significantly reducing the number of data copies required. This increases system availability and minimizes waste of system resources.

[0077] In addition, each consensus set has a consensus master node and at least one backup node. When the consensus master node fails, the backup node is updated to the consensus master node. Therefore, as the system availability increases, if the consensus master node fails, it is only necessary to switch the backup node to the consensus master node, and there is no need to increase the number of consensus nodes in the system. In layman's terms, as long as more than one consensus node in the system is normal, the entire system can work normally. Therefore, the method provided by the present application also avoids the problem of decreased consensus efficiency caused by the fault tolerance limit of the consensus algorithm, greatly improves the consensus efficiency of the system, and thus improves the transaction processing speed. In addition, different pending transactions can be given to different consensus shards for concurrent processing at the same time, which greatly improves the scalability of the system and reduces the waste of system resources.

[0078] The transaction processing method provided in this application is applied to an electronic device, such as a server cluster having a system that uses a consensus algorithm. Figure 1This is a schematic diagram of an application of the transaction processing method provided in this application. In the diagram, the electronic device receives a processing request for a pending transaction from a client, thereby receiving the pending transaction. After receiving the pending transaction, the pending transaction is assigned to a consensus shard in a previously created consensus cluster. A consensus master node in a consensus set within the consensus shard processes the pending transaction and generates a consensus resolution. The replicated state machine in the electronic device is triggered to execute the consensus resolution, and the memory in the electronic device is triggered to store the execution result of the consensus resolution, thereby completing the processing of the pending transaction.

[0079] See Figure 2 One embodiment of the present application provides a transaction processing method, which is applied to a system using a consensus algorithm. The transaction processing method includes:

[0080] S210, receiving pending transactions and assigning the pending transactions to a consensus shard in the created consensus cluster to trigger a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node.

[0081] Pending transactions are generated by the client and sent to a system that uses a consensus algorithm. The system that uses a consensus algorithm includes a processor that is responsible for receiving pending transactions.

[0082] See Figure 3 As shown in the schematic diagram of the architecture of a system using a consensus algorithm, in an optional embodiment, the processor includes a transaction receiving module, which includes multiple transaction pool shards. After receiving the processing request for the pending transaction initiated by the client, the transaction receiving module is triggered to receive the pending transaction, and according to the shard number carried by the pending transaction, the pending transaction is cached to the target transaction pool shard corresponding to the shard number. The processing request for the pending transaction carries information about the pending transaction and the target consensus shard, wherein the pending transaction includes information about the transaction sender, information about the transaction recipient (i.e., the contract address), input information of the transaction, signature information of the transaction, etc. Among them, the information of the target consensus shard includes the shard number of the target consensus shard, and the number of the target consensus shard is the shard number carried by the pending transaction.

[0083] One transaction pool shard corresponds to one transaction queue. Caching pending transactions to the target transaction pool shard corresponding to the shard number is essentially caching pending transactions to the target transaction queue corresponding to the shard number.

[0084] Optionally, upon receiving a pending transaction, the target transaction pool shard is triggered to verify the pending transaction and cache the verified pending transaction in the target transaction queue corresponding to the shard number. If the pending transaction fails verification, a notification indicating failure may be returned to the client. Verification of the pending transaction may involve verification of a portion of the information contained in the pending transaction, or verification of all the information contained in the pending transaction. The specific verification can be determined based on actual needs and is not limited in this embodiment.

[0085] As described above, the information of the target consensus shard includes the shard number of the target consensus shard, which is the shard number carried by the pending transaction. When the pending transaction is cached to the target transaction pool shard (or target transaction queue) corresponding to the shard number according to the shard number carried by the pending transaction, the corresponding relationship between the transaction pool shard (or transaction queue) and the consensus shard is obtained. The target transaction pool shard (or target transaction queue) is actually the transaction shard corresponding to the target consensus shard. For example, if the shard number of the target consensus shard is i, that is, the target consensus shard is consensus shard i, then the target transaction pool shard is transaction pool shard i, or the target transaction queue is transaction queue i.

[0086] After the pending transaction is verified, the consensus shard corresponding to the target transaction pool shard is obtained based on the correspondence between transaction pool shards and consensus shards, and the pending transaction is assigned to a consensus shard corresponding to the target transaction pool shard. The consensus shard corresponding to the target transaction pool shard is the target consensus shard described above, such as consensus shard i in the above example.

[0087] When assigning pending transactions to a consensus shard in an established consensus cluster, the consensus shard in the consensus cluster periodically packages pending transactions from the corresponding transaction queue or transaction pool. If there are pending transactions cached in the target transaction queue or target transaction pool, the target consensus shard can directly obtain the pending transactions.

[0088] After completing the assignment of pending transactions to a consensus shard in the created consensus cluster, the consensus master node in a consensus set in the consensus shard is triggered to process the pending transactions and generate a consensus resolution. Figure 3 In a system using a consensus algorithm, a consensus cluster is created. This consensus cluster consists of multiple independent consensus shards. Each consensus shard consists of multiple consensus sets, each of which has a consensus master node and at least one backup node. When processing pending transactions, the consensus master node, based on the consensus algorithm, packages the pending transactions into a consensus resolution and reaches consensus on the resolution.

[0089] Taking the PBFT consensus process as an example, this article explains the detailed steps by which the consensus masternode packages pending transactions into consensus resolutions based on the consensus algorithm and then reaches consensus on these resolutions. In the following description, consensus shard i is the consensus shard to which the pending transaction is assigned.

[0090] In the first step, the master consensus set for consensus shard i repackages the consensus resolution packaged by the consensus master node into a Pre-Prepare package, which includes a signature. The master consensus set broadcasts this Pre-Prepare package to the consensus master nodes of all consensus sets in consensus shard i. It should be noted that the master consensus set is determined based on the index of each consensus set in consensus shard i.

[0091] In the second step, after receiving the broadcasted Pre-Prepare packet, the consensus master nodes of all consensus sets in consensus shard i first determine whether the Pre-Prepare packet comes from the primary consensus set of consensus shard i and verify the signature of the Pre-Prepare packet. After the signature verification is passed, the Prepare message packet will be broadcast to the consensus master nodes of all other consensus sets in consensus shard i.

[0092] In the third step, after the consensus master node of the consensus set receives the Prepare message package, it first determines whether the Prepare message package is generated by the consensus master node of the consensus set of consensus shard i, and verifies the signature of the Prepare message package. After the verification is passed, the Prepare message package is added to the local cache of the consensus master node of this consensus set. After the consensus master node of this consensus set collects two-thirds of the Prepare packets from different consensus sets (that is, after collecting at least two-thirds of the Prepare packets sent by all consensus sets in all consensus sets), the consensus master node of this consensus set broadcasts the Commit message package to all consensus set master nodes of consensus shard i.

[0093] In the fourth step, after receiving the Commit message packet, the consensus set master node determines whether the Commit message packet was generated by the consensus master node of the consensus set of consensus shard i. It then verifies the signature of the Commit message packet. If the signature is verified, it adds the Commit message packet to the local Commit cache. After collecting two-thirds of the Commit message packets from different consensus sets, it reaches a consensus on the consensus resolution and sends the consensus resolution to the replicated state machine in the system using the consensus algorithm for execution.

[0094] The above four steps provide a detailed description of the consensus process based on the PBFT consensus algorithm. Another consensus algorithm is the Raft consensus algorithm. When implementing the consensus process based on the Raft consensus algorithm, you can refer to the four steps described above and adapt some of the steps accordingly. This embodiment will not describe the specific steps of the consensus process based on the Raft consensus algorithm in detail.

[0095] It's important to note that if a consensus master node fails while processing a pending transaction, the backup node in the consensus set becomes the consensus master, ensuring that pending transactions can be processed normally. Because the master and backup nodes in a consensus set are assigned by all consensus nodes in the system using the consensus algorithm, as long as at least one consensus node in the system using the consensus algorithm is functioning properly, every consensus set in any consensus shard will have a consensus master. Therefore, after receiving a pending transaction, as long as at least one consensus node in the system using the consensus algorithm is functioning properly, the system using the consensus algorithm can process the pending transaction normally. This increases system availability, and if a consensus master node fails, simply switch to the backup node as the consensus master, eliminating the need to increase the number of consensus nodes in the system. In other words, as long as at least one consensus node in the system is functioning properly, the entire system can function normally. This design avoids the problem of consensus efficiency degradation caused by the fault tolerance limits of the consensus algorithm, significantly improving the system's consensus efficiency and, consequently, transaction processing speed.

[0096] In addition, when multiple pending transactions are received within a period of time, different pending transactions can be assigned to different consensus shards for processing, which greatly improves the scalability of the system and reduces the waste of system resources.

[0097] S220: Execute the consensus resolution and store the execution result of the consensus resolution to complete the processing of the pending transaction.

[0098] As described above, after a consensus resolution is generated, the replicated state machine is triggered to execute it, and the memory in the consensus algorithm system is triggered to store the execution result of the consensus resolution, thereby completing the processing of the pending transaction. This memory comprises a distributed storage system. After the replicated state machine executes the consensus resolution, the execution result is written to the backend distributed storage system. Correspondingly, the distributed storage system stores the transaction data and the execution result.

[0099] In summary, the method provided in the embodiment of the present application first creates a consensus cluster, which includes multiple independent consensus shards, and each consensus shard includes multiple consensus sets. All consensus nodes in the distributed system or blockchain system are allocated according to the dimension of consensus sets, and each consensus set is allocated a consensus master node and at least one backup node. When processing pending transactions, the pending transactions are allocated to the corresponding consensus shards, triggering the consensus master node in a consensus set in the consensus shard to process the pending transactions. In this way, only the backup nodes in the consensus set are needed to back up the data, and the number of data copies that need to be maintained is greatly reduced. As the system availability increases, system resources will not be wasted.

[0100] In addition, each consensus set has a consensus master node and at least one backup node. When the consensus master node fails, the backup node is updated to the consensus master node. Therefore, as the system availability increases, if the consensus master node fails, it is only necessary to switch the backup node to the consensus master node, and there is no need to increase the number of consensus nodes in the system. In layman's terms, as long as more than one consensus node in the system is normal, the entire system can work normally. Therefore, the method provided in the embodiment of the present application also avoids the problem of decreased consensus efficiency caused by the fault tolerance limit of the consensus algorithm, greatly improving the consensus efficiency of the system, thereby improving the transaction processing speed.

[0101] In addition, different pending transactions can be given to different consensus shards for concurrent processing, which greatly improves the scalability of the system and reduces the waste of system resources.

[0102] The following focuses on a detailed description of the consensus cluster creation and management process. The consensus cluster must be created before receiving pending transactions. That is, if the system using the consensus algorithm does not create a consensus cluster, the system using the consensus algorithm cannot process transactions.

[0103] First, the process of creating a consensus cluster is described in detail. Figure 4 It should be noted in advance that the system also includes a master selection cluster, which is the Etcd described above.

[0104] When the system is initialized, there are no consensus shards and no transactions can be processed. At this point, the system administrator must initiate a request to create a consensus shard. Accordingly, a consensus shard creation request is obtained, which contains at least the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard. For example, a consensus shard creation request might include {consensus shard i, consensus set 1, consensus set 2, …, consensus set m}, where consensus shard i is the name of the consensus shard to be created and m is the number of consensus sets contained in consensus shard i.

[0105] Then, based on the request to create the consensus shard, the master election cluster is accessed to determine whether the consensus shard to be created has been created. Specifically, based on the request to create the consensus shard, the master election cluster is queried to determine whether the name of the consensus shard to be created exists. Taking the name of the consensus shard to be created as consensus shard i as an example, to query whether the name of the consensus shard to be created exists in the master election cluster is to determine whether a primary key with the prefix "consensus shard i" exists in the master election cluster. If the name of the consensus shard to be created exists in the master election cluster (there is a primary key with the prefix "consensus shard i"), it is determined that the consensus shard to be created (consensus shard i) has been created. At this time, the request to create the consensus shard is ignored or a notification that the consensus shard has been created is returned.

[0106] If the name of the consensus shard to be created does not exist in the leader election cluster (there is no primary key prefixed with "consensus shard i"), it is determined that the consensus shard to be created (consensus shard i) has not been created. If it does not exist, a consensus shard is created in the consensus cluster based on the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request to obtain the created consensus cluster. Specifically, a key-value pair of the consensus shard is generated based on the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request. The key-value pair of the consensus shard is inserted into the leader election cluster to create the consensus shard in the consensus cluster. For example, the key-value pair { / meta / consensus shard i, consensus set 1, consensus set 2, ..., consensus set m} is inserted into the leader election cluster.

[0107] After inserting the key-value pairs of the consensus shard into the leader selection cluster, it is also necessary to assign a consensus master node and a backup node to each consensus set in the consensus shard.

[0108] First, the consensus master node allocation process for the consensus set is executed. Specifically, when allocating a consensus master node for a consensus set, the key-value pair of the consensus shard is inserted into the master election cluster, triggering the master election cluster to receive a master node application message from a consensus node in the system. The master election cluster is triggered to respond to the master node application message and insert the consensus node's serial number information into the key-value pair of the consensus shard, thereby allocating the consensus node as the consensus master node for the consensus set to be allocated in the consensus shard. The consensus set to be allocated is a consensus set that does not have a consensus master node. This completes the allocation of the consensus master node for the consensus set.

[0109] Specifically, each consensus node in the system subscribes to the key-value information for each consensus shard recorded by the leader election cluster. When the key-value information for any consensus shard is updated, each consensus node sends a master node request to the leader election cluster. In simple terms, each consensus node in the system subscribes to changes in the primary key under the / meta path in the leader election cluster. Therefore, when a new key-value pair for a consensus shard is added, all consensus nodes in the system autonomously trigger the consensus master election process. This consensus master election process involves electing to become the consensus master for any consensus set within a consensus shard. For example, for the created consensus shard i, after the consensus master election process is triggered, each consensus node in the system retrieves the consensus set information {consensus set 1, consensus set 2, …, consensus set m} using the primary key / meta / consensus shard i. Each consensus node in the system traverses each consensus set within consensus shard i, attempting to become the consensus master for any consensus set.

[0110] Taking a consensus node in this system as an example, based on the consensus set name "consensus set i," this consensus node attempts to obtain the primary key " / consensus shard i / consensus set i" for the consensus master node information of consensus set i from the leader election cluster. If the primary key " / consensus shard i / consensus set i" already exists, it indicates that a node has already run for the consensus master node for consensus set i. The consensus node then attempts to run for the consensus master node for another consensus set in consensus shard i. If the primary key " / consensus shard i / consensus set i" does not exist, it indicates that consensus set i does not yet have a consensus master node. The consensus node then sends a master node application to the leader election cluster. This master node application includes lease application information and the consensus node's ID information. After triggering the leader election cluster to respond to the master node application, the consensus node attempts to write its own ID information (NodeID) to the primary key " / consensus shard i / consensus set i" in a transaction. If the write operation for the primary key " / consensus shard i / consensus set i" is successful, it indicates that the consensus node has successfully run for the consensus master node for consensus set i of consensus shard i. The consensus node initiates a thread to periodically send lease application information (heartbeat renewal) to the leader election cluster to prevent the lease from expiring and causing the primary key " / consensus shard i / consensus set i" to expire, thereby losing the role of consensus master node. Correspondingly, the leader election cluster responds to the lease application information and maintains the consensus node as the consensus master node. After the consensus node successfully runs for the consensus master node, it starts the consensus process as a consensus set for consensus shard i and participates in the consensus of consensus shard i.

[0111] If the leader election cluster fails to receive a lease request from the consensus node for a predetermined period of time, it is triggered to receive a master node request from a backup node in the consensus set. At this point, the consensus set with the consensus node as the master node is updated to the pending consensus set, and the backup node is assigned as the master node of the pending consensus set.

[0112] When allocating consensus master nodes to each consensus set in a consensus shard, the allocation process is the same as the allocation process described above. The allocation process of consensus master nodes for other consensus sets in the consensus shard will not be elaborated here.

[0113] After assigning a consensus master node to each consensus set in the consensus shard, at least one consensus node is assigned to each consensus set in the consensus shard that has a consensus master node as at least one backup node of the consensus set. It should be noted that the consensus master node in a consensus set is different from the backup node. A backup node refers to a consensus node in the system that did not successfully run for the consensus master node. Assuming that the system contains N consensus nodes (N is a natural number greater than zero) and the consensus shard to be created contains M consensus nodes (M is less than N), then each consensus set in the created consensus shard will have 1 consensus master node and M-1 backup nodes.

[0114] When each consensus set in a consensus shard is assigned a consensus master node and at least one backup node, the consensus shard is created in the consensus cluster. When the required number of consensus shards is created, the consensus cluster is created.

[0115] The following describes in detail the management process of the consensus cluster, which includes the expansion, reduction, and deletion of consensus shards.

[0116] Expanding a consensus shard means adding at least one new consensus set to the consensus shard. The system administrator can issue a new consensus set instruction to the system, and the system responds to the new consensus set instruction, thereby adding a new consensus set to the consensus shard. The new consensus set indicated by the new consensus set instruction includes at least one consensus set.

[0117] Correspondingly, when a new consensus set indication is received, the leader election cluster first determines whether the new consensus set has been created. If the new consensus set has already been created, the new consensus set indication is ignored. If the new consensus set has not yet been created, the new consensus set creation process is executed. This involves updating the key-value information of the target consensus shard in the leader election cluster based on the new consensus set information carried in the new consensus set indication and the indicated target consensus shard, thereby adding the new consensus set to the target consensus shard.

[0118] Suppose there are m consensus sets in consensus shard i, and t new consensus sets are needed: {consensus set (m+1), consensus set (m+2), …, consensus set (m+t)}. Upon receiving a new consensus set instruction, the leader election cluster is first accessed to determine whether the primary key " / meta / consensus-shard-i" exists. If the primary key " / meta / consensus-shard-i" does not exist in the leader election cluster, consensus shard i has not been created, and the new consensus set instruction can be ignored.

[0119] If it is determined that consensus shard i has been created, then determine whether the new consensus set has been created. If it is determined that the new consensus set has not been created, update the value information corresponding to the primary key " / meta / consensus shard i" of consensus shard i based on the newly added t consensus set information {consensus set (m+1), consensus set (m+2), ..., consensus set (m+t)}.

[0120] After adding a new consensus set, a consensus master and backup node must be assigned to the new consensus set. Accordingly, upon receiving a key change notification, the consensus nodes in the system that subscribe to the primary key " / meta / consensus-shard-i" will trigger a new round of consensus master elections. Specifically, each consensus node in the system will traverse the consensus master node information for all consensus sets in consensus shard-i and attempt to become the consensus master node for the new consensus set.

[0121] The above completes the expansion of consensus sharding.

[0122] Scaling down a consensus shard means removing at least one consensus set from the consensus shard. A system administrator can issue a consensus set deletion instruction to the system, and the system responds to the consensus set deletion instruction, thereby removing the consensus set from the consensus shard. The consensus set deleted indicated by the consensus set deletion instruction includes at least one consensus set.

[0123] Correspondingly, when a deletion consensus set indication is received, the system first determines whether a deletion consensus set has been created based on the leader election cluster. If a deletion consensus set has not been created, the deletion consensus set indication is ignored. If the deletion consensus set has already been created, the deletion consensus set creation process is executed. The deletion consensus set creation process is executed, that is, based on the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard, the key value information of the target consensus shard in the leader election cluster is updated to remove the deletion consensus set from the target consensus shard.

[0124] Assume that there are m consensus sets in consensus shard i, and now t new consensus sets {consensus set 1, consensus set 2, …, consensus set t} need to be added. When a consensus set deletion instruction is received, the leader selection cluster is first accessed to determine whether the primary key " / meta / consensus-shard-i" exists. If the primary key " / meta / consensus-shard-i" does not exist in the leader selection cluster, it means that consensus shard i has not been created, and the consensus set deletion instruction can be directly ignored. If it is determined that consensus shard i has been created, the deletion consensus set is then determined to determine whether the deletion consensus set has been created. If it is determined that the deletion consensus set has been created, the value information corresponding to the primary key " / meta / consensus-shard-i" of consensus shard i is updated based on the information of the t deleted consensus sets {consensus set 1, consensus set 2, …, consensus set t}, thereby removing the deleted consensus set from consensus shard i.

[0125] More specifically, the consensus master node for the deletion consensus set is determined based on the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard. If no lease request information is received from the consensus master node for the deletion consensus set within a preset time period, the key value information of the deletion consensus set is filtered out from the key value information of the target consensus shard in the master selection cluster, and the key value information of the deletion consensus set is deleted to eliminate the deletion consensus set from the target consensus shard.

[0126] For example, to delete consensus set t, first determine that the target consensus shard indicated by the consensus set deletion instruction is consensus shard i, to which consensus set t belongs. First, access the leader election cluster. If the primary key " / consensus shard i / consensus set t" does not exist in the leader election cluster, it means that consensus set t is no longer in the consensus set list of consensus shard i, and the instruction to delete consensus set t can be directly ignored.

[0127] When determining the consensus master node for a deleted consensus set based on the deletion consensus set information and the target consensus shard carried in the deletion consensus set instruction, the consensus node actually autonomously determines whether it is the consensus master node for the deleted consensus set based on the deletion consensus set information and the target consensus shard. For example, the deletion consensus set information may be consensus set t, and the target consensus shard may be consensus shard i. When autonomously determining whether the current consensus node is the consensus master node for the deleted consensus set, if the primary key " / consensus_shard_i / consensus_set_t" exists but its corresponding key value is not the current consensus node's NodeID, this indicates that the current consensus node is not the consensus master node for consensus set t for consensus shard i. In this case, the deletion consensus set t instruction is directly ignored. If the primary key " / consensus_shard_i / consensus_set_t" exists and its corresponding key value is the current consensus node's NodeID, this indicates that the current consensus node is the consensus master node for consensus set t for consensus shard i. The current consensus node is then stopped from participating in the consensus process for consensus shard i. Furthermore, the current consensus node is stopped from sending lease request information (renewal heartbeats) to the leader election cluster.

[0128] Correspondingly, when the lease application information sent by the consensus master node for deleting the consensus set is not received within the preset time, the key value information of the consensus set t is filtered out from the key value information of the consensus shard i in the selected master cluster, and the key value information of the consensus set t is deleted to eliminate the consensus set t in the consensus shard i.

[0129] The deletion process of other consensus sets in the deletion consensus set is the same as the deletion process of consensus set t described in the above example, and will not be repeated here. After the deletion of the deletion consensus set is completed, the scale-down of the consensus shard is completed.

[0130] Deleting a consensus shard refers to deleting a consensus shard from a consensus cluster. A system administrator can issue a consensus shard deletion instruction to the system, which responds to the consensus shard deletion instruction, thereby deleting the consensus shard from the consensus cluster. The consensus shard deletion instruction indicates that the consensus shard to be deleted includes at least one consensus shard.

[0131] Correspondingly, when receiving a consensus shard deletion instruction, the system first determines whether the consensus shard to be deleted has been created. For example, if the consensus shard to be deleted is consensus shard i, the system accesses the master selection cluster to determine whether the primary key " / meta / consensus-shardi" exists. If the primary key " / meta / consensus-shardi" does not exist, it indicates that consensus shard i has not been created, and the consensus shard deletion instruction can be ignored.

[0132] If the consensus shard information in the leader election cluster determines that the consensus shard to be deleted has already been created, that is, if the primary key " / meta / consensus-shard i" exists, then, based on the consensus shard deletion information carried in the consensus shard deletion instruction, the key-value information of the deleted consensus shard is removed from the key-value information of the target consensus shard in the leader election cluster. Specifically, based on the information of consensus shard i, the primary key " / meta / consensus-shard i" is deleted. Furthermore, the key-value information of each consensus set corresponding to the deleted consensus shard is removed from the key-value information of the target consensus shard in the leader election cluster, thereby deleting the key-value information of each consensus set in consensus shard i. Specifically, upon receiving the deletion notification for the primary key " / meta / consensus-shard i", the consensus nodes subscribed to the primary key " / meta / " traverse the consensus set list for consensus shard i and perform the consensus set deletion operation. After the consensus set deletion operation is completed, the consensus master nodes of all consensus sets in consensus shard i cease participating in the consensus process for consensus shard i. The consensus set deletion process can refer to the content related to the deletion of the consensus set described above, which will not be repeated here.

[0133] The above completes the deletion of the consensus shard.

[0134] In summary, the creation and management of a consensus cluster primarily involves the interaction between the leader election cluster, the consensus cluster, and the consensus nodes in the system. Expanding, shrinking, and deleting consensus shards involves writing or deleting corresponding primary key information in the leader election cluster. This flexible consensus shard management solution not only reduces the impact of increasing the number of consensus nodes on consensus performance, thereby improving the system's horizontal scalability, but also enhances the overall system's flexibility.

[0135] The following describes in detail how to replace a consensus master node when a faulty consensus master node occurs.

[0136] As described above, when the leader election cluster fails to receive a lease request from the consensus node for a predetermined period of time, the leader election cluster is triggered to receive a master node request from a backup node in the consensus set. The consensus set with the consensus node as the consensus master is updated to the consensus set to be assigned, and the backup node is assigned as the consensus master of the consensus set to be assigned.

[0137] This is explained by taking the failure of the consensus master node of consensus set t in consensus shard i as an example.

[0138] If the consensus master node in consensus set t fails and cannot send heartbeats to the leader-electing cluster for renewal, the leader-electing cluster will automatically clear the consensus master node information corresponding to consensus set t when the lease expires. In other words, the primary key " / consensus shard i / consensus set t" will be cleared.

[0139] When the leader election cluster clears the primary key " / consensus-shard-i / consensus-set-t" held by the consensus master node of consensus set t, all other consensus nodes in the system receive a change notification because they have subscribed to this primary key. At this point, all healthy consensus nodes in the system access the metadata " / meta / consensus-shard-i" for consensus shard i to obtain the latest consensus set list corresponding to consensus shard i. All healthy consensus nodes in the system traverse the latest consensus set list for consensus shard i and determine whether there is a functioning consensus master node in the consensus set by checking the presence of the primary key " / consensus-shard-i / consensus-set-t". Upon reaching consensus set t, if " / consensus-shard-i / consensus-set-t" does not exist, the consensus node (backup node) proactively sends a lease request to the leader election cluster and attempts to write its own serial number information to the leader election cluster, attempting to run for the consensus master node for consensus set t.

[0140] If writing its own number information successfully, the consensus node (backup node) is successfully elected as the consensus master for consensus set t. The consensus node initiates the consensus process for consensus shard i, replacing the failed consensus master. If writing its own number information unsuccessfully, the node continues to check whether " / consensus shard i / consensus set t" already exists. If so, this indicates that another consensus node has replaced the failed consensus master and become the consensus master for consensus set t. At this point, the consensus node exits election mode and awaits notification of new consensus shard changes. If writing its own number information unsuccessfully and confirming that " / consensus shard i / consensus set t" does not exist, this indicates that no consensus node has yet replaced the failed consensus master as the consensus master for consensus set t. At this point, the node attempts to write its own number information again until writing its own number information successfully or confirming that another consensus node has replaced the failed consensus master and become the consensus master for consensus set t.

[0141] In summary, this embodiment organizes all consensus nodes into consensus sets, virtualizing them into a consensus resource pool. This master selection cluster selects the consensus master node for the consensus set, and processes consensus message packets based on the consensus set. Each consensus set has backup nodes and the ability to update failed consensus master nodes, ensuring that all consensus nodes can serve as backup nodes for failed consensus master nodes. Compared to traditional solutions, the replacement solution for failed consensus master nodes provided by this embodiment improves system resource utilization and the high availability of the consensus system.

[0142] See Figure 5 An embodiment of the present application also provides a system 10 that utilizes a consensus algorithm. The system 10 includes at least a processor 11, a replication state machine 12, and a memory 13.

[0143] The processor 11 is used to receive pending transactions and assign the pending transactions to a consensus shard in the created consensus cluster to trigger a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node.

[0144] The replicated state machine 12 is used to execute the consensus decision.

[0145] The memory 13 is used to store the execution result of the consensus resolution to complete the processing of the pending transaction.

[0146] The system 10 also includes a master selection cluster 14. The consensus cluster is created by obtaining a consensus shard creation request, which carries at least the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created. Based on the consensus shard creation request, the master selection cluster 14 is queried to determine whether the name of the consensus shard to be created exists. If not, the consensus shard is created in the consensus cluster based on the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created, thereby obtaining the created consensus cluster.

[0147] More specifically, based on the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request, a key-value pair of the consensus shard is generated; the key-value pair of the consensus shard is inserted into the master selection cluster 14 to create the consensus shard in the consensus cluster.

[0148] More specifically, the key-value pair of the consensus shard is inserted into the master election cluster 14, and the master election cluster 14 is triggered to receive the master node application information sent by a consensus node in the system 10; the master election cluster 14 is triggered to respond to the master node application information, and the number information of the consensus node is inserted into the key-value pair of the consensus shard to assign the consensus node as the consensus master node of the consensus set to be assigned in the consensus shard, wherein the consensus set to be assigned refers to a consensus set that does not have a consensus master node; for each consensus set with a consensus master node in the consensus shard, at least one consensus node is assigned as at least one backup node of the consensus set, and the consensus master node and the backup node in the consensus set are different; when each consensus set in the consensus shard is assigned a consensus master node and at least one backup node, the creation of the consensus shard in the consensus cluster is completed.

[0149] More specifically, the master node application information includes lease application information and the number information of the consensus node; after the consensus node is assigned as the consensus master node, it periodically sends lease application information to the master election cluster 14; the master election cluster 14 responds to the lease application information and maintains the consensus node as the consensus master node. When the master election cluster 14 does not receive the lease application information sent by the consensus node for a preset period of time, the processor 11 is specifically configured to trigger the master election cluster 14 to receive master node application information sent by the backup node in the consensus set; update the consensus set with the consensus node as the consensus master node to the consensus set to be assigned, and assign the backup node as the consensus master node of the consensus set to be assigned.

[0150] More specifically, each consensus node in the system 10 subscribes to the key value information of each consensus shard recorded by the leader election cluster 14. When the key value information of any consensus shard is updated, each consensus node sends the leader node application information to the leader election cluster 14.

[0151] The processor 11 is specifically used to, when receiving a new consensus set indication, if it is determined based on the consensus set information in the master election cluster 14 that the new consensus set has not been created, then, according to the new consensus set information carried by the new consensus set indication and the indicated target consensus shard, update the key value information of the target consensus shard in the master election cluster 14 to add the new consensus set to the target consensus shard, wherein the new consensus set includes at least one consensus set.

[0152] The processor 11 is specifically used to, when receiving a deletion consensus set indication, if it is determined based on the consensus set information in the master election cluster 14 that the deletion consensus set has been created, then, according to the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard, update the key value information of the target consensus shard in the master election cluster 14 to remove the deletion consensus set from the target consensus shard, wherein the deletion consensus set includes at least one consensus set.

[0153] The processor 11 is specifically used to determine the consensus master node of the deletion consensus set based on the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard; when the lease application information sent by the consensus master node of the deletion consensus set is not received for more than a preset time period, the key value information of the deletion consensus set is filtered out from the key value information of the target consensus shard in the master selection cluster 14, and the key value information of the deletion consensus set is deleted to eliminate the deletion consensus set in the target consensus shard.

[0154] The processor 11 is specifically used to, when receiving a deletion consensus shard indication, if it is determined based on the consensus shard information in the master election cluster 14 that the deletion consensus shard has been created, then, based on the deletion consensus shard information carried by the deletion consensus shard indication, remove the key value information of the deletion consensus shard from the key value information of the target consensus shard in the master election cluster 14, and remove the key value information of each consensus set corresponding to the deletion consensus shard from the key value information of the target consensus shard in the master election cluster 14.

[0155] The system 10 also includes a transaction receiving module, which includes multiple transaction pool shards. The processor 11 is specifically used to trigger the transaction receiving module to receive pending transactions, and cache the pending transactions to the target transaction pool shard corresponding to the shard number according to the shard number carried by the pending transactions; according to the correspondence between the transaction pool shards and the consensus shards, obtain the consensus shard corresponding to the target transaction pool shard, and allocate the pending transactions to a consensus shard corresponding to the target transaction pool shard.

[0156] See Figure 6One embodiment of the present application further provides an electronic device 20, which includes a processor 21 and a memory 22 in communication with the processor 21. The memory 22 stores computer-executable instructions, and the processor 21 executes the computer-executable instructions stored in the memory 22 to implement the transaction processing method provided in any of the above embodiments.

[0157] The present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the instructions are executed, the computer-executable instructions are executed by a processor to implement the transaction processing method provided in any of the above embodiments.

[0158] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the transaction processing method provided in any of the above embodiments.

[0159] It should be noted that the computer-readable storage medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface mount storage device, an optical disc, or a compact disc read-only memory (CD-ROM). It may also be various electronic devices that include one or any combination of the above memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0160] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0161] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0163] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0166] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A transaction processing method, characterized in that: Applied to a system utilizing a consensus algorithm, the method comprises: Receive pending transactions and assign them to a consensus shard in a created consensus cluster, triggering a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node; Executing the consensus resolution and storing the execution result of the consensus resolution to complete the processing of the pending transaction; The system also includes a leader cluster selection process, and the consensus cluster is created in the following ways: Obtain a consensus shard creation request, the creation request carrying at least the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created; query whether the name of the consensus shard to be created exists in the selected master cluster based on the consensus shard creation request; if not, create a consensus shard in the consensus cluster based on the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request, so as to obtain the created consensus cluster; The step of creating a consensus shard in the consensus cluster according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request includes: Generate a key-value pair of the consensus shard according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request; insert the key-value pair of the consensus shard into the leader selection cluster to create the consensus shard in the consensus cluster.

2. The method according to claim 1, characterized in that Inserting the key-value pair of the consensus shard into the leader selection cluster to create the consensus shard in the consensus cluster includes: Insert the key-value pair of the consensus shard into the leader election cluster, and trigger the leader election cluster to receive the master node application information sent by a consensus node in the system; Triggering the leader election cluster to respond to the master node application information and insert the number information of the consensus node into the key-value pair of the consensus shard to assign the consensus node as the consensus master node of the consensus set to be assigned in the consensus shard, wherein the consensus set to be assigned refers to a consensus set that does not have a consensus master node; Assign at least one consensus node to each consensus set with a consensus master node in the consensus shard as at least one backup node of the consensus set, where the consensus master node and the backup node in the consensus set are different; When each consensus set in a consensus shard is assigned a consensus master node and at least one backup node, creation of a consensus shard in the consensus cluster is complete.

3. The method according to claim 2, characterized in that The master node application information includes lease application information and the number information of the consensus node; after the consensus node is assigned as the consensus master node, the lease application information is sent to the master election cluster at a regular interval; The leader election cluster responds to the lease application information and maintains the consensus node as the consensus master node; The method further comprises: When the master election cluster fails to receive the lease application information sent by the consensus node for a preset period of time, the master election cluster is triggered to receive the master node application information sent by the backup node in the consensus set; The consensus set with the one consensus node as the consensus master node is updated to be the consensus set to be allocated, and the backup node is allocated as the consensus master node of the consensus set to be allocated.

4. The method according to claim 3, characterized in that Each consensus node in the system subscribes to the key value information of each consensus shard recorded by the leader election cluster. When the key value information of any consensus shard is updated, each consensus node sends the master node application information to the leader election cluster.

5. The method according to any one of claims 1 to 4, characterized in that After creating the consensus cluster, the method further includes: When receiving a new consensus set indication, if it is determined based on the consensus set information in the leader selection cluster that the new consensus set has not been created, then, According to the new consensus set information carried by the new consensus set indication and the indicated target consensus shard, the key value information of the target consensus shard in the master election cluster is updated to add the new consensus set to the target consensus shard, wherein the new consensus set includes at least one consensus set.

6. The method according to any one of claims 1 to 4, characterized in that Also includes: When receiving a consensus set deletion instruction, if it is determined based on the consensus set information in the master selection cluster that the consensus set has been created, then: According to the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard, the key value information of the target consensus shard in the master election cluster is updated to remove the deletion consensus set from the target consensus shard, wherein the deletion consensus set includes at least one consensus set.

7. The method according to claim 6, characterized in that The updating of the key value information of the target consensus shard in the master selection cluster according to the deletion consensus set information carried by the deletion consensus set indication and the indicated target consensus shard, so as to remove the deletion consensus set from the target consensus shard, includes: Determine the consensus master node for deleting the consensus set based on the deletion consensus set information carried by the deletion consensus set instruction and the indicated target consensus shard; When the lease application information sent by the consensus master node of the deletion consensus set is not received within the preset time period, the key value information of the deletion consensus set is filtered out from the key value information of the target consensus shard in the master selection cluster, and the key value information of the deletion consensus set is deleted to eliminate the deletion consensus set in the target consensus shard.

8. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When receiving a consensus shard deletion instruction, if it is determined based on the consensus shard information in the master selection cluster that the consensus shard to be deleted has been created, then, According to the deletion consensus shard information carried by the deletion consensus shard indication, the key value information of the deletion consensus shard is removed from the key value information of the target consensus shard in the master selection cluster, and the key value information of each consensus set corresponding to the deletion consensus shard is removed from the key value information of the target consensus shard in the master selection cluster.

9. The method according to any one of claims 1 to 4, characterized in that The system further includes a transaction receiving module, the transaction receiving module including a plurality of transaction pool shards, and receiving pending transactions and assigning pending transactions to a consensus shard includes: Trigger the transaction receiving module to receive pending transactions, and cache the pending transactions to the target transaction pool shard corresponding to the shard number according to the shard number carried by the pending transactions; According to the correspondence between the transaction pool shards and the consensus shards, the consensus shard corresponding to the target transaction pool shard is obtained, and the to-be-processed transaction is allocated to a consensus shard corresponding to the target transaction pool shard.

10. A system using a consensus algorithm, characterized in that: The system comprises at least: a processor, a replication state machine and a memory; wherein, The processor is configured to receive pending transactions and assign the pending transactions to a consensus shard in the created consensus cluster, thereby triggering a consensus master node in a consensus set in the consensus shard to process the pending transactions and generate a consensus resolution; wherein the consensus cluster includes multiple independent consensus shards, each consensus shard includes multiple consensus sets, and each consensus set has a consensus master node and at least one backup node; The replicated state machine is used to execute the consensus resolution; The memory is used to store the execution result of the consensus resolution to complete the processing of the pending transaction; The system also includes a master selection cluster, and the method for creating the consensus cluster includes: obtaining a creation request for a consensus shard, the creation request carrying at least the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created; according to the creation request for the consensus shard, querying whether the name of the consensus shard to be created exists in the master selection cluster; if it does not exist, creating a consensus shard in the consensus cluster according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request, so as to obtain the created consensus cluster; wherein, creating a consensus shard in the consensus cluster according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request includes: generating a key-value pair of the consensus shard according to the name of the consensus shard to be created and the number of consensus sets contained in the consensus shard to be created carried in the creation request; and inserting the key-value pair of the consensus shard into the master selection cluster to create the consensus shard in the consensus cluster.

11. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the transaction processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed, enable a computer to execute the transaction processing method according to any one of claims 1 to 9.

Images