Cross-shard transaction processing method and system

Abstract

The embodiments of the present description relate to the technical field of data processing, and in particular to a cross-shard transaction processing method and system. The method comprises: after receiving the current Epoch number and a cross-shard transaction corresponding thereto which are sent by a cross-shard service module, each shard node processes the cross-shard transaction; and each shard node determines whether a respective received second transaction sent by a client belongs to a local transaction type of the shard node itself and, if so, processes same or, if not, sends to the cross-shard service module the second transaction as a non-local transaction. The embodiments of the present description solve the problems of existing cross-shard transaction processing technologies that centralized serial sorting results in relatively high sorting costs, and failure to distinguish between intra-shard transactions and cross-shard transactions results in a waste of communication resources.

Description

A cross-shard transaction processing method and system Technical Field

[0001] The embodiments of this specification relate to the field of data processing technology, and in particular to a cross-shard transaction processing method and system. Background Art

[0002] In current blockchain systems, each node stores all state and processes all transactions simultaneously, ensuring security while limiting scalability. The basic idea behind blockchain sharding is to divide the nodes in a blockchain network into several relatively independent shards. A single shard handles smaller transactions or even stores only a portion of the network state. Multiple shards process transactions in parallel, thereby increasing the throughput of the entire network.

[0003] While sharding is an exciting and promising direction, while it improves efficiency, it also introduces new challenges. These include security and efficiency issues within a shard, as well as cross-shard security and efficiency issues arising from cross-shard transactions. Existing blockchain systems primarily focus on improving security, while struggling to simultaneously enhance transaction processing efficiency. A transaction may include multiple inputs and outputs, and cross-shard communication requires a trade-off between the costs of communication and the benefits of improved performance. Furthermore, cross-shard transactions are often unavoidable. In extreme cases, where all transactions within a system are cross-shard, system performance will degrade compared to pre-sharding levels.

[0004] Existing technical solutions for sharded blockchain systems are relatively scarce, and most rely on traditional distributed transaction processing methods. Common approaches include using distributed locks to manage cross-shard transactions and using centralized ordering nodes to sort all transactions. However, this method uses centralized serial ordering, which is costly. Furthermore, sending all transactions to the ordering node wastes communication resources for transactions within the shard.

[0005] There is an urgent need for a cross-shard transaction processing method to solve the problem that the current cross-shard transaction processing technology has high sorting costs due to centralized serial sorting, and does not distinguish between intra-shard transactions and cross-shard transactions, thus wasting communication resources.

[0006] Contents of this application

[0007] To address the current cross-shard transaction processing technology's high sorting costs due to centralized serial sorting, and the failure to distinguish between intra-shard and cross-shard transactions, which wastes communication resources, the present specification provides a cross-shard transaction processing method and system. This method simulates a global logical clock by distributing Epoch numbers, divides transactions arriving at shard nodes in time, and performs deterministic transaction processing on transactions within each Epoch time slice through deterministic transactions, so that transactions within each Epoch time slice can be executed in parallel. Finally, the Epoch module synchronizes cross-shard transactions with Epoch numbers across all shards to achieve cross-shard transaction management.

[0008] In order to solve the above technical problems, the specific technical solutions of the embodiments of this specification are as follows:

[0009] In one aspect, an embodiment of this specification provides a cross-shard transaction processing method, the method comprising:

[0010] After receiving the current Epoch number and the cross-shard transaction of its own local transaction type corresponding to the current Epoch number sent by the cross-shard service module, each shard node processes the cross-shard transaction, wherein the cross-shard transaction is the first transaction of a non-local transaction type received by the cross-shard service module from the client between the previous Epoch number and the current Epoch number and received by other shard nodes. At the same time, each shard node determines whether the second transaction sent by the client it receives belongs to its own local transaction type. If it belongs to its own local transaction type, the second transaction is treated as a local transaction and processed. If it does not belong to its own local transaction type, the second transaction is sent to the cross-shard service module as a non-local transaction.

[0011] On the other hand, the embodiments of this specification also provide a cross-shard transaction processing system, the system including multiple shard nodes and a cross-shard service module;

[0012] The above method is executed when the shard node and the cross-shard service module process the transaction sent by the client.

[0013] Using the embodiment of this specification, the cross-shard service module periodically generates continuous Epoch numbers and sends the Epoch numbers to each shard node. At the same time as sending the Epoch number, the cross-shard transaction corresponding to the current Epoch number and belonging to the local transaction type of each shard node is also sent to each shard node, so that each shard node can process the cross-shard transaction sent by the cross-shard service module. After receiving the Epoch number, the shard node determines whether the second transaction sent by the client it receives belongs to its own local transaction type. If it does, it processes the second transaction. If it does not belong to its own local transaction type, it sends the second transaction to the cross-shard service module, so that the cross-shard service module can send the second transaction to the corresponding shard node for processing when sending the next Epoch number. The embodiment of this specification implements the function of distributed synchronization lock through the Epoch number mechanism, and uses Epoch numbers to divide time slices to control the synchronous execution of cross-shard transactions. Compared with the traditional distributed lock method, it reduces the performance overhead of explicit distributed locks and improves the parallel processing capability of transactions. In addition, compared to the centralized sorting method, the embodiments of this specification do not need to send all transactions to the sorting node for sorting consensus. Instead, they only send transactions that do not belong to their own local transaction types to the cross-shard service module for distribution, reducing the sorting waiting time and network communication overhead of transactions within the shard, and further improving the parallel processing capability of transactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0015] FIG1 is a schematic diagram showing a system for implementing a cross-shard transaction processing method according to an embodiment of this specification;

[0016] FIG2 is a flowchart of a cross-shard transaction processing method according to an embodiment of the present specification;

[0017] FIG3 shows a flowchart of the cross-shard service module execution according to an embodiment of the present specification;

[0018] FIG4 is a schematic diagram showing the structure of a cross-shard service module according to an embodiment of the present specification;

[0019] FIG5 is a flowchart showing the cross-shard service module according to an embodiment of this specification performing a first deterministic transaction processing on the non-local transaction to obtain a first deterministic transaction sequence;

[0020] FIG6 is a schematic diagram showing a flow chart of performing read-write conflict detection on the non-local transactions sent by each shard node in an embodiment of this specification;

[0021] FIG7 is a schematic diagram showing a transaction submission process according to an embodiment of the present disclosure;

[0022] FIG8 is a schematic diagram showing a detailed workflow of a deterministic transaction processing mechanism according to an embodiment of this specification;

[0023] FIG9 is a schematic diagram showing the execution flow of the cross-shard transaction processing method according to an embodiment of this specification;

[0024] FIG10 is a schematic diagram showing a process of processing the cross-shard transaction by a shard node according to an embodiment of this specification;

[0025] FIG11 is a schematic diagram showing a process of executing transactions by a shard node within an Epoch time slice according to an embodiment of this specification;

[0026] FIG12 is a schematic diagram showing a shard transaction processing flow within a shard node according to an embodiment of the present specification;

[0027] FIG13 is a schematic diagram showing the structure of a computer device according to an embodiment of the present specification.

[0028] [Explanation of the accompanying drawings]: 1. Client; 2. Processing system; 21. Processing node; 211. Shard node; 212. Storage shard; 22. Cross-shard service module; 401. Consistency maintenance submodule; 402. Deterministic processing submodule; 403. Epoch service submodule; 404. Transaction management submodule; 405. Data storage submodule; 1302. Computer device; 1304. Processing device; 1306. Storage resource; 1308. Driving mechanism; 1310. Input / output module; 1312. Input device; 1314. Output device; 1316. Presentation device; 1318. Graphical user interface; 1320. Network interface; 1322. Communication link; 1324. Communication bus. DETAILED DESCRIPTION

[0029] FIG1 is a schematic diagram of a system for implementing a cross-shard transaction processing method according to an embodiment of the present specification. The system may include: multiple clients 1 and a processing system 2. The processing system may be a blockchain system or a distributed processing system, and the present specification does not limit this. The client 1 and the processing system 2 communicate via a network. The network may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, and is connected to a website, a user device (e.g., a computing device), and a back-end system. The client 1 sends the transaction to the processing system 2 for processing. The processing system 2 is deployed with multiple processing nodes 21. In a blockchain system, a processing node 21 is a node in the blockchain. In a distributed processing system, a processing node 21 is a distributed processing node in the distributed system. Consensus accounting is required between each processing node 21.

[0030] In order to improve processing efficiency, the existing technology has sharded the processor and memory on the processing node 21, that is, the processing node 21 includes multiple shard nodes 211 and storage shards 212 corresponding to each shard node 211, and consensus accounting is also required between the storage shards 212. Each shard node 211 can have its own business processing scope, that is, each shard node 211 can only handle transactions of its own local transaction type. Although sharding technology can improve processing efficiency, cross-shard transactions are often unavoidable. Cross-shard transactions refer to transactions received by the shard node 211 from the client 1 that do not belong to its own local transaction type, so they need to be handed over to the shard node that can handle the transaction for processing. How to sort and distribute transactions sent by each shard node 211 that do not belong to its own local transaction type is a technical problem that urgently needs to be solved.

[0031] In the existing technology, cross-shard transactions can be managed through distributed locks, but this method has a large performance overhead, resulting in low transaction processing capabilities.

[0032] Alternatively, all transactions may be sorted through a centralized sorting node. However, this method requires each shard node 211 to send all transactions to the sorting node for sorting consensus, resulting in high network communication overhead and poor transaction parallel processing capability.

[0033] In response to the above-mentioned problems, the embodiments of this specification provide a cross-shard transaction processing method, which introduces a cross-shard service module 22. The cross-shard service module 22 introduces the Epoch mechanism to act as a global clock, periodically sending consecutive Epoch numbers to all shard nodes 211. Two adjacent Epoch numbers divide a time slice, and the order of transactions is uniquely determined by combining the numbers of all shard nodes 211 in the consensus cluster, the arrival order of transactions sent by each shard node 211 that do not belong to its own local transaction type and the division of Epoch time slices. No explicit sorting process is required, thereby avoiding the additional overhead brought by the transaction sorting operation in the transaction processing system. Through deterministic transactions, conflict-free execution of transactions within each Epoch time slice is achieved, thereby improving the system's throughput performance and fully utilizing the parallel processing capabilities of the sharding system.

[0034] Specifically, an embodiment of this specification provides a cross-shard transaction processing method, which simulates a global logical clock by distributing Epoch numbers, divides the transactions arriving at the shard nodes in time, and performs deterministic transaction processing on the transactions in each Epoch time slice through deterministic transactions, so that the transactions in each Epoch time slice can be executed in parallel. Finally, the cross-shard transactions are combined with the Epoch numbers through the Epoch module to synchronize all shards to achieve cross-shard transaction management. Figure 2 shows a flow chart of a cross-shard transaction processing method according to an embodiment of this specification. This figure describes the process of cross-shard processing by shard nodes, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many, and does not represent the only execution order. When the system or device product is executed in practice, it can be executed sequentially or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, as shown in Figure 2, it can be executed by the shard node 211, and the method may include:

[0035] Step 201: After receiving the current Epoch number and the cross-shard transaction of its own local transaction type corresponding to the current Epoch number sent by the cross-shard service module, the shard node processes the cross-shard transaction;

[0036] In this step, the cross-shard transaction is the first transaction of a local transaction type that is not of its own and is received by the cross-shard service module between the previous Epoch number and the current Epoch number and is sent by other shard nodes from the client.

[0037] Step 202: At the same time, each shard node determines whether the second transaction sent by the client received by each shard node belongs to its own local transaction type. If it belongs to its own local transaction type, the second transaction is treated as a local transaction and processed. If it does not belong to its own local transaction type, the second transaction is sent to the cross-shard service module as a non-local transaction.

[0038] Using the embodiment of this specification, the cross-shard service module periodically generates continuous Epoch numbers and sends the Epoch numbers to each shard node. At the same time as sending the Epoch number, the cross-shard transaction corresponding to the current Epoch number and belonging to the local transaction type of each shard node is also sent to each shard node, so that each shard node can process the cross-shard transaction sent by the cross-shard service module. After receiving the Epoch number, the shard node determines whether the second transaction sent by the client it receives belongs to its own local transaction type. If it does, it processes the second transaction. If it does not belong to its own local transaction type, it sends the second transaction to the cross-shard service module, so that the cross-shard service module can send the second transaction to the corresponding shard node for processing when sending the next Epoch number. The embodiment of this specification implements the function of distributed synchronization lock through the Epoch number mechanism, and uses Epoch numbers to divide time slices to control the synchronous execution of cross-shard transactions. Compared with the traditional distributed lock method, it reduces the performance overhead of explicit distributed locks and improves the parallel processing capability of transactions. In addition, compared to the centralized sorting method, the embodiments of this specification do not need to send all transactions to the sorting node for sorting consensus. Instead, they only send transactions that do not belong to their own local transaction types to the cross-shard service module for distribution, reducing the sorting waiting time and network communication overhead of transactions within the shard, and further improving the parallel processing capability of transactions.

[0039] According to one embodiment of this specification, in order to improve the parallel processing efficiency of cross-shard transactions, as shown in FIG3 , the steps performed by the cross-shard service module further include:

[0040] Step 301: After receiving the non-local transactions sent by each shard node, the cross-shard service module performs a first deterministic transaction process on the non-local transactions to obtain a first deterministic transaction sequence;

[0041] Step 302: When sending the next Epoch number to each shard node, the cross-shard transaction in the first deterministic transaction sequence is sent to the corresponding shard node for processing.

[0042] The difference between the method of the embodiment of this specification and the traditional method is that:

[0043] First, epochs are not requested by the server actively, but are issued continuously, acting as a global clock. This can strictly divide all transactions in time and avoid becoming a system bottleneck in high-concurrency and large-scale node scenarios.

[0044] Secondly, deterministic transaction processing can eliminate resource conflicts between transactions within the same Epoch time slice, allowing transactions to be executed in parallel.

[0045] Finally, intra-shard and cross-shard transactions are managed differently, eliminating the need to centrally sort all transactions and reducing unnecessary network overhead.

[0046] It should be noted that the embodiments of this specification do not need to consider specific sharding schemes and forms, and can be adapted to any current sharding strategy, such as data element-based sharding, network-based sharding, etc.

[0047] Specifically, the cross-shard transaction processing method of the embodiments of this specification is primarily used in parallel blockchain systems to ensure that each shard node generates blocks at the same time interval, maintains the global transaction order according to certain rules, and ensures the final execution order and consistency of transactions. In addition to its application in parallel blockchain systems, it can also be applied to serially ordered blockchain systems with non-sharded architectures for efficient transaction processing. Furthermore, it can also be applied to traditional distributed systems to parallelize transactions generated by shards, which is not a limitation of the embodiments of this specification.

[0048] Specifically, according to an embodiment of the present specification, as shown in FIG4 , the cross-shard service module may include a consistency maintenance submodule 401, a deterministic processing submodule 402, an Epoch service submodule 403, a transaction management submodule 404, and a data storage submodule 405. The transaction management submodule 404 is primarily responsible for managing received transactions, including pre-processing received transactions, receiving transactions and calling related processing modules, and submitting processed transactions to the data storage module for persistence.

[0049] According to one embodiment of the present specification, in order to ensure that the cross-shard transactions in the first deterministic transaction sequence are sent to the corresponding shard nodes for processing, each shard node also performs consensus verification on the processing transaction scope of its own shard, so that the cross-shard service module determines the shard node corresponding to the cross-shard transaction in the first deterministic transaction sequence based on the results of the consensus verification, and sends the cross-shard transaction to the corresponding shard node for processing.

[0050] This step can be performed by the consistency maintenance submodule 401 in the cross-shard service module. The role of the consistency maintenance submodule 401 is to ensure that the data and state in the distributed system remain consistent across multiple shard nodes. In a distributed environment, due to the involvement of multiple shard nodes and concurrent operations, the system requires a mechanism to solve the consistency problem to prevent inconsistent data or erroneous states. Therefore, in the embodiment of this specification, this part of the function is abstracted into the consistency maintenance submodule 401, which implements data synchronization operations through a consistent consensus algorithm. Its main functions are:

[0051] 1) Responsible for reaching consensus with other shard nodes on Epoch number and cross-shard transaction order.

[0052] 2) Responsible for maintaining the consistency of Epoch numbers, cross-shard transactions, and other network messages received by the shards.

[0053] 3) Responsible for maintaining data consistency of all shard nodes and achieving data synchronization.

[0054] After the first deterministic transaction sequence is generated, the first deterministic transaction sequence is sent to the corresponding sharding node through the consistency maintenance submodule 401.

[0055] In the embodiments of this specification, the deterministic processing submodule 402 can perform deterministic transaction processing on non-local transactions sent by each shard node. The function of the deterministic processing submodule 402 is to generate a deterministic execution sequence using a deterministic algorithm given an arbitrary input sequence. Specifically, as shown in FIG5 , the cross-shard service module performs a first deterministic transaction processing on the non-local transaction, and the first deterministic transaction sequence further includes:

[0056] Step 501: Perform read-write conflict detection on the non-local transactions sent by each shard node;

[0057] Step 502: Abort the non-local transactions with read-write conflicts, and group the non-local transactions without read-write conflicts into the first deterministic transaction sequence.

[0058] In the embodiments of this specification, deterministic transactions aim to use certain principles and conditions to define the conditions under which transaction conflicts occur and how to resolve them. Unlike other concurrency control methods, deterministic transactions primarily determine whether one or more transactions have a read-write conflict within a very short timeframe.

[0059] If there is a read-write conflict in a transaction, the transaction with the read-write conflict is aborted, and the transactions without the read-write conflict are combined into a deterministic transaction sequence.

[0060] Furthermore, as shown in FIG6 , performing read-write conflict detection on the non-local transactions sent by each shard node further includes:

[0061] Step 601: The non-local transaction sent by each shard node is treated as a cross-shard transaction to be processed;

[0062] Step 602: Determine the data elements corresponding to the to-be-processed cross-shard transaction and the read / write types of the data elements.

[0063] Step 603: Perform read-write conflict detection on the read and write types of the same data elements according to the order in which the cross-shard transactions to be processed are received. If there is a read-write conflict in at least one data element of the cross-shard transactions to be processed, then there is a read-write conflict in the cross-shard transactions to be processed.

[0064] For example, as shown in Figure 7, within the same Epoch time slice, if shard node block i submits two non-local transactions T1 and T2 successively, if both T1 and T2 are read-only businesses and neither modifies the data status, then both transactions can be submitted; if transaction T1 is read-only and transaction T2 is an update operation, since T2 has not arrived when T1 is executed, it will not affect the results of the two transactions, so both transactions can be submitted; if transaction T1 is an update operation and transaction T2 is a read-only operation, then because the data value has been changed by T1 when T2 is read, transaction T2 is aborted; if both T1 and T2 are update operations, this will cause the update operation of T1 to be lost, so transaction T2 is aborted.

[0065] Specifically, as shown in Figure 8, the detailed workflow of the deterministic transaction processing mechanism is as follows:

[0066] Step 1: Transaction preprocessing: Preprocess the non-local transaction sent by the received shard node to obtain the read and write sequence of the corresponding data elements contained in the non-local transaction.

[0067] Step 2: Retrieve the data element table: According to the data element read and write sequence contained in the non-local transaction, search the data element read and write record table in the deterministic processing submodule 402.

[0068] In this step, the data element read and write record table records the read and write records of the transaction elements in the order of transaction submission in this Epoch time slice. When a transaction submitted by any shard node is received within this Epoch time slice, the data element read and write record table is checked to see whether there are read and write records for each data element of the transaction.

[0069] Step 3: Determine whether there is a write operation record for the data element. If yes, go to step 4 and execute the abort operation; if no, go to step 5 and execute the retrieval read and write type operation.

[0070] In this step, if a data element already has a write operation record in the data element read and write record table, it means that other transactions have already updated the data element within the Epoch time slice. Then, within the Epoch time slice, any subsequent operations (read or write) on the data element will be in conflict, so execute step 4 to terminate the transaction.

[0071] If there is no write operation record for a data element in the data element read and write record table, it means that no other transaction has updated the data element first within the Epoch time slice. Then, within the Epoch time slice, any subsequent operations (read or write) on the data element are not conflicting, so execute step 5.

[0072] Step 4: Abort the transaction: The read and write sequence included in the current transaction contains secondary read and write operations on data elements that have been changed in the current Epoch time slice, which does not comply with the deterministic transaction processing rules. Therefore, the current transaction is aborted and the process ends.

[0073] Step 5: Retrieve the read and write type of the transaction: Check the operation type of the data element based on the read and write sequence of the data elements contained in the transaction.

[0074] Step 6: Determine whether it is a write operation: If it is, the current operation will modify the data element, then go to step 7 and execute the update data element table operation; if it is not, then go to step 8 and accept the transaction.

[0075] Step 7: Update the data element table: add the modification operation record of the data element to the data element operation record table and mark it as modified.

[0076] In this step, since reading first and writing later (or reading) does not conflict, in order to save resources, the embodiment of this specification only records the write operation type in the data element operation record table, and does not record the read operation type.

[0077] Step 8: Execute the transaction: Complete the deterministic transaction processing and submit the transaction to the data storage submodule 405 for subsequent operations.

[0078] It should be noted that Figure 8 describes the deterministic transaction processing mechanism, not the mechanism for executing transactions on shard nodes. Therefore, the transaction execution described in step 8 refers to the deterministic processing submodule. Transaction execution here refers to generating a deterministic transaction sequence. For the deterministic processing submodule, transaction execution means completing deterministic transaction processing. Transactions are not processed within the deterministic processing submodule.

[0079] In an embodiment of the present specification, because read-write conflict detection is based on the commit order of transactions, after the deterministic processing submodule 402 completes the deterministic processing and obtains the deterministic transaction sequence, each shard node also needs to process the transaction in the order in which the transaction is committed. Therefore, according to an embodiment of the present specification, after the non-local transactions that do not have read-write conflicts are combined into the first deterministic transaction sequence, the method further includes:

[0080] The order in which each pending cross-shard transaction is received is recorded in the first deterministic transaction sequence, so that the shard node processes the pending cross-shard transaction in the order in which the pending cross-shard transaction is received.

[0081] In the embodiment of this specification, the execution process of the cross-shard transaction processing method can be shown in Figure 9, which specifically includes the following steps:

[0082] Step 1: Transaction preprocessing: When a sharding node receives a transaction sent by a client, it first preprocesses the transaction and converts it into a set of read and write operations based on data elements (transaction read and write set).

[0083] Step 2: Determine whether the transaction is cross-shard: Based on the pre-processed transaction information and the local transaction type of the shard node itself, determine whether the transaction contains data elements that require cross-shard operations. If the judgment is yes (as long as there is at least one data element that requires cross-shard processing, the transaction is non-local), proceed to step 4 and execute the cross-shard transaction processing mechanism; if the judgment is no, proceed to step 3 and execute the intra-shard transaction processing mechanism.

[0084] Step 3: Intra-shard transaction processing: Execute the intra-shard transaction processing mechanism. Transactions are ordered and executed locally on the shard node. Intra-shard transactions need to wait until the cross-shard transactions of the current Epoch time slice are processed before execution. Deterministic transaction processing is also required during execution, as shown in Figure 12.

[0085] Step 4: Cross-shard transaction processing: Execute the cross-shard transaction processing mechanism, send the transaction information to the cross-shard service module for cross-shard sorting, and obtain the Epoch message with a deterministic transaction sequence (including the next Epoch number and the corresponding cross-shard transaction).

[0086] Step 5: Deterministic transaction processing: Execute the deterministic transaction processing mechanism and process transactions within the next Epoch time slice according to the deterministic transaction sorting rules to obtain the final transaction sorting sequence. The cross-shard transactions in this transaction sorting sequence need to be sent to the corresponding shard node together with the next Epoch number for processing. The processing process is shown in Figure 12.

[0087] In an embodiment of this specification, the method for the shard node to process the cross-shard transaction and the local transaction further includes:

[0088] The cross-shard transactions and local transactions are processed according to their respective priorities.

[0089] In the embodiments of this specification, the priority of each transaction can be pre-specified, and the high priority transactions are processed first, and the low priority transactions are processed later. For example, if the priority of cross-shard transactions is higher than that of local transactions, then when the current Epoch number and the corresponding cross-shard transaction sent by the cross-shard service module are received, the cross-shard transaction is executed first. It is determined whether the transaction sent by the client received between the current Epoch number and the next Epoch number is a cross-shard transaction. If so, it is sent to the cross-shard service module. If not, the local transaction is executed after the cross-shard transaction corresponding to the current Epoch number is executed. And all transactions are processed and completed before the next Epoch number is received.

[0090] Furthermore, the method for the shard node to process the cross-shard transaction and the local transaction further includes:

[0091] The shard node performs second deterministic transaction processing on the cross-shard transaction and the local transaction to obtain a second deterministic transaction sequence, and processes the transactions in the second deterministic transaction sequence.

[0092] According to one embodiment of the present specification, in order to improve the efficiency of a shard node in processing cross-shard transactions and local transactions within the current Epoch time slice, as shown in FIG10 , before the shard node processes the cross-shard transaction, the method further includes:

[0093] Step 1001: After the transactions before the current Epoch number are processed, a current snapshot is generated to facilitate processing of the cross-shard transactions and local transactions within the current snapshot, and the processing results are stored in the memory space corresponding to the current snapshot;

[0094] Step 1002: Before receiving the next Epoch number, complete processing the cross-shard transaction and the local transaction, end the current snapshot, and store the processing results in the memory space corresponding to the current snapshot on disk.

[0095] In the embodiments of this specification, the epoch mechanism is equivalent to accumulating transactions for batch processing. For blockchain systems, this is the beginning of a block; for general distributed systems, it is the beginning of a batch. If data operations are directly written to disk, I / O will be frequent and hardware pressure will be significant. In the method shown in Figure 10, the current snapshot acts as an isolation mechanism. After the current snapshot begins, all operations are executed in the memory space corresponding to the current snapshot. Only after the current snapshot ends are the processing results in the memory space corresponding to the current snapshot stored on disk.

[0096] The embodiment of this specification uses the Epoch service submodule 403 to regularly send Epoch numbers to each shard node at certain time intervals, and divides a time slice by two consecutive Epoch numbers, which is called an Epoch time slice. The Epoch service submodule 403 can be a single server, a server cluster, or any one of the shard nodes. Before this Epoch time slice, all operations in the previous Epoch time slice must be completed. If not completed, asynchronous non-blocking processing is performed. In fact, as long as the cross-shard transaction at the head of the previous Epoch time slice is completed, there is no conflict in the subsequent local transaction that needs to be rolled back. At this time, it is only necessary to submit the batch processing of the previous Epoch time slice to the disk, which will not affect the transaction processing of the next Epoch time slice. Because each Epoch time slice parallelizes the serial transaction sequence (by eliminating conflict operations) and integrates it into a batch processing.

[0097] For example, FIG11 shows a process for executing a transaction on a shard node within an Epoch time slice, which may include the following steps:

[0098] Step 1: Enter Epoch i: Receive Epoch number i and a cross-shard transaction set, and enter the new Epoch time slice.

[0099] Step 2: Generate a snapshot: After entering a new Epoch time slice, a new snapshot will be generated after the transactions included in the previous Epoch time slice are executed, and a new data element read and write record table will be established at the same time.

[0100] Step 3: Execute cross-shard transactions: After the snapshot is created, execute cross-shard transactions within the current Epoch time slice in the memory space corresponding to the current snapshot, and then execute intra-shard transactions while complying with the deterministic transaction rules of the shard in the current Epoch time slice.

[0101] Step 4: Execute intra-shard transactions: After completing the execution of cross-shard transactions, intra-shard transactions will be executed according to the intra-shard transaction processing mechanism.

[0102] Step 5: End the snapshot: Before entering the next Epoch time slice, complete all transactions, commit the processing results in the current snapshot, and then end the current snapshot version.

[0103] Step 6: Enter Epoch i+1: Complete all tasks in the current Epoch time slice, persist the data, enter the next Epoch time slice, and repeat Step 1.

[0104] After the transactions are sorted by the Epoch mechanism, a triple is used to represent their record order. As shown in Table 1:

[0105] Table 1

[0106] Epoch_id indicates the Epoch time slice number to which the transaction belongs when it is submitted, which is used to determine the temporal order; Shard_id indicates the shard node number to which the transaction belongs, which is used to determine the spatial order; Tx_id indicates the arrival order of the transaction within the Epoch time slice in the current shard node, which is used to distinguish the execution order of conflicting transactions.

[0107] In the embodiment of this specification, each shard node can be composed of a server node, or a cluster composed of multiple server nodes through a consistency maintenance module. The transaction consensus within the shard can also refer to the consistency maintenance submodule 401, and the specific algorithm used can be changed according to the application scenario, such as using the Raft consensus algorithm in the crash fault tolerance scenario, and using the BFT consensus algorithm in the safety fault tolerance scenario. In the embodiment of this specification, the transaction processing within the shard is also implemented with reference to the deterministic processing submodule 402. The processed transaction sequence can be executed in parallel within a single Epoch time slice to complete the transaction processing within the shard. The shard transaction processing process within the shard node can be shown in Figure 12, including the following steps:

[0108] Step 1: Generate a snapshot: After entering a new Epoch time slice, a new snapshot will be generated after the transactions included in the previous Epoch time slice are executed, and a new data element read and write record table will be established at the same time to facilitate the execution of cross-shard transactions and local transactions in the memory space corresponding to the generated new snapshot.

[0109] The data element reading and writing record table in this step mainly records the data element reading and writing records of the shard node within the Epoch time slice, and is not shared with the data element reading and writing record table in the deterministic processing sub-module 402.

[0110] Step 2: Deterministic transaction processing: After the shard node receives the cross-shard transaction sequence from the cross-shard service module, it generates a transaction read-write set based on the cross-shard transaction sequence. Then, it performs deterministic processing based on the transaction read-write set and the local transaction in sequence, and generates a transaction sequence within the current Epoch time slice according to the deterministic transaction rules.

[0111] Step 3: Execute the transaction: Complete the transaction processing within the shard and submit the transaction to the data storage module for subsequent operations.

[0112] It should be noted that the cross-shard transactions and local transactions are processed within the generated snapshot, and the processing results are stored in the memory space corresponding to the current snapshot. Before receiving the next Epoch number, the cross-shard transactions and local transactions for the Epoch time slice corresponding to this Epoch number are processed, and the snapshot is terminated. The processing results in the memory space corresponding to the snapshot are stored on disk (that is, the transactions are submitted to the data storage module). Repetitions are not repeated here.

[0113] Based on the same inventive concept, an embodiment of this specification further provides a cross-shard transaction processing system, the system comprising a plurality of shard nodes and a cross-shard service module;

[0114] The sharding node and the cross-sharding service module execute the above method when processing the transaction sent by the client.

[0115] Since the principle of solving the problem by the above system is similar to that of the above method, the implementation of the above device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0116] FIG13 is a schematic diagram of the structure of a computer device according to an embodiment of the present disclosure. The system according to an embodiment of the present disclosure may be a computer device according to the embodiment, executing the method of the present disclosure described above. Computer device 1302 may include one or more processing devices 1304, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. Computer device 1302 may also include any storage resources 1306 for storing any type of information, such as code, settings, data, and the like. For example, and without limitation, storage resources 1306 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory, hard disks, optical disks, and the like. More generally, any storage resource may use any technology to store information. Furthermore, any storage resource may provide volatile or non-volatile retention of information. Furthermore, any storage resource may represent a fixed or removable component of computer device 1302. In one embodiment, when processing device 1304 executes associated instructions stored in any storage resource or combination of storage resources, computer device 1302 may perform any operation of the associated instructions. The computer device 1302 also includes one or more drive mechanisms 1308 for interacting with any storage resources, such as a hard disk drive mechanism, an optical disk drive mechanism, and the like.

[0117] The computer device 1302 may also include an input / output module 1310 (I / O) for receiving various inputs (via input devices 1312) and for providing various outputs (via output devices 1314). A specific output mechanism may include a presentation device 1316 and an associated graphical user interface (GUI) 1318. In other embodiments, the input / output module 1310 (I / O), input devices 1312, and output devices 1314 may not be included, and the computer device 1302 may simply be a computer device in a network. The computer device 1302 may also include one or more network interfaces 1320 for exchanging data with other devices via one or more communication links 1322. One or more communication buses 1324 couple the components described above together.

[0118] The communication link 1322 can be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1322 can include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0119] The embodiments of this specification also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0120] The embodiments of this specification also provide a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to execute the above method.

[0121] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0122] Specific embodiments are used in this application to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A cross-shard transaction processing method, characterized in that, The method includes: After each shard node receives the current Epoch number sent by the cross-shard service module and the cross-shard transaction of its own local transaction type corresponding to the current Epoch number, it processes the cross-shard transaction. Among them, the cross-shard transaction is the first transaction that does not belong to its own local transaction type and is received by the cross-shard service module from other shard nodes sent by the client between the previous Epoch number and the current Epoch number. At the same time, each shard node respectively determines whether the second transaction sent by the client it receives belongs to its own local transaction type. If it belongs to its own local transaction type, the second transaction is regarded as a local transaction and the local transaction is processed. If it does not belong to its own local transaction type, the second transaction is sent to the cross-shard service module as a non-local transaction.

2. The method according to claim 1, characterized in that, The method further includes: After the cross-shard service module receives the non-local transactions sent by each shard node, it performs a first deterministic transaction process on the non-local transactions to obtain a first deterministic transaction sequence, and when sending the next Epoch number to each shard node, it sends the cross-shard transactions in the first deterministic transaction sequence to the corresponding shard nodes for processing.

3. The method according to claim 2, characterized in that, The cross-shard service module performing a first deterministic transaction process on the non-local transactions to obtain a first deterministic transaction sequence further includes: Performing read-write conflict detection on the non-local transactions sent by each shard node; Aborting the non-local transactions with read-write conflicts, and forming the first deterministic transaction sequence with the non-local transactions without read-write conflicts.

4. The method according to claim 3, characterized in that, Performing read-write conflict detection on the non-local transactions sent by each shard node further includes: Regarding the non-local transactions sent by each shard node as pending cross-shard transactions; Determining the data elements corresponding to the pending cross-shard transactions and the read-write types of the data elements; Performing read-write conflict detection on the read-write types of the same data elements according to the reception order of the pending cross-shard transactions. If there are read-write conflicts in at least one data element of the pending cross-shard transaction, the pending cross-shard transaction has a read-write conflict.

5. The method according to claim 4, characterized in that, After forming the first deterministic transaction sequence with the non-local transactions without read-write conflicts, the method further includes: Recording the reception order of each pending cross-shard transaction in the first deterministic transaction sequence, so that the shard nodes process the pending cross-shard transactions according to the reception order of the pending cross-shard transactions.

6. The method according to claim 2, characterized in that, The method further includes: Each shard node performs consensus verification on the processing transaction scope of its own shard, so that the cross-shard service module determines the shard nodes corresponding to the cross-shard transactions in the first deterministic transaction sequence according to the result of the consensus verification, and sends the cross-shard transactions to the corresponding shard nodes for processing.

7. The method according to claim 1, characterized in that, The method for the shard node to process the cross-shard transaction and the local transaction further includes: Processing the cross-shard transaction and the local transaction according to the respective priorities of the cross-shard transaction and the local transaction.

8. The method according to claim 1, characterized in that, The method for the shard node to process the cross-shard transaction and the local transaction further includes: The shard node performs second deterministic transaction processing on the cross-shard transaction and the local transaction to obtain a second deterministic transaction sequence, and processes the transactions in the second deterministic transaction sequence.

9. The method according to claim 1, characterized in that, Before the shard node processes the cross-shard transaction, the method further includes: After the transactions before the current Epoch number are processed, a current snapshot is generated to facilitate processing the cross-shard transaction and the local transaction within the current snapshot, and the processing results are stored in the memory space corresponding to the current snapshot; Before receiving the next Epoch number, the cross-shard transaction and the local transaction are processed, the current snapshot is ended, and the processing results in the memory space corresponding to the current snapshot are stored on the disk.

10. A cross-shard transaction processing system, characterized in that, The system includes multiple shard nodes and a cross-shard service module; When the shard node and the cross-shard service module process the transaction sent by the client, they execute the method according to any one of claims 1-9.

Images