Transaction management method and related system

Abstract

The invention provides a transaction management method, which comprises the following steps that: a transaction management node receives a transaction request, the transaction request is used for requesting to execute a distributed transaction, the transaction management node segments the distributed transaction according to dependency information and conflict information between operations of the distributed transaction to obtain a plurality of independent transaction slices, and the transaction slices are used for executing the distributed transaction; the transaction management node distributes time sequence information for the multiple transaction slices and distributes the multiple transaction slices and the time sequence information corresponding to the multiple transaction slices to the multiple storage nodes, and the transaction slices are sequentially executed at the storage nodes according to the time sequence information. In the method, a transaction management node divides a transaction into independent transaction slices and then allocates time sequence information to the transaction slices, so that the execution sequence of the transaction slices in a data storage layer is ensured, and frequent communication between a transaction initiating / processing node and a storage node is not needed; the problem that a large amount of communication overhead exists between the transaction initiating / processing node and the storage node is solved, and the transaction processing efficiency is improved through concurrent execution.

Description

Technical Field

[0001] The present application relates to the field of transaction technology, and in particular to a transaction management method, a transaction management node, a transaction management system, a computing device cluster, a computer-readable storage medium, and a computer program product. Background Art

[0002] A transaction is a finite sequence of operations. A distributed transaction is a set of operations involving data in two or more data repositories (especially databases). A distributed transaction system is a system designed to ensure the correct and efficient operation of distributed transactions. Its main function is to coordinate and manage transactions between data repositories deployed on different nodes.

[0003] There are two possible results of executing a transaction: all operations are completed successfully, or some operations cannot be executed normally due to some reasons. In the first case, the transaction will be committed, so that transactions executed after the transaction can see the data results and impacts after the transaction is executed; in the second case, if some operations have been successfully executed before the failed operation, then these operations will be rolled back, so that the data state will be rolled back to the state before these operations were executed, and the transaction will be abandoned. At present, the industry mainly uses two-phase commit (2PC) to ensure the integrity of distributed transactions.

[0004] In the distributed transaction solution based on 2PC design, there is a large amount of communication overhead between the transaction initiating / processing nodes and the storage nodes. As the scale and complexity of transactions increase, the communication overhead will also increase sharply, and the transaction processing efficiency is low. Summary of the invention

[0005] The present application provides a transaction management method, which divides distributed transactions into independent transaction slices, and then assigns timing information to the transaction slices to ensure the execution order of the transaction slices in the data storage layer, without the need for frequent communication between the transaction initiating / processing node and the storage node, solves the problem of a large amount of communication overhead between the transaction initiating / processing node and the storage node, and improves the transaction processing efficiency through concurrent execution. The present application also provides a transaction management node, a transaction management system, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0006] In a first aspect, the present application provides a transaction management method. The method can be executed by a transaction management node of a transaction management layer, wherein the transaction management node can be a transaction manager (TM) or a server on which the TM is deployed. The TM is a coordinator role used to coordinate the behavior of all resource managers (RM) associated with cross-library transactions. The RM is used to provide operation and management interfaces for data resources to ensure data consistency and integrity. Most storage systems or storage nodes deployed with storage systems can be regarded as an RM.

[0007] Specifically, the transaction management node receives a transaction request, which is used to request the execution of a distributed transaction. Then, the transaction management node splits the distributed transaction according to the dependency information and conflict information between the operations of the distributed transaction to obtain multiple independent transaction slices (several transaction segments obtained after the distributed transaction is processed through the transaction splitting process). The transaction management node assigns timing information to the multiple transaction slices, and distributes the multiple transaction slices and the timing information corresponding to the multiple transaction slices to multiple storage nodes. Accordingly, the transaction slices are executed on the storage nodes in the order of the timing information.

[0008] In this method, the transaction management node divides the transaction into independent transaction slices, and then assigns time sequence information to the transaction slices to sequence the transaction slices, thereby ensuring the execution order of the transaction slices in the data storage layer. There is no need for the transaction initiating / processing node (such as the above-mentioned transaction management node) to communicate frequently with the storage node, which solves the problem of large communication overhead between the transaction initiating / processing node and the storage node, and improves the transaction processing efficiency through concurrent execution.

[0009] In some possible implementations, the timing information includes a number representing the timing or a timestamp applied to a clock timing node connected to the transaction management node. Among them, by sequencing the transaction slices by the number representing the timing, resource overhead can be reduced and implementation is easy; by applying a timestamp to the clock timing node connected to the transaction management node to sequence the transaction slices, accuracy can be improved.

[0010] In some possible implementations, the clock calibration node may perform clock synchronization with other clock calibration nodes by running a precise time protocol (PTP) to obtain a timestamp.

[0011] Among them, the accuracy of the timestamp obtained by the clock synchronization node through PTP synchronization can reach the sub-microsecond level, which further improves the accuracy of the timestamp compared with the Network Time Protocol (NTP) clock synchronization (usually, the accuracy is at the microsecond level) improved based on the unilateral Remote Direct Memory Access (RDMA). The time error between the transaction management node and the clock synchronization node in the transaction management cluster can be maintained within a small range. Assuming the time error is [L, U], UL can be at the sub-microsecond level. Even if different transaction management nodes need to be separated by a certain time (for example, T wait =(1+ε)(UL), where ε refers to the error ratio between the local clock and the calibration clock), a distributed transaction can be initiated and processed to reduce the probability of conflict between distributed transactions initiated by different nodes. Since UL is at the sub-microsecond level, the time interval can be greatly shortened, thereby reducing the limitations on transaction processing efficiency and improving transaction processing efficiency.

[0012] In some possible implementations, the clock synchronization node includes a data processing unit DPU, a network card or a switch. Taking the clock synchronization node as an example of DPU, each DPU can be connected to multiple transaction management nodes, and clock synchronization is performed through a small number of DPUs (for example, 3 DPUs). The DPU can be used as the master clock of multiple transaction management nodes, and the transaction management node can apply to the DPU for a high-precision timestamp as a local business timestamp to complete various business designs centered on high-precision timestamps. This method reduces the large demands and restrictions on network performance and cluster scale in the traditional PTP protocol, and is conducive to the implementation of high-precision clock synchronization solutions in large-scale cluster scenarios.

[0013] In some possible implementations, the transaction management node may send multiple transaction slices and timing information corresponding to the multiple transaction slices to a clock synchronization node to which the transaction management node is connected, instructing the clock synchronization node to aggregate transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node according to the destination address.

[0014] In this way, the clock calibration node can serve as the exit of the transaction management cluster to aggregate transaction slices and reduce the computing pressure of the storage layer.

[0015] In some possible implementations, transaction slices with the same destination address in multiple transaction slices can be aggregated after being sorted according to timing information. This can ensure consistency on the one hand, and further reduce the computing pressure of the storage layer on the other hand.

[0016] In some possible implementations, the transaction management node can obtain multiple transaction operation sequences based on the dependency information and conflict information between the operations of the distributed transaction, where the multiple transaction operation sequences are independent of each other, and the transactions or operations within the transaction operation sequences have dependency or conflict relationships. The operations are then merged according to the transaction operation sequences to obtain multiple transaction slices.

[0017] In this way, distributed transactions can be divided into multiple independent transaction slices, and then the concurrent execution of transaction slices can be achieved, thus improving the processing efficiency of distributed transactions.

[0018] In some possible implementations, the distributed transaction is a first distributed transaction, and the transaction management node can obtain multiple operation sequences based on the dependency information between operations of the first distributed transaction, and the multiple operation sequences are independent of each other. That is, there is no dependency relationship between the operation sequences. When the first operation sequence and the second operation sequence in the multiple operation sequences conflict with the second distributed transaction, the transaction management node merges the first operation sequence, the second operation sequence, and the second distributed transaction to obtain a transaction operation sequence.

[0019] In this way, accurate segmentation of distributed transactions can be achieved, and the number of slices can be reduced as much as possible while ensuring the independence of multiple transaction slices, thereby reducing communication overhead.

[0020] In some possible implementations, the transaction management node can perform transaction segmentation on the distributed transaction in a graph manner. Specifically, the transaction management node can construct the dependency edge of the transaction operation relationship graph based on the dependency information between the operations of the first distributed transaction, and the operations represented by the vertices connected by the dependency edge belong to the same operation sequence. According to the conflict information between the first distributed transaction and the second distributed transaction, the conflict edge of the transaction operation relationship graph is constructed, and then the transaction management node can determine multiple connected components in the transaction operation relationship graph, and the connected components include the connected graph formed by the dependency edge or the conflict edge, and each of the multiple connected components represents a transaction operation sequence. Accordingly, the transaction management node can merge the operation vertices other than the transaction vertices in the connected components to obtain transaction slices.

[0021] This method can improve the processing efficiency of distributed transactions by constructing a transaction operation relationship graph and efficiently segmenting distributed transactions through graph processing.

[0022] In a second aspect, the present application provides a transaction management node. The node includes:

[0023] A communication module, used for receiving a transaction request, wherein the transaction request is used for requesting execution of a distributed transaction;

[0024] A transaction splitting module, used to split the distributed transaction according to dependency information and conflict information between operations of the distributed transaction to obtain multiple transaction slices, wherein the multiple transaction slices are independent of each other;

[0025] A sequencing module, configured to assign timing information to the plurality of transaction slices;

[0026] The communication module is also used to distribute the multiple transaction slices and timing information corresponding to the multiple transaction slices to multiple storage nodes, and the transaction slices are executed in the storage nodes in the order of the timing information.

[0027] In some possible implementations, the timing information includes a number representing the timing or a timestamp requested from a clock calibration node connected to the transaction management node.

[0028] In some possible implementations, the clock calibration node includes a data processing unit DPU, a network card, or a switch.

[0029] In some possible implementations, the communication module is specifically used to:

[0030] Send the multiple transaction slices and the timing information corresponding to the multiple transaction slices to the clock calibration node connected to the transaction management node, instruct the clock calibration node to aggregate the transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node according to the destination address.

[0031] In some possible implementations, the transaction slices with the same destination address among the multiple transaction slices are sorted according to the timing information and then aggregated.

[0032] In some possible implementations, the transaction segmentation module is specifically used to:

[0033] According to the dependency information and conflict information between the operations of the distributed transaction, a plurality of transaction operation sequences are obtained, wherein the plurality of transaction operation sequences are independent of each other, and transactions or operations within the transaction operation sequences have dependency relationships or conflict relationships;

[0034] The operations are merged according to the transaction operation sequence to obtain multiple transaction slices.

[0035] In some possible implementations, the transaction segmentation module is specifically used to:

[0036] Obtaining multiple operation sequences according to dependency information between operations of the first distributed transaction, where the multiple operation sequences are independent of each other;

[0037] When a first operation sequence and a second operation sequence among the multiple operation sequences conflict with the second distributed transaction, the first operation sequence, the second operation sequence, and the second distributed transaction are merged to obtain a transaction operation sequence.

[0038] In some possible implementations, the transaction segmentation module is further used to:

[0039] Constructing a dependency edge of a transaction operation relationship graph according to dependency information between operations of the first distributed transaction, wherein operations represented by vertices connected by the dependency edge belong to the same operation sequence, and constructing a conflict edge of the transaction operation relationship graph according to conflict information between the first distributed transaction and the second distributed transaction;

[0040] Determine a plurality of connected components in the transaction operation relationship graph, wherein the connected components include a connected graph formed by the dependent edges or the conflicting edges, and each of the plurality of connected components represents a transaction operation sequence;

[0041] The transaction segmentation module is specifically used for:

[0042] The operation vertices other than the transaction vertices in the connected components are merged to obtain a transaction slice.

[0043] In a third aspect, the present application provides a transaction management system. The transaction management system includes a clock time calibration node and multiple transaction management nodes, the multiple transaction management nodes are respectively connected to the clock time calibration node, at least one of the multiple transaction management nodes includes a processor and a memory, the processor executes computer-readable instructions, so that the transaction management node executes the transaction management method described in the first aspect or any implementation of the first aspect.

[0044] In some possible implementations, the transaction management system may further include a storage node, for example, a plurality of storage nodes. Accordingly, when the storage node receives a transaction slice, it executes the transaction slice in the order of the timing information corresponding to the transaction slice.

[0045] In a fourth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory, so that the computing device or the computing device cluster executes the transaction management method as described in the first aspect or any implementation of the first aspect.

[0046] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and the instructions instruct a computing device or a computing device cluster to execute the transaction management method described in the first aspect or any one of the implementations of the first aspect.

[0047] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device or a computing device cluster, enables the computing device or the computing device cluster to execute the transaction management method described in the first aspect or any one of the implementations of the first aspect.

[0048] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0050] Figure 1 A schematic diagram of the architecture of a transaction management system provided in an embodiment of the present application;

[0051] Figure 2 An interactive flow chart of a transaction management method provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of a transaction segmentation process provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a process for batch transaction segmentation provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of adding a timestamp to a transaction slice for encapsulation provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of clock synchronization between clock calibration nodes provided in an embodiment of the present application;

[0056] Figure 7 A flowchart of a method for determining a timestamp based on a delayed response mechanism provided in an embodiment of the present application;

[0057] Figure 8 A schematic diagram of a process of executing transaction slicing by a storage node provided in an embodiment of the present application;

[0058] Fig. 9 A schematic diagram of a process for collaboratively completing distributed transaction execution by a transaction management layer and a data storage layer provided in an embodiment of the present application;

[0059] Fig.10 A schematic diagram of the structure of a transaction management node provided in an embodiment of the present application;

[0060] Fig.11 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0061] Fig.12 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0063] First, some technical terms involved in the embodiments of the present application are introduced.

[0064] A distributed transaction is a set of operations involving data in two or more data repositories (especially databases). A distributed transaction system is a system designed to ensure the correct and efficient operation of distributed transactions. Its main function is to coordinate and manage transactions between data repositories deployed on different nodes.

[0065] Two-phase commit (2PC) refers to a transaction mode designed based on the distributed transaction specification (XASpecification). Distributed transactions are committed in two stages to ensure the integrity of distributed transactions. XA defines a distributed transaction processing paradigm, which includes four core module designs, as shown below:

[0066] Resource Managers (RM) are used to provide data resource operations and management interfaces to ensure data consistency and integrity. Most storage systems can be considered as an RM.

[0067] Transaction Managers (TM) are coordinators that coordinate the behavior of all RMs associated with cross-database transactions.

[0068] Application Program (AP) is used to call RM interface according to business rules to complete the change of business model data. When the data change involves multiple RMs and the consistency and integrity must be guaranteed through transactions, AP can define the transaction boundary through TM, which is responsible for coordinating the RMs involved in the transaction to complete a global distributed transaction.

[0069] Communication Resource Managers (CRMs) are mainly used to propagate transactions across services.

[0070] The distributed transaction in the XA specification is built on the basis of the RM local transaction (the local transaction is also called the branch transaction), and the TM is responsible for coordinating the successful submission or complete rollback of these branch transactions.

[0071] The XA specification divides the distributed transaction processing process into the preparation phase and the commit / rollback phase.

[0072] The preparation phase includes the following operations:

[0073] 1.TM records the transaction start log and asks each RM whether it can perform the commit preparation operation.

[0074] 2. After receiving the instruction, RM evaluates its own status and attempts to perform the preparatory operations of the local transaction. The preparatory operations include but are not limited to reserving resources, locking resources, and executing operations. However, after the attempt is successful, RM does not commit the transaction, but waits for subsequent instructions from TM. If the attempt fails, RM informs TM that the execution of this stage has failed and rolls back the operation, and then no longer participates in this transaction.

[0075] 3. TM collects RM’s response and records the transaction preparation completion log.

[0076] The commit / rollback phase includes the following operations:

[0077] If all RMs return a successful execution in the previous stage (i.e., the preparation stage), perform the following operations:

[0078] 1. TM records the transaction commit log and initiates a transaction commit instruction to all RMs.

[0079] 2. After receiving the transaction commit instruction, RM commits the transaction, releases resources, and sends a commit completion response to TM.

[0080] 3. If TM receives the submission completion response from all RMs, it records the transaction end log.

[0081] If the RM returns an execution failure or a timeout without a response in the previous phase (preparation phase), the TM will handle the execution failure as follows:

[0082] 1. Record the transaction abandonment (Abort) log and send transaction rollback instructions to all RMs.

[0083] 2. After receiving the transaction rollback instruction, RM rolls back the transaction, releases resources, and sends a rollback completion response to TM.

[0084] 3. If TM receives rollback completion responses from all RMs, it records the transaction end log.

[0085] Since XA / 2PC is a distributed transaction model implemented at the resource management level, it has a low degree of intrusion into the business, so most mainstream databases support the XA specification or are compatible with the XA specification.

[0086] However, in the distributed transaction solution based on 2PC design, there is a lot of communication overhead between the transaction initiating / processing nodes and the storage nodes. As the scale and complexity of transactions increase, the communication overhead will also increase sharply, and the transaction processing efficiency is low.

[0087] In view of this, the present application provides a transaction management method. The method can be executed by a transaction management node (for example, a TM or a server deploying a TM) of a transaction management layer. Specifically, the transaction management node receives a transaction request, which is used to request the execution of a distributed transaction, and then the transaction management node divides the distributed transaction according to the dependency information and conflict information between the operations of the distributed transaction to obtain multiple independent transaction slices (several transaction segments obtained after the distributed transaction is processed by the transaction segmentation process). The transaction management node assigns timing information to multiple transaction slices (for example, a number representing the timing or a timestamp applied to a clock calibration node connected to the transaction management node), distributes multiple transaction slices and the timing information corresponding to the multiple transaction slices to multiple storage nodes, and accordingly, the transaction slices are executed in the storage nodes in the order of the timing information.

[0088] In this method, the transaction management node divides the transaction into independent transaction slices, and then assigns time sequence information to the transaction slices to ensure the execution order of the transaction slices in the data storage layer. There is no need for the transaction initiating / processing node (such as the above-mentioned transaction management node) to communicate frequently with the storage node, which solves the problem of a large amount of communication overhead between the transaction initiating / processing node and the storage node, and improves the transaction processing efficiency through concurrent execution. Furthermore, the transaction slices of different transactions can be aggregated according to the destination address and distributed to the storage node, further reducing the communication overhead. The transaction management node does not need to set a time interval to initiate distributed transactions to reduce the probability of conflicts between distributed transactions initiated by different nodes, which has less restrictions on transaction processing efficiency and better scalability.

[0089] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0090] See also Figure 1The schematic diagram of the architecture of a transaction management system shown in the figure, the transaction management system 10 can be logically divided into a transaction management layer and a data storage layer, the transaction management layer includes a transaction management node 100, and the data storage layer 200 includes a storage node 200. Among them, the transaction management node 100 and the storage node 200 can be hardware, such as a server, including but not limited to a cloud server and a physical server. The server executes a software program to implement functions such as transaction management and data storage. The transaction management node 100 or the storage node 200 can also be software, for example, the transaction management node 100 can be software or a service with a transaction management function, and the storage node 200 can be software or a service with a data storage function. For ease of description, this application takes the transaction management node 100 and the storage node 200 as hardware examples.

[0091] The transaction management node 100 is used to receive a transaction request, which is used to request the execution of a distributed transaction. Among them, the transaction request may be a transaction request from an application APP. The transaction management node 100 is also used to divide the distributed transaction into multiple independent transaction slices according to the dependency information and conflict information between the operations of the distributed transaction. Among them, multiple independent transaction slices may be transaction slices without dependencies. The transaction management node 100 is also used to assign timing information to multiple transaction slices. In some examples, the timing information may be a number representing the timing, and the smaller the number, the earlier the corresponding transaction slice is executed. In other examples, the timing information may also be a timestamp applied to the clock timing node 300 (for ease of description, this application is also referred to as the first clock timing node) connected to the transaction management node 100. The transaction management node 100 can distribute multiple transaction slices and timing information corresponding to multiple transaction slices to multiple storage nodes 200. The above-mentioned transaction slices are executed in the storage node 200 in the order of timing information.

[0092] The storage node 200 is used to receive transaction slices and execute the transaction slices in order according to the timing information of the transaction slices. After the transaction slice is executed, the storage node 200 is also used to send a notification to inform the transaction slice that the execution is completed / abandoned. This embodiment uses the example of the transaction management system 10 including the storage node 200. In actual application, the transaction management system 10 may also not include the above-mentioned storage node 200. For example, the transaction management system 10 may include a transaction management node 100 and a clock calibration node 300. The transaction management system 10 can be connected to a third-party storage node to achieve transaction management of distributed transactions.

[0093] The transaction management node 100 and the clock timing node 300 can be connected via a Compute Express Link (CXL) or a Unified Bus (UB) to shorten the latency. The clock timing node 300 is a node used for clock timing and can provide a high-precision timestamp, for example, returning a high-precision timestamp to the transaction management node 100. The clock timing node can be a network card, a switch, or a data processing unit (DPU). Figure 1 As shown, the clock timing nodes 300 (e.g., the first clock timing node and the second clock timing node) can run the precise time protocol (PTP) for clock synchronization to obtain a timestamp. The timestamp is a high-precision timestamp. The transaction management node 100 can receive the timestamp obtained by synchronizing the first clock timing node with the second clock timing node based on PTP, thereby allocating a timestamp to the transaction slice.

[0094] Furthermore, the clock calibration node 300 of the transaction management layer can serve as an external network exit node of the cluster of the transaction management layer (also referred to as the transaction management cluster), and aggregate the transaction slices sent by the distributed transaction management node 100 connected to the clock calibration node 300 to reduce the computing load pressure of the storage node 200 of the data storage layer. Among them, the aggregation of transaction slices can be the input output (IO) aggregation of transaction slices. For example, if a transaction slice includes multiple write IOs, and the logical addresses of the multiple write IOs are continuous or the interval between the logical addresses is less than the set length, the multiple write IOs can be aggregated into one write IO. In some possible implementations, the clock calibration node 300 can aggregate the transaction slices after sorting them by timestamp to ensure the execution order of the transaction slices.

[0095] In some possible implementations, the data storage layer may also include a metadata management node 400. The metadata management node 400 may be a data processing unit DPU. The DPU is responsible for metadata management of the data storage layer nodes, and is also responsible for distributing data to each storage node. Specifically, after receiving the transaction slices of the distributed transaction, the storage node 200 of the data storage layer may first perform sorting, such as performing a lightweight sorting operation, and execute the transaction slices in sequence after the sorting is completed; after the transaction slice is executed, the storage node 200 sends a notification to the metadata management node 400 of the data storage layer to inform the transaction slice execution completion / abandonment, and the metadata management node 400 (such as DPU) can send a notification to the transaction management node 100 of the transaction management layer based on the notifications collected from each storage node 200 to inform the transaction submission / abandonment.

[0096] Furthermore, the DPU deployed in the transaction management layer and the DPU deployed in the data storage layer can be connected via a Remote Direct Memory Access (RDMA) link, providing distributed transaction processing support through a high-speed network link.

[0097] It should be noted that the distributed transaction segmentation and improved two-phase commit method provided by the transaction management system 10 of the embodiment of the present application can be connected to the distributed database in the form of a functional module or cloud service as a related solution for distributed transactions. For example, in scenarios such as financial big data, the above-mentioned distributed transaction segmentation and improved two-phase commit method can be connected as a transaction improvement solution design module to provide better database performance without changing the semantics of the upper and lower data flows of the database.

[0098] Furthermore, in the embodiment of the present application, the high-precision clock synchronization method assisted by DPU (or other types of clock timing nodes) can be used as an architectural deployment solution designed around the high-precision clock function that can be provided by the new device DPU. Based on this, the embodiment of the present application also provides a cluster management solution design around DPU. In this solution design, DPU can be responsible for managing and maintaining several server nodes connected to it (the server node can be a transaction management node). For example, a single clock timing node (such as DPU) with low operation and maintenance deployment overhead can be interconnected with multiple transaction management nodes to form a cluster of transaction management layers (i.e., transaction management cluster). After the customer powers on the clock timing nodes and transaction management nodes in the cluster, he can directly use the cluster function and save the cluster construction overhead such as secret-free authentication between cluster nodes. At the same time, clock timing nodes such as DPU can also be used as cluster management nodes. Users can access DPU to carry out subsequent operation and maintenance work such as rapid flow control management on the cluster.

[0099] Based on the aforementioned transaction management system 10, the present application provides a transaction management method. The transaction management method of the present application is introduced below in conjunction with an embodiment.

[0100] See also Figure 2 A flowchart of a transaction management method is shown, the method comprising the following steps:

[0101] S202: The transaction management node 100 receives a transaction request.

[0102] The transaction request is used to request the execution of a distributed transaction. The transaction request may be a transaction request generated by an APP, for example, the APP may generate a transaction request in response to a user operation. Taking the APP as an example of mobile banking, a user may trigger a transfer operation through mobile banking, and the mobile banking may generate a transfer transaction, which includes a deduction operation on the balance of account A and an increase operation on the balance of account B, where the deduction amount is equal to the increase amount.

[0103] In specific implementation, the transaction management node 100 can receive transaction requests generated by different user trigger operations, or receive transaction requests generated by user trigger operations at different times. Further, the transaction management node 100 can also be connected to different applications to receive transaction requests generated by different applications.

[0104] S204. The transaction management node 100 divides the distributed transaction into multiple independent transaction slices according to the dependency information and conflict information between the operations of the distributed transaction.

[0105] Distributed transactions are usually sequences of operations with a temporal relationship. The dependency information between operations in an operation sequence may be the order dependency of the operations. For example, the read operation and the write operation on the same address have an order dependency. In specific implementation, the transaction management node 100 may obtain multiple transaction operation sequences based on the dependency information and conflict information between the operations of the distributed transaction. Among them, the transaction operation sequence includes at least one operation, and further, the transaction operation sequence may also include at least one transaction. For example, for the distributed transaction Tx1, a transaction operation sequence obtained by the transaction management node 100 may be OP1-OP3-TX2-OP4, wherein OP1, OP3, and OP4 are operations in Tx1, respectively, and Tx2 is other distributed transactions that conflict with OP3 and OP4. Multiple transaction operation sequences are independent of each other. There is a dependency or conflict relationship between the transactions or operations within the transaction operation sequence. Specifically, there may be dependencies between operations in a transaction operation sequence, and there may be conflicts between operations and transactions (other distributed transactions). For example, in the transaction operation sequence OP1-OP3-TX2-OP4, there is a dependency between OP1-OP3, and there is a conflict between OP3 and Tx2, and between OP4 and Tx2. Then, the transaction management node 100 may merge operations according to the transaction operation relationship sequence to obtain multiple transaction slices.

[0106] Among them, the distributed transaction may include a first distributed transaction. When splitting the first distributed transaction, the transaction management node 100 may obtain multiple operation sequences based on the dependency information between the operations of the first distributed transaction. The multiple operation sequences are independent of each other. In other words, operations with dependencies are divided into the same operation sequence. When the first operation sequence and the second operation sequence in the multiple operation sequences conflict with the second distributed transaction, the transaction management node 100 may merge the first operation sequence, the second operation sequence, and the second distributed transaction to obtain a transaction operation sequence.

[0107] Among them, the transaction management node 100 can perform transaction segmentation on the distributed transaction in a graph manner. Specifically, the transaction management node 100 can construct the dependency edge (denoted as S-Edge, also called S edge) of the transaction operation relationship graph according to the dependency information between the operations of the first distributed transaction, and the operations represented by the vertices connected by the dependency edge belong to the same operation sequence. According to the conflict information between the first distributed transaction and the second distributed transaction, the conflict edge (denoted as C-Edge, also called C edge) of the transaction operation relationship graph is constructed, and then the transaction management node 100 can determine multiple connected components in the transaction operation relationship graph. The connected components include a connected graph formed by the dependency edge or the conflict edge, and each connected component represents a transaction operation sequence. Accordingly, the transaction management node 100 can merge the operation vertices other than the transaction vertices in the connected components (such as merge) to obtain transaction slices.

[0108] For ease of understanding, this application also provides an example for illustration. Figure 3 As shown in the figure, the transaction chopping process can be processed at the transaction level. For example, each transaction in a batch of transactions can be independently chopped. Figure 3 The figure shows a schematic diagram of transaction segmentation for five transactions (recorded as Tx1 to Tx5) of the same batch. Taking the transaction segmentation of Tx 1 as an example, the transaction management node 100 can perform rough segmentation according to the operations of the distributed transaction. Each operation after rough segmentation has a conflict relationship with the remaining four transactions, and there is a dependency relationship between the operations, thereby forming S edges and C edges. The transaction management node 100 can remove the Tx vertex and retain the operation vertex for the connected components formed by the S edge and the C edge to obtain transaction slices. In this example, the transaction management node 100 performs transaction segmentation on Tx1 and obtains the following transaction slices: Write (D), Read (A)-Write (E)-Write (A), and Write (C)-Read (B).

[0109] It should be noted that Figure 3The example of splitting a transaction in a batch of transactions is used to illustrate that in actual applications, since different transactions are executed independently of each other, each transaction in the batch can be independently executed by the splitting algorithm through a concurrent / parallel solution.

[0110] like Figure 4 As shown, the transaction management node 100 can preliminarily divide the distributed transaction to be divided into multiple operations according to the order of operations. For example, transaction Tx1 can be preliminarily divided into Read (D), Write (A), and Read (B), and transaction Tx2 can be preliminarily divided into Write (B) and Read (C). Then the transaction management node 100 can construct C-Edge according to the conflict information (such as conflict relationship) between the operations of the distributed transaction and other transactions, and construct S-Edge according to the dependency information (such as dependency relationship) between the operations within the distributed transaction. For example, the transaction management node 100 can construct an S edge between Write (A) and Read (B) of Tx1, and a C edge between Read (B) of Tx1 and Write (B) of Tx2. In this way, a transaction operation relationship graph can be formed.

[0111] The transaction operation relationship graph includes multiple connected components. For connected components with dependency conflict cycles (S edges and C edges exist in a cycle, recorded as SC-Cycle), the transaction management node 100 eliminates the SC-Cycle that appears in the graph based on the Merge transaction operation method, and finally different transactions are divided into multiple sub-operation fragments. Figure 4 As shown, the transaction management node 100 can merge Write(A) and Read(B), thereby eliminating the SC-Cycle in the transaction operation relationship diagram. Accordingly, each sub-operation fragment can be regarded as a transaction slice. For example, Read(D) forms a transaction slice P1, Write(A) and Read(B) can form a transaction slice P2, Write(B) can form a transaction slice P3, and Read(C) can form a transaction slice P4. The above P1 and P2 are multiple independent transaction slices obtained by transaction segmentation of Tx1, and the above P3 and P4 are multiple independent transaction slices obtained by transaction segmentation of Tx2. Among them, the execution order between different operation fragments can be modified to further improve the execution efficiency of the transaction.

[0112] S206 : The transaction management node 100 applies for a timestamp from the clock calibration node 300 to which the transaction management node 100 is connected.

[0113] S208 . The clock calibration node 300 synchronizes with other clock calibration nodes 300 based on PTP to obtain a timestamp.

[0114] S210 : The clock calibration node 300 sends a timestamp to the transaction management node 100 .

[0115] The clock synchronization nodes 300 can achieve sub-microsecond time synchronization accuracy by running PTP. In this way, the timestamp requested by the transaction management node 100 can achieve sub-microsecond accuracy, which further improves the accuracy of the timestamp compared to the Network Time Protocol (NTP) clock synchronization based on the unilateral RDMA improvement (usually, the accuracy is at the microsecond level). In other words, the time error between the transaction management node 100 and the clock synchronization node 300 in the transaction management cluster is maintained within a small range. In this embodiment, assuming that the time error is [L, U], UL can be at the sub-microsecond level. Even if different transaction management nodes 100 need to be separated by a certain time (for example, T wait =(1+ε)(UL), where ε refers to the error ratio between the local clock and the calibration clock), a distributed transaction can be initiated and processed to reduce the probability of conflict between distributed transactions initiated by different nodes. Since UL is at the sub-microsecond level, the time interval can be greatly shortened, thereby reducing the limitations on transaction processing efficiency and improving transaction processing efficiency.

[0116] It should be noted that the above S206 to S208 are a specific implementation method for the transaction management node 100 to obtain timing information (for example, a high-precision timestamp). In other possible implementation methods of the embodiments of the present application, the transaction management node 100 may also obtain timing information in other ways. For example, the transaction management node 100 may obtain a timestamp in other ways, or the transaction management node 100 may obtain a number representing the timing. Taking the example of obtaining a number representing the timing as an example, for the transaction management node 100, assuming that five distributed transactions are received in a time period, namely Tx1, Tx2, Tx3, Tx4, and Tx5, the transaction management node 100 can obtain 5 numbers representing the timing, namely 01, 02, 03, 04, and 05, so as to be subsequently assigned to the transaction slices of the corresponding transactions.

[0117] S212. The transaction management node 100 assigns timestamps to multiple transaction slices.

[0118] After the transaction is split, for each transaction slice (Transaction Slice), the transaction management node 100 can assign the timestamp obtained by the application (for example, a high-precision timestamp applied to the first clock calibration node) together with the transaction information to the transaction slice. Among them, the transaction information is used to indicate the transaction to which the transaction slice belongs. In some possible implementations, the transaction information can be transaction hash information, specifically a hash value obtained by performing a hash operation on the transaction operation data, also called a transaction ID (referred to as Tx_ID).

[0119] like Figure 5 As shown, the transaction management node 100 can apply for a timestamp from a clock calibration node such as a DPU, and add a timestamp and transaction information (such as Tx_ID) to the transaction slices obtained by transaction segmentation, thereby encapsulating and formatting the transaction slices. Among them, the transaction slices of different transactions can have different timestamps and transaction information. The above-mentioned encapsulated transaction slices can eventually form a set of transaction slice sequences with a time sequence relationship.

[0120] It should be noted that the transaction management node 100 may also record the correspondence between the transaction slice and the timestamp, thereby allocating timestamps to multiple transaction slices. Furthermore, the embodiment of the present application uses the timing information as an example of the timestamp. When the timing information is other information such as a number representing the timing, the transaction management node 100 may also encapsulate and format the number and the transaction slice. Alternatively, the transaction management node 100 may record the correspondence between the transaction slice and the number, thereby allocating timing information to multiple transaction slices.

[0121] S214. The transaction management node 100 distributes multiple transaction slices and timestamps corresponding to the multiple transaction slices to multiple storage nodes 200.

[0122] Specifically, the transaction management node 100 can send the encapsulated transaction slices (including transaction slices and timestamps corresponding to the transaction slices) to the storage node 200 storing the slice operation data according to the slice operation data, so that the storage node 200 executes the received transaction slices. Among them, the storage node 200 can execute the transaction slices in the order of timing information such as timestamps. Alternatively, when the transaction slices are not encapsulated and formatted in the above manner, the transaction management node 100 can distribute the transaction slices and the corresponding timestamps to the storage node 200 without encapsulating the timestamps and transaction slices before sending them.

[0123] In some possible implementations, the clock timing node 300, such as the DPU, can also serve as the external network exit node of the cluster, and aggregate the transaction slices sent by the transaction management node 100 connected to it to reduce the computing load pressure of the storage node 200. Specifically, the transaction management node 100 sends multiple transaction slices and timestamps (or other timing information, such as numbers) corresponding to the multiple transaction slices to the clock timing node 300 (such as the first clock timing node) connected to the transaction management node, instructing the first clock timing node to aggregate the transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node 200 according to the destination address. In order to reduce the computing pressure of the data storage layer, the transaction slices with the same destination address among the multiple transaction slices can be aggregated after being sorted by timestamp.

[0124] Based on the above description, the transaction management method of the present application divides distributed transactions into independent transaction slices, and then assigns timestamps to the transaction slices to ensure the execution order of the transaction slices in the data storage layer. There is no need for frequent communication between the transaction initiating / processing nodes and the storage nodes, which solves the problem of a large amount of communication overhead between the transaction initiating / processing nodes and the storage nodes, and improves transaction processing efficiency through concurrent execution. Furthermore, transaction slices of different transactions can be aggregated and distributed to storage nodes according to the destination address, further reducing communication overhead, with less restrictions on transaction processing efficiency and better scalability.

[0125] Next, the process of obtaining timestamps based on PTP in this application is described in detail. PTP is a time synchronization protocol, which is usually used for high-precision time synchronization between devices. This application proposes a high-precision clock synchronization method based on clock timing nodes (such as DPUs) assisted by PTP. This method is implemented on the basis of PTP and combined with the DPU deployment architecture, which reduces the large requirements and restrictions on network performance and cluster scale in the traditional PTP protocol, and is conducive to the implementation of high-precision clock synchronization and timing solutions in large-scale cluster scenarios.

[0126] After the transaction management node 100 applies for a timestamp from the first clock timing node to which the transaction management node 100 is connected, the first clock timing node can serve as a master clock and initiate a message receiving and sending process based on a delay request-response mechanism to determine the timestamp. Figure 6 As shown, the first clock calibration node (DPU) can be connected to the second clock calibration node (such as the DPU connected to other transaction management nodes) and synchronize time through PTP to obtain a timestamp. Then the DPU can serve as the master clock for the nodes connected to it (such as the transaction management node 100), and other nodes in the cluster can apply to the DPU for a high-precision timestamp as their local business timestamp to complete various business designs centered on high-precision timestamps. Figure 6 In the example, high-precision clock synchronization can be achieved through a small number of clock synchronization nodes (such as DPU), which improves efficiency and can meet the business needs in large-scale cluster scenarios.

[0127] Among them, the PTP messages involved in the delay response mechanism include the following: (1) Sync synchronization message (2) Follow_Up follow message (3) Delay_Req delay request message (4) Delay_Resp delay request response message.

[0128] The message sending and receiving process of the delayed response mechanism is introduced below with reference to the accompanying drawings.

[0129] See also Figure 7 The flowchart of a method for determining a timestamp based on a delayed response mechanism shown in FIG. 1 includes the following steps:

[0130] 1. The master clock periodically sends out Sync messages and records the precise sending time t1 when the Sync messages leave the master clock.

[0131] Among them, Sync is usually a message that is sent periodically, and the clock calibration node as the master clock may or may not carry the sending time information. It should be noted that if the master clock carries the sending time information, the sending time information may be an estimated sending timestamp OriginTimeStamp.

[0132] 2. The master clock encapsulates the precise sending time t1 into the Follow_up message and sends it to the slave clock.

[0133] Since the Sync message does not carry the precise message departure time, that is, the precise sending time t1 when the Sync message leaves the master clock in step 1, the precise sending time t1 of the Sync message can be encapsulated in the subsequent Follow_up message and sent to the slave clock.

[0134] 3. The slave clock records the precise arrival time t2 of the Sync message at the slave clock.

[0135] 4. The slave clock sends a Delay_req message and records the exact sending time t3.

[0136] 5. The master clock records the precise arrival time t4 of the Delay_req message reaching the master clock.

[0137] 6. The master clock sends a Delay_resp message carrying precise timestamp information t4 to the slave clock.

[0138] In this way, the slave clock can obtain four accurate message receiving and sending times including t1, t2, t3, and t4. The slave clock can obtain the master-slave clock deviation Offset and transmission delay Delay through the four accurate timestamp information including t1, t2, t3, and t4. Among them, Offset represents the deviation between clocks. There is a time deviation between the master and slave clocks. The deviation value is Offset. Figure 7 In this example, the dotted line connecting the master and slave clocks is the time consistency point of the two clocks. Therefore, the deviation of the time consistency point on the time axis is the Offset. Delay is the delay caused by the transmission of the message in the network. Among them, Offset and Delay can be determined by the following formulas respectively:

[0139]

[0140]

[0141] A high-precision timestamp can be obtained based on the local clock of the transaction management node 100 and the clock deviation such as offset. For example, the clock calibration node 300 can subtract the clock deviation offset from the timestamp generated by the local clock of the transaction management node 100 to obtain a high-precision timestamp.

[0142] It should be noted that in the case of possible breakthroughs in network communication bottlenecks in the future, such as in the unified bus (UB) / Compute Express Link (CXL) network scenario, the memory on the DPU device can be used as a shared cache for a small-scale server node cluster connected to it. The design of this shared cache can be used as a high-speed cache for cluster access to remote hot data on the one hand, and on the other hand, after the DPU intercepts the cluster's external network access IO information, some pre-processing operations can be performed based on the cache, such as encryption, decryption, or indexing, etc., to reduce the cluster's outbound and inbound traffic requirements. In this case, the DPU will not only act as a timestamp allocation node, but can also be used as a cluster management node with full functions.

[0143] The above introduces the transaction segmentation and transaction slice scheduling process of distributed transactions. The following describes the transaction execution and submission process in the data storage layer.

[0144] After assigning high-precision timestamps to transaction slices, the transaction management node 100 (a node responsible for batch processing and segmenting distributed transactions, also called a proxy in some cases) can distribute each transaction slice to the storage node 200 of the data storage layer, and the storage node 200 can execute the transaction slices according to the timestamp order.

[0145] Specifically, the transaction management node 100 can send the transaction slices to the corresponding storage node 200 according to the slice operation data, and the storage node 200 can maintain a set of cache queues to receive the sent transaction slices. Furthermore, the storage node 200 can sort the transaction slices in the cache queue. When the number of transaction slices in the cache queue of the storage node 200 reaches a set number, for example, the cache queue reaches a full queue, the storage node 200 can take out the sorted transaction slices from the cache queue and execute the operations in the transaction slices (for example, IO operations) in sequence. After the operations in the transaction slices are executed, the storage node 200 can inform the transaction management node 100 that the operation of this node is completed. When the transaction management node 100 receives relevant notifications of all transaction slices of a distributed transaction, it can commit or abandon the transaction.

[0146] For ease of understanding, this application also provides an example for illustration. Figure 8 The data storage layer includes the following storage nodes: DB1, DB2, DB3. The application sends a transaction request to the transaction management node 100, and the transaction management node 100 splits the distributed transaction requested by the transaction request into multiple transaction slices. The transaction management node 100 can apply to the DPU connected to the transaction management node 100 for a high-precision timestamp to be assigned to the above transaction slices. The specific implementation of the DPU determining the high-precision timestamp can be found in Figure 7 The relevant contents of the embodiment shown are described. Then the transaction management node 100 can send a Try message to DB1, DB2, and DB3, and the Try message carries a transaction slice with a timestamp. The data operated by the transaction slice carried by each Try message is stored in the storage node where the Try message arrives. If DB1, DB2, and DB3 all return a confirm message of successful execution, the transaction management node 100 can send a commit instruction to DB1, DB2, and DB3. Furthermore, DB1, DB2, and DB3 can also send a commit completion notification after the commit is completed.

[0147] When a storage node times out (overtime) when executing a transaction slice, or cancels (cancel) the execution of a transaction slice, for example, DB2 times out or cancels the execution of a transaction slice, the transaction management node 100 can instruct the storage nodes that have successfully executed to perform a rollback (rollback), for example, DB1 and DB3 can perform a rollback operation when DB2 times out or cancels the execution of a transaction slice. It should be noted that since different transaction slices are independent, when a storage node times out or cancels the execution of a transaction slice, the transaction slice can also be re-executed without rolling back other storage nodes that have successfully executed. When multiple transaction slices of a distributed transaction are successfully executed, the distributed transaction can be committed.

[0148] Next, the transaction management method of the present application is described from the perspective of the interaction between the transaction management layer and the data storage layer.

[0149] See also Fig. 9 The flowchart of a transaction management method shown in the figure shows that for a network request initiated by an application, after semantic analysis, the distributed transaction part of the network request can be taken out separately for subsequent processing. The processing of distributed transactions can include transaction segmentation and datagram sending at the transaction management layer, and datagram analysis and transaction slicing execution at the data storage layer.

[0150] Transaction segmentation specifically includes batch processing of distributed transactions, segmenting distributed transactions into independent transaction slices by batch, and then assigning a high-precision timestamp to each transaction slice. Figure 2 The description of the contents of S204, S206 to S210 in the embodiment shown in the figure, and the assignment of a high-precision timestamp to each transaction slice can be found in Figure 2 The description of the relevant contents of S206 to S210 in the illustrated embodiment will not be repeated here.

[0151] Datagram sending and datagram parsing are responsible for implementing the basic functions of communication and interaction between the transaction management layer and the data storage layer based on RDMA. Specifically, datagram sending includes encapsulating multiple transaction slices obtained by transaction segmentation, and then sending the encapsulated datagram (carrying transaction slices and timestamps). The datagram encapsulation method can be referred to Figure 5 Related content description. After encapsulating the datagram, the transaction management node can send the encapsulated datagram to the clock calibration node such as DPU. DPU and others can sort and aggregate the transaction slices in the above datagram to reduce the computing pressure of the storage layer. Datagram parsing includes decapsulating the datagram to obtain transaction slices. For example, when the DPU of the storage layer receives a datagram, it can identify the message header of the datagram to extract the transaction slices in the datagram.

[0152] Transaction slice execution includes performing read and write operations on the KV storage system according to the received transaction slice to read or write data (including adding or modifying data). The backend KV storage system can periodically persist the transaction commit log to the secondary storage device for persistent storage.

[0153] This method divides transactions into independent transaction slices and then assigns timestamps to the transaction slices to ensure the execution order of the transaction slices in the data storage layer. It does not require frequent communication between the transaction initiating / processing nodes and the storage nodes, which solves the problem of large communication overhead between the transaction initiating / processing nodes and the storage nodes. In addition, through concurrent execution, the transaction processing efficiency is improved. Furthermore, the transaction slices of different transactions can be aggregated according to the destination address and distributed to the storage node, which further reduces the communication overhead, has less restrictions on transaction processing efficiency, and has better scalability.

[0154] Based on the transaction management method of the above embodiment, the present application also provides a transaction management node 100. The transaction management node 100 provided by the present application will be introduced below from the perspective of functional modularization in combination with an embodiment.

[0155] See also Fig.10 A schematic diagram of the structure of a transaction management node 100 is shown, and the transaction management node 100 includes

[0156] The communication module 1002 is used to receive a transaction request, where the transaction request is used to request execution of a distributed transaction;

[0157] A transaction splitting module 1004 is used to split the distributed transaction into multiple transaction slices according to dependency information and conflict information between operations of the distributed transaction, and the multiple transaction slices are independent of each other;

[0158] A sequencing module 1006, configured to assign timing information to the plurality of transaction slices;

[0159] The communication module 1002 is also used to distribute the multiple transaction slices and timing information corresponding to the multiple transaction slices to multiple storage nodes, and the transaction slices are executed in the storage nodes in the order of the timing information.

[0160] Exemplarily, the communication module 1002, the transaction segmentation module 1004, and the sequencing module 1006 may be implemented by software.

[0161] When implemented by software, the communication module 1002, the transaction splitting module 1004, and the sequencing module 1006 can be applications running on a computing device. For example, the transaction splitting module 1004 can be a computing engine. The application can be provided to users in the form of virtualized services. Virtualized services may include virtual machine (VM) services, bare metal server (BMS) services, and container services. Among them, the VM service can be a service that virtualizes a virtual machine (VM) resource pool on multiple physical hosts (such as computing devices) through virtualization technology to provide users with VMs on demand for use. The BMS service is a service that virtualizes a BMS resource pool on multiple physical hosts to provide users with BMS on demand for use. The container service is a service that virtualizes a container resource pool on multiple physical hosts to provide users with containers on demand for use. VM is a simulated virtual computer, that is, a logical computer. BMS is a high-performance computing service that can be elastically scalable, and its computing performance is no different from that of a traditional physical machine, and it has the characteristics of secure physical isolation. Container is a kernel virtualization technology that can provide lightweight virtualization to achieve the purpose of isolating user space, processes and resources. It should be understood that the VM service, BMS service and container service in the above virtualization services are only used as specific examples. In actual applications, the virtualization service can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0162] The communication module 1002, the transaction splitting module 1004, and the sequencing module 1006 can also be implemented by hardware. For example, the communication module 802 can be a communication interface, including but not limited to a transceiver. The transaction splitting module 1004 and the sequencing module 1006 can be a processor, or a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Among them, the above-mentioned PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0163] In some possible implementations, the timing information includes a number representing the timing or a timestamp requested from a clock calibration node connected to the transaction management node.

[0164] In some possible implementations, the clock calibration node 300 includes a data processing unit DPU, a network card, or a switch.

[0165] In some possible implementations, the communication module 1002 is specifically configured to:

[0166] Send the multiple transaction slices and the timing information corresponding to the multiple transaction slices to the clock calibration node connected to the transaction management node, instruct the clock calibration node to aggregate the transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node according to the destination address.

[0167] In some possible implementations, the transaction slices with the same destination address among the multiple transaction slices are sorted according to the timing information and then aggregated.

[0168] In some possible implementations, the transaction segmentation module 1004 is specifically used to:

[0169] According to the dependency information and conflict information between the operations of the distributed transaction, a plurality of transaction operation sequences are obtained, wherein the plurality of transaction operation sequences are independent of each other, and transactions or operations within the transaction operation sequences have dependency relationships or conflict relationships;

[0170] The operations are merged according to the transaction operation sequence to obtain multiple transaction slices.

[0171] In some possible implementations, the transaction segmentation module 1004 is specifically used to:

[0172] Obtaining multiple operation sequences according to dependency information between operations of the first distributed transaction, where the multiple operation sequences are independent of each other;

[0173] When a first operation sequence and a second operation sequence among the multiple operation sequences conflict with the second distributed transaction, the first operation sequence, the second operation sequence, and the second distributed transaction are merged to obtain a transaction operation sequence.

[0174] In some possible implementations, the transaction segmentation module 1004 is further configured to:

[0175] Constructing a dependency edge of a transaction operation relationship graph according to dependency information between operations of the first distributed transaction, wherein operations represented by vertices connected by the dependency edge belong to the same operation sequence, and constructing a conflict edge of the transaction operation relationship graph according to conflict information between the first distributed transaction and the second distributed transaction;

[0176] Determine a plurality of connected components in the transaction operation relationship graph, wherein the connected components include a connected graph formed by the dependent edges or the conflicting edges, and each of the plurality of connected components represents a transaction operation sequence;

[0177] The transaction segmentation module 1004 is specifically used for:

[0178] The operation vertices other than the transaction vertices in the connected components are merged to obtain a transaction slice.

[0179] The transaction management node 100 is introduced above from the perspective of functional modularization. The transaction management node 100 is introduced below from the perspective of hardware entity.

[0180] The present application also provides a computing device 1100. The computing device 1100 is used to implement the functions of the transaction management node 100. Fig.11 As shown, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate through the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1100.

[0181] The bus 1102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (eg, the memory 1106, the processor 1104, and the communication interface 1108).

[0182] The processor 1104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0183] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The memory 1106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the aforementioned transaction management method. Specifically, the memory 1106 stores instructions for the transaction management node 100 to execute the transaction management method.

[0184] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0185] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0186] like Fig.12 As shown, the computing device cluster includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster may store the same transaction management node 100 for executing instructions of the transaction management method.

[0187] In some possible implementations, one or more computing devices 1100 in the computing device cluster may also be used to execute part of the instructions of the transaction management node 100 for executing the transaction management method. In other words, a combination of one or more computing devices 1100 may jointly execute the instructions of the transaction management node 100 for executing the transaction management method.

[0188] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster can store different instructions for executing partial functions of the transaction management node 100 .

[0189] Based on the aforementioned transaction management node 100, the present application also provides a transaction management system 10. Figure 1 As shown, the transaction management system 10 includes a clock calibration node 300 and a plurality of transaction management nodes 100. The plurality of transaction management nodes 100 are respectively connected to the clock calibration node 300, and at least one of the plurality of transaction management nodes 100 includes a processor and a memory, and the processor executes computer-readable instructions, so that the transaction management node executes the aforementioned transaction management method.

[0190] In some possible implementations, the transaction management system 10 may further include a storage node 200, for example, multiple storage nodes 200. Accordingly, when the storage node 200 receives a transaction slice, it executes the transaction slice in the order of the timing information corresponding to the transaction slice.

[0191] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned application to the transaction management node 100 for executing the transaction management method.

[0192] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the transaction management method described above.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A transaction management method, characterized in that: The method comprises: The transaction management node receives a transaction request, where the transaction request is used to request execution of a distributed transaction; The transaction management node divides the distributed transaction into multiple transaction slices according to the dependency information and conflict information between the operations of the distributed transaction, and the multiple transaction slices are independent of each other; The transaction management node assigns timing information to the multiple transaction slices, distributes the multiple transaction slices and the timing information corresponding to the multiple transaction slices to multiple storage nodes, and the transaction slices are executed at the storage nodes in the order of the timing information.

2. The method according to claim 1, characterized in that The timing information includes a number representing the timing or a timestamp requested from a clock calibration node connected to the transaction management node.

3. The method according to claim 2, characterized in that The clock timing node includes a data processing unit DPU, a network card or a switch.

4. The method according to any one of claims 1 to 3, characterized in that: The transaction management node distributes the multiple transaction slices and timing information corresponding to the multiple transaction slices to the multiple storage nodes, including: The transaction management node sends the multiple transaction slices and the timing information corresponding to the multiple transaction slices to the clock synchronization node connected to the transaction management node, instructing the clock synchronization node to aggregate the transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node according to the destination address.

5. The method according to claim 4, characterized in that The transaction slices with the same destination address among the multiple transaction slices are sorted according to the timing information and then aggregated.

6. The method according to any one of claims 1 to 5, characterized in that: The transaction management node divides the distributed transaction into multiple transaction slices according to the dependency information and conflict information between the operations of the distributed transaction, including: According to the dependency information and conflict information between the operations of the distributed transaction, a plurality of transaction operation sequences are obtained, wherein the plurality of transaction operation sequences are independent of each other, and transactions or operations within the transaction operation sequences have dependency relationships or conflict relationships; The operations are merged according to the transaction operation sequence to obtain multiple transaction slices.

7. The method according to claim 6, characterized in that The distributed transaction is a first distributed transaction, and obtaining multiple transaction operation sequences according to dependency information and conflict information between operations of the distributed transaction includes: Obtaining multiple operation sequences according to dependency information between operations of the first distributed transaction, where the multiple operation sequences are independent of each other; When a first operation sequence and a second operation sequence among the multiple operation sequences conflict with the second distributed transaction, the first operation sequence, the second operation sequence, and the second distributed transaction are merged to obtain a transaction operation sequence.

8. The method according to claim 7, characterized in that The method further comprises: Constructing a dependency edge of a transaction operation relationship graph according to dependency information between operations of the first distributed transaction, wherein operations represented by vertices connected by the dependency edge belong to the same operation sequence, and constructing a conflict edge of the transaction operation relationship graph according to conflict information between the first distributed transaction and the second distributed transaction; Determine a plurality of connected components in the transaction operation relationship graph, wherein the connected components include a connected graph formed by the dependent edges or the conflicting edges, and each of the plurality of connected components represents a transaction operation sequence; The step of performing operation merging according to the transaction operation sequence to obtain a plurality of transaction slices includes: The operation vertices other than the transaction vertices in the connected components are merged to obtain a transaction slice.

9. A transaction management node, characterized in that: The nodes include: A communication module, used for receiving a transaction request, wherein the transaction request is used for requesting execution of a distributed transaction; A transaction splitting module, used to split the distributed transaction according to dependency information and conflict information between operations of the distributed transaction to obtain multiple transaction slices, wherein the multiple transaction slices are independent of each other; A sequencing module, configured to assign timing information to the plurality of transaction slices; The communication module is also used to distribute the multiple transaction slices and timing information corresponding to the multiple transaction slices to multiple storage nodes, and the transaction slices are executed in the storage nodes in the order of the timing information.

10. The node according to claim 9, characterized in that: The timing information includes a number representing the timing or a timestamp requested from a clock calibration node connected to the transaction management node.

11. The node according to claim 10, characterized in that: The clock timing node includes a data processing unit DPU, a network card or a switch.

12. The node according to any one of claims 9 to 11, characterized in that: The communication module is specifically used for: Send the multiple transaction slices and the timing information corresponding to the multiple transaction slices to the clock calibration node connected to the transaction management node, instruct the clock calibration node to aggregate the transaction slices with the same destination address among the multiple transaction slices, and route the aggregated transaction slices to the corresponding storage node according to the destination address.

13. The node according to claim 12, characterized in that: The transaction slices with the same destination address among the multiple transaction slices are sorted according to the timing information and then aggregated.

14. The node according to any one of claims 9 to 13, characterized in that: The transaction segmentation module is specifically used for: According to the dependency information and conflict information between the operations of the distributed transaction, a plurality of transaction operation sequences are obtained, wherein the plurality of transaction operation sequences are independent of each other, and transactions or operations within the transaction operation sequences have dependency relationships or conflict relationships; The operations are merged according to the transaction operation sequence to obtain multiple transaction slices.

15. The node according to claim 14, characterized in that: The transaction segmentation module is specifically used for: Obtaining multiple operation sequences according to dependency information between operations of the first distributed transaction, where the multiple operation sequences are independent of each other; When a first operation sequence and a second operation sequence among the multiple operation sequences conflict with the second distributed transaction, the first operation sequence, the second operation sequence, and the second distributed transaction are merged to obtain a transaction operation sequence.

16. The node according to claim 15, characterized in that The transaction segmentation module is also used for: Constructing a dependency edge of a transaction operation relationship graph according to dependency information between operations of the first distributed transaction, wherein operations represented by vertices connected by the dependency edge belong to the same operation sequence, and constructing a conflict edge of the transaction operation relationship graph according to conflict information between the first distributed transaction and the second distributed transaction; Determine a plurality of connected components in the transaction operation relationship graph, wherein the connected components include a connected graph formed by the dependent edges or the conflicting edges, and each of the plurality of connected components represents a transaction operation sequence; The transaction segmentation module is specifically used for: The operation vertices other than the transaction vertices in the connected components are merged to obtain a transaction slice.

17. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the method as described in any one of claims 1 to 8.

18. A transaction management system, characterized in that: The transaction management system includes a clock calibration node and multiple transaction management nodes, the multiple transaction management nodes are respectively connected to the clock calibration node, at least one of the multiple transaction management nodes includes a processor and a memory, the processor executes computer-readable instructions, so that the transaction management node executes the method as described in any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 8.

20. A computer program product, characterized in that The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 8.

Images