Two-phase transaction commit method and device for distributed database based on rdma

Abstract

The application discloses a kind of distributed database two-stage transaction submission method and device based on RDMA, it is related to distributed database high-performance transaction processing technical field, comprising: in pre-commitment stage, data jump is completed using RDMA unilateral writing operation;Through polling, transaction redo log data is written successfully;In the submission stage, the atomic update submission state flag bit is carried out through the RDMA atomic comparison and exchange operation;When the number of successful times reaches the majority threshold value through polling RDMA completion queue, exit waiting loop and execute consensus achievement processing. It can eliminate the multiple network round trip and software processing delay in traditional protocol, so that the end-to-end transaction delay is reduced from the software protocol level to the hardware network level, the system CPU resources are released so that it does not need to participate in the protocol processing, so that the computing resources can be efficiently utilized, and the overall throughput and concurrent processing capacity of the distributed cluster are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of high-performance transaction processing technology for distributed databases, and in particular to a two-phase transaction commit method and apparatus for distributed databases based on RDMA. Background Technology

[0002] Distributed databases often achieve storage scalability and high availability through data sharding and multi-replica mechanisms. To ensure strong data consistency across replicas, a two-phase commit based on consensus algorithms such as Raft is widely adopted for distributed transactions: In the pre-commit phase of log replication, the transaction coordinator (Leader node) synchronizes the redo log of the transaction to all participants (Follower nodes) and waits for a majority of nodes to confirm successful persistence; then, in the state commit phase, after obtaining a majority of confirmations, the Leader node initiates the commit, notifying all participants to apply the persisted log to their local state machines, making the transaction effective. Simultaneously, to overcome traditional network performance bottlenecks, RDMA (Remote Direct Memory Access) is used, leveraging its high bandwidth, low latency, kernel bypass, and zero-copy characteristics to accelerate data transmission between nodes. This allows the computer's network card to directly read and write the user-space memory of remote nodes without the involvement of the remote operating system's kernel protocol stack and CPU.

[0003] However, simply applying RDMA to the communication layer of traditional TCP / IP network protocols—that is, simply replacing the original TCP Socket communication module in a one-to-one manner—still results in new performance bottlenecks due to the inherent, serialized "request-response" interaction logic (such as multiple rounds of broadcasting and acknowledgment) in the two-phase commit protocol. This is because the logic of multiple network round trips and blocking waits still exists, causing delays in the microsecond-level single operation provided by the RDMA hardware. Especially in high-bandwidth, low-latency RDMA network environments, the Follower CPU may still need to participate in receiving acknowledgments and requires complex multi-round software interactions to reach consensus. Relying on multiple network round trips and software interaction logic involving deep CPU involvement from both sides, the Follower node's CPU still needs to be frequently interrupted to process protocol logic, resulting in significant overall CPU overhead. This fails to fully utilize the potential of RDMA hardware to achieve low-latency transaction processing and CPU overhead optimization. Summary of the Invention

[0004] This invention provides a two-phase transaction commit method and apparatus for distributed databases based on RDMA, to solve the technical problem that in the two-phase commit distributed transaction protocol, although RDMA hardware is used, CPU interaction is still required, and the issuance of commit instructions and consensus require multiple rounds of network message interaction between the Leader and all Followers, resulting in high CPU load, high replication latency, and performance bottlenecks that block critical paths.

[0005] In a first aspect, embodiments of the present invention provide a two-phase transaction commit method for distributed databases based on RDMA, comprising:

[0006] S101, In the pre-commit phase, the Leader node uses RDMA one-sided write operation to directly write the transaction redo log data into the remote memory pre-registered by the Follower node, thus completing the data jump.

[0007] S102, by polling the RDMA completion queue, confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes, and complete the pre-commit phase;

[0008] S103, Entering the commit phase, the Leader node uses RDMA atomic comparison and exchange operation to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data, so as to atomically update the value of the commit status flag.

[0009] S104, the Leader node polls the RDMA completion queue in a waiting loop to count the number of successful and failed atomic comparison and exchange operations of each Follower node. When the count of successful operations reaches the preset majority threshold, the Leader node immediately exits the waiting loop to execute consensus processing and returns a transaction commit success response.

[0010] Secondly, embodiments of the present invention provide a two-phase transaction commit device for a distributed database based on RDMA, comprising:

[0011] The data jump module is used in the pre-commit phase to enable the Leader node to use RDMA one-sided write operations to directly write the transaction redo log data to the remote memory pre-registered by the Follower node.

[0012] The pre-commit confirmation module is used to poll the RDMA completion queue to confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes.

[0013] The atomic comparison and exchange operation module is used to enter the commit phase, enabling the Leader node to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data through the RDMA atomic comparison and exchange operation, so as to atomically update the value of the commit status flag.

[0014] The consensus module enables the Leader node to poll the RDMA completion queue through a waiting loop, count the number of successful and failed atomic comparison and exchange operations of each Follower node, and immediately exit the waiting loop to execute the consensus process and return a transaction commit success response when the count of successful operations reaches a preset majority threshold.

[0015] Thirdly, embodiments of the present invention provide an electronic device, including:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described two-phase transaction commit method for distributed databases based on RDMA.

[0019] Fourthly, embodiments of the present invention provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute the above-described two-phase transaction commit method for distributed databases based on RDMA.

[0020] This invention provides a two-phase transaction commit method and apparatus for distributed databases based on RDMA. The method reconstructs the traditional two-phase commit protocol into a direct memory copy of log data (data jump) completed by a single-sided write operation of RDMA, and achieves the commit consensus through hardware-based atomic comparison and exchange operations of RDMA. This constructs a completely new, hardware-driven distributed transaction commit paradigm. It reduces end-to-end latency in transaction commits, eliminates the millisecond-level overhead of multiple network round trips and software processing in traditional protocols, and reduces transaction latency from the software protocol level to the hardware network level. This significantly frees up system CPU resources, especially the CPU of Follower nodes. Throughout the critical path (data copying and consensus achievement), the CPU of Follower nodes does not need to participate in protocol processing, achieving zero remote CPU participation. This allows their computing resources to be used as much as possible for processing business queries, significantly improving the overall throughput and concurrent processing capabilities of the system. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0022] Figure 1 This is a flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 1 of the present invention.

[0023] Figure 2 This is a flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 2 of the present invention.

[0024] Figure 3 This is a flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 3 of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of a two-phase transaction commit device for a distributed database based on RDMA, as described in Embodiment 4 of the present invention.

[0026] Figure 5 This is a structural diagram of the electronic device described in Embodiment 5 of the present invention. Detailed Implementation

[0027] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0028] Example 1

[0029] Figure 1 The flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 1 of the present invention, specifically includes the following steps:

[0030] S101, during the pre-commit phase, the Leader node uses RDMA one-sided write operations to directly write the transaction redo log data into the remote memory pre-registered by the Follower node, thus completing the data jump.

[0031] When the Leader node receives a transaction request from a client (such as a set of SQL write operations), its transaction engine first executes the transaction locally. To ensure the persistence and replicability of the transaction, corresponding redo log records are generated during execution, forming transaction redo log data, which is temporarily stored in a dedicated, contiguous memory buffer on the Leader node, also known as the local log buffer. Unlike traditional solutions that require sending transaction redo log data through the network protocol stack and having it received and processed by the Follower's CPU, this embodiment utilizes RDMA's one-sided write (WRITE) operation. The Leader node's RDMA network card directly initiates the process, sending the transaction redo log data from the local log buffer directly to the remote memory pre-registered on the Follower node, without waking up the Follower's CPU. The transaction redo log data is then directly written to a specified location in the Follower node's user-space receive buffer, achieving efficient data transfer. It can completely bypass the operating system kernel protocol stack (TCP / IP) at both the Leader and Follower ends, achieving zero-copy transmission. During the process, the CPU can continue to execute other computing tasks (such as processing read-only queries), completely eliminating the overhead of interrupt handling, buffer copying, protocol parsing, etc. of the Follower node CPU on the data receiving path in the traditional method.

[0032] S102, by polling the RDMA completion queue, confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes, and complete the pre-commit phase.

[0033] After the Leader node initiates all RDMA one-sided write requests to the target Followers, these requests are executed asynchronously by the RDMA network interface card (NIC). During this time, the Leader node can continue processing new transaction requests. The Leader node can obtain the operation completion status by actively polling the Completion Queue (CQ) associated with its RDMA NIC. Whenever an RDMA one-sided write operation is successfully executed by the NIC hardware (i.e., the data has been confirmed to have been delivered and written to the Follower's remote memory), the NIC places a Completion Entry in the CQ. By continuously polling and checking the CQ, the Leader can determine which Follower nodes have completed their write operations. According to distributed consistency principles (such as Raft's majority rule), when the Leader confirms that the current transaction redo log data has been successfully persisted in the memory of a majority (e.g., N / 2+1, where N is the total number of nodes) of Follower nodes, the pre-commit phase can be considered successful. At this point, the modification log data of this transaction (i.e., the transaction redo log data) has been safely stored in most replicas and is ready for final commit.

[0034] S103, Entering the commit phase, the Leader node uses RDMA atomic comparison and exchange operations to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data, so as to atomically update the value of the commit status flag.

[0035] Based on a successful pre-commit, the Leader node logically decides to commit the redo log data for this transaction. At this point, it needs to notify all Follower nodes holding the redo log data of this commit decision. Notification involves atomically modifying a reserved commit status flag in the memory area where the redo log data resides. This flag is initialized to an "uncommitted" state (e.g., 0) when the redo log data is written. During the commit phase, the Leader node initiates an RDMA atomic Compare and Swap (CAS) operation for each relevant Follower node. This operation is executed by the Leader's RDMA network card hardware. It atomically (i.e., indivisibly) reads the current value of the corresponding commit status flag in the remote memory of the Follower node holding the redo log data for that transaction. If the value matches the comparison value (0 if not committed), it replaces the value with the swap value (1 if committed). At the RDMA network card hardware level, it ensures that the "read-compare-write" sequence for this flag is an uninterrupted continuous operation, ensuring that even under concurrent access, the flag can only be successfully changed from 0 to 1 once.

[0036] S104, the Leader node polls the RDMA completion queue in a waiting loop to count the number of successful and failed atomic comparison and exchange operations of each Follower node. When the count of successful operations reaches the preset majority threshold, the Leader node immediately exits the waiting loop to execute consensus processing and returns a transaction commit success response.

[0037] After the Leader node initiates CAS operations sequentially (or concurrently) to all relevant Follower nodes, it enters a waiting loop, polling the Completion Queue (CQ) of the RDMA network interface to count the results of each CAS operation. Two counters are maintained: success_count and failure_count. Each time a successful CAS operation completion entry is received, success_count is incremented; conversely, when a failed CAS operation completion entry is received, failure_count is incremented. During this process, it continuously checks: once success_count reaches the threshold (quorum_size) required to form a majority in the distributed cluster, according to the principles of distributed consensus (such as the core ideas of Paxos and Raft), the proposal to "commit the transaction" has been accepted by a majority of the distributed cluster, and consensus is formally reached. At this point, the Leader immediately exits the waiting loop, without waiting for the operation results of the remaining Followers. Afterwards, the Leader node performs the cleanup work of the critical path: recording the transaction commit log, releasing all locks and memory resources occupied by the transaction locally, and immediately returning a "transaction committed successfully" response to the client that initiated the transaction. From the client's perspective, this can be considered a completed transaction redo, which can significantly reduce end-to-end latency.

[0038] This embodiment reconstructs the traditional two-phase commit protocol into a hardware-driven distributed transaction commit paradigm by refactoring it into a direct memory copy of log data (data jump) via RDMA one-sided write operations and achieving commit consensus via RDMA atomic comparison and exchange operations. This reduces end-to-end transaction commit latency, eliminates the millisecond-level overhead of multiple network round trips and software processing in traditional protocols, and reduces transaction latency from the software protocol level to the hardware network level. This significantly frees up system CPU resources, especially the CPU of follower nodes. Throughout the critical path (data copying and consensus achievement), the CPU of follower nodes does not need to participate in protocol processing, achieving zero remote CPU involvement. This allows their computing resources to be used as much as possible for processing business queries, significantly improving the overall system throughput and concurrency capabilities.

[0039] Example 2

[0040] Figure 2 This is a flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 2 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, S101 is specifically optimized as follows:

[0041] During the pre-commit phase, the Leader node encapsulates the logs generated by the transaction redo in batches to form transaction redo log data.

[0042] Through asynchronous RDMA one-sided write operations, transaction redo log data is directly written from the local memory of the Leader node to the receive buffer slot pre-allocated in the remote memory of the Follower node, thus completing the data jump.

[0043] Accordingly, the two-phase transaction commit method for distributed databases based on RDMA provided in this embodiment specifically includes:

[0044] S201, during the pre-commit phase, the Leader node batch encapsulates the logs generated by the transaction redo to form transaction redo log data.

[0045] To improve network utilization and reduce the number of RDMA operations, the Leader node does not initiate a separate network operation for each log record. The transaction redo process includes reading relevant data, performing constraint checks, and calculating the updated value. To ensure persistence and replication, the transaction engine generates redo log records in local memory corresponding to all modification operations of the transaction. Each log record typically contains metadata and payload data such as a unique transaction identifier (Transaction ID), Log Sequence Number (LSN), modified data page identifier, modification type (insert / update / delete), and before and after images of the data. Log records belonging to multiple concurrent transactions generated within a certain period (or when the accumulated log size reaches a preset threshold, such as 32KB or 64KB) are organized and batch-packaged in the local log buffer to form transaction redo log data.

[0046] An optional implementation of this embodiment is that batch packaging includes:

[0047] Based on the preset batch encapsulation threshold, the log records of all transactions within the threshold range are encapsulated in the local log buffer.

[0048] The preset batch encapsulation threshold can be set for multiple transactions that are active within a specified time window and whose log generation times overlap. The log records of all transactions within the time window are then organized and batch encapsulated in the local log buffer. Encapsulation strategies include: sequential arrangement, strictly following the log LSN order to ensure consistent log order across replicas; and batch packaging, packaging multiple log records into a larger, contiguous logical data unit called a log batch or log segment.

[0049] Simultaneously, a header is added to each log batch. This header includes the start log sequence number (start_lsn), end log sequence number (end_lsn), log batch size (size), checksum, and consensus metadata (such as a pre-commit flag) for subsequent consensus determination, thus forming the transaction redo log data. The encapsulated log batch is the complete data unit that will be replicated to all Follower nodes. It resides as a contiguous byte array in the Leader node's user-space memory, and its memory address, length, and other key information have been registered so that RDMA operations can directly access it.

[0050] S202 uses asynchronous RDMA one-sided write operations to directly write transaction redo log data from the local memory of the Leader node to the pre-allocated receive buffer slot in the remote memory of the Follower node, thus completing the data transfer.

[0051] For each target Follower node, the Leader node performs an asynchronous RDMA one-sided write operation. This is achieved by constructing an RDMA WRITE work request, which specifies the memory address of the redo log data for this batch, the data length, the remote memory address of the pre-allocated receive buffer slot on the target Follower node, and the corresponding memory access key (rkey). After submitting this request to the send queue of the local RDMA network card, the network card, independently of the CPU, directly transfers the redo log data from the Leader node's local log buffer and writes it to the specified receive buffer slot in the remote memory, completing one data transfer. Write operations from multiple Followers can be initiated concurrently.

[0052] During the initialization or connection establishment phase, each Follower node pre-registers one or more fixed-size, contiguous user-space memory regions with the operating system and RDMA network card as remote memory log receive buffers. These buffers are divided into several slots, i.e., receive buffer slots, each slot can hold one log batch (i.e., one transaction redo log data). During the registration process, these memory regions obtain a memory key (containing access permissions and address mapping information). The Leader node, through the earlier handshake or discovery process, has already obtained the remote memory addresses and corresponding memory keys of these buffers on all Follower nodes. The Leader determines an available slot in the target Follower's receive buffer according to a predetermined buffer management strategy (such as sequential allocation based on circular buffers). The Leader then constructs an RDMA WRITE work request in its user-space RDMA communication library, specifying the local source data address (pointing to the starting address of the encapsulated log batch (transaction redo log data) in the Leader's local memory), the local data length (the size of the log batch), the remote target address, the remote memory address of the selected receive buffer slot on the target Follower node, the remote memory key (the key required to access this remote memory region), and operation flags, including a flag indicating whether to request completion notification. The constructed RDMAWRITE work request is then submitted to the transmit queue associated with the local RDMA network card. The RDMA network card retrieves the request from the transmit queue, and the CPU, independent of the Leader, directly reads the transaction redo log data from local memory via the RDMA network and transmits it to the target Follower node's network card. Upon receiving the data, the Follower node's network card also does not need to wake up the Follower's CPU; instead, it directly writes the data to the specified slot in its user-space receive buffer according to the target address and memory key in the request, completing the data transfer.

[0053] Specifically, the Leader node actively polls the completion queue associated with the RDMA network card. Based on the completion entries in the queue, it confirms the completion status of the corresponding write operation. It can also confirm that the transaction redo log data has been successfully written when the write operation that meets the preset write operation completion threshold is completed.

[0054] When the Leader needs to confirm whether the current log batch has been successfully replicated to a majority of nodes, it can actively poll the Completion Queue (CQ) associated with the RDMA network interface card (NIC). Each time an RDMA WRITE operation is completed by the NIC hardware, a completion entry is placed in the CQ. The execution status of each RDMA WRITE operation is confirmed based on the completion entries and their status in the completion queue. Alternatively, it can be configured to receive an asynchronous event notification after all or a specified number (a preset write operation completion threshold) of RDMA WRITE operations have been completed. When the Leader confirms through this mechanism that the RDMA WRITE operations of the current log batch have been successfully written to the remote memory of a majority (N / 2+1) of the follower nodes, it means that the transaction redo log data of this log batch has been reliably persisted in the volatile memory of these nodes.

[0055] For failed RDMA WRITE operations (returning an error status in the CQ, such as due to network interruption, invalid remote buffer, etc.), the RDMA completion queue will return an error status. At this point, the Leader node records the failure information and selects a new write address (choosing a new free buffer slot) to trigger a retry. If the write fails after exceeding the preset maximum number of retries (e.g., 3 times), the corresponding Follower node is marked as unavailable or unhealthy, and it is determined whether there are still a sufficient number of healthy Follower nodes to form a majority. If a majority cannot be reached (e.g., only 2 healthy nodes in a 5-node cluster), the transaction abort process is directly initiated, and corresponding rollback and resource release operations are performed. Even a majority may still include faulty nodes, failing to guarantee data persistence. A healthy majority, on the other hand, means that a majority has been reached while explicitly excluding faulty nodes, ensuring that all nodes reaching consensus are usable and that data has been securely persisted. For buffer management, Follower nodes need to have their own background threads or mechanisms to periodically check the status of each receive buffer slot, mark slots that have been processed by subsequent processes as free for later reuse, and prevent buffer exhaustion.

[0056] S203, by polling the RDMA completion queue, confirms that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes, thus completing the pre-commit phase.

[0057] S204, Entering the commit phase, the Leader node uses RDMA atomic comparison and exchange operations to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores the transaction redo log data, so as to atomically update the value of the commit status flag.

[0058] S205, the Leader node uses a waiting loop to poll the RDMA completion queue, counts the number of successful and failed atomic comparison and exchange operations of each Follower node, and when the count of successful operations reaches a preset majority threshold, it immediately exits the waiting loop to execute consensus processing and returns a transaction commit success response.

[0059] This embodiment employs a strategy of batch encapsulating multiple transaction log records into log batches during the pre-commit phase. It then utilizes asynchronous RDMA one-sided write operations to directly write these batches into pre-allocated buffer slots in the Follower. Combined with an active polling mechanism for the RDMA completion queue and robust write failure retry and node state management strategies, this achieves an efficient data jump pre-commit process. This significantly improves network bandwidth utilization and transmission efficiency. Batch encapsulation reduces the number of RDMA operations, lowers protocol overhead, and allows a single network transmission to carry more valid data, enhancing system availability. Through active polling and real-time acknowledgment mechanisms, the Leader can accurately perceive the data replication status, ensuring that in the event of partial node or network anomalies, transactions are only committed and executed by a healthy majority of nodes that have reached consensus in the distributed database; otherwise, they are aborted promptly. This avoids system blockage or inconsistency due to single or minority points of failure.

[0060] Example 3

[0061] Figure 3 This is a flowchart of a two-phase transaction commit method for a distributed database based on RDMA, as described in Embodiment 3 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, S103 is specifically optimized as follows:

[0062] Confirm the address, key, comparison value, and exchange value of the commit status flag corresponding to the receive buffer slot containing the transaction redo log data in the remote memory of the Follower node;

[0063] The RDMA atomic comparison and swap operation is submitted to the RDMA transmit queue. It atomically compares the comparison values. If the comparison values ​​match, the swap value is used to replace and write the data to update the value of the submission status flag. If the comparison values ​​do not match, the operation is marked as a failure and no replacement is written.

[0064] Accordingly, the two-phase transaction commit method for distributed databases based on RDMA provided in this embodiment specifically includes:

[0065] S301, during the pre-commit phase, the Leader node uses RDMA one-sided write operations to directly write the transaction redo log data into the remote memory pre-registered by the Follower node, thus completing the data jump.

[0066] S302, by polling the RDMA completion queue, confirms that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes, thus completing the pre-commit phase.

[0067] S303, confirm the address, key, comparison value, and exchange value of the commit status flag corresponding to the receive buffer slot containing transaction redo log data in the remote memory of the Follower node.

[0068] Building upon the success of the pre-commit phase, the two logical steps of "issuing the commit command" and "achieving commit consensus" are merged into an indivisible action guaranteed by RDMA atomic operation hardware. This allows the cluster to reach eventual consensus on whether to commit the transaction in a way that minimizes latency and maximizes efficiency, thus triggering the transaction to take effect. Before initiating the RDMA atomic CAS operation, precise operation parameters need to be prepared. Based on the transaction redo log data encapsulated in the pre-commit phase (usually a structure such as log_batch_info), when transaction redo log data is written to each Follower node, in addition to the data itself, a commit status flag (CommitStatus Flag) is reserved at a fixed position in each slot (e.g., the header) and initialized to 0 (uncommitted). When the Leader node decides to commit, it modifies this flag on all relevant Follower nodes from 0 to 1 (committed).

[0069] When the Leader constructs an RDMA atomic CAS operation request in its RDMA communication library, it needs to specify the remote target address of the receive buffer slot corresponding to the current transaction redo log data, the remote memory key required to access the address, the comparison value (set to 0 to indicate uncommitted), the exchange value (set to 1 to indicate committed), and the specific address of the commit status flag bit inside the slot.

[0070] S304: Submit the RDMA atomic comparison and swap operation to the RDMA transmit queue. Atomically, based on the corresponding comparison value, if the comparison value matches, the swap value is used to perform replacement writing to update the value of the submission status flag. If the comparison value does not match, it is recorded as an operation failure and no replacement writing is performed.

[0071] The constructed CAS request is then submitted to the RDMA send queue and executed. When the RDMA network card hardware executes this operation, it atomically completes the following sequence: First, it reads the current value of the flag bit specified in the remote memory; then it compares the read flag bit value with the comparison value (0). If they are equal, the swap value (1) is written to the same address; if they are not equal, no writing is performed and the operation is marked as failed. The entire process is guaranteed to be atomic by the hardware, ensuring that even under high concurrency, the same flag bit can only be successfully changed from 0 to 1 once, thus accurately indicating that the Follower node has accepted the commit instruction. When the Leader's RDMA network card completes this CAS operation and generates a "successful" completion entry in the completion queue, it indicates that the commit instruction has been issued, that is, the Leader has successfully sent the commit instruction to the Follower (implemented by writing the flag bit). The individual consensus of the Follower node has been reached, that is, the Follower node has accepted the commit decision, because the success of the CAS operation indicates that its state has changed from uncommitted to committed. At this point, the successful completion of the CAS operation itself is a hardware-level confirmation of the Leader, proving that the operation instruction has been executed and the state has changed.

[0072] S305, the Leader node uses a waiting loop to poll the RDMA completion queue, counts the number of successful and failed atomic comparison and exchange operations of each Follower node, and when the count of successful operations reaches the preset majority threshold, it immediately exits the waiting loop to execute consensus processing and returns a transaction commit success response.

[0073] The Leader node initiates RDMA atomic write operations sequentially (or concurrently) to all relevant Followers. It then waits and confirms that the operation has been successfully executed on a majority (N / 2+1) of the Follower nodes. Once this condition is met, consensus is considered formally achieved based on distributed consistency principles (such as the core ideas of Paxos and Raft). At this point, the final state (commit) of the transaction is irrevocably determined. Each Follower node runs one or more Log Applier threads, which periodically poll the local log receive buffer to check the commit status flag values ​​of each receive buffer slot. Memory monitoring or operating system-provided memory event mechanisms can also be utilized (although RDMA writes do not trigger CPU interrupts, changes in memory values ​​may be detected by specific hardware or software mechanisms). When an application thread polls and finds that the flag of a slot changes from 0 to 1, it indicates that the corresponding transaction has achieved cluster consensus and can take effect. At the moment majority consensus is achieved, the Leader node does not need to wait for any subsequent Follower responses or log application completion. Then, a "transaction committed successfully" response is returned to the client application that initiated the transaction, and resources such as locks and temporary memory occupied by the transaction in the Leader local area are released, allowing other transactions to access the relevant data. The log entry for this transaction in the local log buffer is also marked as committed, allowing it to be cleaned up at an appropriate time (such as log archiving). This completes the entire end-to-end transaction latency from client initiation to receiving a successful response. The latency mainly includes the local transaction execution time, the network transmission time for the first-stage RDMA one-sided write, and the network transmission time for the second-stage RDMA atomic write, eliminating the time-consuming, multi-round software interaction confirmation process.

[0074] An optional implementation of this embodiment is that the consensus-reaching process includes:

[0075] Based on the transaction redo log data, the data modification operations are replayed in the Follower node's database, and the same transaction redo is performed as on the Leader node.

[0076] The application thread reads the complete transaction redo log data from the receive buffer slot and, according to the log sequence number (LSN) order, replays (redos) the data modification operations contained in all log records within the batch into the local database storage engine (i.e., state machine) of the Follower. This is equivalent to performing the same data update on the replica as on the Leader, thereby ensuring data consistency.

[0077] Mark the receive buffer slot corresponding to the current transaction redo log data as available, and release the occupied resources.

[0078] After the log application is complete, the application thread marks the corresponding receive buffer slot as idle or processed to prepare for receiving new log data. It also releases any temporary resources that the transaction might have occupied on the Follower side. By decoupling and asynchronously implementing the path between the Leader and the client, even a slight delay in the log application of a Follower will not affect the confirmation of the transaction commit or the readability of the data on the Leader side (because read requests can usually be handled by the Leader or an applied Follower), further improving the overall throughput and response speed of the system.

[0079] Optionally, when the number of successful counts reaches a preset majority threshold and consensus is reached, the remaining Follower nodes that have not completed the atomic comparison and exchange operation continue to perform the atomic comparison and exchange operation. If the operation is successful, the corresponding node will still be processed according to the formula.

[0080] When the success_count reaches the quorum_size, and the Leader exits the loop and responds to the client, the committed status of any CAS operations that are still in progress or will return later, regardless of their eventual success or failure, remains unaffected. The system can choose to continue waiting for their completion or cancel them (if supported). For Followers that ultimately succeed, their log application threads will still asynchronously detect the commit flag and apply it to the log.

[0081] If the number of remaining Follower nodes that have not completed atomic comparison and exchange operations cannot reach the preset majority threshold even if all operations are successful, the loop will be terminated early to wait and execute consensus failure handling, record the transaction abort log, trigger transaction rollback to the Follower nodes that have successfully completed the operations, release the occupied resources, and return a transaction commit failure response.

[0082] During the waiting loop, the Leader continuously assesses the number of remaining Followers without results. If, even assuming all incomplete operations succeed, the success_count cannot reach the quorum_size (e.g., due to too many failed operations), the Leader will prematurely exit the loop, deeming it a consensus failure. At this point, consensus failure handling is performed: a transaction abort log is recorded; an abort signal is sent to Follower nodes that have successfully executed the CAS operation (flag set to 1) to trigger a rollback (this can be achieved by initiating another CAS operation or relying on subsequent timeout cleanup mechanisms); all relevant local resources of the Leader must be released, and a transaction commit failure response is returned to the client.

[0083] For Follower nodes that fail the atomic comparison and swap operation, the Leader node performs a status check on the Follower node and performs corresponding node removal, crash recovery, or failover operations based on the check results.

[0084] For Followers whose CAS operations fail (e.g., due to abnormal modification of flags), the Leader logs the failure and checks the overall status of that Follower node. If the node is determined to be in a persistently abnormal or inconsistent state (e.g., network partition, crash), the Leader may initiate a process to remove it from the current replica group. For Followers that recover and rejoin after a crash, they need to pull the missing committed logs from the Leader or other healthy replicas using a traditional log synchronization mechanism (which can also be accelerated using RDMA) to catch up, and correctly initialize their buffers and flags during the catch-up process. If the Leader itself fails after reaching consensus but before some Followers apply the logs, the new Leader, after being elected, will check the flag status of each replica. The new Leader (or any Follower that has applied the logs) knows which transactions have been committed (because the commit flag has been set) and ensures that all committed transactions are eventually applied on all surviving replicas, guaranteeing eventual data consistency.

[0085] For example, a core function "AtomicCommitConsensus" is defined to implement the data processing from commit decision to consensus. This function encapsulates the set of operations in the commit phase, condensing the complex multi-round software negotiation into a single efficient hardware operation.

[0086] Function name: AtomicCommitConsensus;

[0087] Function objective: By initiating a series of RDMA atomic write operations, enable the cluster to reach consensus on the commit of a specific transaction and complete critical path processing;

[0088] Input parameters:

[0089] transaction_id: A unique identifier for the transaction to be committed;

[0090] log_batch_info: A structure describing the log batch information that has been copied during the pre-commit phase, containing the remote memory address, memory key, and address list of slot commit status flags on each Follower node.

[0091] quorum_size: The number of majority nodes required to reach consensus;

[0092] Output / Return Value:

[0093] COMMIT_SUCCESS: Indicates that consensus has been reached and the transaction has been successfully committed;

[0094] COMMIT_FAILURE: Indicates that consensus could not be reached and the transaction commit failed;

[0095] PARTIAL_FAILURE: Consensus was reached, but operations on a few non-critical Followers failed (the transaction is still considered successful, but the exception needs to be logged).

[0096] Function internal processing flow:

[0097] (1) Initialization and verification: At the beginning of the function, the validity of log_batch_info is verified to confirm that the transaction is in the pre-commit success state;

[0098] (2) Prepare atomic operation parameters: Iterate through the Follower list in log_batch_info and prepare an RDMA CAS operation parameter packet for each Follower. Each parameter packet contains the target flag address, memory key, comparison value (UNCOMMITTED), and exchange value (COMMITTED).

[0099] (3) Batch submission of atomic operations: The prepared CAS operation parameter packets are submitted in batches to the network card's transmission queue through the RDMA communication interface. These operations are sent asynchronously and concurrently to each Follower node;

[0100] (4) Waiting for and collecting completion notifications: Enter a waiting loop, poll the RDMA completion queue (CQ), and collect the results of completed CAS operations. Maintain two counters: success_count (number of successes) and failure_count (number of failures);

[0101] (5) Real-time judgment and early return: During the collection process, the success_count is checked in real time. Once the success_count reaches the quorum_size, the waiting loop is immediately exited and (7) consensus is reached. If the remaining uncompleted operations cannot reach the quorum_size even if all of them are successful, the loop is exited early and (8) consensus failure is reached.

[0102] (6) Handling incomplete operations: For operations that have not returned results after consensus has been reached, they can continue to wait or be cancelled (if supported), but the result will no longer affect the consensus decision;

[0103] (7) Consensus processing: a. Record the transaction commit log; b. Release all locks and memory resources of the transaction on the Leader node; c. Put the "transaction committed successfully" response into the queue to be sent to the client; d. The function returns COMMIT_SUCCESS. If failure_count>0 exists, the PARTIAL_FAILURE status code can be appended.

[0104] (8) Consensus failure handling: a. Record the transaction abort log; b. Trigger the transaction rollback process, which may require sending an abort signal to the Follower that has successfully executed CAS (this can be achieved by setting the flag to ABORTED through another atomic operation, or by relying on the cleanup mechanism after timeout); c. Release local resources; d. Return `COMMIT_FAILURE` to the function.

[0105] (9) Resource cleanup: Clean up temporary data structures used within the function;

[0106] (10) Return: Returns the final status code to the caller.

[0107] This embodiment uses RDMA atomic compare-and-swap operations during the commit phase to atomically update the commit status flag in the Follower's memory, and processes committed transactions through asynchronous polling and application logs on the Follower side. The Leader can respond to clients immediately after hardware consensus is reached, while time-consuming operations such as log application are left to the Followers for asynchronous background execution. This ensures that the response latency of transaction commits is no longer constrained by the log application speed of the slowest replica. This guarantees strong atomicity and correctness of distributed consensus, decouples critical and non-critical paths, further optimizes response time, provides a complete exception handling framework, and guarantees eventual data consistency.

[0108] Example 4

[0109] Figure 4 This is a schematic diagram of a two-phase transaction commit device for a distributed database based on RDMA, as described in Embodiment 4 of the present invention. In this embodiment, the two-phase transaction commit device for a distributed database based on RDMA includes:

[0110] The data jump module 810 is used to enable the Leader node to use RDMA one-sided write operation to directly write the transaction redo log data to the remote memory pre-registered by the Follower node during the pre-commit phase.

[0111] The pre-commit confirmation module 820 is used to poll the RDMA completion queue to confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes.

[0112] The atomic comparison and exchange operation module 830 is used to enter the commit phase, enabling the Leader node to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data through the RDMA atomic comparison and exchange operation, so as to atomically update the value of the commit status flag.

[0113] The consensus module 840 is used to enable the Leader node to poll the RDMA completion queue through a waiting loop, count the number of successful and failed atomic comparison and exchange operations of each Follower node, and when the count of successful operations reaches the preset majority threshold, it immediately exits the waiting loop to execute the consensus process and returns a transaction commit success response.

[0114] This embodiment uses a data jump module to directly write transaction redo log data to Follower nodes during the pre-commit phase. A pre-commit confirmation module polls the RDMA completion queue to confirm successful writing of the transaction redo log data. An atomic comparison and exchange operation module performs atomic comparison and exchange operations to write a commit value to the commit status flag containing the transaction redo log data. Finally, a consensus-reaching module executes consensus processing when the number of successful transactions reaches a preset majority threshold. This reduces end-to-end latency in transaction commits, eliminating the millisecond-level overhead of multiple network round trips and software processing in traditional protocols. It reduces transaction latency from the software protocol level to the hardware network level, significantly freeing up system CPU resources, especially the Follower node's CPU. Throughout the critical path (data replication and consensus-reaching), the Follower node's CPU does not need to participate in protocol processing, achieving zero remote CPU involvement. This allows its computing resources to be used as much as possible for processing business queries, significantly improving the overall system throughput and concurrency capabilities.

[0115] The RDMA-based distributed database two-phase transaction commit device provided in this embodiment of the invention can execute the RDMA-based distributed database two-phase transaction commit method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Example 5

[0117] Figure 5 This is a structural diagram of an electronic device according to Embodiment 5 of the present invention. Figure 5 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 5 The electronic device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0118] like Figure 5As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0119] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0120] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0121] System memory 28 may include computer system readable media in the form of volatile memory, such as RAM 30 and / or cache 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0122] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0123] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the electronic device 12 / server / computer, and / or with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through I / O interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 5 As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0124] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the two-phase transaction commit method for distributed databases based on RDMA provided in this embodiment of the invention.

[0125] Example 6

[0126] Embodiment 6 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute the two-phase transaction commit method for distributed databases based on RDMA as provided in the above embodiments.

[0127] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0129] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0130] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A two-phase transaction commit method for distributed databases based on RDMA, characterized in that, include: S101, In the pre-commit phase, the Leader node uses RDMA one-sided write operation to directly write the transaction redo log data into the remote memory pre-registered by the Follower node, thus completing the data jump. S102, by polling the RDMA completion queue, confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes, and complete the pre-commit phase; S103, Entering the commit phase, the Leader node uses RDMA atomic comparison and exchange operation to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data, so as to atomically update the value of the commit status flag. S104, the Leader node polls the RDMA completion queue in a waiting loop to count the number of successful and failed atomic comparison and exchange operations of each Follower node. When the count of successful operations reaches the preset majority threshold, the Leader node immediately exits the waiting loop to execute consensus processing and returns a transaction commit success response.

2. The method according to claim 1, characterized in that, S101 includes: During the pre-commit phase, the Leader node encapsulates the logs generated by the transaction redo in batches to form transaction redo log data. Through asynchronous RDMA one-sided write operations, transaction redo log data is directly written from the local memory of the Leader node to the receive buffer slot pre-allocated in the remote memory of the Follower node, thus completing the data jump.

3. The method according to claim 2, characterized in that, The batch packaging includes: Based on the preset batch encapsulation threshold, the log records of all transactions within the threshold range are encapsulated in the local log buffer. Add a header to each log batch. The header includes the start log sequence number, end log sequence number, log batch size, checksum, and consensus metadata.

4. The method according to claim 1, characterized in that, S102 includes: The leader node actively polls the completion queue associated with the RDMA network card. Based on the completion entries in the queue, it confirms the completion status of the corresponding write operation. It can also confirm that the transaction redo log data has been successfully written when the write operation that meets the preset write operation completion threshold is completed. For a failed write operation, the Leader node records the failure information and selects a new write address to trigger a retry of the write operation. If the write still fails after exceeding the preset maximum number of retries, the corresponding Follower node is marked as unavailable, and it is determined whether there are still enough healthy Follower nodes to form a majority. If a majority cannot be reached, the transaction is aborted.

5. The method according to claim 2, characterized in that, S103 includes: Confirm the address, key, comparison value, and exchange value of the commit status flag corresponding to the receive buffer slot containing the transaction redo log data in the remote memory of the Follower node; The RDMA atomic comparison and swap operation is submitted to the RDMA transmit queue. It atomically compares the comparison values. If the comparison values ​​match, the swap value is used to replace and write the data to update the value of the submission status flag. If the comparison values ​​do not match, the operation is marked as a failure and no replacement is written.

6. The method according to claim 2, characterized in that, The consensus-reaching process includes: Based on the transaction redo log data, the data modification operations are replayed in the database of the Follower node, and the same transaction redo is performed as in the Leader node. Mark the receive buffer slot corresponding to the current transaction redo log data as available, and release the occupied resources.

7. The method according to claim 1, characterized in that, S104 further includes: When the number of successful counts reaches the preset majority threshold and consensus is reached, the remaining Follower nodes that have not completed the atomic comparison and swap operation continue to perform the atomic comparison and swap operation. If the operation is successful, the corresponding node will still be processed according to the formula. If the number of remaining Follower nodes that have not completed atomic comparison and exchange operations cannot reach the preset majority threshold even if all operations are successful, the loop will be terminated early and consensus failure will be handled. The transaction abort log will be recorded, and the transaction will be rolled back to the Follower nodes that have successfully completed the operations. The occupied resources will be released and a transaction commit failure response will be returned. For Follower nodes that fail the atomic comparison and swap operation, the Leader node performs a status check on the Follower node and performs corresponding node removal, crash recovery, or failover operations based on the check results.

8. A two-phase transaction commit device for a distributed database based on RDMA, used to implement the two-phase transaction commit method for a distributed database based on RDMA as described in any one of claims 1-7, characterized in that, include: The data jump module is used in the pre-commit phase to enable the Leader node to use RDMA one-sided write operations to directly write the transaction redo log data to the remote memory pre-registered by the Follower node. The pre-commit confirmation module is used to poll the RDMA completion queue to confirm that the transaction redo log data has been successfully persisted and written to a majority of Follower nodes. The atomic comparison and exchange operation module is used to enter the commit phase, enabling the Leader node to write the commit value to the preset commit status flag in the remote memory of each Follower node that stores transaction redo log data through the RDMA atomic comparison and exchange operation, so as to atomically update the value of the commit status flag. The consensus module enables the Leader node to poll the RDMA completion queue through a waiting loop, count the number of successful and failed atomic comparison and exchange operations of each Follower node, and immediately exit the waiting loop to execute the consensus process and return a transaction commit success response when the count of successful operations reaches a preset majority threshold.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the two-phase transaction commit method for distributed databases based on RDMA as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the two-phase transaction commit method for a distributed database based on RDMA as described in any one of claims 1-7.

Images