Decentralized Two-Phase Commit via Partition Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional distributed data transaction systems rely on a centralized coordinator for two-phase commit transactions, which leads to inefficient use of computing resources, increased network latency, and additional network failures.
Innovation Solution
A decentralized two-phase commit module using metadata associated with partition keys to manage transactions, allowing for decentralized locking and completion of transactions without a central coordinator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized coordinator is used to manage two-phase commit transactions, then transaction coordination is achieved, but computing resource efficiency deteriorates and network latency increases
Solution Approach 1:
The patent extracts the coordinator function from the two-phase commit protocol, removing the centralized coordination requirement. Each partition independently manages its own commit operations using metadata associated with partition keys, eliminating the need for a central coordinator while maintaining transaction integrity.
Solution Approach 2:
The system enables each partition to autonomously perform commit operations without external coordination. By using metadata tied to partition keys, each partition self-manages its transaction state, locking mechanisms, and commit/rollback operations, achieving self-service transaction management.
2Reliability
If a centralized coordinator manages transactions, then coordination is possible, but network latency increases
Solution Approach 1:
The coordinator component is extracted from the system, eliminating the network communication overhead associated with centralized coordination. Transactions are managed locally at each partition using metadata, removing the time-consuming round-trip communications to a central coordinator.
Solution Approach 2:
The transaction management function is segmented and distributed across multiple partitions. Each partition independently handles its own commit operations, parallelizing the transaction processing and eliminating the sequential bottlenecks inherent in centralized coordination.
3Ease of operation
If a centralized coordinator is used, then transaction management is centralized, but system reliability deteriorates due to additional network failure points
Solution Approach 1:
The centralized coordinator is extracted and replaced with distributed metadata-based management. By removing the single point of failure (the coordinator), the system gains resilience against network failures while maintaining ease of transaction management through partition-level autonomy.
Solution Approach 2:
Each partition maintains local metadata associated with its partition keys, enabling independent transaction management without relying on centralized coordination. This local quality approach ensures that failures in one partition do not propagate to others, enhancing overall system reliability.
4Productivity
If decentralized locking is implemented using metadata with partition keys, then computing resource efficiency improves, but transaction coordination complexity increases
Solution Approach 1:
The locking mechanism uses metadata specifically associated with partition keys, creating a localized coordination approach. Each partition manages its own locks through metadata, simplifying the coordination complexity by confining it to partition-level operations rather than system-wide coordination.
Solution Approach 2:
Partitions autonomously manage their own locking and commit operations using metadata, eliminating the need for complex centralized coordination logic. The self-service approach reduces computational overhead while maintaining transaction integrity through partition-level autonomy.
Data Source
AI summary
Systems, methods, and computer readable media for decentralized two-phase commit in a datastore or database where the methods include determining pre-validating a transaction, writing first metadata to a leader partition, the first metadata including a state indicating preparing and mutations of the transaction, and writing second metadata of a follower partition to indicate the follower partition is locked by the leader partition. The methods further include updating the state of the first metadata to indicate committed and performing the transaction on the leader partition, performing the transaction on the follower partition and unlocking the follower partition, and updating the first metadata to remove the mutations of the transaction and updating the state of the first metadata to indicate the transaction is completed. The methods may further include selecting a leader partition key and updating the state of the first metadata to indicate committed unapplied.