Distributed Transaction Management Without Central Log
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-performance transaction systems with multiple data stores face performance bottlenecks due to the use of centralized logs for global atomic transactions, which can degrade when the number of data stores increases.
Innovation Solution
A distributed data storage system that operates without a central log, using a central database management module and data store nodes with write-ahead logs to manage transaction requests, allowing for distributed transaction processing and checkpointing operations to ensure data integrity and fault tolerance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized log with 2-phase commit is used to perform global atomic transactions across multiple data stores, then transaction atomicity and data integrity are improved, but system performance deteriorates due to the centralized log becoming a performance bottleneck when the number of data stores is high
Solution Approach 1:
The patent divides the centralized log into distributed write-ahead logs at each data store node. Each node maintains its own log independently, eliminating the single centralized log bottleneck. The system segments the transaction coordination function across multiple nodes while preserving atomicity through distributed 2-phase commit protocol.
Solution Approach 2:
The patent transitions from a single-point centralized log architecture to a distributed log architecture where logs are spread across multiple dimensions (different data store nodes). This dimensional distribution allows parallel processing of transactions across different nodes, improving overall system throughput while maintaining coordination through the distributed 2-phase commit protocol.
2Ease of operation
If a centralized log is used to unify each transaction domain into a global transaction domain, then transaction coordination is simplified, but device complexity increases due to the centralized log managing all transactions
Solution Approach 1:
The patent segments the centralized log functionality into distributed write-ahead logs at each data store node. Each node's log is independent and locally managed, reducing the complexity burden on any single centralized component. Transaction coordination is maintained through the distributed 2-phase commit protocol that operates across these segmented logs.
Solution Approach 2:
Each data store node manages its own write-ahead log independently, performing self-coordination of transactions. The nodes autonomously handle their local transaction domains while the distributed 2-phase commit protocol coordinates globally. This self-service approach reduces centralized log complexity by distributing management responsibilities.
3Reliability
If the number of data stores is increased to improve system capacity and availability, then fault tolerance is improved, but the performance bottleneck of the centralized log worsens
Solution Approach 1:
The patent segments the centralized log into distributed write-ahead logs at each data store node. This segmentation allows the system to scale horizontally with the number of data stores without creating a single performance bottleneck. Each node's independent log processing enables parallel transaction handling, maintaining high throughput even as the number of data stores increases for improved fault tolerance.
Solution Approach 2:
The patent distributes the log management function across multiple dimensions (multiple nodes) rather than concentrating it at a single centralized location. This dimensional distribution enables the system to handle increased numbers of data stores without performance degradation, as transactions can be processed in parallel across the distributed log architecture.
Data Source
AI summary
Data storage system and method for managing transaction requests in the data storage system utilizes prepare requests for a transaction request for multiple data storage operations. The prepare requests are sent to selected destination storage nodes of the data storage system to handle the multiple data storage operations. Each prepare request includes at least one of the multiple data storage operations to be handled by a particular destination data store node and a list of the destination storage nodes involved in the transaction request.