Decentralized Transaction Processing via Commit Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current transactional data management systems face challenges in scaling out processing to handle higher transaction rates while maintaining full transactional consistency, leading to contention and bottlenecks due to sequential commit processing and synchronization issues.
Innovation Solution
A decentralized transactional management system is decomposed into components such as commit servers, snapshot servers, transaction managers, conflict managers, loggers, and data managers, allowing for parallel transaction processing and asynchronous commitment without blocking, with each component handling a subset of transactions and data, ensuring consistency and durability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sequential commit processing is used to guarantee transactional consistency, then transactional consistency is maintained, but processing throughput and scalability are limited
Solution Approach 1:
The system segments the commit processing function by introducing separate commit servers that issue commit labels independently from transaction managers. This segmentation allows multiple transactions to be committed in parallel without blocking each other, as each transaction manager can receive commit labels from the commit server without waiting for sequential processing. The decomposition of the monolithic commit processing into separate services enables high throughput while maintaining consistency through the ordered commit label sequence.
Solution Approach 2:
The commit label acts as an intermediary mechanism that mediates between transaction managers and the commit process. Instead of direct sequential coordination, transaction managers obtain commit labels from the commit server and apply them to their transactions. This intermediary approach decouples the transaction managers from each other and from the commit process, allowing parallel processing while ensuring consistency through the ordered sequence of commit labels.
2Reliability
If synchronous synchronization between transactions is implemented, then transactional consistency is ensured, but system latency and processing time increase
Solution Approach 1:
The system transitions from static synchronous synchronization to dynamic asynchronous operation. Transaction managers operate independently and asynchronously, obtaining commit labels from the commit server at their own pace. The snapshot server dynamically provides snapshots based on commit labels without requiring transactions to wait for each other. This dynamic approach maintains consistency through the ordered commit label sequence while eliminating synchronous blocking and reducing processing latency.
Solution Approach 2:
The commit server issues commit labels in advance before transactions actually commit, and the snapshot server prepares snapshots ahead of time. This preliminary action allows transactions to proceed without waiting for synchronization. The commit labels are issued in order but do not block transaction execution, and snapshots are ready before transactions need them, eliminating synchronous waiting and reducing latency.
3Device complexity
If centralized transaction management is used, then coordination and consistency control are simplified, but system scalability and fault isolation are reduced
Solution Approach 1:
The centralized transaction management system is segmented into multiple independent services: transaction managers handle individual transactions, a commit server manages commit labeling, and a snapshot server manages database snapshots. This segmentation distributes the coordination complexity across multiple lightweight components rather than a single complex central manager. Each component has a simple, well-defined role, making the system more scalable and easier to fault-isolate while maintaining overall coordination through the commit label sequence.
Solution Approach 2:
The commit label and snapshot serve as intermediaries that mediate between transaction managers and the database. Instead of direct centralized coordination, transaction managers interact with these intermediary objects to achieve consistency. The commit label sequence provides a simple ordering mechanism that eliminates complex centralized coordination, while the snapshot mechanism provides efficient data access. This intermediary approach simplifies component interactions and enables scalability.
4Reliability
If contentions are resolved by aborting transactions, then data consistency is maintained, but system throughput and efficiency decrease
Solution Approach 1:
The system performs preliminary conflict detection using commit labels before transactions actually execute. By ordering transactions according to their commit labels, the system identifies potential conflicts in advance and resolves them through the snapshot mechanism rather than aborting. This preliminary ordering action allows the system to maintain consistency while avoiding the throughput penalty of transaction aborts, as conflicts are managed through the snapshot system rather than by rejecting transactions.
Data Source
AI summary
A system and method for decentralized transaction processing that reduce contention by a number of techniques. First, the transactional management system is decomposed in a number of components that can be scaled out and/or scaled up independently and in a composable way. Second, transactions are committed in parallel without blocking each other. Third, applications can progress when the transactions updates are durable even if the writes of the transaction have not yet completed. Fourth, transactional consistency is guaranteed by making readable only gap-free prefixes of committed update transactions, without blocking the commit of new transactions. Fifth, session consistency is guaranteed by delaying the start of a new transaction till the snapshot of any previous update transaction in the same session is readable.