Thread Decoupling for Database Transaction Commit Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in computational efficiency and network bandwidth utilization when committing transactions due to waiting for disk I/O and network I/O operations, leading to wasted processing resources.
Innovation Solution
Implementing thread decoupling and job grouping in a distributed database system, where processor-intensive operations are handled by one set of threads and I/O-intensive operations by another, allowing for asynchronous processing and reducing the number of I/O requests through queue management and synchronous/asynchronous network requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the server waits for completion of disk I/O and network I/O operations to commit transactions, then transaction durability and consistency are ensured, but computational efficiency deteriorates and processing resources are wasted
Solution Approach 1:
The patent segments the transaction commit process into distinct phases: writing prepare commit logs (PCLs) to slave nodes and writing commit logs (CLs) to the master node. This segmentation allows the server to send PCL write requests to slave nodes and continue processing without waiting for their completion, while still ensuring durability through the two-phase commit protocol. The separation of PCL and CL writing operations enables asynchronous processing while maintaining transaction reliability.
Solution Approach 2:
The patent implements preliminary action by writing PCLs to slave nodes before finalizing the transaction commit. The server sends PCL write requests to slave nodes and can continue processing other transactions without waiting for these I/O operations to complete. The durability is ensured because the PCLs are written before the transaction is considered committed, and the master node writes CLs only after receiving acknowledgments from slave nodes. This preliminary writing of PCLs allows the server to proceed with computational tasks while I/O operations complete in the background.
2Reliability
If the server sends individual I/O requests for each transaction log entry, then I/O operations are completed reliably, but network bandwidth utilization deteriorates due to excessive requests
Solution Approach 1:
The patent merges multiple I/O requests by having the server send a single PCL write request to each slave node that includes multiple log entries. Instead of sending separate I/O requests for each transaction log entry, the server batches multiple PCLs into a single network request to each slave node. This merging reduces the number of network round trips and improves bandwidth utilization while maintaining reliability through the two-phase commit protocol that ensures all PCLs are written before transaction commitment.
3Device complexity
If the server uses a single thread for both processor-intensive operations and I/O operations, then thread management is simplified, but computational efficiency deteriorates due to thread blocking during I/O waits
Solution Approach 1:
The patent segments thread responsibilities into separate threads: one thread handles processor-intensive operations (executing SQL queries, processing transactions) while another thread handles I/O operations (writing PCLs and CLs to disk, sending network requests). This segmentation allows the processor-intensive thread to continue executing transactions without blocking during I/O waits, significantly improving computational efficiency. The I/O thread manages all disk and network I/O operations independently, ensuring that processor resources are not wasted waiting for I/O completion.
Solution Approach 2:
The patent introduces an intermediary mechanism where the I/O thread acts as a mediator between the processor-intensive operations and the disk/network I/O operations. The I/O thread receives requests from the processor thread, manages the I/O operations asynchronously, and provides completion notifications without blocking the processor thread. This intermediary approach allows the processor-intensive operations to continue executing while I/O operations complete in the background, eliminating thread blocking and improving overall computational efficiency.
Data Source
AI summary
Innovations in the area of server-side processing when committing transactions to disk in a distributed database system can improve computational efficiency at database nodes and/or reduce network bandwidth utilization. For example, when transactions are committed in a database system, at a master node of the database system, a server uses different threads for certain processor-intensive operations and certain I/O-intensive operations.