Transaction Log Round-Robin Block Allocation for Contention Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data storage systems, concurrent write operations to a transaction log can lead to significant contention and performance issues due to threads competing for access to the same lockable units, resulting in delayed acknowledgments and increased latency.
Innovation Solution
The system partitions the log into chunks with lockable units, ensuring that data from sequential requests is stored in different units, reducing contention by using a round-robin approach for segment selection and pre-allocating chunks for specific request sizes, thereby minimizing thread pauses and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple threads concurrently write to the same region in the transaction log, then throughput is improved, but contention increases causing threads to pause and latency to increase
Solution Approach 1:
The transaction log is divided into multiple regions, with each region further divided into pages that can be independently locked. This segmentation allows multiple threads to concurrently write to different pages within the same region without blocking each other, thereby maintaining high throughput while minimizing thread pause time due to contention.
2Productivity
If multiple threads concurrently write to the same region in the transaction log, then throughput is improved, but IO request latency increases due to delayed acknowledgments
Solution Approach 1:
By segmenting the log into regions and pages with fine-grained locking, the system enables parallel write operations that complete faster. Threads can acquire locks on different pages simultaneously, reducing the time to complete write operations and send acknowledgments back to hosts, thus reducing IO request latency while maintaining high throughput.
3Reliability
If threads wait for pages to be released by other threads, then data integrity is maintained through locking, but system performance deteriorates
Solution Approach 1:
The log structure divides data into small page units that can be independently locked. This fine-grained segmentation reduces the scope of locking, allowing threads to access different pages concurrently without waiting for each other. Data integrity is maintained through locking protocols, while system performance improves due to reduced thread waiting time.
Solution Approach 2:
The patent introduces a log buffer as an intermediary layer between write requests and the actual log storage. The log buffer temporarily holds write operations, allowing threads to proceed without immediately contending for log pages. This intermediary mechanism decouples the writing threads from the locking mechanism, maintaining data integrity while improving overall system performance.
Data Source
AI summary
A technique for storing data in a log receives a set of sequential requests, each request specifying data to be written to a data object served by a data storage system. The data specified by respective ones of the requests is placed into respective lockable units of storage for the log, such that the data of no two sequential requests are stored within the same lockable unit. Each lockable unit is locked for access by a writer when being written, and unlocked when not being written, and is configured to store data specified by multiple requests. Completion of each of the requests is acknowledged in response to placing the data specified by the respective request in one of the lockable units of storage. The log may also be partitioned, with individual partitions being used to store data specified by requests having specific request sizes.