Dynamic Shared Write Buffer Selection to Reduce Lock Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in managing write requests efficiently, leading to latency issues due to lock contention and inefficient space utilization in shared write buffers.
Innovation Solution
A storage system that dynamically selects between shared and private write segments based on latency and load characteristics, using a decision-making process that considers the nature of the write stream, contention delay, and pending requests to optimize latency and space utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple compute entities write to a shared current segment simultaneously, then space utilization is improved, but lock contention increases causing latency
Solution Approach 1:
The write buffer is divided into multiple segments (first write segment and second write segment). When lock contention is detected on the current segment, the system switches to an alternative segment, allowing concurrent write operations to proceed without contention while maintaining space utilization.
Solution Approach 2:
A segment selector acts as an intermediary that manages write operations by selecting appropriate segments. It monitors lock contention and redirects writes to alternative segments when needed, mediating between the compute entities and the shared buffer to reduce contention while maintaining efficient space usage.
2Reliability
If a locking mechanism is used to manage shared write buffer access, then data consistency is improved, but lock contention delay increases
Solution Approach 1:
The write buffer is segmented into multiple independent write segments. By distributing write operations across segments, the system reduces lock contention while maintaining data consistency within each segment. Compute entities can write to different segments concurrently without conflicting.
Solution Approach 2:
The system pre-allocates multiple write segments and maintains a segment selector that tracks which segment to use next. This preliminary organization allows write operations to proceed efficiently without acquiring locks, as each segment is pre-designated for specific write operations.
3Loss of time
If write operations are acknowledged immediately, then latency is reduced, but data safety may be compromised
Solution Approach 1:
The write buffer is divided into segments with dedicated ownership. When a compute entity is assigned ownership of a segment, it can immediately acknowledge write operations without waiting for locks, as it has exclusive access to that segment. This segmentation enables immediate acknowledgment while maintaining data safety through ownership tracking.
Solution Approach 2:
The segment selector and compute entities work together in a self-service manner where the segment selector assigns segments to compute entities, and compute entities independently manage their assigned segments without requiring continuous lock management. This enables immediate acknowledgment while ensuring data safety through the ownership model.
Data Source
AI summary
A method for writing data to a shared write buffer, the method includes (i) receiving, by a processing circuit of a compute node of a storage system, a write request for writing to the shared write buffer an information unit associated with an accessing entity located outside the storage system; wherein the shared write buffer is stored in a non-volatile memory of a storage node of the storage system; the storage node is in communication with the compute node; (ii) determining, by the processing circuit whether to (a) store the information unit in a non-shared segment of the shared write buffer, or (b) store the information unit in a shared segment of the shared write buffer while applying a locking mechanism; wherein the determining is based on parameters, the parameters include a writing parameter of the accessing entity, a writing latency parameter associated with the accessing entity, and a locking timing parameter; and (iii) storing the information unit according to the determination.