In-Memory Reservation State Caching for Block Storage I/O Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing block storage services face challenges in maintaining low-latency input/output (I/O) operations due to the need to check reservation status for read and/or write access to shared volumes, which introduces latency in the I/O path.
Innovation Solution
A coherent in-memory cache is implemented on the storage server to maintain per-attachment permission sets and reservation-state related attributes, allowing for frequent access during I/O requests to perform reservation checks and fence decisions. Additionally, a mutation history store and command sequence numbers are used to handle re-submitted mutation commands after network connectivity is re-established.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reservation status is checked for read/write access to shared volumes, then access control and data consistency are ensured, but I/O latency increases
Solution Approach 1:
The patent implements preliminary action by pre-checking reservation status and caching permission sets before I/O operations occur. The storage server maintains an in-memory cache of reservation state information, allowing rapid decision-making during I/O processing without performing full reservation status checks at the time of data access, thus reducing latency while maintaining consistency.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of an in-memory cache layer between the I/O processing path and the full reservation status check system. This cache stores permission sets and reservation state information, acting as a mediator that provides quick access control decisions without requiring direct queries to the complete reservation management system, thereby reducing I/O latency.
2Loss of time
If in-memory cache is implemented for reservation state, then I/O latency is reduced, but system complexity increases
Solution Approach 1:
The patent applies copying by creating a simplified copy of reservation state information in an in-memory cache structure. Instead of maintaining complex distributed reservation management across multiple systems, the storage server creates a local copy of the necessary permission set data, enabling fast I/O processing while the full reservation state remains managed centrally, thus balancing performance improvement with acceptable complexity.
Solution Approach 2:
The patent segments the reservation state management into two parts: a simplified in-memory cache copy for rapid I/O processing and the full reservation management system for authoritative state storage. This segmentation allows the I/O path to operate with minimal complexity using cached data, while the complex reservation management remains isolated in the background, reducing the perceived system complexity from the I/O processing perspective.
3Reliability
If mutation commands are processed during network connectivity issues, then data integrity is maintained, but consistency across distributed systems becomes difficult
Solution Approach 1:
The patent implements feedback mechanisms where the storage server tracks mutation command processing status and network connectivity state. When network issues occur, the system uses feedback information to determine whether to retry, discard, or hold mutation commands, ensuring data integrity while managing consistency through adaptive responses based on real-time system state information.
Solution Approach 2:
The patent applies beforehand cushioning by preparing handling strategies for mutation commands before network connectivity issues occur. The system maintains buffers and retry mechanisms that can accommodate interrupted operations, ensuring that data integrity is preserved even when network issues temporarily disrupt consistency protocols, thus reducing the impact of such issues without requiring complex real-time consistency management.
Data Source
AI summary
A thread of a storage server determines that a command to modify a reservation record which comprises access permissions of an object stored at the server has been received. A version of the record is stored in an in-memory cache. The thread acquires a first lock associated with the record in exclusive mode, and a second lock in shared mode. The thread reads the cached version, constructs a modified version indicated in the command, and releases the second lock without updating the cache. After entries representing the modified version are stored in an operations journal, the thread acquires the second lock in exclusive mode and updates the cache. The thread then releases both locks. An indication that the command has succeeded is provided without verifying that the second version is stored at a metadata store.