Storage Pool Allocator Latency via Dynamic Transaction Classes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network latency has become a bottleneck in storage systems, particularly in solid-state array (SSA) environments, where traditional storage protocols struggle to keep up with the increased performance demands, leading to unpredictable client latency and higher costs due to the need for faster storage networks and more expensive hardware.
Innovation Solution
The method involves creating multiple transaction classes within write transaction groups, dynamically adjusting their capability weights based on IOPS, MBPS, and time slices during storage pool allocator sync times, and redistributing unspent time slices to optimize system resource allocation and predict client latency through staged processing distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional storage protocols are used in SSA environments, then storage performance can be maintained, but network latency becomes a bottleneck and client latency becomes unpredictable
Solution Approach 1:
The patent segments storage operations into different transaction classes (foreground and background operations) with different priority levels. Foreground operations that affect client latency are separated from background operations, allowing the system to guarantee latency bounds for critical operations while still performing background maintenance tasks.
Solution Approach 2:
The system dynamically changes operational parameters including IOPS limits, MBPS rates, and time slice allocations based on transaction class and system state. This allows the storage system to adapt resource allocation to maintain predictable latency under varying workloads while maximizing throughput when possible.
2Productivity
If more system resources are allocated to handle increased performance demands, then storage performance improves, but costs increase due to faster storage networks and more expensive hardware
Solution Approach 1:
The patent implements dynamic resource allocation where IOPS, MBPS, and time slice parameters are adjusted in real-time based on workload characteristics and system state. This allows the system to maximize performance when needed while reducing resource consumption during normal operations, avoiding the need for permanently over-provisioned expensive hardware.
Solution Approach 2:
The system changes operational parameters dynamically based on transaction class and system conditions. By adjusting IOPS limits, data rates, and time slices, the system can achieve high performance when required without permanently allocating resources at maximum capacity, thereby reducing hardware cost requirements.
3Ease of manufacture
If background file system operations are allowed to proceed without restrictions, then storage maintenance tasks can be completed, but client latency becomes unpredictable and sync times increase
Solution Approach 1:
The patent divides storage operations into foreground transactions that serve client requests and background operations that perform maintenance tasks. This segmentation allows background operations to proceed without interfering with foreground latency guarantees, as background operations are executed in separate time slices with limited resource allocation.
Solution Approach 2:
Background file system operations are scheduled periodically in dedicated time slices rather than running continuously. This periodic execution ensures that background maintenance tasks get necessary resource allocation while preventing them from interfering with foreground client operations, thereby maintaining predictable sync times.
Data Source
AI summary
A method, system and computer program product for managing an amount of storage written from a transaction group comprises creating a plurality of transaction classes within each of a plurality of write transaction classes and discovering an initial underlying drive capability assignable to the transaction classes when a storage pool is allocated (SPA). Each transaction class capability weight is adjusted via computing an instruction and operations per second, a data rate per second and a time slice at every storage pool allocator sync time. An unspent time slice from an adjusted transaction class is lent to a plurality of remaining transaction classes yet to be adjusted via a staged processing distribution during sync of the SPA. The disclosed method further includes assigning system resources and underlying drive capability to each transaction class per its capability weight based on a ratio of weights for each transaction class of operations.