Dynamic IO Queue Allocation for NVMe Storage Targets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems face challenges in configuring appropriate network resources, particularly when using advanced protocols like NVMe-oF or NVMe/TCP, leading to overallocation and undermining performance.
Innovation Solution
Dynamic adjustment of network resources, such as IO queues and TCP connections, based on actual usage to optimize storage system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If network resources such as IO queues and TCP connections are statically allocated for NVMe targets, then configuration is simple and initial setup is fast, but overallocation occurs leading to degraded storage system performance
Solution Approach 1:
The patent implements dynamic adjustment of IO queues and TCP connections based on real-time utilization monitoring. The system transitions from static allocation to dynamic reconfiguration, where network resources are adjusted according to actual workload demands, preventing both overallocation and underallocation while optimizing storage system performance
Solution Approach 2:
The system employs feedback mechanisms by monitoring utilization levels of network resources and using this information to dynamically adjust the number of IO queues and TCP connections. This closed-loop control ensures resources are optimized based on actual usage patterns, resolving the contradiction between simple static configuration and performance-optimized dynamic allocation
2Productivity
If the number of IO queues is increased to handle higher IO operations, then storage system throughput improves, but network resource overallocation occurs undermining performance
Solution Approach 1:
The system dynamically adjusts the number of IO queues based on real-time utilization monitoring rather than maintaining a fixed high number. This dynamic approach allows the system to scale IO queue capacity with actual throughput demands, achieving high performance when needed while avoiding resource waste during lower utilization periods
Solution Approach 2:
The patent changes the parameter of IO queue count from a static value to a dynamically adjusted value based on utilization metrics. This parameter change enables the system to optimize throughput by increasing queues when utilization is high while reducing queues to avoid waste when utilization is low, directly addressing the contradiction between throughput and resource waste
Data Source
AI summary
An apparatus comprises at least one processing device that includes a processor coupled to a memory. The at least one processing device is configured to receive in a storage system from a host device a command specifying an initial number of input-output (IO) queues to be configured for a given target of the storage system, to configure in the storage system the initial number of IO queues for the given target responsive to the received command, and to dynamically adjust in the storage system the number of IO queues configured for the given target based at least in part on an amount of IO operations received by the storage system for the given target. Dynamically adjusting the number of IO queues illustratively comprises increasing the number of IO queues relative to the initial number of IO queues based at least in part on a utilization level of the IO queues.