NVMe Queue Management for Multi-Tier Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-tier storage systems face performance degradation due to core overload or underload conditions, particularly during data migration between storage tiers, as existing systems fail to effectively balance I/O queues across cores based on workload and storage device capabilities.
Innovation Solution
A computer-implemented method that identifies potential core imbalances in multi-tier storage systems using the NVMe protocol, remaps I/O queues to balance workload across cores, and adjusts core assignments during extent migration to maintain optimal performance by generating a migration plan that accounts for relative workload and storage tier capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If I/O queues are not dynamically remapped based on workload, then system complexity is reduced, but core overload or underload conditions occur leading to performance degradation
Solution Approach 1:
The patent implements dynamic remapping of I/O queues to cores based on real-time workload assessment. The system continuously monitors extent access patterns and recalculates queue-to-core mappings to balance workload across cores, transforming the static queue management into a dynamic adaptive system that responds to changing storage conditions and prevents core overload
Solution Approach 2:
The system employs feedback mechanisms by monitoring extent workload and access patterns, then using this information to drive remapping decisions. The workload assessment feedback loop continuously evaluates current queue assignments and triggers remapping when imbalances are detected, ensuring optimal core utilization while maintaining system performance
2Productivity
If I/O queues are statically assigned to cores, then ease of operation is improved, but parallelism is reduced during extent migration causing performance degradation
Solution Approach 1:
The system performs preliminary workload assessment and queue remapping before extent migration operations. By proactively identifying and balancing workload distributions in advance, the system prepares optimal queue-to-core mappings that prevent core overload during migration, maintaining parallelism without requiring complex real-time adjustments during the actual migration process
3Productivity
If core workload is not balanced during extent migration, then device complexity is reduced, but throughput is degraded due to core overload
Solution Approach 1:
The system changes the assignment parameters of I/O queues by dynamically remapping them to different cores based on workload conditions. During extent migration, the system modifies queue-to-core mappings to distribute workload evenly, preventing core overload and maintaining optimal throughput by adjusting the parameter of core assignment rather than adding complex load balancing infrastructure
Data Source
AI summary
A computer-implemented method manages I/O queues in a multi-tier storage system. The method includes identifying a set of subsystems in a multi-tier storage system, and each subsystem in the set of subsystems are communicatively connected to the storage system via a non-volatile memory express (NVMe) protocol and correlated to a tier of the multi-tier storage system. The method includes, determining a workload of each extent, wherein each extent of the set of extents are stored on one subsystem and the extents are accessed by an application. The method further includes, mapping, based on the workload of each extent, each extent to a core of the plurality of cores, wherein the mapping is configured to such that each core is balanced. The method includes, establishing, based on the mapping, an IOQ for each extent, wherein the IOQ is processed by the core to which it is mapped.