Connection Virtualization for Storage Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in dynamic resource allocation for non-volatile memory resources among hosts with varying workloads in multi-tenant systems, leading to suboptimal utilization of storage resources.
Innovation Solution
A connection virtualization layer dynamically allocates host storage connections and commands, enabling the pooling of storage resources across connections and supporting more connections and commands than individual devices can handle, by managing queue-pairs and completion identifiers to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed number of queue-pairs are allocated to each host storage connection, then the storage device can manage connection resources, but the system cannot support more connections than the configured limits of individual devices
Solution Approach 1:
A connection virtualization layer is introduced as an intermediary between hosts and storage devices. This layer receives connection requests from multiple hosts, virtualizes the limited queue-pair resources of individual storage devices, and dynamically allocates completion identifiers to host connections. The virtualization layer enables more host connections to be supported than the physical queue-pair limits would normally allow, while maintaining manageable complexity through centralized virtualization logic.
2Productivity
If individual storage devices are used with fixed queue-pair limits, then device configuration is simple, but storage resources cannot be pooled across connections
Solution Approach 1:
The connection virtualization layer merges storage resources from multiple individual storage devices into a pooled resource pool. By combining the queue-pair resources of multiple devices and managing them through a unified virtualization layer, the system achieves improved storage resource utilization and allocation flexibility, allowing dynamic assignment of completion identifiers across the pooled resources.
3Ease of operation
If hosts are not diligent about load balancing and terminating unused connections, then connection establishment is simple, but storage resources are used inefficiently
Solution Approach 1:
The connection virtualization layer implements feedback mechanisms to monitor and manage storage resource usage across host connections. The system tracks completion identifier assignments, queue depths, and resource utilization metrics, using this information to dynamically adjust resource allocation and encourage efficient host behavior through informed resource management decisions.
Data Source
AI summary
Systems and methods for connection virtualization in data storage device arrays are described. A host connection identifier may be determined for a storage connection request. A target storage device and corresponding completion connection identifier may be determined for a storage command including the host connection identifier. A command tracker may be stored that associates the storage command with the host connection identifier and the completion connection identifier and the storage command may be sent to the processing queue associated with the completion connection identifier.