Stream Management in Multi-Host Virtualized Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stream management mechanisms in storage systems are suboptimal when serving multiple hosts, leading to delayed read latencies and impaired throughput, as they are designed for single-host environments and cannot efficiently assign stream identifiers to all virtual hosts, resulting in wasted resources and performance gaps between sequential and random read operations.
Innovation Solution
A method for adaptive stream management in multi-host virtualized storage systems that analyzes usage history, including data volume, sequential vs. random data ratio, and read look-ahead hit rates, to assign stream identifiers to the most deserving virtual hosts, optimizing stream allocation and management based on priority and resource availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stream identifiers are assigned to all virtual hosts, then read latency and throughput are improved, but the maximum number of streams assignable by the storage system is limited, resulting in resource exhaustion
Solution Approach 1:
The patent changes the parameter of stream identification by introducing a hierarchical structure where stream identifiers are no longer uniquely assigned to each virtual host but are instead shared and dynamically allocated based on usage patterns. This transforms the static one-to-one mapping into a dynamic many-to-one relationship, allowing more virtual hosts to access stream functionality than the original limit of 4096 streams would permit.
Solution Approach 2:
The system dynamically determines whether to assign stream identifiers to virtual hosts based on their I/O patterns and characteristics. Rather than a fixed assignment, the system adapts stream allocation in real-time, assigning streams to virtual hosts that exhibit sequential I/O patterns while leaving random I/O virtual hosts without stream assignments, thereby optimizing resource utilization.
2Adaptability or versatility
If stream identifiers are assigned to virtual hosts with random I/O patterns, then resource allocation is maximized, but sequential read performance is degraded due to improper stream placement
Solution Approach 1:
The patent applies different stream management strategies to different virtual hosts based on their local characteristics. Virtual hosts exhibiting sequential I/O patterns receive stream identifier assignments with optimized placement, while virtual hosts with random I/O patterns are excluded from stream assignments. This localized differentiation ensures that stream functionality benefits sequential workloads without being degraded by random access patterns.
Solution Approach 2:
Rather than assigning stream identifiers to all virtual hosts, the system selectively assigns them only to those that benefit from sequential access patterns. This partial action approach avoids the performance degradation that would result from applying stream placement optimization to inappropriate workloads, while still providing stream functionality where it is most effective.
3Reliability
If the storage system assigns stream identifiers to all virtual hosts, then Quality of Service is improved, but system complexity increases due to managing more than 4096 streams
Solution Approach 1:
The patent introduces a stream determination module that acts as an intermediary between virtual hosts and the stream management system. This intermediary evaluates virtual host I/O patterns and determines whether stream identifier assignment is appropriate, thereby simplifying the overall system architecture by filtering out virtual hosts that would not benefit from stream functionality and reducing the burden on stream management resources.
4Ease of manufacture
If stream identifiers are limited to 4096, then resource management is simplified, but virtual hosts exceeding this limit experience performance degradation
Solution Approach 1:
The patent segments the virtual host population into different categories based on their I/O characteristics. Virtual hosts are divided into those that receive stream identifier assignments (typically sequential workloads) and those that do not (typically random workloads). This segmentation allows the system to maintain the simple 4096 stream limit while serving a much larger number of virtual hosts by directing stream resources only to those that need them.
Data Source
AI summary
A storage system and method for stream management in a multi-host virtualized storage system are provided. In one embodiment, a method for stream management is provided that is performed in a storage system in communication with a host comprising a plurality of virtual hosts. The method comprises: receiving, from the host, identification of each virtual host of the plurality of virtual hosts; analyzing usage history of each virtual host of the plurality of virtual hosts; and assigning streams to a subset of the plurality of virtual hosts based on the usage history, wherein a maximum number of streams assignable by the storage system is less than a total number of virtual hosts in the plurality of virtual hosts. Other embodiments are provided.