Write Stream Detection for Multi-Tenant Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face complexity in identifying and managing Write Streams across multiple hosts and applications, leading to inefficient data placement and reduced SSD performance due to unawareness of application-specific data associations.
Innovation Solution
A method involving random assignment of extent identifiers to RAID groups, using clustering algorithms like k-means to detect Write Streams, and tagging write operations with stream group identifiers for optimized data allocation across solid state drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple host systems access a single storage device to share data, then data sharing capability is improved, but system complexity increases due to the need for additional software and management overhead
Solution Approach 1:
The patent segments write streams into distinct groups (first write stream group, second write stream group) with different service requirements. By dividing the storage system into multiple service classes with differentiated quality of service parameters, the system can handle multiple host systems efficiently without requiring complex centralized management software, thus resolving the contradiction between data sharing capability and system complexity.
2Speed
If write operations are processed without stream detection, then processing speed is improved, but data placement efficiency deteriorates leading to reduced SSD performance
Solution Approach 1:
The patent performs preliminary stream detection and classification of write operations into different stream groups before data placement. By pre-identifying write stream characteristics and assigning appropriate quality of service parameters in advance, the system optimizes data placement efficiency without adding significant processing overhead during actual write operations, thus resolving the contradiction between processing speed and data placement efficiency.
3Productivity
If stream detection processing is performed for each RAID group, then data placement optimization is improved, but processing overhead increases
Solution Approach 1:
The patent applies local quality by performing stream detection and assigning quality of service parameters at the RAID group level rather than globally. Each RAID group can be independently optimized based on its specific write stream characteristics, allowing efficient data placement while minimizing processing overhead by limiting the scope of stream detection to local RAID groups rather than the entire storage system.
Data Source
AI summary
Techniques for processing I/O operations may include: randomly assigning extent identifiers of extents to RAID groups; receiving write activity information for the extents for a sampling period; for each of the RAID groups, performing stream detection processing in accordance with the first write activity information and determining stream groups, wherein each stream group of one of the RAID groups includes extents of the RAID group belonging to a same write stream in the RAID group, wherein stream groups across all RAID groups are uniquely identified using stream group identifiers; tagging a write operation directed to one of the extents with a first stream group identifier denoting a stream group including the one extent; and issuing the write operation to a solid state drive of one of the RAID groups. Stream detection may use K-Means clustering algorithm. An aggregate address range of logical devices is mapped to the extent identifiers.