Automated Storage System Performance Modeling via Self-Discovery I/O
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in modeling and optimizing performance across multiple storage tiers and external data storage systems, particularly in predicting workload performance and avoiding overload without explicit configuration information or human intervention.
Innovation Solution
A method for data storage system modeling that involves discovery processing to obtain performance characteristics by issuing and observing I/O operations, generating performance metrics, and tracking limits to predict component utilization and drive technology selection, using non-exploratory I/O operations to build models without requiring explicit input from human administrators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If discovery processing is performed to obtain performance characteristics without explicit configuration information, then the system can automatically model performance and avoid overload, but the complexity of the system increases due to the need for automated discovery and modeling mechanisms
Solution Approach 1:
The data storage system performs self-discovery by automatically issuing I/O operations to external storage systems and analyzing the performance characteristics without requiring manual configuration or explicit input from administrators. The system models its own performance by observing its behavior under various workload conditions, thereby achieving automated performance modeling through self-service mechanisms.
Solution Approach 2:
The system performs preliminary discovery processing before actual workload execution to establish performance models. By conducting exploratory I/O operations and analyzing responses in advance, the system pre-characterizes the performance of external storage systems, enabling subsequent workload optimization without requiring real-time manual intervention.
2Measurement precision
If I/O operations are issued to discover performance characteristics, then accurate performance modeling is achieved, but the time required for modeling and system overhead increases
Solution Approach 1:
The system issues a limited set of I/O operations with specific characteristics (different sizes, block densities, and patterns) to discover performance characteristics. Rather than exhaustively testing all possible conditions, the system performs partial discovery using representative workloads that capture the essential performance behavior, thereby achieving adequate modeling accuracy with reduced time investment.
3Reliability
If multiple performance metrics are tracked for different storage tiers and components, then comprehensive performance prediction is achieved, but the complexity of data collection and analysis increases
Solution Approach 1:
The system segments the data storage system into distinct components (front-end components, back-end components, and external storage systems) and tracks performance metrics for each segment separately. By dividing the complex system into manageable parts and modeling their individual performance characteristics, the system reduces the overall modeling complexity while maintaining comprehensive performance prediction capability.
Data Source
AI summary
Described is data storage system modeling. Received at a first data storage system is information representing a workload for I/O operations directed to a logical devices having storage provisioned on physical devices of a second data storage system. Information representing the workload may be obtained by performing discovery processing to discover performance characteristics of the physical devices of the second data storage system. Discovery processing may include receiving, at the first data storage system, I/O operations from a client directed to the set of one or more logical devices having storage provisioned on the physical devices of the second data storage system. The I/O operations are then issued to the second data storage system. In response, performance data is obtained at the first data storage system representing the workload for the plurality of I/O operations. Performance of the second data storage system is modeled in accordance with the workload.