Storage System Configuration Based on Workload Telemetry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sizing techniques for storage devices in storage area networks (SANs) fail to predict whether a storage device's infrastructure can meet customers' performance requirements under varying workloads, such as those from end-user applications.
Innovation Solution
Collecting telemetry data from storage systems to model relationships between storage system configurations and workload/performance characteristics using machine learning techniques, predicting response times, and recommending optimal system configurations based on anticipated workloads and service level agreements (SLAs).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional sizing techniques are used for storage devices, then the infrastructure can be sized based on peak data activities, but the system cannot predict whether it will meet performance requirements under varying workloads
Solution Approach 1:
The system performs preliminary actions by collecting telemetry data from multiple storage devices under various workload conditions before making sizing recommendations. Machine learning models are trained in advance on this historical data to predict performance characteristics, enabling the system to assess whether a proposed infrastructure will meet performance requirements before deployment occurs.
Solution Approach 2:
The system implements feedback by continuously collecting telemetry data from field-deployed storage devices and using this information to refine machine learning models. The predicted performance characteristics are compared against actual performance, and the models are updated to improve accuracy over time, creating a closed-loop system that learns from operational experience.
2Measurement precision
If machine learning techniques are implemented to predict performance characteristics, then accurate predictions can be made, but the system complexity and data processing requirements increase
Solution Approach 1:
The system applies universality by using a single machine learning framework that can predict multiple performance characteristics (response times, throughput, IOPS) across different storage device types and workload conditions. The same core infrastructure handles data collection, processing, model training, and prediction generation, making the complex system multi-functional rather than requiring separate specialized systems for each task.
Solution Approach 2:
The patent introduces an intermediary layer between raw telemetry data and performance predictions. This layer includes data processing pipelines that aggregate, clean, and normalize telemetry information, as well as feature engineering components that transform raw data into meaningful inputs for machine learning models. This intermediary structure manages complexity by organizing the data flow and processing logic in a systematic way.
3Adaptability or versatility
If telemetry data is collected from multiple storage devices to build prediction models, then more accurate recommendations can be provided, but the data collection and processing overhead increases
Solution Approach 1:
The system performs preliminary data collection by gathering telemetry information from multiple storage devices over extended periods before models need to be updated or new predictions are required. This advance data collection ensures that the machine learning models are trained on comprehensive workload scenarios, reducing the need for intensive real-time data gathering when predictions are needed.
Solution Approach 2:
The patent merges data collection efforts by aggregating telemetry data from multiple field-deployed storage devices into a centralized repository. This combined dataset from diverse sources is processed together to train universal prediction models, achieving broader workload coverage more efficiently than collecting and processing data from individual devices separately.
Data Source
AI summary
One or more aspects of the present disclosure relate to providing storage system configuration recommendations. System configurations of one or more storage devices can be determined based on their respective collected telemetry information. Performance of storage devices having different system configurations can be predicted based on one or more of: the collected telemetry information and each of the different system configurations. In response to receiving one or more requested performance characteristics and workload conditions, one or more recommended storage device configurations can be provided for each request based on the predicted performance characteristics, the requested performance characteristics, and the workload conditions.