Storage Workload Bucket Distribution Using Correlation Factors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining how to distribute an increased workload portion across buckets representing operations of different characteristics, such as I/O sizes, to simulate its impact on storage system performance is difficult.
Innovation Solution
A storage system workload distribution engine computes factor values based on correlation coefficients to distribute quantities of the increased workload portion into buckets representing operations of different characteristics, using a predictive model to predict performance and adjust resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If workload distribution is simulated without factor values based on correlation coefficients, then the simulation process is simpler, but the accuracy of performance prediction deteriorates
Solution Approach 1:
The system pre-computes factor values based on correlation coefficients between different workload buckets during a calibration phase. These pre-computed factor values are stored and later used to quickly distribute increased workload portions without performing complex real-time correlation analysis, thus maintaining prediction accuracy while reducing operational complexity
Solution Approach 2:
Factor values serve as an intermediary that translates complex correlation relationships between workload buckets into simple multiplication factors. Instead of directly computing correlations during workload distribution, the system uses these pre-computed intermediaries to accurately allocate increased workload portions across different bucket types
2Speed
If resource allocation is adjusted reactively after performance degradation, then the control logic is simpler, but the system responsiveness to workload changes deteriorates
Solution Approach 1:
The system performs proactive resource allocation adjustments by predicting performance impacts before they occur. When workload increases are detected, the system uses the pre-computed factor values to anticipate resource requirements and adjust allocation in advance, preventing performance degradation rather than reacting to it
Solution Approach 2:
The system implements a feedback mechanism where actual performance metrics are continuously monitored and compared against predicted values. This feedback loop allows the system to refine its workload distribution predictions and adjust resource allocation proactively, improving responsiveness while maintaining manageable complexity through iterative optimization
Data Source
AI summary
In some examples, a computing system receives an indication of an increased workload portion to be added to a workload of a storage system, the workload comprising buckets of operations of different characteristics. The computing system computes, based on quantities of operations of the different characteristics in the workload, factor values that indicate distribution of operations of the increased workload portion to the buckets of operations of the different characteristics, and distributes, according to the factor values, the operations of the increased workload portion into the buckets of operations of the different characteristics.