Autocorrelation-Based Background Workflow Scheduling in Storage Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storage systems experience performance drops due to background operations competing with initiator I/O operations for resources, especially during high load periods, and existing technologies lack effective methods for intelligent scheduling to mitigate this.
Innovation Solution
A system that schedules background workflows by analyzing historical load statistics to identify predominant periodicities, using a linear predictor to estimate future system loads, and allocating background operations quotas to minimize impact on primary I/O operations, thereby reducing the likelihood of background operations during high initiator load periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If background operations are scheduled during high I/O activity periods, then background maintenance tasks can be performed, but system performance drops due to resource competition
Solution Approach 1:
The system dynamically adjusts background operation scheduling based on real-time system load conditions. The load predictor continuously monitors historical I/O patterns and dynamically determines optimal scheduling windows, transforming the static background operation schedule into a dynamic one that adapts to changing system conditions, thereby resolving the contradiction between maintaining background productivity and preserving initiator I/O reliability
Solution Approach 2:
The system performs preliminary analysis of historical load statistics to predict future system load patterns before scheduling background operations. By using autocorrelation analysis and linear predictors to forecast load conditions in advance, the system can proactively schedule background tasks during predicted low-load periods, preventing performance degradation before it occurs
2Reliability
If background operations are scheduled during low load periods, then initiator I/O performance is maintained, but background operations may be delayed beyond optimal maintenance windows
Solution Approach 1:
The system implements feedback mechanisms by continuously monitoring actual system load and comparing it with predicted load patterns. This feedback loop allows the system to learn from past scheduling decisions and refine its predictions, ensuring that background operations are scheduled at optimal times that balance initiator performance requirements with background maintenance timing needs, thereby minimizing time loss while maintaining reliability
3Productivity
If background operations compete for resources with initiator I/O, then resource utilization increases, but system overload occurs during peak periods
Solution Approach 1:
The system changes the temporal parameter of background operation execution by scheduling them during predicted low-load periods rather than uniformly or on-demand. This parameter change in timing allows the system to maintain high overall resource utilization while avoiding system overload during peak initiator I/O periods, as background tasks are shifted to temporal windows when resource demand is naturally lower
Data Source
AI summary
Technologies are described for implementing a system that schedules background workflows in a storage system. Historical load statistics can be analyzed to estimate the preferred times in the future during which to schedule background workflows. After collecting load statistics related to the storage system, natural cyclical patterns in the load data can be determined as the predominant periodicities of the load data are selected from peaks in an autocorrelation of the collected data. A linear predictor may be formed as a weighted sum of the historical data sampled at the determined periodicities. Error minimization, such as the method of least squares, may be used to determine the values of the weighting in order to minimize an error between the load predictor and the measured load. Predicting future loads can improve the scheduling of background operations by avoiding conflicts with normal system operations.