Data Set Reorganization Forecasting for Capacity-Outage Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data sets in storage systems experience increased size due to gaps created by record additions and deletions, leading to a point where they cannot be extended, causing system outages as records can no longer be written.
Innovation Solution
A reorganization analysis using regression analysis and linear extrapolation to predict the time duration for reorganizing data sets, allowing reorganization before the data set exceeds a predetermined percentage of its maximum size, thereby mitigating latency and preventing system outages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reorganization is performed immediately when data set is full, then system outage is prevented, but latency and downtime increase due to immediate intervention
Solution Approach 1:
The system performs reorganization in advance by predicting future data set capacity using linear extrapolation. The analysis determines the optimal reorganization time before the data set actually becomes full, allowing the system to proactively reallocates space and prevents the need for immediate emergency reorganization when capacity is exhausted.
Solution Approach 2:
The system continuously monitors data set growth patterns and uses this feedback to update predictions about when capacity will be reached. By analyzing historical growth rates and comparing them against current data set size, the system dynamically adjusts the predicted time to capacity, enabling timely reorganization decisions that balance reliability with minimal latency.
2Loss of time
If reorganization is delayed until data set is full, then operational latency is reduced, but system outage occurs when data set cannot be extended
Solution Approach 1:
The system performs reorganization in advance by predicting future data set capacity using linear extrapolation. The analysis determines the optimal reorganization time before the data set actually becomes full, allowing the system to proactively reallocates space and prevents the need for immediate emergency reorganization when capacity is exhausted.
Solution Approach 2:
The system continuously monitors data set growth patterns and uses this feedback to update predictions about when capacity will be reached. By analyzing historical growth rates and comparing them against current data set size, the system dynamically adjusts the predicted time to capacity, enabling timely reorganization decisions that balance reliability with minimal latency.
3Productivity
If reorganization is performed frequently to prevent gaps, then data set efficiency improves, but system complexity and overhead increase
Solution Approach 1:
The system uses feedback from monitoring data set growth patterns to determine the optimal timing for reorganization. By analyzing actual growth rates and comparing them against predicted capacity, the system can identify the best moment to perform reorganization, avoiding both premature frequent reorganizations and delayed interventions that would increase complexity.
Solution Approach 2:
The system changes the parameter of reorganization timing from immediate or frequent to optimized based on predicted capacity. By using linear extrapolation to forecast when data set capacity will be reached, the system adjusts the reorganization schedule to occur only when necessary, reducing overall complexity while maintaining efficiency.
Data Source
AI summary
A computer-implemented method (CIM), according to one embodiment, includes performing a reorganization analysis for mitigating latency during reorganization of a data set. The reorganization analysis includes using regression analysis to predict a time duration that reorganization of the data set will take based on a total size of the data set, relating the predicted time duration to a previous data set reorganization time duration, and performing linear extrapolation to estimate a time at which the total size of the data set will exceed a predetermined percentage of a maximum size of the data set. The method further includes reorganizing the data set before the estimated time, where the data set is reorganized at a time determined based on the reorganization analysis or derivative thereof.