Dynamic Resource Allocation via Multivariate Forecasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current networked data backup systems face challenges in optimizing compute resource allocation due to unpredictable future demand levels, leading to inefficient resource utilization and the need for manual user input for throttling, which is impractical and not always optimal.
Innovation Solution
A dynamic resource allocation system that uses multivariate time series forecasting to automatically adjust resource availability based on predicted load, avoiding over-revision and single point spikes, and continuously maintains accuracy through relative deviation adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual throttling mechanisms are used to allow direct user control, then user control over resource allocation is improved, but system complexity and operational burden increase as users must repeatedly change throttle values as load changes
Solution Approach 1:
The system performs self-service by automatically monitoring compute load metrics, forecasting future demand levels, and adjusting throttle values without requiring direct user intervention. The autonomous resource allocation system handles the repetitive task of throttle adjustment itself, eliminating the operational burden on users while maintaining effective control.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring current compute load, comparing it against forecasted demand, and automatically adjusting throttle values based on the deviation. This closed-loop feedback system replaces manual user control with automated responses to changing system conditions.
2Productivity
If throttling is used to regulate resources in real time, then resource allocation control is improved, but the system cannot truly determine optimal throttling because it cannot know future demand levels
Solution Approach 1:
The system performs preliminary action by forecasting future compute load demand before it actually occurs. Using time-series analysis and predictive models, the system anticipates upcoming demand spikes or drops and proactively adjusts throttle values in advance, rather than merely reacting to current conditions. This allows optimal resource allocation before the need arises.
Solution Approach 2:
The system implements dynamic resource allocation by continuously adapting throttle values based on changing system conditions and forecasted demand. Rather than using static or manually-set thresholds, the system dynamically adjusts resource availability in real-time according to predicted workload patterns, enabling optimal productivity without time loss.
3Adaptability or versatility
If dynamic throttling is implemented without accurate forecasts, then resource flexibility is improved, but resource allocation accuracy deteriorates leading to over-allocation or under-allocation
Solution Approach 1:
The system performs preliminary forecasting of compute load demand using historical data and predictive models before making allocation decisions. This advance prediction enables the system to maintain both flexibility in resource adjustment and precision in allocation, as the forecast provides an accurate target for throttle setting rather than relying on reactive or guesswork-based adjustments.
Solution Approach 2:
The system replaces manual or rule-based throttle adjustment mechanisms with data-driven predictive analytics. By substituting the mechanical approach of setting fixed thresholds with intelligent forecasting algorithms, the system achieves both high adaptability to changing conditions and high measurement precision in allocation accuracy.
Data Source
AI summary
Embodiments are described for an autonomously and dynamically allocating resources in a distributed network based on forecasted a-priori CPU resource utilization, rather than a manual throttle setting. A multivariate (CPU idle %, disk I/O, network and memory) rather than single variable approach for Probabilistic Weighted Fuzzy Time Series (PWFTS) is used for forecasting compute resources. The dynamic throttling is combined with an adaptive compute change rate detection and correction. A single spike detection and removal mechanism is used to prevent the application of too many frequent throttling changes. Such a method can be implemented for several use cases including, but not limited to: cloud data migration, replication to a storage server, system upgrades, bandwidth throttling in storage networks, and garbage collection.