Cloud Resource Failure Prediction via Hybrid ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based platforms face challenges in managing failures of resources allocated to tenants due to high entropy, requiring extensive human intervention and resulting in delayed repair and poor user experience, especially with increasing tenant subscriptions.
Innovation Solution
An anti-entropy system that uses machine learning models to predict failures and automate the detection and repair of resources, leveraging a hybrid approach with Holt-Winters' and Long Short-Term Memory (LSTM) models to compute overall predictions for proactive resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual monitoring and repair methods are used for cloud resources, then human intervention can detect and address failures, but the system experiences delayed repair and poor user experience due to high entropy and extensive human intervention requirements
Solution Approach 1:
The system performs preliminary actions by training multiple prediction models (Holt-Winters, LSTM, ARIMA) on historical resource data to predict failures before they occur. The models analyze patterns and trends to generate early warnings, enabling proactive resource reallocation and repair scheduling, thus reducing actual repair time when failures occur.
Solution Approach 2:
The system implements continuous feedback loops where prediction model outputs are monitored, and model performance is evaluated against actual failure outcomes. This feedback mechanism allows the system to learn from past predictions and improve future accuracy, while also triggering automated responses to predicted failures before they impact service availability.
2Measurement precision
If multiple prediction models are used to improve prediction accuracy, then the system can compute overall predictions using weighted combinations of Holt-Winters, LSTM, and ARIMA models, but the device complexity increases due to multiple models and weight determination criteria
Solution Approach 1:
The system segments the prediction task into multiple specialized models, each handling different aspects of failure prediction. Holt-Winters handles seasonal patterns, LSTM captures temporal dependencies, and ARIMA models linear trends. This segmentation allows each model to excel at specific prediction aspects while maintaining manageable individual model complexities.
Solution Approach 2:
The system merges multiple prediction model outputs through weighted combinations to produce overall failure predictions. The weight determination criteria (equal weights, performance-based weights, or expert-defined weights) provide a systematic approach to combining models, achieving higher prediction accuracy while managing complexity through structured integration rather than ad-hoc combinations.
3Productivity
If automated failure prediction and repair systems are implemented, then the system can reduce human intervention and improve response time, but the extent of automation increases system complexity and requires sophisticated prediction models
Solution Approach 1:
The system enables self-service automation where the prediction models autonomously analyze resource data, generate failure predictions, and trigger repair workflows without human intervention. The automated resource reallocation and repair scheduling mechanisms allow the system to manage itself, improving productivity while the modular architecture keeps automation complexity manageable through standardized processes.
Data Source
AI summary
The present disclosure relates to system and techniques for prediction of failures in resources deployed in a data plane of a cloud based infrastructure. The resource are selected from a plurality of cloud based resources arranged in a hierarchical manner and allocated to a client device. A predictor employs a first prediction model to obtain a first prediction of a failure of a resource, and a second prediction model to obtain a second prediction of the failure of the resource. Weights are assigned to the first prediction and second prediction based at least in part on a criterion. The predictor computes an overall prediction of the failure of the resource based at least in part on at least one of the first prediction, the second prediction or the respective weights assigned to the predictions. The overall prediction is utilized for restoring the failure of the resource.