ML Incident Prediction for IT Resource Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing incident prediction systems are reactive and fail to anticipate future incidents, leading to inefficient resource utilization and increased energy consumption due to the need to address unplanned disruptions and degradations in IT environments.
Innovation Solution
Implementing machine-learning (ML) models, specifically k-nearest neighbors models, trained on historical data to predict incidents and their triggering services, allowing for proactive identification of potential incidents and services likely to cause incidents within a prediction window.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reactive incident prediction systems are used, then system simplicity is maintained, but resource utilization efficiency deteriorates due to unnecessary processing of unplanned disruptions
Solution Approach 1:
The system performs preliminary actions by training ML models on historical incident data to predict future incidents before they occur. The model analyzes patterns in past incidents and proactively identifies services likely to experience incidents, allowing the system to prepare and respond in advance rather than reacting after incidents occur, thereby improving resource utilization efficiency.
2Productivity
If proactive incident prediction using ML models is implemented, then resource utilization is optimized, but computational energy consumption increases during model training and operation
Solution Approach 1:
The system performs computationally intensive model training in advance during a preliminary phase, separate from the prediction phase. Historical incident data is trained on ML models beforehand, and the trained models are then deployed for efficient prediction. This separates the high-energy training process from the low-energy prediction process, optimizing resource utilization during operation while acknowledging the upfront energy investment in training.
3Reliability
If traditional reactive monitoring is used, then system complexity is low, but incident response time deteriorates due to unplanned disruptions
Solution Approach 1:
The system implements feedback by continuously monitoring current service states and comparing them against predictions from the ML model. The model receives feedback from actual incident occurrences and service states, refining its predictions over time. This feedback mechanism enables the system to maintain low complexity while improving incident response time through learned patterns from historical data.
4Measurement precision
If historical data is collected and stored for ML training, then prediction accuracy is improved, but data storage requirements increase
Solution Approach 1:
The system extracts only the essential features and patterns from historical incident data that are most relevant for prediction, rather than storing and processing all raw historical data. The ML model learns to extract key predictive features from the data, reducing storage requirements while maintaining prediction accuracy by focusing on the most informative aspects of historical incidents.
Data Source
AI summary
Incidents in a lookback window from a current time are identified based on selection criteria. A current state is identified based on the incidents. A subset of objects of interest that are likely to occur in a prediction window is identified using a machine-learning (ML) model and based on the current state. The ML model is a k-nearest neighbors model that is trained based on training data obtained from historical data. Each training datum of the training data includes a training lookback window and a training prediction window. Each training lookback window is used to identify incidents occurring in the each training lookback window. Each training prediction window is used to identify which of the objects of interest occurred in the each training prediction window. A notification is transmitted or displayed indicating the subset of the objects of interest.