ML Ensemble for Failure Event Prediction in Computing Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for monitoring computing systems are inefficient in predicting and addressing failure events, leading to delayed resolution, increased costs, and customer dissatisfaction due to reliance on human teams and complex log analysis.
Innovation Solution
The implementation of a machine learning-based method that trains multiple models on combinations of event indications, evaluates their quality, and forms an ensemble to predict failure events, generating alerts when a failure is detected, thereby reducing downtime and improving predictive accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If human monitoring teams are used to analyze system alerts and identify failure events, then the system can detect failures, but the cost increases significantly and response time is delayed
Solution Approach 1:
The patent replaces human monitoring teams with an automated machine learning-based system. The system uses ML models to analyze system logs and alerts, automatically identifying failure events and predicting their occurrence. This substitution eliminates the need for human experts to continuously monitor systems, thereby reducing response time while maintaining reliable failure detection.
Solution Approach 2:
The system enables self-service by automatically detecting, analyzing, and predicting failure events without requiring human intervention. The ML models continuously monitor system behavior, identify patterns indicative of failures, and provide predictions proactively, allowing the system to serve itself in detecting and responding to failures autonomously.
2Reliability
If human monitoring teams analyze system alerts, then failure events can be identified, but recurring expenditures increase significantly
Solution Approach 1:
The patent replaces the expensive human monitoring team with a cost-effective automated ML-based system. The system uses machine learning models that, once trained, can analyze system alerts and logs without requiring continuous human expertise, thereby eliminating recurring personnel costs while maintaining reliable failure identification capability.
Solution Approach 2:
The system replaces expensive human resources with a disposable software-based ML model infrastructure. The ML models are trained once using historical data and then deployed to continuously monitor system behavior, eliminating the need for ongoing human expert salaries while providing continuous reliable monitoring.
3Loss of information
If manual analysis of server logs is performed, then detailed information can be obtained, but the logs are too large and complex to analyze effectively
Solution Approach 1:
The patent replaces manual log analysis with an automated ML-based analysis system. The ML models process large volumes of server logs automatically, filtering and analyzing the data to identify failure patterns. This substitution enables complete log analysis without being overwhelmed by the volume and complexity of the logs, as the ML algorithms efficiently handle large datasets.
Solution Approach 2:
The system extracts meaningful information from the complex and voluminous server logs by using ML algorithms to identify and isolate failure-related patterns. The models filter out noise and irrelevant data, extracting only the critical information needed for failure detection and prediction, thereby making the analysis manageable and effective.
4Productivity
If operations continue during failure events, then service is maintained, but sub-optimal performance occurs leading to customer dissatisfaction
Solution Approach 1:
The system performs preliminary action by predicting failure events before they actually occur. The ML models analyze system behavior patterns and identify indicators of impending failures, allowing proactive intervention. This enables the system to take preventive measures to avoid failures, ensuring both service continuity and optimal performance by addressing issues before they impact customer experience.
Data Source
AI summary
Techniques for failure prediction are provided. A plurality of event indications is received, where each respective event indication corresponds to a respective failure in a computing system. A plurality of machine learning (ML) models is trained based on combinations of event indications in the plurality of event indications, and the ML models are evaluated to generate a respective quality score for each respective ML model. An ensemble of ML models is defined from the plurality of ML models, based on identifying ML models of the plurality of ML models with corresponding quality scores exceeding a predefined threshold. Current data logs from the computing system are processed using the ensemble of ML models, and upon determining that any ML model of the ensemble of ML models predicted a failure based on the current data logs, an alert is generated.