Root Cause Analysis Using Gradient Boosted Trees and SHAP
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems often stop at anomaly detection in computing environments, failing to provide effective root cause analysis, which hinders troubleshooting and decision-making.
Innovation Solution
A multi-stage root cause analysis framework that integrates anomaly detection with advanced correlation analysis, using boosted tree-based models and SHAP values to provide interpretable explanations of anomalies, enabling efficient and accurate determination of underlying causes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If existing anomaly detection systems are used, then anomaly detection capability is provided, but root cause analysis capability is missing
Solution Approach 1:
The patent segments the anomaly analysis process into multiple stages: anomaly detection, sub-signal decomposition, root cause identification, and explanation generation. Each stage processes specific aspects of the signal data independently, allowing the system to detect anomalies while preserving and analyzing root cause information through the segmented sub-signals.
Solution Approach 2:
The patent transforms the problem from a single-dimension anomaly detection task into a multi-dimensional analysis by decomposing signals into multiple sub-signals representing different aspects of system performance. This dimensional expansion enables simultaneous detection of anomalies and identification of their root causes across different signal dimensions.
2Measurement precision
If complex machine learning models are used for root cause analysis, then analysis accuracy is improved, but model interpretability deteriorates
Solution Approach 1:
The patent introduces an interpretable intermediary layer that connects the complex machine learning model to human understanding. This intermediary generates natural language explanations that translate the model's internal reasoning about sub-signal contributions into comprehensible root cause descriptions, maintaining both accuracy and interpretability.
Solution Approach 2:
The patent changes the output parameters of the machine learning model from opaque probability scores to interpretable explanations. By transforming the model output to include human-readable descriptions of which sub-signals contributed to anomalies and why, the system maintains analytical accuracy while improving ease of operation for engineers.
Data Source
AI summary
A data processing system implements performing a root cause analysis that includes identifying a first anomalous signal data predictive of a root cause of a first anomaly in signal data received from a computing system, analyzing the sub-signals of the first anomalous signal data to generate labeled training data, training a gradient boosted tree model using the labeled training data, generating a decision tree based approximating a predictive performance of the gradient boosted tree model, determining insights data predictive of the root cause of the first anomaly based on the gradient boosted tree model and the decision tree, aggregating the insights and analyzing the aggregated insights data to determine a predicted root cause for the first anomaly, determining a confidence level associated with the predicted root cause, and categorizing the predicted root cause into one of a plurality of categories based on the confidence level.