Incident Investigation Using Cross-Layer Anomaly Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing incident investigation systems are manually implemented and domain-based, lacking awareness of application and platform layers, leading to inaccurate responses and inefficient incident resolution.
Innovation Solution
An automated incident investigation system that performs end-to-end analysis across application, managed infrastructure, and platform layers, utilizing anomaly detection, troubleshooting, and enrichment to swiftly identify root causes and generate comprehensive summaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual incident investigation processes are used, then human judgment and flexibility are maintained, but investigation time and efficiency deteriorate
Solution Approach 1:
The system enables self-service through automated anomaly detection that autonomously identifies incidents, determines their severity, and initiates investigation procedures without requiring manual intervention from operators, thereby reducing both investigation time and manual effort
Solution Approach 2:
Manual mechanical investigation processes are replaced with an automated computer-based system that uses algorithms to detect anomalies, analyze incident data, and generate resolution recommendations, significantly improving productivity while reducing time loss
2Adaptability or versatility
If domain-based incident investigation systems are used, then system simplicity is maintained, but awareness of application and platform layers deteriorates
Solution Approach 1:
The system achieves universality by designing a multi-layer architecture that can detect and investigate incidents across application, platform, and infrastructure layers simultaneously, making the system adaptable to various incident types without requiring separate domain-specific tools for each layer
Solution Approach 2:
The architecture employs a nested structure where domain-based investigation modules are contained within a unified multi-layer framework, allowing each domain to maintain its simplicity while being integrated into a broader system that provides cross-layer awareness
3Speed
If automated anomaly detection is implemented, then investigation speed is improved, but accuracy and contextual understanding deteriorate
Solution Approach 1:
The system implements feedback mechanisms where detected anomalies are continuously fed back into the investigation process for refinement, allowing the automated detection to learn from contextual information and improve its accuracy while maintaining high speed through iterative processing
Solution Approach 2:
An intermediary analysis layer is introduced between raw anomaly detection and final root cause determination, where contextual information is synthesized and cross-referenced to enhance the precision of accuracy measurements while preserving the speed advantage of automated processing
Data Source
AI summary
Systems and methods are provided for automated incident investigation. Anomaly detection is used to identify anomalies in incident data (e.g., alerts, changes, metrics, logs, and/or system health), and the identified anomalies are converted into facts (or textual prompt inputs for a large language model (“LLM”)). A troubleshooting or diagnostic system is run on the anomalies to provide additional facts to identify a root cause of an incident. The facts from the diagnostics, the facts from the anomaly detections are entered into a consolidated explainer that generates a summary of what happened, what is a likely cause, and what to do next to resolve the issue. In examples, anomaly enrichment data including a time correlation result, a weighted list of abnormal transaction patterns, a list of abnormal trace patterns, a list of exception patterns, a difference pattern, and/or region data are input as further facts to enhance the incident investigation process.