Entity-State Matrix for Log Correlation Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data centers, administrators face challenges in monitoring and identifying issues due to overwhelming logs and metrics, with rule-based alerts being ineffective in capturing all problems and anomaly detection generating false alarms, leading to inefficient notification management.
Innovation Solution
An automated log analysis tool that identifies domain-specific entities and their correlations with states, generating an entity-state matrix and graph to provide actionable insights, allowing for domain-agnostic alerting and reducing the administrative burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based alerts are used to monitor system issues, then specific known problems can be detected, but the system cannot capture all problems and generates ineffective alerts
Solution Approach 1:
The patent introduces an entity-state matrix as an intermediary structure that connects system entities (processors, storages, network links) with their states (failed, overloaded, underutilized). This matrix serves as a mediator between raw log data and alert generation, enabling the system to detect both known and unknown problems by analyzing correlations in the matrix rather than relying on predefined rules.
Solution Approach 2:
The system dynamically changes parameters by computing entity-state matrices from incoming log streams and adjusting the matrix based on changing system conditions. This allows the monitoring system to adapt to new problem types and system configurations without requiring manual rule updates, thereby improving both reliability and versatility.
2Adaptability or versatility
If anomaly detection is used to identify system issues, then comprehensive problem coverage is achieved, but false alarms are generated
Solution Approach 1:
The patent implements feedback mechanisms where the entity-state matrix is continuously updated based on incoming logs and previous states. The system uses this feedback to refine correlation strength measurements and adjust alert thresholds dynamically, reducing false alarms while maintaining comprehensive problem coverage. The feedback loop allows the system to learn from past anomalies and distinguish between genuine issues and normal variations.
3Reliability
If detailed log analysis is performed to identify all issues, then comprehensive monitoring is achieved, but administrative burden increases
Solution Approach 1:
The patent segments the complex log analysis task into structured entity-state pairs organized in a matrix format. By dividing the monitoring space into discrete entities (processors, storages, network links) and their possible states, the system makes comprehensive monitoring manageable and automates the analysis process, significantly reducing administrative burden while maintaining monitoring completeness.
Solution Approach 2:
The entity-state matrix serves multiple functions simultaneously: it stores system state information, detects anomalies, generates alerts, and provides a framework for correlation analysis. This multi-functionality eliminates the need for separate systems for each task, reducing administrative complexity while achieving comprehensive monitoring.
Data Source
AI summary
In an example, there is disclosed a logging server computing apparatus, having: a processor; a memory; and a logging engine to: analyze a network; build an entity-state matrix M from an entity vector e and a state vector s; determine that there is a strong correlation between an entity ec and a state sc; and report the strong correlation.