Log-Based Status Modeling for Distributed Application Diagnosis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems lack an efficient method for performing log-based status modeling and problem diagnosis for distributed applications, making it difficult to identify anomalies and diagnose issues across diverse and distributed logging systems.
Innovation Solution
A computer-implemented method that collects logs from distributed sources, groups them based on related sources, generates feature vectors, and compares them to a reference status model built from historical logs to detect anomalies, using techniques like TF-IDF and clustering algorithms to model behavior and identify deviations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If logs are collected and analyzed from multiple distributed sources without grouping, then comprehensive monitoring coverage is improved, but diagnostic complexity and computational overhead increase significantly
Solution Approach 1:
The patent segments logs from distributed sources by grouping them according to their sources. This organization divides the complex task of analyzing all logs into manageable groups, reducing diagnostic complexity while maintaining comprehensive monitoring coverage across all distributed systems.
2Measurement precision
If all current logs are used to build status models, then model accuracy improves, but computational resources and processing time increase
Solution Approach 1:
The patent builds status models separately for each group of logs rather than processing all logs together. This segmented approach maintains model accuracy for each specific log source while significantly reducing overall computational resources and processing time required.
Solution Approach 2:
The patent processes logs in manageable groups rather than attempting to process all logs simultaneously. This partial processing approach achieves sufficient model accuracy for diagnostic purposes while avoiding the excessive computational burden of processing the entire log set at once.
3Measurement precision
If detailed analysis of individual log entries is performed, then anomaly detection precision improves, but processing speed decreases
Solution Approach 1:
The patent performs detailed anomaly analysis on segmented groups of logs rather than individual entries across all sources. This approach maintains high anomaly detection precision within each group while achieving practical processing speeds through parallelization and reduced per-group computational load.
Data Source
AI summary
Techniques include collecting current logs from distributed sources, selecting a group of the current logs that are from a related source of the distributed sources, and generating a feature vector using the group of the current logs. A current status model is created for the feature vector using the group of the current logs. One or more anomalies are determined in the group of the current logs based on a difference between the current status model and a reference status model, the reference status model being based on history logs.