Log Processing System for Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of modern communication networks generates a vast array of log data in diverse formats, making comprehensive review and analysis of log files impractical due to the variability in devices and applications, leading to challenges in event detection and monitoring.
Innovation Solution
A method and system for processing log messages that involves establishing log processing rules to selectively process logs based on content, parsing data fields using tokens, and determining field content to match patterns, enabling intelligent, real-time processing and improved monitoring across platforms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive review of all log files across multiple devices and applications is attempted, then complete event detection capability is improved, but the complexity and impracticality of manual review increases
Solution Approach 1:
The patent introduces an automated log analysis system that acts as an intermediary between raw log files and event detection. This system includes a log collection module that gathers logs from multiple devices, a parsing module that standardizes different log formats, and an analysis module that automatically detects events. This intermediary automation resolves the contradiction by eliminating the need for manual comprehensive review while maintaining complete event detection capability.
2Loss of information
If diverse log formats from various communication devices are processed manually, then complete information extraction is improved, but the time and resources required increase significantly
Solution Approach 1:
The patent implements a universal log parsing framework that can handle multiple log formats from different communication devices through a single standardized interface. The parsing module includes format detection capabilities and automatic adaptation to various log structures (CSV, JSON, XML, plain text), allowing complete information extraction from diverse sources without requiring separate processing procedures for each format, thus reducing processing time while maintaining information completeness.
3Productivity
If automated processing rules are established for selective log processing, then processing efficiency is improved, but the complexity of rule configuration increases
Solution Approach 1:
The patent implements preliminary action by providing pre-configured processing rules and templates for common event types and log formats. The system includes a library of standardized parsing rules for different device types and event categories, which can be selected and activated without requiring users to create rules from scratch. This preliminary preparation maintains high processing efficiency while reducing configuration complexity through template-based deployment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention generally relates to log messages (1 1) processing such that event s cant be detected and alarms can be generated. The system (20) provides for a log manager (13) that is communicatively to various computers (10) to receiv the log messages (1 1) generated thereform. The log messages (11) are generated by a variety of network platforms. A log managers described herein collect such log data/message using various protocols to determine events. The system (20) is generally configured with the event manager (14) to process the event messages to determine whether an alarm should be generated (analysis) therefrom.