Log Record Recognizer Expression for Merged Stream Parsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of parsing log records from diverse sources into uniform events is hindered by the lack of common standards, leading to CPU-intensive processing and scalability limitations, particularly when dealing with large systems and merged log streams.
Innovation Solution
A method is introduced to create a log record recognizer expression by normalizing and reassembling regular expressions for each log record type, sorting them based on operator precedence, and using the resulting expression to identify and parse log records efficiently, thereby reducing false positives and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple log record type regular expressions are used to handle diverse log sources, then parsing accuracy and adaptability improve, but CPU time and processing complexity increase significantly
Solution Approach 1:
The patent pre-processes and normalizes multiple regular expressions into a unified log record recognizer expression before actual parsing occurs. This preliminary action involves merging multiple type-specific regex patterns into a single optimized expression that can handle all log types, thereby avoiding the CPU-intensive process of evaluating multiple separate expressions during runtime processing.
2Adaptability or versatility
If multiple log record type regular expressions are used to handle diverse log sources, then parsing accuracy improves, but device complexity and scalability limitations worsen
Solution Approach 1:
The patent merges multiple separate regular expressions for different log record types into a single unified log record recognizer expression. This combining process normalizes the patterns and integrates them into one expression that can accurately identify and parse various log types without requiring the system to maintain and manage multiple separate regex patterns, thereby reducing device complexity and improving scalability.
3Productivity
If log records from multiple sources are merged into a single stream, then correlation discovery and issue recognition improve, but log record identification complexity increases
Solution Approach 1:
The patent creates a universal log record recognizer expression that serves multiple functions: it can identify and parse various log record types from different sources within a single unified expression. This multi-functional approach enables the system to handle merged log streams from multiple sources efficiently, facilitating correlation discovery while avoiding the complexity of maintaining separate identification mechanisms for each source.
Data Source
AI summary
A computing device automatically creates a log record recognizer expression and uses the log record recognizer expression to identify a log record type for a log record to parse the log record. A log record type regular expression is selected from log record type regular expressions and is separated into subexpressions that are normalized and are reassembled into an expression recognizer for each log record type regular expression. The expression recognizer for each is read into a data structure. The recognizer expressions are sorted based on an order associated with an expression operator of each subexpression. A log recognizer expression is created from each read expression recognizer included in the sorted recognizer expressions. A log record type of a log record is identified using the created log recognizer expression. A log record type regular expression is selected. The log record is parsed using the selected log record type regular expression.