Anchor Token Trees for Event Log Parser Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing number of assets in enterprise systems makes it difficult to monitor and detect security threats efficiently, as the process of generating and selecting parsers for event logs is time-consuming and resource-intensive, affecting the ability to process security events per second (EPS).
Innovation Solution
A data-driven model that splits incoming event logs into tokens, identifies anchor tokens, and uses anchor token trees to select appropriate parsers for data extraction, enabling faster detection of security threats and remediation actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional parser matching methods are used for event logs, then security threats can be detected, but the processing speed and efficiency decrease as the number of assets increases
Solution Approach 1:
The patent segments the event log parsing process into distinct phases: tokenization of incoming event logs, identification of anchor tokens, and traversal through anchor token trees to select parsers. This segmentation allows the system to process only relevant portions of event logs using optimized data structures, thereby maintaining high detection reliability while improving processing throughput as assets scale
Solution Approach 2:
The system performs preliminary actions by pre-building anchor token trees from historical event log data before actual threat detection begins. These pre-computed trees enable rapid parser selection during runtime without requiring full parser matching against every new event log, thus maintaining reliable detection while significantly boosting events-per-second capacity
2Measurement precision
If comprehensive parser sets are generated to handle all event log patterns, then detection accuracy improves, but resource consumption and processing time increase
Solution Approach 1:
The patent extracts only the most discriminative tokens from event logs to form anchor tokens, which then serve as keys for rapid parser selection through pre-built trees. This extraction approach maintains high pattern matching accuracy by focusing on critical identifying features while eliminating unnecessary processing of redundant log components, thereby reducing parser selection time
Solution Approach 2:
The system applies local quality by creating specialized anchor token trees for different event log types and patterns rather than using a single universal parser matching process. Each tree is optimized for specific patterns encountered in particular asset types or threat scenarios, maintaining high detection precision while enabling parallel processing paths that reduce overall selection time
3Adaptability or versatility
If the number of assets in the enterprise system increases, then monitoring coverage improves, but the complexity of generating and selecting parsers increases
Solution Approach 1:
The patent implements universality through anchor token trees that serve multiple functions: they enable rapid parser selection, provide structured organization for diverse event log types, and facilitate scalable adaptation to new assets. The same anchor token tree structure can handle event logs from different asset types and security devices, allowing the system to expand monitoring coverage without proportionally increasing parser generation and selection complexity
Data Source
AI summary
A method includes obtaining an incoming event log, splitting the incoming event log into a set of tokens, and identifying a subset of the tokens as anchor tokens. The method also includes utilizing an ordered list of the anchor tokens to traverse through a set of anchor token trees and, responsive to identifying a path from (i) a root node of a given one of the anchor token trees to (ii) a given leaf node of the given anchor token tree corresponding to the ordered list of the anchor tokens, selecting a given parser associated with the given leaf node. The method further includes extracting data from the incoming event log utilizing the given parser, detecting one or more security threats affecting at least one asset in an enterprise system based on the extracted data, and applying at least one remediation action to mitigate the detected security threats.