Event Log Graph Clustering for Efficient Threat Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing large volumes of event data from digital files and executable software results in storage challenges and inefficiencies in anomaly or suspicious activity detection.
Innovation Solution
A method to cluster event logs into categories using graph data structures, normalizing event records, and generating event clusters based on common attributes and hierarchical relationships, allowing for data compression and efficient storage and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large volumes of event data are stored and managed, then complete event records are preserved for analysis, but storage costs increase and data analysis efficiency decreases
Solution Approach 1:
The patent merges multiple event records that share common attributes (such as same process ID, parent process ID, or file path patterns) into single clustered event records. This consolidation reduces the total volume of stored data while preserving the essential information needed for security analysis, directly addressing the contradiction between data completeness and storage volume.
Solution Approach 2:
The clustered event records serve multiple functions: they maintain the integrity needed for anomaly detection, reduce storage requirements, and enable efficient querying. By creating a universal data structure that can fulfill multiple requirements simultaneously, the patent resolves the trade-off between preserving complete event information and reducing storage costs.
2Reliability
If large volumes of event data are stored, then comprehensive analysis is possible, but computing resources and analysis time increase
Solution Approach 1:
The patent segments the large volume of event data into meaningful clusters based on common attributes and hierarchical relationships. This segmentation organizes the data into manageable groups that can be analyzed more efficiently while maintaining the contextual information necessary for accurate anomaly detection, thus resolving the contradiction between detection accuracy and analysis efficiency.
Solution Approach 2:
The patent introduces a new dimensional organization of event data by creating hierarchical clusters that group events based on multiple attributes simultaneously (process relationships, file paths, timestamps). This dimensional transformation allows analysts to view and analyze data patterns more efficiently without losing the detail needed for accurate threat detection.
3Quantity of substance
If event data is normalized and clustered into categories, then storage efficiency improves, but data processing complexity increases
Solution Approach 1:
The patent performs normalization and clustering operations as preliminary actions during the event data ingestion phase, before the data needs to be stored or analyzed. By pre-processing the data into normalized, clustered formats, the system reduces the complexity of subsequent operations while achieving efficient storage, thus resolving the contradiction between storage efficiency and processing complexity.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing event log data. An example event log processing method includes receiving an event log comprising a plurality of event records describing events that have occurred on each of one or more computer systems over a period of time; converting the event log into a graph, comprising: normalizing the plurality of event records, including anonymizing a unique identifier value in each event record and replacing a variable value in each event record with a predetermined value; representing each normalized event record as one or more nodes in the graph; and generating a plurality of event clusters, wherein each event cluster includes an aggregated group of nodes and is generated based on common attributes of and hierarchical relationships between the normalized event records represented by the nodes in the aggregated group.