Event Stream Correlation Using Join Keys for Threat Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cybersecurity threat detection systems face challenges in accurately correlating events due to non-unique join keys, leading to false positives, data volume management, data diversity, time sensitivity, and lack of event context, which complicates the identification of cyber threats.
Innovation Solution
The method involves identifying common join keys among cybersecurity events using a timeframe, event field value cardinality, entropy, and correlation confidence score to correlate events effectively, reducing false positives and improving threat detection efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional event correlation methods are used with non-unique join keys, then event correlation can be performed, but false positives increase and detection accuracy decreases
Solution Approach 1:
The patent changes the parameters for selecting join keys by using cardinality and entropy thresholds to filter and select unique join keys from event fields. This parameter-based filtering ensures that only fields with sufficient uniqueness and information content are used for correlation, thereby reducing false positives while maintaining detection accuracy.
2Productivity
If more event fields are considered for correlation, then correlation completeness improves, but data volume and processing complexity increase
Solution Approach 1:
The patent extracts and selects only the most relevant event fields for correlation by applying cardinality and entropy threshold filters. This extraction process removes unnecessary fields with low uniqueness or low information content, reducing the data volume processed while maintaining correlation efficiency through the use of only essential join keys.
3Ease of operation
If join key selection is performed manually, then control over correlation is maintained, but time consumption and operational complexity increase
Solution Approach 1:
The patent implements self-service join key selection where the system automatically identifies and selects appropriate join keys based on predefined cardinality and entropy thresholds. This automated self-service approach eliminates manual configuration requirements, reducing operational complexity and time consumption while maintaining ease of use through transparent, rule-based selection.
Data Source
AI summary
Automatically identifying join keys to correlate events is provided. A group of fields of a plurality of fields corresponding to a plurality of events having a percentage of cardinality and entropy of values greater than a defined threshold percentage level for the cardinality and the entropy is identified as a group of join keys corresponding to the plurality of events that occurred within a defined timeframe. It is determined which particular join keys of the group of join keys corresponding to the plurality of events that occurred within the defined timeframe are common join keys among the plurality of events. Pairs of events in the plurality of events that occurred within the defined timeframe are correlated based on the common join keys between each pair of events to form a set of correlated event pairs for detecting threats.