Deep Learning SIEM Event Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Security Information and Event Management (SIEM) systems face challenges in properly normalizing and categorizing log data due to their reliance on regular expressions, which fail to identify new or variant events, leading to a significant percentage of events being overlooked, potentially missing critical security incidents.
Innovation Solution
A method using a deep learning classification model, trained with parsed logs, to generate a vectorized log and classify event types, which includes cleaning, tokenizing, and padding the logs, and determining a confidence level to parse the event log effectively, even when it cannot be identified by the SIEM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If regular expressions are used for event normalization and categorization in SIEM systems, then processing speed and simplicity are improved, but the ability to identify new or variant events deteriorates, leading to 5-20% of events being overlooked
Solution Approach 1:
The patent replaces the mechanical pattern-matching system (regular expressions) with a neural network-based semantic analysis system. The neural network model learns semantic representations of event logs and can identify event types based on meaning rather than exact pattern matches, thereby resolving the contradiction between processing efficiency and adaptability to new event variants
Solution Approach 2:
The patent transforms the event log data from raw text format into vector representations (embeddings) that capture semantic meaning. By changing the parameter representation from discrete pattern strings to continuous vector spaces, the system gains the ability to generalize to new event types while maintaining efficient processing through learned similarity metrics
2Adaptability or versatility
If deep learning classification models are used to identify event types, then the ability to categorize unrecognized logs is improved, but the complexity of the system increases
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network model on a large corpus of labeled event logs before deployment. This offline training phase captures the complexity of event pattern recognition in advance, allowing the deployed system to make predictions with simpler inference computations, thus reducing operational complexity while maintaining high identification accuracy
Solution Approach 2:
The patent introduces an intermediary component (the neural network model) that sits between the raw event logs and the parsing system. This intermediary translates unstructured log data into structured event type predictions, managing the complexity of pattern recognition internally while presenting a simplified interface to the rest of the SIEM system
Data Source
AI summary
Embodiments of a method are disclosed. The method includes determining that the event type of an event log of a security information and event management (SIEM) cannot be identified. The method further includes generating a vectorized log using a cleaned, tokenized, and padded version of the event log. Additionally, the method includes generating a classification for the vectorized log using a deep learning classification model that is trained to identify a potential event type for the event log based on deep learning training using multiple parsed logs. The method also includes determining that a confidence level of the classification meets a predetermined threshold. The method further includes parsing the event log based on the classification.