Masked Log Anomaly Detection for Evasive Cyber-Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud systems are vulnerable to cyber-attacks, requiring a detection system that can handle large volumes of events efficiently, adapt to changes, and minimize false alarms without relying on extensive historical data or being easily bypassed by attackers.
Innovation Solution
A self-supervised machine learning model that generates probability distributions for masked attributes in event logs, allowing rapid detection of anomalous events without explicit historical data lookup, and is resistant to evasion by attackers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional detection systems use extensive historical data and explicit lookup tables, then they can detect known attack patterns, but they become easily bypassed by attackers and require large storage resources
Solution Approach 1:
The patent replaces traditional mechanical detection systems (rule-based engines, signature matching, explicit lookup tables) with a neural network-based machine learning system. The neural network learns attack patterns implicitly from training data rather than relying on pre-programmed rules, making the system adaptable to new threats while maintaining detection accuracy for known attacks.
Solution Approach 2:
The patent transforms the detection approach by changing from static rule-based parameters to dynamic learned representations. The neural network processes event logs through multiple layers, transforming raw event data into abstract feature representations that capture complex attack patterns without requiring explicit rule definitions.
2Productivity
If the system processes large volumes of events in real-time, then it can detect attacks quickly, but it increases computational complexity and processing time
Solution Approach 1:
The patent performs preliminary action by pre-training the neural network offline on extensive training data containing both normal and malicious event patterns. This pre-training phase computes and stores learned weights and biases that encapsulate complex detection logic, allowing the system to perform rapid real-time inference without repeating complex computations during actual event processing.
Solution Approach 2:
The patent segments the detection system into distinct components: an offline training phase that learns from historical data, and an online inference phase that processes events in real-time. The neural network is divided into multiple layers (input layer, hidden layers, output layer) that process information in stages, allowing parallel computation and efficient real-time processing.
3Adaptability or versatility
If the detection system adapts to changes in user behavior, then it reduces false alarms, but it may become less reliable for detecting known attack patterns
Solution Approach 1:
The patent implements feedback mechanisms where the neural network's detection results and performance metrics are used to refine and retrain the model. The system continuously learns from new data, adjusting its decision boundaries to distinguish between legitimate user behavior changes and actual attacks, thereby reducing false alarms while maintaining detection reliability through iterative improvement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of detecting anomalous events indicative of malicious activity is described. The method comprises receiving a log of an event comprising a plurality of values, the plurality of values comprising known values corresponding to each of a plurality of attributes of the event and generating a masked log by masking a value in the received log, the masked value corresponding to one of the attributes. The method further comprises, based on the masked log and a trained machine learning model, generating a distribution of probabilities for possible values of the masked value, wherein the trained machine learning model is based on a plurality of masked logs of events, and determining that the event is an anomalous event based on a comparison of the known value of the masked value and the distribution of probabilities.