Stateful Rules Engine for Real-Time Endpoint Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing security technologies for end points are limited in detecting correlated security breaches in real time, as they typically analyze log data in batch mode, leading to delays in detecting malicious behavior.
Innovation Solution
A system and method for analyzing log data in real time using stateful rules, where events are streamed from end points, enriched with process information, and analyzed using an Event Query Language (EQL)-based rules engine to detect sequences of events, enabling immediate action upon malicious activity detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If batch mode analysis of log data is used, then system complexity is reduced, but detection speed and real-time capability deteriorate
Solution Approach 1:
The system dynamically adapts its analysis approach by implementing both batch processing for historical data and real-time streaming processing for current events. The rules engine continuously evaluates new events as they arrive, transitioning from static batch analysis to dynamic real-time analysis, thereby improving detection speed while maintaining manageable system complexity through phased implementation.
Solution Approach 2:
The analysis system is segmented into multiple independent components: batch processing modules for historical data, streaming processing modules for real-time events, and a rules engine that can operate in both modes. This segmentation allows each component to be optimized independently, enabling real-time detection capabilities without overwhelming system complexity.
2Measurement precision
If all events are stored in memory for analysis, then detection accuracy is improved, but memory usage increases
Solution Approach 1:
The system extracts only the essential fields and characteristics needed for detection from complete event data. Instead of storing all event details in memory, it extracts key parameters such as event type, source, destination, and timestamp, maintaining detection accuracy while significantly reducing memory consumption through selective data retention.
Solution Approach 2:
Different storage strategies are applied to different types of data based on their importance and access patterns. Frequently accessed event patterns and rules are kept in fast memory, while less critical historical data is stored in slower storage media. This local quality differentiation optimizes memory usage by allocating high-speed memory only where absolutely necessary for real-time detection.
3Ease of operation
If simple rule-based detection is used, then ease of operation is improved, but ability to detect correlated events deteriorates
Solution Approach 1:
The detection system implements nested rules where complex detection patterns are built by combining simpler rules. Basic event detection rules are nested within sequence rules, which are in turn nested within correlation patterns. This nested structure allows the system to detect complex correlated events while maintaining the simplicity of individual rule definitions, preserving ease of operation while enhancing detection capability.
Solution Approach 2:
The rules engine is designed with universal functionality that can handle both simple single-event detection and complex multi-event correlation using the same rule framework. By making the rule system multi-functional, it can operate effectively whether detecting isolated events or sophisticated attack sequences, maintaining ease of operation across different detection scenarios while improving overall reliability.
Data Source
AI summary
Provided are methods and systems for preventing malicious behavior of an end point. An example method commences with monitoring a stream of events associated with the end point. The method further includes processing the stream to record a set of events to a memory. Processing an event of the stream includes determining that the event satisfies at least one rule in a sequence of rules and, in response to the determination, adding the event to the set of events in the memory. The method further includes determining that the set of events includes a sequence of events. Each state in the sequence of events corresponds to at least one rule in the sequence of rules. The method continues with executing at least one action on the end point in response to the determination that the set of events includes the sequence of events.