Anomaly Detection via Multi-Level Tolerance Relations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in identifying and analyzing sequences of events, particularly in large or complex data sets, as they often require numerical features and struggle with human behavior patterns that deviate from statistical distributions, leading to false positives and delayed detection of new behavior patterns, such as network intrusion events.
Innovation Solution
A method using a minimal directed acyclic graph (DAG) data structure to represent event sequences, defining categories of behavior, and calculating a Jaccard similarity coefficient to cluster entities based on their association with these categories, allowing for efficient filtering and prediction of future events.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical methods or machine learning approaches are used to model event information, then numerical features can be processed, but they require significant numbers of examples to form meaningful models and struggle to detect new behavior patterns quickly
Solution Approach 1:
The patent pre-processes event sequences during an initialization phase to build a comprehensive event model that captures all observed event types and their relationships. This preliminary action stores the complete event space structure, enabling rapid detection of new behavior patterns without requiring additional training time when anomalies occur.
Solution Approach 2:
The patent segments the event detection process into distinct phases: initialization (building the event model from historical data) and detection (comparing new events against the pre-built model). This segmentation allows the system to accumulate data efficiently during initialization and then perform rapid anomaly detection by simply comparing new events against the established model, avoiding the need for continuous retraining.
2Loss of information
If detailed analysis and comparison of individual events is performed to determine relationships, then relationship identification can be achieved, but false positive determinations increase
Solution Approach 1:
The patent creates a universal event model that serves multiple functions: it captures all event types, defines their relationships, and provides a framework for comparing any new event sequence. This multi-functional model reduces false positives by providing consistent criteria for relationship determination across all events, rather than applying ad-hoc analysis to each individual event comparison.
Solution Approach 2:
The patent transforms individual event attributes into a standardized event model structure during initialization, changing the parameters from raw event data to structured relationships and patterns. This parameter transformation enables reliable comparison by converting diverse event types into a common framework, reducing false positives caused by inconsistent analysis criteria.
3Quantity of substance
If vast quantities of data are stored and processed at greater rates with greater complexity, then more comprehensive event information is available, but the big data problem of storing, handling, processing or using the data arises
Solution Approach 1:
The patent extracts only the essential structural elements from vast quantities of event data during initialization, building a compact event model that captures the relationships and patterns without storing all raw data. This extraction reduces processing complexity by working with the distilled model structure rather than the complete voluminous event dataset during detection operations.
Solution Approach 2:
The patent performs the complex data processing and relationship extraction in advance during the initialization phase, building the event model structure before detection is needed. This preliminary action consolidates the complexity of handling vast data quantities into a pre-computed model, allowing subsequent detection operations to proceed efficiently with reduced processing requirements.
Data Source
AI summary
A method for partitioning a plurality of entities each associated with a plurality of ordered sequences of events received by a computer system, the method including: defining a minimal directed acyclic graph data structure representing the sequences of events to define a plurality of categories of behavior of the entities; defining a threshold degree of similarity as an xmu number, the xmu number having cardinality that is able to vary across a normalized range; defining a relation for each entity including a degree of association of the entity with each of the categories; defining a cluster of entities as a set of entities comprising a first entity; comparing a relation for the first entity with a relation for a second entity to define a xmu Jaccard similarity coefficient for the first and second entities; and responsive to the coefficient meeting the threshold degree of similarity, adding the second entity to the cluster.