Cloud User Activity Modeling for Scalable Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern User and Entity Behavior Analytics (UEBA) solutions face challenges with unwieldy models due to incorporating multiple aspects of user behavior, leading to increased training complexity and compute costs, limiting their applicability, especially when handling large datasets with diverse user actions.
Innovation Solution
A two-stage activity characterization system using escort clustering and latent Dirichlet allocation (LDA) to automatically determine the number of behavioral groups and activities, enabling efficient handling of large datasets and identifying anomalous user behavior through group-level and user-level modeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If progressively more aspects of user behavior are incorporated into a single UEBA model, then the comprehensiveness of behavioral modeling improves, but the model becomes unwieldy and training complexity increases quadratically or cubically
Solution Approach 1:
The patent divides the comprehensive behavioral modeling task into multiple specialized models, each focusing on specific aspects of user behavior (e.g., authentication patterns, file access, application usage). This segmentation allows each model to remain manageable in size while collectively providing comprehensive coverage of user behavior analytics.
Solution Approach 2:
The patent introduces an intermediary layer that coordinates multiple specialized behavioral models. This intermediary manages the integration of insights from various models without requiring a single monolithic model, thereby maintaining tractability while achieving comprehensive behavioral analysis.
2Measurement precision
If larger models with more input features are used to achieve sufficient sensitivity and confidence, then detection accuracy improves, but compute cost increases quadratically, cubically, or higher
Solution Approach 1:
The patent segments the feature set into multiple specialized models, each processing a subset of features relevant to specific behavioral aspects. This approach maintains detection sensitivity by ensuring each model has sufficient features for its domain while avoiding the cubic complexity increase of a single large model processing all features.
Solution Approach 2:
Each specialized model is tailored with locally optimized features and parameters specific to its behavioral domain, achieving high detection confidence for that particular aspect without requiring all features to be processed at full complexity across all models.
3Adaptability or versatility
If a single comprehensive model is used to model all user behavior aspects, then behavioral analysis coverage improves, but the model becomes difficult to train on realistic data
Solution Approach 1:
The patent segments the training process into multiple smaller training tasks, each focused on a specific behavioral aspect with relevant data subsets. This makes training more manageable and achievable with realistic data volumes, while the collection of specialized models provides comprehensive behavioral analysis coverage.
Data Source
AI summary
Systems, methods, and other embodiments associated with self-reliant characterization of activities of users are described. In one embodiment, a method includes generating a dataset of data points from a batch of electronic log messages that describe electronic actions taken by various accounts. A data point collectively describes actions of a single account. The method includes modeling distinct activities based on clustering of the data points into M behavioral groups and inferring M or more distinct activities from the dataset by probabilistic activity modeling of the actions. The value of M is derived automatically during the clustering. The method includes predicting activity of a user account to be non-conformant based on other accounts in a behavioral group satisfying a threshold for similarity. And, the method includes generating an electronic alert that indicates the user account to have non-conformant activity.