Anomaly Detection Using Binary Clusters and Incremental Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current user behavior analytics (UBA) systems face challenges in accurately evaluating machine learning-based anomaly detectors due to resource inefficiencies and limitations in tracking users and events, leading to costly system maintenance and laborious traditional clustering methods.
Innovation Solution
A computer-implemented method that creates binary clusters of high-dimensional datapoints using a rule engine and a trained multiclass classifier, reducing system resources by incremental learning and employing fuzzy labels, allowing for accurate anomaly detection with robust security orchestration and automation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional clustering methods are used to evaluate anomaly detectors, then detection accuracy can be achieved, but system resource consumption increases and maintenance costs rise
Solution Approach 1:
The patent creates binary clusters as simplified copies of complex high-dimensional data groups. Instead of performing computationally intensive traditional clustering on all data, the system creates compact binary cluster representations that capture essential patterns while consuming minimal resources during anomaly detection operations
Solution Approach 2:
The patent transforms high-dimensional datapoints into binary representations, fundamentally changing the parameter space from continuous high-dimensional values to discrete binary states. This parameter transformation enables efficient comparison and clustering operations that maintain detection accuracy while dramatically reducing computational resource requirements
2Reliability
If the number of users and events tracked is increased to improve detection coverage, then anomaly detection capability improves, but system resource requirements increase
Solution Approach 1:
The patent segments the large dataset of high-dimensional datapoints into multiple binary clusters, each representing a distinct group of user behaviors or event patterns. This segmentation allows the system to track numerous users and events by organizing them into manageable cluster groups, maintaining comprehensive detection coverage while using resources efficiently through cluster-based processing rather than individual data point processing
3Productivity
If more system resources are allocated to UBA systems, then tracking and evaluation capabilities improve, but maintenance costs and system complexity increase
Solution Approach 1:
The patent replaces complex traditional clustering algorithms with a binary cluster-based system. Instead of using resource-intensive mechanical computing approaches for data grouping and analysis, the system uses simplified binary representations and comparisons, reducing both computational complexity and maintenance requirements while maintaining or improving tracking and evaluation capabilities
Data Source
AI summary
One or more computer processors create a binary cluster of events by bootstrapping a set of ground truths contained with a rule engine applied to a set of high-dimensional datapoints, wherein the binary cluster contains two clusters each containing a plurality of high-dimensional datapoints; determine one or more peer groups for a set of unknown high-dimensional datapoints utilizing a trained multiclass classifier, wherein the high-dimensional datapoints are assigned to one or more peer groups by the trained multiclass classifier using an incremental learning algorithm in order to reduce system resources; create an activity distribution for each unknown high-dimensional datapoint associated with a user in the set of unknown high-dimensional datapoints and each peer group; calculate a deviation percentage between the activity distribution of the user and each peer group associated with the user; and responsive to exceeding a deviation threshold, classify the user or associated high-dimensional datapoints as risky.