Multivariate Bin Rareness Metric for Network Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods in communication networks are inadequate for high-dimensional network metadata due to complex multivariate relationships, irregular distributions, and the inability to handle mixed feature types, leading to insufficient detection of suspicious behavior.
Innovation Solution
A nonparametric approach using multivariate histograms with varying bin widths for density estimation, which groups features into multivariate bins, computes rareness metrics based on probability, and identifies anomalies based on these metrics, providing a pervasive view of network behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection methods are used, then the detection process is simple, but the detection accuracy is insufficient for high-dimensional network metadata
Solution Approach 1:
The patent segments the high-dimensional feature space into multiple bins along each dimension, creating a grid structure that divides complex multivariate relationships into manageable discrete cells. This segmentation enables accurate density estimation by counting samples in each bin while handling high-dimensional data through systematic partitioning.
Solution Approach 2:
The patent transforms the continuous high-dimensional feature space into a discrete multidimensional binning structure, adding the dimension of bin indices to represent sample distributions. This dimensional transformation allows the system to handle mixed feature types and irregular distributions by mapping continuous variables into discrete bin categories.
2Reliability
If density estimation techniques are applied, then anomaly detection capability improves, but the ability to handle mixed feature types and irregular distributions deteriorates
Solution Approach 1:
The patent changes the parameter representation by converting continuous feature values into discrete bin indices. This parameter transformation enables the density estimation to handle mixed feature types (categorical and continuous) and irregular distributions by operating on bin counts rather than raw feature values, thus improving both reliability and adaptability.
3Measurement precision
If multivariate bins are created for density estimation, then complex relationships are captured, but computational complexity increases
Solution Approach 1:
The patent segments the feature space into a grid of multivariate bins, where each bin represents a discrete cell in the multidimensional space. This segmentation captures complex multivariate relationships by counting samples in each bin independently, avoiding the need for complex continuous density calculations while maintaining accuracy through the structured partitioning of the feature space.
Data Source
AI summary
In one embodiment, a method includes receiving network data at an analytics device, grouping features of the network data into multivariate bins, generating a density for each of the multivariate bins, computing a rareness metric for each of the multivariate bins based on a probability of obtaining a feature in a bin and the probability for all other of the multivariate bins with equal or smaller density, and identifying anomalies based on computed rareness metrics. An apparatus and logic are also disclosed herein.