Network Threat Detection via Traffic Anomaly Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional network administration systems rely on knowledge of known bad endpoints and command and control structures to identify threats, which is ineffective against evolving botnets and malware, as they require direct knowledge of these elements to detect malicious activity.
Innovation Solution
A method that accumulates traffic data, groups it by keys, calculates metric values, fits a mixture distribution to each set, and identifies outliers based on anomalous behavior patterns, allowing for the detection of potential threats without relying on pre-known malicious endpoints or structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional threat management systems rely on known bad endpoints and command and control structures, then detection accuracy for known threats is improved, but the system becomes ineffective against evolving botnets and malware
Solution Approach 1:
Instead of identifying threats by matching known malicious patterns (traditional approach), the system inverts the problem by identifying normal behavior patterns and flagging deviations as potential threats. This allows detection of unknown and evolving malware without requiring prior knowledge of specific threat signatures.
Solution Approach 2:
The system enables endpoints to effectively monitor themselves by comparing their communication behavior against the modeled normal patterns. Endpoints that deviate from expected behavior automatically flag themselves for investigation, eliminating the need for external knowledge bases of known threats.
2Ease of operation
If threat management systems use known threat patterns and command and control structures, then identification of malicious activity is simplified, but the system cannot detect novel botnets without direct knowledge
Solution Approach 1:
The system performs preliminary modeling of normal communication behavior patterns during a baseline period before threat detection begins. This pre-established behavioral model serves as the foundation for automatic anomaly detection, enabling the system to identify threats without requiring prior knowledge of specific malicious patterns.
Solution Approach 2:
The system replaces the mechanical approach of manually updating threat signatures and command and control lists with an automated statistical modeling system that continuously learns normal behavior patterns and automatically detects deviations, eliminating the need for direct knowledge of specific threats.
3Measurement precision
If manual analysis of endpoint behavior is performed, then accurate identification of threats is achieved, but the process is time-consuming and cannot scale to large networks
Solution Approach 1:
The system replaces manual analysis with automated statistical modeling and anomaly detection algorithms. The mixture distribution models automatically analyze communication patterns and identify outliers, achieving both high accuracy and scalability that manual processes cannot match.
Solution Approach 2:
The system transforms the analysis from examining individual communication events to analyzing aggregated statistical parameters (mean, variance, skewness, kurtosis) of communication patterns. This parameter transformation enables efficient automated detection while maintaining high accuracy in identifying anomalous behavior.
Data Source
AI summary
Methods, systems, and computer-readable media for identifying potential threats on a network based on anomalous behavior in communication between endpoints are provided. Traffic data for a network is accumulated over some period of time. The traffic data is grouped by one or more keys, such as source IP address, and sets of metric values are calculated for the keys. A mixture distribution, such as a negative binomial mixture distribution, is fitted to each set of metric values, and outlying metric values are determined based on the mixture distribution(s). A list of outliers is then generated comprising key values having outlying metric values in one or more of the sets of metric values.