Latent Factor Model for Suspicious Network Traffic Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network threat detection methods are narrowly focused and fail to adapt to new or changing threats, often missing 'low and slow' attacks and suffering from high false positive and false negative rates due to reliance on rules-based or supervised learning approaches.
Innovation Solution
The use of latent factor models on network data combined with heuristics and user feedback to identify anomalous activity, employing a machine learning process that learns normal behavior and assesses the likelihood of new connections, allowing for the identification of suspicious network traffic without prior labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rules-based approaches are used for network threat detection, then specific types of attacks can be identified, but low and slow attacks are missed and the system cannot adapt to new threats
Solution Approach 1:
The patent transforms network traffic analysis from rule-based detection to statistical parameter analysis. Instead of checking against fixed rules, the system calculates statistical parameters (mean, variance, skewness, kurtosis) of network traffic flows and compares them against learned baselines. This allows the system to detect both known attack patterns and novel anomalies by identifying deviations in statistical parameters, thereby simultaneously improving detection accuracy and adaptability to new threats.
Solution Approach 2:
The patent replaces the mechanical rule-based detection system with a statistical learning system. Rather than manually configured rules that require updates for new threats, the system uses statistical models that automatically adapt to new attack patterns by learning from historical traffic data. This substitution enables the system to maintain high detection accuracy while gaining versatility against evolving threats.
2Reliability
If supervised learning approaches are used, then models can be trained on labeled data, but false positive and false negative rates remain high due to lack of labeled examples
Solution Approach 1:
The patent inverts the traditional supervised learning approach by using unsupervised statistical analysis. Instead of requiring labeled attack data to train models, the system learns normal traffic patterns statistically and identifies anomalies as deviations from these patterns. This inversion eliminates the dependency on scarce labeled examples, significantly reducing both false positives and false negatives while maintaining high detection reliability.
Solution Approach 2:
The system performs self-service by automatically learning traffic patterns and detecting anomalies without requiring manual labeling or external training data. The statistical models continuously adapt to new normal patterns in the network environment, enabling the system to maintain high reliability and precision autonomously without human intervention for data labeling or model retraining.
3Ease of manufacture
If narrow rule-based detection is used, then implementation is simple, but the system fails to detect evolving and diverse network threats
Solution Approach 1:
The patent implements a universal statistical detection framework that can identify multiple types of network threats using the same core methodology. The system calculates statistical parameters for all network flows and applies anomaly detection algorithms uniformly across different traffic types. This multi-functional approach enables simple implementation of a single system that can detect various attack patterns (port scans, data exfiltration, command and control, etc.) without requiring separate rules for each threat type, thereby achieving both simplicity and broad detection coverage.
Data Source
AI summary
Apparatuses, methods and storage medium associated with techniques to identify suspicious network connections. In embodiments, an apparatus may include an analysis function to be operated by the one or more processors to receive a collection of network data records, and apply a latent factor model to the network data records to identify a subset of the network data records as suspicious network connections. Other embodiments may be disclosed or claimed.