Anomaly Detection Using Network Traffic Feature Spaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection methods in computer networks face challenges such as the lack of ground truth data, defining normal behavior in high-dimensional spaces, dynamic network behaviors, and differentiating between noise and anomalies, especially in the context of Denial of Service (DoS) attacks like DDoS, which are difficult to distinguish due to their distributed nature.
Innovation Solution
A self-learning network (SLN) infrastructure that utilizes machine learning techniques to construct a model of normal network behavior, populating a feature space with network traffic metrics, and adjusting traffic routing to capture missing data sets, allowing for anomaly detection without active probing, using features constructed from network traffic data to infer non-observed properties and differentiate between normal and anomalous behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If active probing is used to collect network traffic data for anomaly detection, then measurement precision is improved, but device complexity and energy consumption increase
Solution Approach 1:
The network devices themselves generate and provide the traffic data needed for anomaly detection, eliminating the need for external probing systems. Normal network operations naturally produce the feature space data required by the machine learning model, so the system serves its own detection needs without additional complexity
Solution Approach 2:
The patent extracts anomaly detection capabilities from complex active probing systems and implements them using passive observation of naturally occurring network traffic. The machine learning model extracts meaningful features from existing traffic data without requiring active intervention or additional probing infrastructure
2Reliability
If comprehensive network traffic monitoring is implemented to detect DDoS attacks, then reliability is improved, but loss of time increases due to data processing overhead
Solution Approach 1:
The machine learning model is pre-trained offline with extensive network traffic data to learn normal behavior patterns and anomaly signatures. This preliminary training phase prepares the model in advance, so that during actual deployment, detection decisions can be made rapidly by comparing live traffic against the pre-established model without time-consuming analysis
Solution Approach 2:
The anomaly detection system is designed to adapt dynamically to changing network conditions while maintaining fast response times. The model can update its understanding of normal behavior patterns without requiring complete retraining, allowing it to respond to new attack vectors and network changes in real-time
3Measurement precision
If machine learning models are trained with complete feature spaces for all applications, then measurement precision is improved, but device complexity increases due to data collection requirements
Solution Approach 1:
The system accepts that not all applications will generate traffic at all times, and the machine learning model is designed to handle partial feature spaces gracefully. Rather than requiring complete data from all applications continuously, the model learns from the subset of data that is naturally available, using partial observations to build a sufficient representation for anomaly detection
Solution Approach 2:
The anomaly detection model is designed to be universally applicable across multiple applications and network conditions. A single trained model can detect anomalies in traffic from different applications using the same feature space representation, eliminating the need for separate collection and processing infrastructure for each application
Data Source
AI summary
In one embodiment, a device in a network receives traffic metrics for a plurality of applications in the network. The device populates a feature space for a machine learning-based anomaly detector. The device identifies a missing dataset in the feature space for a particular one of the plurality of applications. The device adjusts how traffic is sent in the network, to capture the missing dataset.