Network Security Event Detection via Normalized Distance Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for detecting anomalous network traffic struggle with identifying rare and evolving malicious patterns due to over-fitting with irrelevant features, leading to false alarms and failure to distinguish between benign and malicious traffic, especially in heterogeneous internet environments.
Innovation Solution
An encoder-decoder neural network is employed to generate reconstructed vectors from input aggregate vectors of DNS traffic records, calculating distances to identify anomalies, and normalized distance-based clustering is used to classify network traffic data, allowing for real-time detection and remedial actions without prior knowledge of attack types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional detection methods use numerous measurements and features to investigate anomalous traffic patterns, then the ability to detect potential attacks is improved, but over-fitting with irrelevant or redundant features occurs leading to excessive false alarms
Solution Approach 1:
The patent extracts only the most relevant features from the dataset by training an autoencoder neural network to learn a compressed representation of normal traffic patterns. The model automatically identifies and retains important features while discarding irrelevant or redundant ones, thereby reducing over-fitting and false alarms while maintaining detection accuracy.
Solution Approach 2:
The patent transforms the feature space by applying dimensionality reduction through the autoencoder's latent space. By changing the parameter representation from high-dimensional raw features to low-dimensional encoded features, the system achieves better generalization and reduces false positives while preserving the ability to detect anomalies.
2Ease of operation
If conventional methods reduce anomalous behavior to few parameters for human understanding, then the complexity of interpretation is reduced, but mitigation efforts may block benign traffic or permit malicious traffic
Solution Approach 1:
The patent introduces a new dimension of analysis by using the reconstruction error from the autoencoder as an anomaly score. This additional dimension allows the system to maintain detailed feature information for accurate classification while providing a simplified scalar metric for human interpretation and decision-making.
3Measurement precision
If fixed signatures are used for attack detection, then detection of known attack patterns is improved, but the system is easily circumvented by evolving attack patterns
Solution Approach 1:
The patent implements a dynamic detection system using an autoencoder that learns normal traffic patterns continuously. Unlike static signatures, the model adapts to evolving traffic characteristics and can detect previously unseen attack patterns by identifying deviations from learned normal behavior, providing both accuracy for known attacks and versatility for new threats.
Data Source
AI summary
A method may include a processing system assigning samples of network traffic data to positions in a list, where each of the samples is assigned a cluster identifier corresponding to the respective position, and traversing the list, where for each position, the processing system: increments an order indicator, and when the cluster identifier is not less than the order indicator, computes a distance between a sample assigned to the position and other samples, records a cluster identifier of another sample when a distance between the sample and the other sample is less than a threshold distance, and assigns a minimum cluster identifier that is recorded to all of the samples with cluster identifiers that are recorded. The processing system may determine clusters from cluster identifiers in the list after the traversing and identify at least one cluster as representing anomalous network traffic data.