Synthetic Traffic Training for Sparse Network Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network anomaly detection systems struggle to detect stealthy attacks and attacks that exploit new vulnerabilities due to reliance on heuristic rules and lack of knowledge about unknown threats.
Innovation Solution
A network anomaly detection system that generates synthetic benign and anomalous data, adjusts the density of anomalous data, and trains the anomaly detection model to improve detection of sparse anomalous patterns using a combination of real and synthetic data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If heuristic rules are used for anomaly detection, then detection of known attacks is improved, but detection of stealthy attacks and new vulnerabilities deteriorates
Solution Approach 1:
The patent transforms the detection approach by changing from fixed heuristic rules to dynamic machine learning models with adjustable parameters. The system uses synthetic data generation to create diverse training samples, allowing the model to adapt its detection parameters for both known and unknown attack patterns. This parameter transformation enables the system to maintain high accuracy for known attacks while gaining versatility against novel threats.
Solution Approach 2:
The patent applies preliminary action by generating synthetic training data before deployment. The system pre-generates diverse attack scenarios including stealthy attacks and new vulnerability exploits, then trains the anomaly detection model on this prepared dataset. This preliminary training enables the model to recognize patterns of unknown threats before they appear in real network traffic, resolving the contradiction between detecting known attacks and adapting to new threats.
2Measurement precision
If anomaly detection sensitivity is increased to detect sparse anomalous patterns, then detection capability for stealthy attacks is improved, but false positive rate increases
Solution Approach 1:
The patent applies partial action by using synthetic data to supplement rather than replace real training data. The system generates synthetic anomalous patterns at various densities and combines them with real network traffic data in controlled proportions. This partial use of synthetic data allows the model to learn sensitivity to sparse patterns while being grounded in real traffic characteristics, thereby improving detection capability without excessively increasing false positives.
Solution Approach 2:
The patent implements feedback mechanisms through iterative model training and evaluation. The system trains the anomaly detection model, evaluates its performance on both known attacks and normal traffic, then uses the results to refine synthetic data generation and model parameters. This feedback loop allows the system to optimize detection sensitivity while monitoring and controlling false positive rates, resolving the contradiction between detecting sparse patterns and maintaining reliability.
3Measurement precision
If more real anomalous network data is used for training, then model accuracy is improved, but data availability and privacy concerns worsen
Solution Approach 1:
The patent directly applies copying by generating synthetic copies of anomalous network traffic patterns. Instead of requiring additional real attack data, the system creates synthetic replicas of various attack types including stealthy attacks and exploits of new vulnerabilities. These synthetic copies preserve the essential characteristics of real attacks while eliminating privacy and data availability constraints, allowing extensive model training without needing more real-world compromised data.
Solution Approach 2:
The patent uses synthetic data as an intermediary between the need for training data and the constraints of data availability and privacy. The synthetic data generation system acts as a mediator that translates limited real attack samples into diverse training datasets. This intermediary approach allows the model to achieve high training accuracy without direct access to large quantities of real anomalous traffic, resolving the contradiction between model accuracy and data availability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method of training a network anomaly detection system is disclosed. The method involves generating (301) synthetic benign network data and synthetic anomalous network data and combining (302) the synthetic benign network data and synthetic anomalous network data to generate combined synthetic network data having a predetermined density of anomalous network data. The combined synthetic network data is provided (303) to a trained anomaly detection model, and an accuracy score is determined (304) that is representative of how accurately the trained anomaly detection model recognises anomalous activity in the combined synthetic network data. If the accuracy score is less than a threshold value, the anomaly detection model is trained (305) with additional network data and a new accuracy score is determined. Otherwise, the predetermined density of anomalous network data is reduced (306) and a new accuracy score is determined until a predetermined stopping criterion is met.