Network Traffic Analysis via Concept Drift Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network traffic analysis methods face challenges in real-time data processing and accuracy due to concept drift and dynamic changes in network environments, leading to lag in data and model transmission and deployment.
Innovation Solution
A network traffic analysis method and device that deploy a classifier pool with trained online classifiers, perform feature processing and transformation, and use a concept drift detection module to update classifiers in real-time based on detected drifts, utilizing Kullback-Leibler divergence and Jensen-Shannon divergence calculations to determine drift and adjust classifiers accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data collection and processing methods are used, then data can be collected and analyzed, but the analysis has strong lag and cannot process network traffic in real time
Solution Approach 1:
The patent pre-deploys multiple online classifiers in a classifier pool before data arrives. When data comes in, the pre-deployed classifiers can immediately process it without waiting for model training and deployment, thus eliminating the time lag in data transmission and deployment while maintaining real-time processing capability
Solution Approach 2:
The patent implements a dynamic classifier pool where classifiers can be added, removed, or updated based on traffic patterns and concept drift detection. This dynamic structure allows the system to adapt to changing network conditions in real-time, improving both responsiveness and processing efficiency
2Measurement precision
If fixed classifiers are deployed, then initial classification accuracy can be achieved, but accuracy deteriorates when concept drift occurs in dynamic network environments
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors classification performance and detects concept drift. When drift is detected, the system triggers classifier updates or replacements from the pool, creating a closed-loop feedback system that maintains high accuracy despite changing network conditions
Solution Approach 2:
The patent changes the parameter of classifier adaptability by implementing a pool of diverse classifiers with different characteristics. When concept drift occurs, the system can switch to classifiers better suited for the new conditions, effectively changing the operational parameters to maintain accuracy in dynamic environments
3Adaptability or versatility
If a single classifier is used, then device complexity is low, but the system cannot handle multi-source network traffic data effectively
Solution Approach 1:
The patent segments the classification task by dividing it into multiple specialized classifiers, each potentially optimized for different types of network traffic or data sources. This segmentation allows the system to handle diverse multi-source traffic effectively while keeping each individual classifier relatively simple
Solution Approach 2:
The classifier pool serves multiple functions: it provides initial classification, adapts to concept drift, and handles different data sources. This multi-functionality allows a single pool structure to address various requirements without proportionally increasing complexity
Data Source
AI summary
Disclosed are a network traffic analysis method and a device based on multi-source network traffic data. The method includes: deploying a pre-training classifier pool in a network stream data source; receiving multi-source network stream data at a current moment for each data source, classifying the multi-source network stream data through an online classifier, performing feature processing and transformation on data collected by each network stream data source at each preset time interval, and transmitting processed traffic data features and a feature transformation matrix to a traffic drift detection module. The traffic drift detection module contains historical concept data to detect a concept drift according to the traffic data features, the feature transformation matrix and the historical concept data; if the concept drift is detected, the online classifier deployed by multiple sources is reset. This method is used for continuous real-time and accurate analysis of the multi-source network traffic data.