Adaptive Ensemble Traffic Classification With Memory-Aware Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network traffic classification methods face challenges in achieving high classification performance while meeting resource demands, particularly in terms of memory overhead and processing time, especially with the increasing prevalence of encrypted traffic and varying network traffic volumes.
Innovation Solution
An adaptive ensemble classification framework (ACDC) that curates a pool of classifiers with varying feature requirements, using a heuristic-based feature exploration method to identify optimal feature combinations, and an adaptive scheduler to select the most suitable classifier and batch size based on current traffic rates and memory availability, balancing performance and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If representation learning-based methods are employed to classify traffic based directly from raw packet captures, then classification performance is improved, but processing time and memory overhead increase significantly
Solution Approach 1:
The system segments the classification task by maintaining a pool of multiple classifiers, each specialized for specific traffic types. This allows the system to process traffic in parallel using the most suitable classifier for each flow, improving overall throughput while maintaining high classification accuracy through specialized models rather than a single monolithic approach.
Solution Approach 2:
The system dynamically adjusts the number and type of classifiers activated based on current network traffic conditions and available computational resources. By changing the parameter of active classifier count and selecting appropriate batch sizes, the system optimizes the balance between classification performance and processing speed under varying load conditions.
2Measurement precision
If representation learning-based methods are employed to classify traffic based directly from raw packet captures, then classification performance is improved, but memory overhead increases significantly
Solution Approach 1:
Instead of loading a single large classifier model into memory, the system segments the classification functionality into multiple smaller specialized classifiers. Each classifier handles specific traffic types and can be loaded into memory as needed, reducing peak memory requirements while maintaining the capability to handle diverse traffic patterns with high accuracy.
Solution Approach 2:
The system dynamically manages memory by selectively loading and unloading classifiers based on current traffic composition and available memory resources. The adaptive scheduler monitors system state and adjusts which classifiers are active, allowing the system to maintain high classification performance while optimizing memory usage according to actual operational needs.
3Device complexity
If a single classifier is used for all traffic types, then device complexity is reduced, but adaptability to varying traffic patterns deteriorates
Solution Approach 1:
The system divides the classification functionality into multiple specialized classifiers, each designed for specific traffic types. This segmentation allows each individual classifier to be relatively simple while the collective system achieves high adaptability through the coordinated operation of specialized models that can handle diverse traffic patterns effectively.
Solution Approach 2:
The system implements a universal classifier management framework that can dynamically select and activate appropriate classifiers based on traffic type. This multi-functional approach allows a single system to handle multiple traffic patterns and protocols by deploying the necessary specialized classifiers, achieving both structural simplicity in individual components and high versatility in overall system capability.
Data Source
AI summary
An embodiment may involve determining a network traffic rate for a link; determining an amount of available memory for classification in a computing system operationally coupled to the link; selecting a classifier from a plurality of classifiers, wherein the classifiers are respectively associated with a time usage and a memory usage, wherein the classifiers were trained to predict network traffic types based on network traffic flows, and wherein the selection is based on: the network traffic rate, an amount of available memory, and the time usage and the memory usage of the classifier; and deploying the classifier to receive incoming network traffic flows by way of the link.