Ensemble Model Splitting for Distributed Network Traffic Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic classification methods are centralized, leading to challenges in deploying high-precision large-scale models due to limited network resources, resulting in wasted resources on processing already classified traffic and suboptimal classification accuracy.
Innovation Solution
The method involves splitting an initial ensemble model into multiple classification paths, enhancing these paths, and deploying them across multiple switches based on an allocation scheme determined in both offline and online phases, using a combination of genetic algorithms and deep reinforcement learning to optimize resource usage and classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a complete but cumbersome model is deployed on a single switch, then classification accuracy can be maintained, but network device resources are excessively consumed and the switch cannot support high-precision large-scale models
Solution Approach 1:
The patent divides the complete classification model into multiple sub-models and distributes them across different switches in the network. Each switch runs a lightweight sub-model rather than a complete model, reducing the computational burden and resource consumption on individual switches while maintaining overall classification accuracy through coordinated decision-making among multiple switches.
2Adaptability or versatility
If the same model is repeatedly deployed on multiple switches to cover all traffic, then full-network classification coverage is achieved, but network device resources are wasted on processing traffic that has already been classified
Solution Approach 1:
The patent assigns different specialized sub-models to different switches based on their specific roles and traffic patterns. Each switch's sub-model is optimized for local classification needs rather than running identical complete models everywhere. This local optimization reduces redundant processing of already-classified traffic while maintaining comprehensive network-wide coverage.
Solution Approach 2:
The system dynamically adjusts model deployment and traffic routing based on real-time network conditions and classification results. Traffic that has been classified by one switch can be redirected or marked to avoid reprocessing by subsequent switches, making the classification process adaptive and efficient across the dynamic network environment.
3Ease of operation
If a centralized classification approach is used, then implementation is simple, but it cannot improve classification accuracy in scenarios with limited network resources
Solution Approach 1:
The patent introduces a centralized controller or coordination mechanism that manages the distributed sub-models across switches. This intermediary component handles model distribution, updates, and coordination without requiring complex peer-to-peer communication between switches, thus maintaining ease of operation while enabling improved classification accuracy through distributed processing.
Data Source
AI summary
Disclosed are a method, an apparatus and a device for distributively classifying network traffic, and a storage medium. The method includes: splitting an initial ensemble model into a plurality of classification paths, and reorganizing and enhancing the initial ensemble model based on the plurality of classification paths to obtain a plurality of enhanced base models, the initial ensemble model is configured to perform classification on the network traffic; determining an enhanced base model allocation scheme through an allocation scheme in the offline phase and an allocation scheme in the online phase; and deploying the plurality of enhanced base models to a plurality of switches based on the enhanced base model allocation scheme, and classifying the network traffic through the plurality of switches to obtain a classification result.