Compressed Network Telemetry for ML Traffic Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise networks face challenges in optimizing network performance due to the similarity in protocols used by business and non-business critical traffic, and the presence of malicious traffic such as DoS attacks and malware, which complicates traffic classification and optimization.
Innovation Solution
A device in the network receives telemetry data, compresses features individually, and uses a machine learning-based classifier to classify traffic flows by looking up classifier inputs from an index, enabling efficient classification of traffic without losing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional network traffic classification methods are used, then network performance optimization can be achieved for known applications, but the system cannot accurately distinguish between business and non-business critical traffic flows that use the same protocols
Solution Approach 1:
The patent introduces compressed network telemetry data as an intermediary representation between raw network traffic and machine learning classifiers. This compressed telemetry data serves as a mediator that captures essential traffic characteristics while reducing complexity, enabling accurate classification without requiring direct analysis of complex raw traffic streams.
Solution Approach 2:
The patent transforms network traffic parameters into compressed telemetry data representations. By changing the parameter format from raw network flow data to compressed telemetry features, the system achieves better classification accuracy while reducing the complexity of processing and storage requirements.
2Measurement precision
If comprehensive network telemetry data is collected for accurate traffic analysis, then traffic classification accuracy improves, but data volume and processing requirements increase significantly
Solution Approach 1:
The patent extracts and selects only the essential features from comprehensive network telemetry data. By taking out only the necessary compressed features for classification rather than processing all raw data, the system maintains analysis accuracy while significantly reducing data volume and processing requirements.
Solution Approach 2:
The patent segments network telemetry data into individual compressed features that can be processed independently. This segmentation allows the system to handle data in manageable units, reducing overall data volume while preserving the information needed for accurate traffic classification.
3Measurement precision
If machine learning classifiers are trained on detailed traffic features, then classification accuracy improves, but training time and computational resources increase
Solution Approach 1:
The patent performs preliminary compression of network telemetry data into feature representations before training the machine learning classifier. This preliminary action reduces the amount and complexity of data that needs to be processed during training, thereby reducing training time while maintaining classification accuracy.
Solution Approach 2:
The patent creates compressed copies of network telemetry data as simplified representations for training purposes. These compressed feature copies serve as substitutes for the full raw data during training, reducing computational resource requirements and training time while preserving the essential patterns needed for accurate classification.
Data Source
AI summary
In one embodiment, a device in a network receives telemetry data regarding a traffic flow in the network. One or more features in the telemetry data are individually compressed. The device extracts the one or more individually compressed features from the received telemetry data. The device performs a lookup of one or more classifier inputs from an index of classifier inputs using the one or more individually compressed features from the received telemetry data. The device classifies the traffic flow by inputting the one or more classifier inputs to a machine learning-based classifier.