Fully Segmented Decision Tree for Network Traffic Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network traffic classification methods face challenges in accurately identifying malicious traffic flows due to the bias-variance tradeoff in decision tree machine learning algorithms, which often fail to capture both large-scale and small-scale patterns effectively, leading to reduced classification accuracy.
Innovation Solution
The approach involves generating a fully segmented decision tree ruleset by adjusting hyperparameters to prioritize capturing small-scale, idiosyncratic patterns, and modifying the training dataset to remove troublesome items if necessary, allowing for more accurate classification of network traffic flows as benign or non-benign.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If decision tree ML algorithms are used to classify network traffic flows, then classification automation is improved, but classification accuracy deteriorates due to inability to capture both large-scale and small-scale patterns
Solution Approach 1:
The patent divides the classification task into two distinct stages: a fast classification stage using a simplified decision tree for large-scale patterns, and a secondary classification stage using a more complex model for small-scale patterns. This segmentation allows the system to maintain both automation and accuracy by handling different pattern scales separately rather than attempting to capture both in a single model.
2Measurement precision
If the decision tree model is trained to capture small-scale, idiosyncratic patterns, then classification accuracy for malicious traffic is improved, but the model complexity increases
Solution Approach 1:
The patent segments the decision tree into two parts: a primary tree with limited depth for capturing large-scale patterns, and a secondary tree for capturing small-scale patterns. This segmentation prevents the single model from becoming overly complex while still achieving high accuracy by distributing the pattern-capturing task across two specialized structures.
Solution Approach 2:
The patent applies partial action by having the primary decision tree handle only the most prominent large-scale patterns, while delegating the more complex small-scale pattern recognition to the secondary classification stage. This partial specialization allows each component to focus on its strengths without unnecessary complexity.
3Measurement precision
If hyperparameters are adjusted to prioritize small-scale patterns, then detection of complex traffic patterns is improved, but the training process becomes more difficult
Solution Approach 1:
The patent segments the training process into two phases: training the primary decision tree on large-scale patterns with standard hyperparameters, and then training the secondary decision tree on residual small-scale patterns. This segmentation simplifies the overall training difficulty by breaking down the complex task of capturing all patterns into two more manageable training stages with different focus areas.
Data Source
AI summary
Systems and methods are provided for classifying network traffic flows across a network. Specifically, the network traffic flows are classified under a fully-segmented ruleset, wherein the fully segmented ruleset was generated by training a decision tree machine learning (“ML”) algorithm with a training dataset, and wherein each item of the training dataset satisfies the complete rule pathway to different leaf nodes of the fully segmented ruleset. Classification under a fully-segmented ruleset allowing for capture of idiosyncratic patterns specific to a given malicious source of network traffic flows. Further, systems and methods are provided allowing for a user to designate network traffic flows for classification of network traffic flows at different network devices, wherein the classification at different network devices may allow for more computationally intensive classification.