Outlier Classification for Network Traffic Flows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional traffic flow analytics methods lack refinement in outlier detection and are delayed in identifying malicious or problematic activity due to their reliance on bulk data exports over long time intervals, leading to delayed precautionary actions.
Innovation Solution
Implementing a multiple sub-time window sampling architecture with outlier detection at the network device level, allowing for real-time analysis of traffic flow data in short intervals to reduce average value convergence and false positives, and enabling quicker identification of malicious activity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traffic flow analytics are performed over long time intervals using bulk data exports, then network resources are conserved, but outlier detection refinement and real-time identification of malicious activity are compromised
Solution Approach 1:
The patent segments the long time interval into multiple sub-time windows, allowing outlier detection to be performed on shorter intervals while still conserving network resources through selective bulk exports. This segmentation enables refined outlier detection without requiring continuous real-time analysis of all traffic data.
Solution Approach 2:
The patent applies partial action by performing outlier detection only on sampled traffic flows rather than all traffic flows. This selective approach maintains measurement precision for outlier detection while reducing the overall processing load and network resource consumption.
2Loss of energy
If traffic flow analytics are performed over long time intervals, then network resources are conserved, but the speed of identifying malicious activity is reduced
Solution Approach 1:
By dividing the analysis into multiple sub-time windows within the bulk export interval, the system can identify malicious activity more quickly within each sub-window while still exporting data in bulk at longer intervals to conserve network resources.
Solution Approach 2:
The patent performs preliminary outlier detection analysis on sampled traffic flows during the bulk export interval, identifying potential malicious activity before the actual bulk export occurs. This preliminary action enables faster identification and response to threats.
3Measurement precision
If multiple sub-time window sampling is implemented with outlier detection at network device level, then outlier detection accuracy is enhanced, but device complexity increases
Solution Approach 1:
The patent extracts only sampled traffic flows for outlier detection analysis rather than processing all traffic flows. This extraction approach enhances detection accuracy for the sampled subset while limiting the processing complexity to only the necessary portion of traffic data.
Solution Approach 2:
The network device performs outlier detection on its own sampled traffic flows without requiring external analytics services. This self-service capability enhances detection accuracy by keeping analysis local while avoiding the complexity of integrated external analytics systems.
4Reliability
If outlier detection is performed on sampled traffic flows using multiple sub-time windows, then false positives are reduced through aggregation, but processing time increases
Solution Approach 1:
The patent segments outlier detection into multiple sub-time windows that process traffic flows in smaller batches. Aggregation of results across these segments reduces false positives while the segmented approach limits the processing time required for each individual segment.
Solution Approach 2:
The patent applies partial action by performing outlier detection only on sampled traffic flows rather than all traffic flows. This selective processing reduces the total processing time while still achieving reliable false positive reduction through aggregation across multiple sub-time windows.
Data Source
AI summary
Systems and methods are provided for combining a multiple sub-time window sampling architecture with machine learning to detect outlier traffic flow behavior which may indicate malicious/problematic network activity. For example, a network device may obtain a sample of traffic flow data during a defined time window. The sample of traffic flow data may comprise information associated with a sampled subset of traffic flows transferred by a network device in the defined time window. The network device may partition the defined time window into two or more sub-time windows. In each sub-time window, using machine learning, the network device may assign an outlier-related classification to each sampled traffic flow based on the relative behavioral characteristics of all the sampled traffic flows. The network device may aggregate the outlier-related classifications for each sampled traffic flow across multiple sub-time windows, and process traffic flows based on the aggregated outlier-related classifications.