Adaptive Network Traffic Sampling for Malicious Channel Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity and volume of network traffic, particularly due to cloud data centers, make it difficult to monitor and identify malicious botnets, as existing sampling techniques often fail to capture rare data flows associated with Command & Control (C2) channels, leading to incomplete reconstructions and inefficient resource utilization.
Innovation Solution
The implementation of late, adaptive, flow-based sampling, which extracts features from network traffic prior to sampling to intentionally bias the selection of data flows, ensuring that sampled data includes relevant malicious communication channels, thereby preserving rare flows and optimizing the reconstruction of C2 networks with minimal resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional packet-based or flow-based sampling is used to monitor network traffic, then resource consumption is reduced, but malicious communication channels are missed due to insufficient capture of rare data flows
Solution Approach 1:
The patent changes the sampling parameter from fixed probability to adaptive probability based on flow characteristics. By calculating entropy values and other statistical features of network flows, the system dynamically adjusts sampling rates to prioritize rare and suspicious flows while maintaining lower rates for common flows, thereby improving detection accuracy without proportionally increasing sampled data volume
Solution Approach 2:
The patent performs preliminary analysis of network flow characteristics (entropy, packet counts, duration) before applying sampling. This preliminary action allows the system to identify and prioritize rare flows that are more likely to contain malicious communication channels, ensuring these critical flows are captured in the sample before the actual sampling occurs
2Reliability
If all network traffic is monitored to ensure complete detection of malicious botnets, then detection completeness is improved, but computational demands and resource consumption increase significantly
Solution Approach 1:
The patent applies different sampling qualities to different parts of the network traffic. Rare and suspicious flows receive high-quality sampling (high probability of capture), while common benign flows receive low-quality sampling (low probability of capture). This localized quality approach ensures detection completeness for malicious channels while minimizing overall resource consumption by not uniformly sampling all traffic at high rates
Solution Approach 2:
The patent extracts and focuses on the most critical subset of network flows for sampling by identifying rare flows with unusual characteristics. Instead of sampling all traffic uniformly, the system extracts only the flows that are most likely to contain malicious communication channels based on statistical features, thereby achieving detection completeness with minimal resource investment
3Measurement precision
If sampling rate is increased to capture rare malicious flows, then detection accuracy is improved, but resource consumption and processing overhead increase
Solution Approach 1:
The patent implements dynamic sampling rates that adapt to the characteristics of each network flow. Rather than using a static high sampling rate for all traffic, the system dynamically adjusts the sampling probability based on real-time flow analysis, entropy calculations, and rarity metrics. This dynamic approach maintains high measurement precision for rare malicious flows while preserving processing efficiency by using lower rates for common flows
Data Source
AI summary
Identifying malicious communications by generating data representative of network traffic based on adaptive sampling includes, at a computing device having connectivity to a network, obtaining a set of data flows representing network traffic between one or more nodes in the network and one or more domains outside of the network, wherein each data flow in the set of data flows includes a plurality of data packets. One or more features are extracted from the set of data flows based on statistical measurements of the set of data flows. The set of data flows are adaptively sampled based on at least the one or more features. Then, data representative of the network traffic is generated based on the adaptively sampling to identify malicious communication channels in the network traffic.