Adaptive Network Traffic Sampling via Collaborative Intelligence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network traffic monitoring techniques, such as packet-based and flow-based sampling, often miss malicious traffic, including threats from botnets and other malware, due to their random sampling methods, making it difficult to detect and prevent malicious activities effectively.
Innovation Solution
Collaborative sampling techniques that adaptively adjust sampling criteria using global intelligence data from multiple networks, correlating statistical properties of network traffic with known malicious patterns to enhance detection and reduce the volume of data for analysis, allowing for more efficient and precise identification of malicious traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If random sampling methods (packet-based or flow-based) are used to monitor network traffic, then the monitoring process becomes feasible given the high volume of traffic, but malicious network traffic may be missed due to the random nature of sampling
Solution Approach 1:
The system performs preliminary actions by collecting statistical properties of network traffic flows before final detection. It aggregates flow data, computes statistics (packets per flow, bytes per flow, duration, etc.), and pre-processes this information to create a structured dataset that can be efficiently queried and analyzed for malicious patterns, thereby improving both feasibility and detection accuracy
Solution Approach 2:
The patent introduces an intermediary layer between raw network traffic and detection analysis. This intermediary computes and aggregates statistical properties of traffic flows, creating a intermediate representation that preserves essential characteristics while reducing data volume. This intermediary statistical layer enables efficient monitoring while maintaining detection capability for malicious traffic patterns
2Reliability
If all network traffic is monitored to ensure complete detection of malicious activity, then detection accuracy improves, but the complexity and resource requirements of the monitoring system become prohibitively high
Solution Approach 1:
The system extracts only the essential statistical properties from complete network traffic data. Instead of monitoring all raw packets, it extracts key features such as packets per flow, bytes per flow, flow duration, source/destination addresses, and protocol information. This extraction maintains detection accuracy for malicious patterns while dramatically reducing system complexity and resource requirements
Solution Approach 2:
The patent applies partial action by monitoring a carefully selected subset of traffic characteristics rather than all possible parameters. It focuses on extracting and analyzing specific statistical properties that are most indicative of malicious activity, achieving effective detection with reduced complexity compared to comprehensive full-traffic analysis
3Reliability
If the sampling rate is increased to capture more malicious traffic, then detection accuracy improves, but the volume of data requiring analysis increases, consuming more resources
Solution Approach 1:
The system performs preliminary aggregation and statistical computation on network traffic flows before final analysis. By pre-computing statistics such as packets per flow, bytes per flow, and flow duration, it creates a condensed representation that captures malicious patterns while minimizing the volume of data requiring detailed analysis, thus improving detection without proportionally increasing resource consumption
Solution Approach 2:
The patent segments network traffic into discrete flows and computes statistics for each flow independently. This segmentation allows efficient processing by grouping related packets together and analyzing them as unified units with aggregated properties, reducing overall data volume while maintaining detection capability for malicious traffic patterns across the network
Data Source
AI summary
Identifying malicious network traffic based on distributed, collaborative sampling includes, at a computing device having connectivity to a network, obtaining a first set of data flows, based on sampling criteria, that represents network traffic between one or more nodes in the network and one or more domains outside of the network, each data flow in the first set of data flows including a plurality of data packets. The first set of data flows is forwarded for correlation with a plurality of other sets of data flows from other networks to generate global intelligence data. Adjusted sampling criteria is generated based on the global intelligence data and a second set of data flows is obtained based on the adjusted sampling criteria.