Netflow Analysis Using Tuple Sorting for Attack Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current netflow analyzing engines are inefficient in identifying network attacks like distributed denial of service and port scanning due to data filtering methods that lose essential flow information, such as source and destination ports, leading to missed detections and unrealistic processing times for analyzing all flows.
Innovation Solution
The method involves storing and sorting flow information to identify associations between IP addresses and ports, ranking conversations by volume, and using tuple data structures to efficiently process and analyze netflow data, enabling real-time detection of attacks by counting distinct associations and patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all flows are analyzed to identify attack patterns, then detection accuracy is improved, but processing time and storage requirements become unrealistic
Solution Approach 1:
The patent segments the analysis process into two phases: a filtering phase that quickly eliminates normal flows using criteria like octet size thresholds and connection state filters, and a detailed analysis phase that focuses only on suspicious flows. This segmentation allows the system to maintain high detection accuracy for attacks while avoiding the computational burden of analyzing all flows in detail.
Solution Approach 2:
The patent applies partial action by analyzing only the subset of flows that meet specific suspicious criteria rather than all flows. By using filters that identify potentially malicious flows (e.g., small octet sizes, unusual connection patterns), the system performs sufficient analysis to detect attacks without the excessive processing time required for complete flow analysis.
2Quantity of substance
If flow information is filtered to retain only top IP addresses by volume, then storage requirements are reduced, but distribution information and attack context are lost
Solution Approach 1:
The patent applies local quality by applying different retention criteria to different types of flow information. Instead of uniformly filtering all flows by volume, the system preserves detailed information for flows that exhibit suspicious characteristics (such as small octet sizes or unusual connection patterns) while summarizing or filtering normal high-volume flows. This allows the system to maintain attack detection capability without storing all flow details.
Solution Approach 2:
The patent performs preliminary filtering and classification of flows before detailed analysis, identifying which flows require full information retention based on their characteristics. By pre-identifying suspicious flows using lightweight criteria, the system ensures that context information is preserved for attacks while reducing storage for normal traffic.
3Quantity of substance
If flow subset is selected based on highest octet size, then storage is reduced and some distribution information retained, but attacks with small octet sizes become invisible
Solution Approach 1:
The patent performs preliminary classification of flows based on multiple criteria including octet size, connection state, and traffic patterns before selecting flows for detailed analysis. By pre-identifying flows with characteristics typical of attacks (such as small octet sizes combined with unusual connection patterns), the system ensures that malicious flows are not filtered out despite their small size, while still maintaining reduced storage requirements for normal traffic.
Solution Approach 2:
The patent changes the selection parameters from solely octet size-based filtering to a multi-parameter approach that includes connection state, traffic patterns, and anomaly detection metrics. This allows the system to identify and retain flows with small octet sizes that exhibit attack characteristics, while filtering out normal small flows, thereby maintaining detection reliability without excessive storage requirements.
4Speed
If netflow data is analyzed in real-time to identify attacks, then response time is improved, but processing complexity and resource requirements increase
Solution Approach 1:
The patent segments the real-time processing into multiple stages: initial filtering using simple criteria, intermediate analysis of suspicious flows, and detailed investigation of confirmed threats. This segmentation allows real-time response while managing complexity by applying different processing intensities at different stages, avoiding the need for complex analysis on all flows simultaneously.
Solution Approach 2:
The patent introduces intermediary filtering mechanisms and heuristics that simplify real-time decision-making. By using intermediate classification layers (such as connection state filters and traffic pattern matchers) between raw flow data and detailed analysis, the system achieves real-time detection without requiring full complex analysis of every flow, thereby reducing processing complexity while maintaining response speed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A flow based detection system for detecting networks attacks on data networks. Flow records are collected in a novel data structure that facilitates efficient sorting. The sorted data structure can be subsequently analyzed in an efficient manner to find out if the network is under attack. An attack is identified if the numbers of unique corresponding addresses or conversations are too large.